-
Notifications
You must be signed in to change notification settings - Fork 0
feat(sampler): iCKB sampling tool, config, README, and initial data #9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| **/*{_,.}{test,spec}.* |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| # iCKB Sampler | ||
|
|
||
| An utility to help sampling iCKB rate across time. | ||
|
|
||
| ## Run the sampler on mainnet | ||
|
|
||
| 1. Download this repo in a folder of your choice: | ||
|
|
||
| ```bash | ||
| git clone https://github.com/ickb/stack.git | ||
| ``` | ||
|
|
||
| 2. Enter into the repo folder: | ||
|
|
||
| ```bash | ||
| cd stack/apps/sampler | ||
| ``` | ||
|
|
||
| 3. Install dependencies: | ||
|
|
||
| ```bash | ||
| pnpm install | ||
| ``` | ||
|
|
||
| 4. Build project: | ||
|
|
||
| ```bash | ||
| pnpm build | ||
| ``` | ||
|
|
||
| 5. Start the sampler utility: | ||
|
|
||
| ```bash | ||
| pnpm start | ||
| ``` | ||
|
|
||
| ## Licensing | ||
|
|
||
| This source code, crafted with care by [Phroi](https://phroi.com/), is freely available on [GitHub](https://github.com/ickb/stack) and it is released under the [MIT License](../../LICENSE). | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| { | ||
| "name": "@ickb/sampler", | ||
| "version": "1001.0.0", | ||
| "description": "iCKB sampler built on top of CCC", | ||
| "keywords": [ | ||
| "ickb", | ||
| "ccc", | ||
| "ckb", | ||
| "blockchain" | ||
| ], | ||
| "author": "phroi", | ||
| "license": "MIT", | ||
| "homepage": "https://ickb.org", | ||
| "repository": { | ||
| "type": "git", | ||
| "url": "https://github.com/ickb/stack" | ||
| }, | ||
| "bugs": { | ||
| "url": "https://github.com/ickb/stack/issues" | ||
| }, | ||
| "sideEffects": false, | ||
| "type": "module", | ||
| "main": "dist/index.js", | ||
| "types": "dist/index.d.ts", | ||
| "exports": { | ||
| ".": { | ||
| "import": "./dist/index.js", | ||
| "types": "./dist/index.d.ts" | ||
| } | ||
| }, | ||
| "scripts": { | ||
| "test": "vitest", | ||
| "test:ci": "vitest run", | ||
| "build": "tsc", | ||
| "lint": "eslint ./src", | ||
| "clean": "rm -fr dist", | ||
| "clean:deep": "rm -fr dist node_modules pnpm-lock.yaml", | ||
phroi marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| "start": "node dist/index.js | tee rate.csv" | ||
| }, | ||
| "files": [ | ||
| "dist", | ||
| "src" | ||
| ], | ||
| "publishConfig": { | ||
| "access": "public", | ||
| "provenance": true | ||
| }, | ||
| "devDependencies": { | ||
| "@types/node": "^24.7.0" | ||
| }, | ||
| "dependencies": { | ||
| "@ckb-ccc/core": "catalog:", | ||
| "@ickb/core": "workspace:*", | ||
| "@ickb/utils": "workspace:*" | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| BlockNumber, Date, Value, Note | ||
| 0, 2019-11-15T21:09:50.812Z, 1.00082, Genesis | ||
| 413943, 2020-01-01T00:00:36.936Z, 1.00553737, | ||
| 1311192, 2020-04-01T12:00:02.732Z, 1.01529582, | ||
| 2225181, 2020-07-02T00:00:01.791Z, 1.02475181, | ||
| 2887376, 2020-10-01T12:00:03.001Z, 1.03388539, | ||
| 3583555, 2021-01-01T00:00:01.008Z, 1.04279524, | ||
| 4048565, 2021-04-02T06:00:46.073Z, 1.05146502, | ||
| 4697782, 2021-07-02T12:00:00.086Z, 1.05990675, | ||
| 5503834, 2021-10-01T18:00:03.408Z, 1.06816083, | ||
| 6194506, 2022-01-01T00:00:01.554Z, 1.07621462, | ||
| 6822996, 2022-04-02T06:00:46.874Z, 1.08407983, | ||
| 7548564, 2022-07-02T12:00:01.029Z, 1.09176325, | ||
| 8188955, 2022-10-01T18:00:08.715Z, 1.09928404, | ||
| 8877418, 2023-01-01T00:00:00.729Z, 1.10664516, | ||
| 9562759, 2023-04-02T06:00:11.059Z, 1.11387639, | ||
| 10360745, 2023-07-02T12:00:27.581Z, 1.12095011, | ||
| 11084807, 2023-10-01T18:00:58.702Z, 1.12788772, | ||
| 11850353, 2024-01-01T00:00:04.666Z, 1.13469412, | ||
| 12600876, 2024-04-01T12:00:04.633Z, 1.1414592, | ||
| 13377494, 2024-07-02T00:00:18.459Z, 1.14815852, | ||
| 14010067, 2024-09-12T15:13:19.574Z, 1.15343076, iCKB Launch | ||
| 14160825, 2024-10-01T12:00:24.531Z, 1.15479538, | ||
| 15001370, 2025-01-01T00:00:12.070Z, 1.1613766, | ||
| 15799566, 2025-04-02T06:00:09.602Z, 1.16787482, | ||
| 16605969, 2025-07-02T12:00:00.609Z, 1.17431822, | ||
| 17426528, 2025-10-01T18:00:20.609Z, 1.18071088, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,192 @@ | ||
| /** | ||
| * @packageDocumentation | ||
| * | ||
| * Entry-point script that samples block headers from a CKB mainnet public client | ||
| * and prints a CSV report (BlockNumber, Date, Value, Note). | ||
| * | ||
| * Summary of behavior: | ||
| * - Constructs a `ccc.ClientPublicMainnet` client and queries the genesis and tip headers. | ||
| * - Builds a set of Date samples between genesis and tip (including a small set of | ||
| * named dates such as "Genesis", "iCKB Launch", and the "Tip"). | ||
| * - For each sample date, performs a binary search over block numbers to find | ||
| * the first block whose timestamp is greater than or equal to the sample date. | ||
| * - Logs CSV lines with block number, ISO timestamp, converted value, and an optional note. | ||
| * | ||
| * Remarks: | ||
| * - The sampling functions accept timestamps as bigint millisecond values. | ||
| * - This file runs in Node.js (uses top-level await) and exits on completion or error. | ||
| * - Failures in fetching blocks will throw. | ||
| * | ||
| * Example output (CSV): | ||
| * BlockNumber, Date, Value, Note | ||
| * 0, 2019-11-15T21:09:50.812Z, 1.00082, Genesis | ||
| * | ||
| * @public | ||
| */ | ||
|
|
||
| import { ccc } from "@ckb-ccc/core"; | ||
| import { convert } from "@ickb/core"; | ||
| import { asyncBinarySearch } from "@ickb/utils"; | ||
|
|
||
| /** | ||
| * Main program that orchestrates sampling and logging. | ||
| * | ||
| * - Constructs a public mainnet client. | ||
| * - Fetches genesis and tip headers (throws if missing). | ||
| * - Computes an upper bound `n` for the block-number binary search using the | ||
| * bit-length of tip.number (a simple power-of-two bound). | ||
| * - Generates date samples (per-year, `n` samples per year) and inserts a | ||
| * named "iCKB Launch" sample. | ||
| * - For each date sample, finds the earliest block whose timestamp >= sample | ||
| * date via `asyncBinarySearch` and logs a CSV row for that header. | ||
| * | ||
| * Notes on error handling: | ||
| * - Missing blocks will cause this function to throw. | ||
| * | ||
| * @returns Promise<void> that resolves when sampling and logging complete. | ||
| * | ||
| * @public | ||
| */ | ||
| export async function main(): Promise<void> { | ||
| // Create a public mainnet client (network I/O happens on method calls). | ||
| const client = new ccc.ClientPublicMainnet(); | ||
|
|
||
| // Fetch genesis header (block 0). If absent, abort early. | ||
| const genesis = await client.getHeaderByNumber(0); | ||
| if (!genesis) { | ||
| throw new Error("Genesis block not found"); | ||
| } | ||
|
|
||
| // Fetch tip header to bound our searches. | ||
| const tip = await client.getTipHeader(); | ||
|
|
||
| // Compute an upper bound `n` for the binary search using the bit-length | ||
| // of the tip number. This yields a power-of-two >= tip.number. | ||
| const n = 1 << tip.number.toString(2).length; | ||
|
|
||
| // Generate date samples between genesis and tip (timestamps are bigints in ms). | ||
| // The samples(...) helper returns Date instances; attach optional notes here. | ||
| const dates = samples(genesis.timestamp, tip.timestamp, 4).map( | ||
| (d) => [d, ""] as [Date, string], | ||
| ); | ||
| // Insert a named event sample (kept as an example of adding special dates). | ||
| dates.push([new Date("2024-09-12T15:13:19.574Z"), "iCKB Launch"]); | ||
| // Ensure chronological order across all samples (safety). | ||
| dates.sort((a, b) => a[0].getTime() - b[0].getTime()); | ||
|
|
||
| // Emit CSV header and the genesis row. | ||
| console.log(["BlockNumber", "Date", "Value", "Note"].join(", ")); | ||
| logRow(genesis, "Genesis"); | ||
|
|
||
| // For each sample date, find the earliest block whose timestamp is >= date. | ||
| for (const [date, note] of dates) { | ||
| // asyncBinarySearch expects a predicate that returns true when the index i | ||
| // is at or past the desired condition. We provide a predicate that fetches | ||
| // the header and compares timestamps. | ||
| const blockNumber = await asyncBinarySearch( | ||
| n, | ||
| async (i: number): Promise<boolean> => { | ||
| const header = await client.getHeaderByNumber(i); | ||
| if (!header) { | ||
| // If there's no header at i, signal "true" so the search moves left. | ||
| return true; | ||
| } | ||
| // header.timestamp is numeric-like; convert to Number and compare to Date. | ||
| return date <= new Date(Number(header.timestamp)); | ||
| }, | ||
| ); | ||
|
|
||
| // Fetch header for the found block number and log it. | ||
| const header = await client.getHeaderByNumber(blockNumber); | ||
| if (!header) { | ||
| throw Error("Header not found"); | ||
| } | ||
|
|
||
| logRow(header, note); | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Log a CSV row for a header. | ||
| * | ||
| * Behavior: | ||
| * - Converts the header value via `convert(false, ccc.One, header)`, | ||
| * formats it with `ccc.fixedPointToString`, and writes a CSV line. | ||
| * - This helper is intentionally lightweight and will throw only on programmer errors | ||
| * (e.g. unexpected undefined header when called). | ||
| * | ||
| * @param header - Block header to log. | ||
| * @param note - Optional short note to include in the CSV row (e.g. "Genesis"...). | ||
| * | ||
| * @internal | ||
| */ | ||
| function logRow(header: ccc.ClientBlockHeader, note: string): void { | ||
| // Compute ISO timestamp from header timestamp (milliseconds). | ||
| const date = new Date(Number(header.timestamp)); | ||
| // Convert the header's monetary value to a fixed-point representation. | ||
| const val = convert(false, ccc.One, header); | ||
| // Emit CSV row: blockNumber, ISO date, formatted value, note. | ||
| console.log( | ||
| [ | ||
| String(header.number), | ||
| date.toISOString(), | ||
| ccc.fixedPointToString(val), | ||
| note, | ||
| ].join(", "), | ||
| ); | ||
| } | ||
|
|
||
| /** | ||
| * Generate a set of sample Dates between two millisecond-based bigints. | ||
| * | ||
| * The function: | ||
| * - Splits the overall [startMs, endMs] span by UTC calendar years. | ||
| * - Emits `n` evenly-spaced samples within each year span [Y0, Y1). | ||
| * - Uses integer-rounded millisecond timestamps and returns Date objects. | ||
| * | ||
| * @param startMs - Inclusive start of the sampling range as a bigint (ms since epoch). | ||
| * @param endMs - Inclusive end of the sampling range as a bigint (ms since epoch). | ||
| * @param n - Number of evenly-spaced samples to generate per year span. Must be >= 1. | ||
| * | ||
| * @returns An array of Date objects. Samples are generated year-by-year; calling | ||
| * code may sort again for global ordering (the caller does so). | ||
| * | ||
| * @throws Error if endMs < startMs or if n < 1. | ||
| * | ||
| * @public | ||
| */ | ||
| export function samples(startMs: bigint, endMs: bigint, n: number): Date[] { | ||
| if (endMs < startMs) throw Error("endMs must be bigger than startMs"); | ||
| if (n < 1) throw Error("n must be a positive number"); | ||
|
|
||
| // Convert bigints (ms) to Dates for year extraction. | ||
| const start = new Date(Number(startMs)); | ||
| const end = new Date(Number(endMs)); | ||
| const startYear = start.getUTCFullYear(); | ||
| const endYear = end.getUTCFullYear(); | ||
| const out: Date[] = []; | ||
|
|
||
| // For each UTC year in the covered range, generate n samples inside that year. | ||
| for (let year = startYear; year <= endYear; year++) { | ||
| // Y0 is start of `year` in ms (UTC), Y1 is start of next year. | ||
| const Y0 = Date.UTC(year, 0, 1); | ||
| const Y1 = Date.UTC(year + 1, 0, 1); | ||
| const span = Y1 - Y0; | ||
|
|
||
| for (let i = 0; i < n; i++) { | ||
| // Evenly space n samples in [Y0, Y1). Round to nearest millisecond. | ||
| const t = Y0 + Math.round((span * i) / n); | ||
| const sample = new Date(t); | ||
| // Only include samples that fall within the inclusive overall range. | ||
| if (sample >= start && sample <= end) { | ||
| out.push(sample); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| return out; | ||
| } | ||
|
|
||
| await main(); | ||
|
|
||
| process.exit(0); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| { | ||
| "extends": "../../tsconfig.json", | ||
| "compilerOptions": { | ||
| "rootDir": "src", | ||
| "outDir": "dist", | ||
| "sourceRoot": "../src" | ||
| }, | ||
| "include": ["src"], | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| { | ||
| "$schema": "https://typedoc.org/schema.json", | ||
| "entryPoints": ["./src/index.ts"], | ||
| "extends": ["../../typedoc.base.json"], | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| import { defineConfig } from "vitest/config"; | ||
|
|
||
| export default defineConfig({ | ||
| test: { | ||
| include: ["src/**/*.test.ts"], | ||
| coverage: { | ||
| include: ["src/**/*.ts"], | ||
| }, | ||
| }, | ||
| }); |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.