{"id":21019461,"url":"https://github.com/ceramicnetwork/benchie","last_synced_at":"2026-04-27T09:31:04.758Z","repository":{"id":42425742,"uuid":"466173439","full_name":"ceramicnetwork/benchie","owner":"ceramicnetwork","description":null,"archived":false,"fork":false,"pushed_at":"2022-04-09T13:00:04.000Z","size":76,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-12-29T02:35:08.336Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ceramicnetwork.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-03-04T15:25:18.000Z","updated_at":"2022-03-18T13:59:21.000Z","dependencies_parsed_at":"2022-08-31T03:31:20.765Z","dependency_job_id":null,"html_url":"https://github.com/ceramicnetwork/benchie","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ceramicnetwork/benchie","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ceramicnetwork%2Fbenchie","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ceramicnetwork%2Fbenchie/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ceramicnetwork%2Fbenchie/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ceramicnetwork%2Fbenchie/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ceramicnetwork","download_url":"https://codeload.github.com/ceramicnetwork/benchie/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ceramicnetwork%2Fbenchie/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32331305,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-26T23:26:28.701Z","status":"online","status_checked_at":"2026-04-27T02:00:06.769Z","response_time":128,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-19T10:32:22.839Z","updated_at":"2026-04-27T09:31:04.737Z","avatar_url":"https://github.com/ceramicnetwork.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Ceramic performance testing suite\n\nMeasuring performance of Ceramic nodes.\n\n## Table of contents\n\n- [Background](#background)\n- [Usage](#usage)\n- [Contributing](#contributing)\n- [License](#license)\n\n## Background\n\nAs a developers of a Ceramic network, we would like to have a repeatable way of measuring node performance.\nThis repository contains performance scenarios as well as a toolkit to run them.\n\nIn essence, a scenario executes a single function many times (configurable). We measure how much time each run takes.\nAfter all runs are processed, we calculate statistics - average, minimum, maximum.\nWe understand that some preparations might be required for the function to run. Current API allows a developer to specify\n`beforeAll`, `beforeEach`, `afterAll`, `afterEach`, similar to Jest - to perform adequate tear-up and tear-down.\nThese do not contribute to measurements.\n\nThe performance suite is a macro-benchmark, as opposed to a micro-benchmark. We do not have to worry much about GC interference.\nAll code is written as ES modules, to be able to use Ceramic packages written as ES modules.\n\n## Usage\n\nBefore running the suite:\n\n- download the repository\n- `npm install`\n- `npm run build`\n\n### Running performance suite\n\nThe whole suite is written in TypeScript. We do not want to deal with TypeScript compilation inside our code.\nWe expect an engineer to compile the suite to JavaScript via `npm run build` before running.\n\nThere are two options to run the suite:\n\n1. `node ./dist/benchie/bin/benchie.js`\n\n   It will find all the `*.bench.js` files in the current directory and use them as a source of performance scenarios.\n   These `*.bench.js` files are compiled from TypeScript. Sources are in `*.bench.ts` files.\n\n2. `npm run start`\n\n   It is an alias for the command above.\n\nThe results are printed to `STDOUT` by default. You could see additional options by running the same `benchie` executable with `--help` flag:\n\n- `node ./dist/benchie/bin/benchie.js --help` or\n- `npm run start -- --help`.\n\nThe configuration can be adjusted through your `.env` file. Defaults are provided in `.env.defaults`.\n\n### Writing scenario\n\nA scenario file name should have a name conforming to `*.bench.ts` glob.\nFeel free to put it anywhere inside `/src` folder. One file could contain multiple scenarios.\n\nWe strive to make a scenario code similar to [Jest](https://jestjs.io) test. Generally, a scenario looks like this:\n\n```typescript\nimport { scenario } from \"./benchie/benchmark.js\"; // If the scenario file is in `/src`\n\nscenario(\"Scenario Title\", (perform) =\u003e {\n  perform.tagged('first', 'second'); // Optional. Can later run only scenarios with specific tags.\n  // Place for ancillary stuff: variables, helper functions, etc.\n\n  perform.beforeAll(async () =\u003e {\n    // Code to be executed once before all the runs.\n    // It does not affect performance metrics.\n  });\n\n  perform.beforeEach(async () =\u003e {\n    // Code to be executed before each run.\n    // It does not affect performance metrics.\n  });\n\n  perform.afterAll(async () =\u003e {\n    // Code to be executed once after all the runs.\n    // It does not affect performance metrics.\n  });\n\n  perform.afterEach(async () =\u003e {\n    // Code to be executed after each run.\n    // It does not affect performance metrics.\n  });\n\n  perform.times(1).run(async () =\u003e {\n    // Measured code: single run.\n    // Executed just `1` time here. You could pass any suitable number instead.\n  });\n});\n```\n\nHooks like `beforeAll`, `beforeEach`, `afterAll`, `afterEach` are optional. Feel free to only use them when needed.\nYou do not have to worry about how much time a hook takes. Hook execution time do not contribute to performance metrics.\n\n### Internals\n\n**Processing:** `benchie` executable by default tries to load every `*.bench.js` file in the current folder.\nIt then parses every scenario there. After all the loading is done,\nit starts to sequentially run the scenarios.\n\n**Folder layout:** All the code should be written in TypeScript. The files should be located in `/src` folder.\nA file that contains a scenario should be named `*.bench.ts`. A bench-file should not import other bench-file.\nIt could import any other files though, so feel free to extract valuable shared functionality into own files.\nWhile you are free to use any folder structure for the bench-files in `/src`, it is advised not to put any scenarios\nto `/src/benchie`. This folder contains infrastructure for the scenario runner.\n\n## Contributing\n\nWe are happy to accept small and large contributions, feel free to make a suggestion or submit a pull request.\n\n## License\n\n[MIT](https://tldrlegal.com/license/mit-license) or [Apache-2.0](\u003chttps://tldrlegal.com/license/apache-license-2.0-(apache-2.0)\u003e)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fceramicnetwork%2Fbenchie","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fceramicnetwork%2Fbenchie","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fceramicnetwork%2Fbenchie/lists"}