{"id":35731223,"url":"https://github.com/eventdbx/eventdbx-perf","last_synced_at":"2026-01-06T11:04:30.860Z","repository":{"id":322047276,"uuid":"1087847824","full_name":"eventdbx/eventdbx-perf","owner":"eventdbx","description":"Benchmarking and testing EventDBX versus other database systems","archived":false,"fork":false,"pushed_at":"2025-11-08T05:45:07.000Z","size":113,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-11-11T07:19:47.095Z","etag":null,"topics":["benchmark","database-systems","eventdbx","performance","typescript"],"latest_commit_sha":null,"homepage":"https://docs.eventdbx.com/benchmarking","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eventdbx.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-01T19:08:48.000Z","updated_at":"2025-11-11T07:15:52.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/eventdbx/eventdbx-perf","commit_stats":null,"previous_names":["thachp/eventdbx-perf","eventdbx/eventdbx-perf"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/eventdbx/eventdbx-perf","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eventdbx%2Feventdbx-perf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eventdbx%2Feventdbx-perf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eventdbx%2Feventdbx-perf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eventdbx%2Feventdbx-perf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eventdbx","download_url":"https://codeload.github.com/eventdbx/eventdbx-perf/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eventdbx%2Feventdbx-perf/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28225027,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2026-01-06T02:00:07.049Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","database-systems","eventdbx","performance","typescript"],"created_at":"2026-01-06T11:03:37.470Z","updated_at":"2026-01-06T11:04:30.854Z","avatar_url":"https://github.com/eventdbx.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# EventDBX Performance Benchmarks\n\nThis repository contains the TypeScript benchmark harnesses we use to exercise\nEventDBX alongside the databases it most often competes or interoperates with\n(PostgreSQL, MongoDB, Microsoft SQL Server, and the EventDBX control API).\nEvery suite is designed to be an apples-to-apples comparison:\n\n- All containers run locally on the same Docker bridge network.\n- Each service is started with identical CPU and memory limits (see\n  `src/docker-compose.yml`).\n- Dataset sizes are identical across backends at each benchmark tier.\n- The Tinybench harness, operation labels, and success validation are shared\n  verbatim between implementations.\n\nThe specs themselves seed synthetic data, execute a consistent set of\nCRUD-style operations with [Tinybench](https://github.com/tinylibs/tinybench),\nand print summarised throughput/latency statistics.\n\nAll benchmark sources live in `src/__tests__`. They must be transpiled to\n`dist/__tests__` before AVA consumes them; the provided pnpm scripts wrap this\nflow so you rarely have to invoke `tsc` directly.\n\n## Prerequisites\n\n- Node.js 18.18+ (the project uses ES modules and top-level `await`).\n- Local or remote database instances that match the connection details in your\n  environment (see below). A `docker-compose` file is available under\n  `src/docker-compose.yml` for quick local provisioning.\n\n## Installing \u0026 running\n\n```bash\npnpm install         # install dependencies\npnpm test            # build the TypeScript sources and run all benchmarks\n```\n\nRunning `pnpm test` performs a TypeScript build (`pnpm run build` →\n`tsc --project tsconfig.json`)\nand then invokes AVA against `dist/__tests__/**/*.spec.js`. Each suite attempts\nto connect to its backend and will log a skip message (rather than fail) if the\nservice is unreachable or the optional driver is not installed.\n\n## Test layout\n\n| Spec                     | Backend              | Highlights                                                                       |\n| ------------------------ | -------------------- | -------------------------------------------------------------------------------- |\n| `bench-eventdbx.spec.ts` | EventDBX control API | Exercises the `eventdbxjs` client with a fixed auth token.                       |\n| `bench-postgres.spec.ts` | PostgreSQL           | Seeds zero-padded string aggregate IDs so ordering works without casts.          |\n| `bench-mongodb.spec.ts`  | MongoDB              | Bulk inserts synthetic documents and benchmarks common event-store operations.   |\n| `bench-mssql.spec.ts`    | Microsoft SQL Server | Detects `ETIMEOUT` responses and treats them as skips rather than hard failures. |\n| `bench-shared.ts`        | Shared utilities     | Dataset definitions, optional module loader, formatting helpers, etc.            |\n\nDefault dataset sizes are defined in `bench-shared.ts` as\n`[1_000, 10_000, 100_000, 1_000_000]`.\n\n## Running a single suite\n\nYou can limit AVA to individual suites (or specific benchmark names) by calling\nit directly or by forwarding arguments through the pnpm script:\n\n```bash\n# Build once, then run only the PostgreSQL benchmarks\npnpm run build\npnpm exec ava \"dist/__tests__/bench-postgres.spec.js\"\n\n# Same suite, but constrain to operations whose labels contain \"apply\"\npnpm exec ava \"dist/__tests__/bench-postgres.spec.js\" --match=\"*apply*\"\n```\n\nWhile iterating, you can also use AVA’s watch mode (`pnpm exec ava --watch ...`)\nin combination with `pnpm exec tsc --watch` if you prefer continuous feedback.\n\n### Controlling the operation mix\n\nBenchmarks execute a shared set of read and write operations by default. You can\ntoggle the workload with the `BENCH_RUN_MODE` environment variable (alias:\n`BENCH_MODE`). Set it inline, or add it to your project’s `.env` file (the harness\nloads `.env` automatically):\n\n```bash\n# Only measure write-heavy tasks (apply/create/archive/restore/patch)\nBENCH_RUN_MODE=write-only pnpm test\n\n# Focus on read paths (list/get/select/events)\nBENCH_RUN_MODE=read-only pnpm exec ava \"dist/__tests__/bench-mongodb.spec.js\"\n```\n\nAccepted values are `all` (default), `read` / `read-only`, and `write` /\n`write-only`. When a mode excludes an operation, the suite logs a skip message\ninstead of running the task, so summaries only include the operations that match\nthe selected workload.\n\n## EventDBX Test Token\n\nEventDBX automatically generates a token on first start; you can retrieve it by reading the cli.token file.\n\n```bash\ndocker compose -f src/docker-compose.yml exec eventdbx sh -c 'cat /var/lib/eventdbx/.eventdbx/cli.token'\n```\n\n## Environment configuration\n\n`bench-shared.ts` loads environment variables once via `dotenv/config`. A sample\n`.env` lives at the project root and already mirrors the defaults shown below;\nfeel free to copy/modify it for your environment.\n\n| Variable                              | Default value                                                                 | Description                                  |\n| ------------------------------------- | ----------------------------------------------------------------------------- | -------------------------------------------- |\n| `EVENTDBX_MSSQL_CONN`                 | `Server=localhost;User Id=SA;Password=\u003cPassword\u003e;TrustServerCertificate=True` | Connection string for MSSQL.                 |\n| `EVENTDBX_MONGO_URI`                  | `mongodb://localhost:27017`                                                   | MongoDB connection URI.                      |\n| `EVENTDBX_MONGO_DB`                   | `bench`                                                                       | Mongo database name.                         |\n| `EVENTDBX_MONGO_COLLECTION`           | `events`                                                                      | Events collection name.                      |\n| `EVENTDBX_MONGO_AGGREGATE_COLLECTION` | `events_aggregates`                                                           | Aggregates collection name.                  |\n| `EVENTDBX_PG_DSN`                     | `postgresql://bench:bench@localhost:5432/bench`                               | PostgreSQL DSN consumed by `pg.Pool`.        |\n| `EVENTDBX_TEST_IP`                    | `127.0.0.1`                                                                   | EventDBX control client host.                |\n| `EVENTDBX_TEST_PORT`                  | `6363`                                                                        | EventDBX control client port.                |\n| `EVENTDBX_TEST_TOKEN`                 | _static JWT in repo_                                                          | Authentication token for the control client. |\n| `BENCH_RUN_MODE` / `BENCH_MODE`       | `all`                                                                         | Operation mix (`all`, `read`, `write`).      |\n| `BENCH_OPERATION_LIMIT`               | `100`                                                                         | Shared page/event window size.               |\n| `BENCH_DATASET_SIZES`                 | `1,10000,100000`                                                              | Comma/space-separated dataset sizes.         |\n\nTo spin up local dependencies quickly, use the provided compose file (passwords\nmatch the defaults above):\n\n```bash\ndocker compose -f src/docker-compose.yml up -d postgres mongodb mssql\n```\n\nRemember to tear the stack down afterwards (`docker compose down`).\n\n## Troubleshooting tips\n\n- **Timeouts when talking to MSSQL**: the harness treats `ETIMEOUT` responses as\n  skips and logs the failure. Double-check that port `1433` is reachable and the\n  SA password matches `EVENTDBX_MSSQL_CONN` if you expect the benchmark to run.\n- **`Cannot find module 'pg'` (or similar)**: optional drivers are loaded at\n  runtime. Install the relevant package (`npm install pg`, `npm install mssql`,\n  etc.) or unset the corresponding environment variables to skip that backend.\n- **AVA/npm operation timeouts**: if long-running suites exceed the default\n  timeout, re-run with a larger budget (for example\n  `AVA_TIMEOUT=10m pnpm test --timeout=10m`) so AVA’s watchdog doesn’t abort the\n  process mid-run.\n- **Postgres progress parser warnings about casts**: aggregate IDs are seeded as\n  zero-padded strings and queries order lexicographically. If you still see\n  warnings, ensure you are running the updated suite (no explicit casts remain).\n\n## Cleaning up\n\nThe benchmark specs delete existing synthetic data before seeding, but long\nrunning sessions can still leave behind large datasets. Clean up per-backend\ndata directories as needed to reclaim space.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feventdbx%2Feventdbx-perf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feventdbx%2Feventdbx-perf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feventdbx%2Feventdbx-perf/lists"}