https://github.com/eventdbx/eventdbx-perf
Benchmarking and testing EventDBX versus other database systems
https://github.com/eventdbx/eventdbx-perf
benchmark database-systems eventdbx performance typescript
Last synced: 6 months ago
JSON representation
Benchmarking and testing EventDBX versus other database systems
- Host: GitHub
- URL: https://github.com/eventdbx/eventdbx-perf
- Owner: eventdbx
- Created: 2025-11-01T19:08:48.000Z (8 months ago)
- Default Branch: main
- Last Pushed: 2025-11-08T05:45:07.000Z (8 months ago)
- Last Synced: 2025-11-11T07:19:47.095Z (8 months ago)
- Topics: benchmark, database-systems, eventdbx, performance, typescript
- Language: TypeScript
- Homepage: https://docs.eventdbx.com/benchmarking
- Size: 110 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# EventDBX Performance Benchmarks
This repository contains the TypeScript benchmark harnesses we use to exercise
EventDBX alongside the databases it most often competes or interoperates with
(PostgreSQL, MongoDB, Microsoft SQL Server, and the EventDBX control API).
Every suite is designed to be an apples-to-apples comparison:
- All containers run locally on the same Docker bridge network.
- Each service is started with identical CPU and memory limits (see
`src/docker-compose.yml`).
- Dataset sizes are identical across backends at each benchmark tier.
- The Tinybench harness, operation labels, and success validation are shared
verbatim between implementations.
The specs themselves seed synthetic data, execute a consistent set of
CRUD-style operations with [Tinybench](https://github.com/tinylibs/tinybench),
and print summarised throughput/latency statistics.
All benchmark sources live in `src/__tests__`. They must be transpiled to
`dist/__tests__` before AVA consumes them; the provided pnpm scripts wrap this
flow so you rarely have to invoke `tsc` directly.
## Prerequisites
- Node.js 18.18+ (the project uses ES modules and top-level `await`).
- Local or remote database instances that match the connection details in your
environment (see below). A `docker-compose` file is available under
`src/docker-compose.yml` for quick local provisioning.
## Installing & running
```bash
pnpm install # install dependencies
pnpm test # build the TypeScript sources and run all benchmarks
```
Running `pnpm test` performs a TypeScript build (`pnpm run build` →
`tsc --project tsconfig.json`)
and then invokes AVA against `dist/__tests__/**/*.spec.js`. Each suite attempts
to connect to its backend and will log a skip message (rather than fail) if the
service is unreachable or the optional driver is not installed.
## Test layout
| Spec | Backend | Highlights |
| ------------------------ | -------------------- | -------------------------------------------------------------------------------- |
| `bench-eventdbx.spec.ts` | EventDBX control API | Exercises the `eventdbxjs` client with a fixed auth token. |
| `bench-postgres.spec.ts` | PostgreSQL | Seeds zero-padded string aggregate IDs so ordering works without casts. |
| `bench-mongodb.spec.ts` | MongoDB | Bulk inserts synthetic documents and benchmarks common event-store operations. |
| `bench-mssql.spec.ts` | Microsoft SQL Server | Detects `ETIMEOUT` responses and treats them as skips rather than hard failures. |
| `bench-shared.ts` | Shared utilities | Dataset definitions, optional module loader, formatting helpers, etc. |
Default dataset sizes are defined in `bench-shared.ts` as
`[1_000, 10_000, 100_000, 1_000_000]`.
## Running a single suite
You can limit AVA to individual suites (or specific benchmark names) by calling
it directly or by forwarding arguments through the pnpm script:
```bash
# Build once, then run only the PostgreSQL benchmarks
pnpm run build
pnpm exec ava "dist/__tests__/bench-postgres.spec.js"
# Same suite, but constrain to operations whose labels contain "apply"
pnpm exec ava "dist/__tests__/bench-postgres.spec.js" --match="*apply*"
```
While iterating, you can also use AVA’s watch mode (`pnpm exec ava --watch ...`)
in combination with `pnpm exec tsc --watch` if you prefer continuous feedback.
### Controlling the operation mix
Benchmarks execute a shared set of read and write operations by default. You can
toggle the workload with the `BENCH_RUN_MODE` environment variable (alias:
`BENCH_MODE`). Set it inline, or add it to your project’s `.env` file (the harness
loads `.env` automatically):
```bash
# Only measure write-heavy tasks (apply/create/archive/restore/patch)
BENCH_RUN_MODE=write-only pnpm test
# Focus on read paths (list/get/select/events)
BENCH_RUN_MODE=read-only pnpm exec ava "dist/__tests__/bench-mongodb.spec.js"
```
Accepted values are `all` (default), `read` / `read-only`, and `write` /
`write-only`. When a mode excludes an operation, the suite logs a skip message
instead of running the task, so summaries only include the operations that match
the selected workload.
## EventDBX Test Token
EventDBX automatically generates a token on first start; you can retrieve it by reading the cli.token file.
```bash
docker compose -f src/docker-compose.yml exec eventdbx sh -c 'cat /var/lib/eventdbx/.eventdbx/cli.token'
```
## Environment configuration
`bench-shared.ts` loads environment variables once via `dotenv/config`. A sample
`.env` lives at the project root and already mirrors the defaults shown below;
feel free to copy/modify it for your environment.
| Variable | Default value | Description |
| ------------------------------------- | ----------------------------------------------------------------------------- | -------------------------------------------- |
| `EVENTDBX_MSSQL_CONN` | `Server=localhost;User Id=SA;Password=;TrustServerCertificate=True` | Connection string for MSSQL. |
| `EVENTDBX_MONGO_URI` | `mongodb://localhost:27017` | MongoDB connection URI. |
| `EVENTDBX_MONGO_DB` | `bench` | Mongo database name. |
| `EVENTDBX_MONGO_COLLECTION` | `events` | Events collection name. |
| `EVENTDBX_MONGO_AGGREGATE_COLLECTION` | `events_aggregates` | Aggregates collection name. |
| `EVENTDBX_PG_DSN` | `postgresql://bench:bench@localhost:5432/bench` | PostgreSQL DSN consumed by `pg.Pool`. |
| `EVENTDBX_TEST_IP` | `127.0.0.1` | EventDBX control client host. |
| `EVENTDBX_TEST_PORT` | `6363` | EventDBX control client port. |
| `EVENTDBX_TEST_TOKEN` | _static JWT in repo_ | Authentication token for the control client. |
| `BENCH_RUN_MODE` / `BENCH_MODE` | `all` | Operation mix (`all`, `read`, `write`). |
| `BENCH_OPERATION_LIMIT` | `100` | Shared page/event window size. |
| `BENCH_DATASET_SIZES` | `1,10000,100000` | Comma/space-separated dataset sizes. |
To spin up local dependencies quickly, use the provided compose file (passwords
match the defaults above):
```bash
docker compose -f src/docker-compose.yml up -d postgres mongodb mssql
```
Remember to tear the stack down afterwards (`docker compose down`).
## Troubleshooting tips
- **Timeouts when talking to MSSQL**: the harness treats `ETIMEOUT` responses as
skips and logs the failure. Double-check that port `1433` is reachable and the
SA password matches `EVENTDBX_MSSQL_CONN` if you expect the benchmark to run.
- **`Cannot find module 'pg'` (or similar)**: optional drivers are loaded at
runtime. Install the relevant package (`npm install pg`, `npm install mssql`,
etc.) or unset the corresponding environment variables to skip that backend.
- **AVA/npm operation timeouts**: if long-running suites exceed the default
timeout, re-run with a larger budget (for example
`AVA_TIMEOUT=10m pnpm test --timeout=10m`) so AVA’s watchdog doesn’t abort the
process mid-run.
- **Postgres progress parser warnings about casts**: aggregate IDs are seeded as
zero-padded strings and queries order lexicographically. If you still see
warnings, ensure you are running the updated suite (no explicit casts remain).
## Cleaning up
The benchmark specs delete existing synthetic data before seeding, but long
running sessions can still leave behind large datasets. Clean up per-backend
data directories as needed to reclaim space.