{"id":50592307,"url":"https://github.com/exzvor/ide99-bench","last_synced_at":"2026-06-05T11:30:17.527Z","repository":{"id":356825179,"uuid":"1234197668","full_name":"exzvor/ide99-bench","owner":"exzvor","description":"Reproducible performance benchmarks for ide99, the PostgreSQL desktop IDE.","archived":false,"fork":false,"pushed_at":"2026-05-19T11:13:54.000Z","size":31,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-19T14:14:33.479Z","etag":null,"topics":["benchmarks","database-tools","desktop-app","ide99","nodejs","performance-testing","postgres","postgresql","python","reproducible-research","rust","sql-editor","tauri","tokio-postgres"],"latest_commit_sha":null,"homepage":"https://ide99.io","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/exzvor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-09T21:49:47.000Z","updated_at":"2026-05-19T11:16:28.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/exzvor/ide99-bench","commit_stats":null,"previous_names":["exzvor/ide99-bench"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/exzvor/ide99-bench","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/exzvor%2Fide99-bench","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/exzvor%2Fide99-bench/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/exzvor%2Fide99-bench/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/exzvor%2Fide99-bench/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/exzvor","download_url":"https://codeload.github.com/exzvor/ide99-bench/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/exzvor%2Fide99-bench/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33939225,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-05T02:00:06.157Z","response_time":120,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmarks","database-tools","desktop-app","ide99","nodejs","performance-testing","postgres","postgresql","python","reproducible-research","rust","sql-editor","tauri","tokio-postgres"],"created_at":"2026-06-05T11:30:16.678Z","updated_at":"2026-06-05T11:30:17.502Z","avatar_url":"https://github.com/exzvor.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ide99-bench\n\nReproducible performance benchmarks for [ide99](https://ide99.ru) — the desktop\nPostgreSQL IDE built by the SPG99 team. Every number on the\n[ide99.ru](https://ide99.ru) **Speed** section can be regenerated with the\nscripts in this repo against a fresh PostgreSQL container.\n\nThe point of this repo is **transparency**: anyone can pull the scripts, point\nthem at a Postgres they trust, run them on their own hardware, and see the same\nnumbers (within the noise of their machine).\n\n## What we measure\n\n| Metric                | What it captures                                                            | Script                                  |\n| --------------------- | --------------------------------------------------------------------------- | --------------------------------------- |\n| Boot to ready         | Wall-clock from `ide99` process spawn to the `READY` handshake on stdout    | `scripts/boot_and_idle.py`              |\n| Idle RSS              | Sum of resident set size across the ide99 process tree, 4 s after `READY`   | `scripts/boot_and_idle.py`              |\n| First page (10M rows) | TCP query → tokio-postgres decode → `Vec\u003cVec\u003cValue\u003e\u003e` → JSON serialise      | `scripts/read_first_page/`              |\n| Grid scroll @ 50M     | Per-frame compute for the virtualised result grid scrolling a 50M-row table | `scripts/grid_scroll.mjs`               |\n| Result-grid format    | `query_result_to_json` over 1k / 10k / 100k row pages                       | `ide/src-tauri/benches/result_grid_format.rs` (criterion) |\n| SQL parser            | DDL / SELECT parse latency budget for the autocomplete inner loop           | `ide/src-tauri/benches/parser_bench.rs` (criterion) |\n| Autocomplete          | Parse + scope walk for a 64-table JOIN — adversarial worst case             | `ide/src-tauri/benches/autocomplete_latency.rs` (criterion) |\n| EXPLAIN render        | Parse + insights walk for a 500-node real-world EXPLAIN JSON                | `ide/src-tauri/benches/explain_render.rs` (criterion) |\n\nThe Rust criterion benches live in the main ide99 repo and run on every CI\nbuild. The integration benches (boot, idle RSS, first page, grid) live here\nbecause they need a real ide99 binary and a real Postgres.\n\n## Reproducing the numbers\n\n### 1. Spin up the bench Postgres\n\n```bash\ndocker run -d --name ide99-bench-pg \\\n  -p 55433:5432 \\\n  -e POSTGRES_PASSWORD=bench \\\n  -e POSTGRES_DB=bench \\\n  -e POSTGRES_USER=bench \\\n  --shm-size=512m \\\n  postgres:17-alpine\n\nPGPASSWORD=bench psql -h 127.0.0.1 -p 55433 -U bench -d bench \\\n  -v ON_ERROR_STOP=1 -f scripts/seed.sql\n```\n\nThis creates `public.events_10m` (10,000,000 rows, ~2.6 GB on disk, mixed\ntypes incl. `jsonb`) and `public.lookup_1k`. Seed time on Apple silicon is\n~30 s. The fixture is `UNLOGGED` — we don't need crash safety in a throwaway\nbench DB and it cuts seed time roughly in half.\n\n### 2. Build the ide99 release binary\n\n```bash\ncd ../ide\ncargo tauri build --release --no-bundle\n# binary lands in target/release/ide99\n```\n\nThe `--no-bundle` flag skips DMG/AppImage assembly — we only need the\nexecutable. Set `IDE99_BINARY` if you put the binary somewhere else.\n\n### 3. Run the integration benches\n\n```bash\n# from this repo's root\n./scripts/run-all.sh\n```\n\nOr run the individual benches:\n\n```bash\n# boot + idle RSS (12 + 8 iterations)\npython3 scripts/boot_and_idle.py \u003e results/boot_and_idle.json\n\n# first-page reads against the seeded DB (3 scenarios × 30 iters each)\n(cd scripts/read_first_page \u0026\u0026 cargo build --release)\n./scripts/read_first_page/target/release/read_first_page \u003e results/read_first_page.json\n\n# virtualised grid scroll over 50M rows (600 frames × 88-row window)\nnode scripts/grid_scroll.mjs \u003e results/grid_scroll.json\n```\n\n### 4. Read the report\n\n`reports/REPORT.md` is the human-readable summary of the latest run, with\nmedians, p95s, hardware notes and how each number maps to a claim on\n[ide99.ru](https://ide99.ru).\n\n## Methodology notes\n\n- **Median over mean.** Every bench runs ≥10 iterations and reports the\n  median; means and stdev are included for completeness.\n- **One untimed warm-up** before each measured loop to absorb cold filesystem\n  cache, JIT compilation, and tokio-postgres's first-request schema fetch.\n- **Honest scope.** The first-page bench measures the same code path the IDE\n  uses (Rust → tokio-postgres → JSON). We do **not** include the Tauri IPC\n  frame or the React render in that number, because those are bench-able\n  separately (`grid_scroll.mjs`) and lumping them together hides where time\n  goes. The README flags every stage we leave out.\n- **Worst-case Postgres.** The 10M-row fixture has wide rows: bigint, tstz,\n  ints, numeric, enum-text, jsonb (with nested arrays + objects), and a\n  long sparse `note`. JSON-serialised payload is ~160 KB per 1k-row page —\n  representative of a real analytics workload, not a synthetic narrow table.\n\n## Hardware\n\nThe numbers committed to `results/` were captured on:\n\n- **Apple M-series, macOS 24.6.0** (Darwin), 16 GB RAM\n- PostgreSQL 17.9 in Docker (`postgres:17-alpine`) on the same host\n- ide99 release build, `cargo tauri build --release --no-bundle`\n\nRe-run on your hardware: numbers should land within ±2× on consumer CPUs.\nPRs welcome with results from other platforms.\n\n## Layout\n\n```\nide99-bench/\n├── README.md                       — this file\n├── scripts/\n│   ├── seed.sql                    — Postgres fixture (10M rows + indexes)\n│   ├── boot_and_idle.py            — Tauri spawn + RSS sampler\n│   ├── grid_scroll.mjs             — virtualised grid frame-time bench\n│   ├── read_first_page/            — Rust harness for the read path\n│   │   ├── Cargo.toml\n│   │   └── src/main.rs\n│   └── run-all.sh                  — runs everything end-to-end\n├── results/                        — JSON outputs from the latest run\n│   ├── boot_and_idle.json\n│   ├── read_first_page.json\n│   ├── grid_scroll.json\n│   └── criterion_summary.json\n└── reports/\n    └── REPORT.md                   — human-readable summary\n```\n\n## License\n\nMIT. Use it, fork it, send PRs.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fexzvor%2Fide99-bench","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fexzvor%2Fide99-bench","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fexzvor%2Fide99-bench/lists"}