{"id":13503721,"url":"https://github.com/erikgrinaker/toydb","last_synced_at":"2026-10-18T23:06:32.446Z","repository":{"id":37623733,"uuid":"183929744","full_name":"erikgrinaker/toydb","owner":"erikgrinaker","description":"Distributed SQL database in Rust, written as an educational project","archived":false,"fork":false,"pushed_at":"2026-07-27T22:10:04.000Z","size":5064,"stargazers_count":7283,"open_issues_count":0,"forks_count":624,"subscribers_count":91,"default_branch":"main","last_synced_at":"2026-09-22T10:14:35.461Z","etag":null,"topics":["database","distributed","mvcc","raft","rust","sql"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/erikgrinaker.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2019-04-28T16:02:59.000Z","updated_at":"2026-09-22T03:42:10.000Z","dependencies_parsed_at":"2024-04-13T09:49:00.628Z","dependency_job_id":"92f77e99-1163-49ff-ac07-91a808cdfb5c","html_url":"https://github.com/erikgrinaker/toydb","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/erikgrinaker/toydb","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erikgrinaker%2Ftoydb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erikgrinaker%2Ftoydb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erikgrinaker%2Ftoydb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erikgrinaker%2Ftoydb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/erikgrinaker","download_url":"https://codeload.github.com/erikgrinaker/toydb/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erikgrinaker%2Ftoydb/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":344872684,"owners_count":38343411,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-10-03T21:59:58.778Z","status":"online","status_checked_at":"2026-10-18T02:00:06.448Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","distributed","mvcc","raft","rust","sql"],"created_at":"2024-07-31T23:00:44.027Z","updated_at":"2026-10-18T23:06:32.440Z","avatar_url":"https://github.com/erikgrinaker.png","language":"Rust","funding_links":[],"categories":["Applications","Databases","Rust","sql","\u003ca name=\"Rust\"\u003e\u003c/a\u003eRust"],"sub_categories":["Database"],"readme":"# \u003ca\u003e\u003cimg src=\"./docs/architecture/images/toydb.svg\" height=\"40\" valign=\"top\" /\u003e\u003c/a\u003e toyDB\n\nDistributed SQL database in Rust, built from scratch as an educational project. Main features:\n\n* [Raft distributed consensus][raft] for linearizable state machine replication.\n\n* [ACID transactions][txn] with MVCC-based snapshot isolation.\n\n* [Pluggable storage engine][storage] with [BitCask][bitcask] and [in-memory][memory] backends.\n\n* [Iterator-based query engine][query] with [heuristic optimization][optimizer] and time-travel \n  support.\n\n* [SQL interface][sql] including joins, aggregates, and transactions.\n\ntoyDB is intended to be simple and understandable, and also functional and correct. Other aspects\nlike performance, scalability, and availability are non-goals -- these are major sources of\ncomplexity in production-grade databases, and obscure the basic underlying concepts. Shortcuts have\nbeen taken where possible.\n\nI originally wrote toyDB in 2020 to learn more about database internals. Since then, I've spent\nseveral years building real distributed SQL databases at\n[CockroachDB](https://github.com/cockroachdb/cockroach) and\n[Neon](https://github.com/neondatabase/neon). Based on this experience, I've rewritten toyDB as a\nsimple illustration of the architecture and concepts behind distributed SQL databases.\n\n[raft]: https://github.com/erikgrinaker/toydb/blob/main/src/raft/mod.rs\n[txn]: https://github.com/erikgrinaker/toydb/blob/main/src/storage/mvcc.rs\n[storage]: https://github.com/erikgrinaker/toydb/blob/main/src/storage/engine.rs\n[bitcask]: https://github.com/erikgrinaker/toydb/blob/main/src/storage/bitcask.rs\n[memory]: https://github.com/erikgrinaker/toydb/blob/main/src/storage/memory.rs\n[query]: https://github.com/erikgrinaker/toydb/blob/main/src/sql/execution/executor.rs\n[optimizer]: https://github.com/erikgrinaker/toydb/blob/main/src/sql/planner/optimizer.rs\n[sql]: https://github.com/erikgrinaker/toydb/blob/main/src/sql/parser/parser.rs\n\n## Documentation\n\n* [Architecture guide](docs/architecture/index.md): a guided tour of toyDB's code and architecture.\n\n* [SQL examples](docs/examples.md): walkthrough of toyDB's SQL features.\n\n* [SQL reference](docs/sql.md): reference documentation for toyDB's SQL dialect.\n\n* [References](docs/references.md): research materials used while building toyDB.\n\n## Usage\n\nWith a [Rust compiler](https://www.rust-lang.org/tools/install) installed, a local five-node \ncluster can be built and started as:\n\n```\n$ ./cluster/run.sh\nStarting 5 nodes on ports 9601-9605 with data under cluster/*/data/.\nTo connect to node 1, run: cargo run --release --bin toysql\n\ntoydb4 21:03:55 [INFO] Listening on [::1]:9604 (SQL) and [::1]:9704 (Raft)\ntoydb1 21:03:55 [INFO] Listening on [::1]:9601 (SQL) and [::1]:9701 (Raft)\ntoydb2 21:03:55 [INFO] Listening on [::1]:9602 (SQL) and [::1]:9702 (Raft)\ntoydb3 21:03:55 [INFO] Listening on [::1]:9603 (SQL) and [::1]:9703 (Raft)\ntoydb5 21:03:55 [INFO] Listening on [::1]:9605 (SQL) and [::1]:9705 (Raft)\ntoydb2 21:03:56 [INFO] Starting new election for term 1\n[...]\ntoydb2 21:03:56 [INFO] Won election for term 1, becoming leader\n```\n\nA command-line client can be built and used with node 1 on `localhost:9601`:\n\n```\n$ cargo run --release --bin toysql\nConnected to toyDB node n1. Enter !help for instructions.\ntoydb\u003e CREATE TABLE movies (id INTEGER PRIMARY KEY, title VARCHAR NOT NULL);\ntoydb\u003e INSERT INTO movies VALUES (1, 'Sicario'), (2, 'Stalker'), (3, 'Her');\ntoydb\u003e SELECT * FROM movies;\n1, 'Sicario'\n2, 'Stalker'\n3, 'Her'\n```\n\ntoyDB supports most common SQL features, including joins, aggregates, and transactions. Below is an\n`EXPLAIN` query plan of a more complex query (fetches all movies from studios that have released any\nmovie with an IMDb rating of 8 or more):\n\n```\ntoydb\u003e EXPLAIN SELECT m.title, g.name AS genre, s.name AS studio, m.rating\n  FROM movies m JOIN genres g ON m.genre_id = g.id,\n    studios s JOIN movies good ON good.studio_id = s.id AND good.rating \u003e= 8\n  WHERE m.studio_id = s.id\n  GROUP BY m.title, g.name, s.name, m.rating, m.released\n  ORDER BY m.rating DESC, m.released ASC, m.title ASC;\n\nRemap: m.title, genre, studio, m.rating (dropped: m.released)\n└─ Order: m.rating desc, m.released asc, m.title asc\n   └─ Projection: m.title, g.name as genre, s.name as studio, m.rating, m.released\n      └─ Aggregate: m.title, g.name, s.name, m.rating, m.released\n         └─ HashJoin: inner on m.studio_id = s.id\n            ├─ HashJoin: inner on m.genre_id = g.id\n            │  ├─ Scan: movies as m\n            │  └─ Scan: genres as g\n            └─ HashJoin: inner on s.id = good.studio_id\n               ├─ Scan: studios as s\n               └─ Scan: movies as good (good.rating \u003e 8 OR good.rating = 8)\n```\n\n## Architecture\n\ntoyDB's architecture is fairly typical for a distributed SQL database: a transactional\nkey/value store managed by a Raft cluster with a SQL query engine on top. See the\n[architecture guide](./docs/architecture/index.md) for more details.\n\n[![toyDB architecture](./docs/architecture/images/architecture.svg)](./docs/architecture/index.md)\n\n## Tests\n\ntoyDB mainly uses [Goldenscripts](https://github.com/erikgrinaker/goldenscript) for tests. These \nscript various scenarios, capture events and output, and later assert that the behavior remains the \nsame. See e.g.:\n\n* [Raft cluster tests](https://github.com/erikgrinaker/toydb/tree/main/src/raft/testscripts/node)\n* [MVCC transaction tests](https://github.com/erikgrinaker/toydb/tree/main/src/storage/testscripts/mvcc)\n* [SQL execution tests](https://github.com/erikgrinaker/toydb/tree/main/src/sql/testscripts)\n* [End-to-end tests](https://github.com/erikgrinaker/toydb/tree/main/tests/scripts)\n\nRun tests with `cargo test`, or have a look at the latest \n[CI run](https://github.com/erikgrinaker/toydb/actions/workflows/ci.yml).\n\n## Benchmarks\n\ntoyDB is not optimized for performance, but comes with a `workload` benchmark tool that can run \nvarious workloads against a toyDB cluster. For example:\n\n```sh\n# Start a 5-node toyDB cluster.\n$ ./cluster/run.sh\n[...]\n\n# Run a read-only benchmark via all 5 nodes.\n$ cargo run --release --bin workload read\nPreparing initial dataset... done (0.179s)\nSpawning 16 workers... done (0.006s)\nRunning workload read (rows=1000 size=64 batch=1)...\n\nTime   Progress     Txns      Rate       p50       p90       p99      pMax\n1.0s      13.1%    13085   13020/s     1.3ms     1.5ms     1.9ms     8.4ms\n2.0s      27.2%    27183   13524/s     1.3ms     1.5ms     1.8ms     8.4ms\n3.0s      41.3%    41301   13702/s     1.2ms     1.5ms     1.8ms     8.4ms\n4.0s      55.3%    55340   13769/s     1.2ms     1.5ms     1.8ms     8.4ms\n5.0s      70.0%    70015   13936/s     1.2ms     1.5ms     1.8ms     8.4ms\n6.0s      84.7%    84663   14047/s     1.2ms     1.4ms     1.8ms     8.4ms\n7.0s      99.6%    99571   14166/s     1.2ms     1.4ms     1.7ms     8.4ms\n7.1s     100.0%   100000   14163/s     1.2ms     1.4ms     1.7ms     8.4ms\n\nVerifying dataset... done (0.002s)\n```\n\nThe available workloads are:\n\n* `read`: single-row primary key lookups.\n* `write`: single-row inserts to sequential primary keys.\n* `bank`: bank transfers between various customers and accounts. To make things interesting, this\n  includes joins, secondary indexes, sorting, and conflicts.\n\nFor more information about workloads and parameters, run `cargo run --bin workload -- --help`.\n\nExample workload results are listed below. Write performance is atrocious, due to\n[fsync](https://en.wikipedia.org/wiki/Sync_(Unix)) and a lack of write batching in the Raft layer.\nDisabling fsync, or using the in-memory engine, significantly improves write performance (at the\nexpense of durability).\n\n| Workload | BitCask     | BitCask w/o fsync | Memory      |\n|----------|-------------|-------------------|-------------|\n| `read`   | 14163 txn/s | 13941 txn/s       | 13949 txn/s |\n| `write`  | 35 txn/s    | 4719 txn/s        | 7781 txn/s  |\n| `bank`   | 21 txn/s    | 1120 txn/s        | 1346 txn/s  |\n\n## Debugging\n\n[VSCode](https://code.visualstudio.com) and the [CodeLLDB](https://marketplace.visualstudio.com/items?itemName=vadimcn.vscode-lldb)\nextension can be used to debug toyDB, with the debug configuration under `.vscode/launch.json`.\n\nUnder the \"Run and Debug\" tab, select e.g. \"Debug executable 'toydb'\" or \"Debug unit tests in\nlibrary 'toydb'\".\n\n## Credits\n\nThe toyDB logo is courtesy of [@jonasmerlin](https://github.com/jonasmerlin).","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ferikgrinaker%2Ftoydb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ferikgrinaker%2Ftoydb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ferikgrinaker%2Ftoydb/lists"}