{"id":30030096,"url":"https://github.com/twilson63/bobsled","last_synced_at":"2026-04-19T06:35:25.639Z","repository":{"id":308085905,"uuid":"1031260490","full_name":"twilson63/bobsled","owner":"twilson63","description":"High-performance Erlang NIF for Sled embedded database","archived":false,"fork":false,"pushed_at":"2025-08-04T02:07:49.000Z","size":103,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-04T04:52:41.212Z","etag":null,"topics":["beam","database","embedded-database","erlang","key-value","nif","rust","sled"],"latest_commit_sha":null,"homepage":null,"language":"Erlang","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/twilson63.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-03T11:15:37.000Z","updated_at":"2025-08-04T02:07:53.000Z","dependencies_parsed_at":"2025-08-04T04:52:42.700Z","dependency_job_id":"0a433951-150b-4c3b-8171-6d589a7b247a","html_url":"https://github.com/twilson63/bobsled","commit_stats":null,"previous_names":["twilson63/bobsled"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/twilson63/bobsled","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twilson63%2Fbobsled","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twilson63%2Fbobsled/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twilson63%2Fbobsled/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twilson63%2Fbobsled/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/twilson63","download_url":"https://codeload.github.com/twilson63/bobsled/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twilson63%2Fbobsled/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269137306,"owners_count":24366552,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-06T02:00:09.910Z","response_time":99,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["beam","database","embedded-database","erlang","key-value","nif","rust","sled"],"created_at":"2025-08-06T19:00:21.545Z","updated_at":"2026-04-19T06:35:25.585Z","avatar_url":"https://github.com/twilson63.png","language":"Erlang","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Bobsled 🛷\n\nA high-performance Erlang NIF (Native Implemented Function) for the [Sled](https://github.com/spacejam/sled) embedded database, providing blazing-fast key-value storage with ACID guarantees for BEAM applications.\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Erlang/OTP](https://img.shields.io/badge/Erlang%2FOTP-24%2B-blue.svg)](http://www.erlang.org)\n[![Rust](https://img.shields.io/badge/Rust-1.70%2B-orange.svg)](https://www.rust-lang.org)\n\n## Features\n\n- 🚀 **Blazing Fast**: Over 500K writes/sec and 1.9M reads/sec\n- 🔒 **Thread-Safe**: Lock-free operations with built-in concurrency\n- 💾 **ACID Compliance**: Atomic operations with crash recovery\n- 🌳 **Hierarchical Keys**: Efficient path-based key organization  \n- 🔄 **Transactions**: ACID transactions with optimistic concurrency\n- 📦 **Batch Operations**: Atomic batch writes for bulk updates\n- 🎯 **Compare-and-Swap**: Atomic CAS operations for coordination\n- 🛡️ **Panic Safety**: Rust panics won't crash the BEAM VM\n\n## Performance\n\nBased on benchmarks with 100,000 operations:\n\n| Operation | Throughput | Latency |\n|-----------|------------|---------|\n| Sequential Writes | 578,292 ops/sec | 1.73 μs |\n| Sequential Reads | 1,962,092 ops/sec | 0.51 μs |\n| Concurrent Writes (10 proc) | 804,388 ops/sec | 12.4 μs |\n| Concurrent Reads (10 proc) | 2,974,331 ops/sec | 3.4 μs |\n| Compare-and-Swap | 763,533 ops/sec | 1.31 μs |\n| List Operations (1K items) | 4,553 ops/sec | 219.6 μs |\n\n## Installation\n\n### Prerequisites\n\n- Erlang/OTP 24 or later\n- Rust 1.70 or later\n- Cargo (Rust package manager)\n\n### Building from Source\n\n```bash\ngit clone https://github.com/twilson63/bobsled.git\ncd bobsled\nmake compile\n```\n\n### Rebar3 Integration\n\nAdd bobsled to your `rebar.config`:\n\n```erlang\n{deps, [\n    {bobsled, {git, \"https://github.com/twilson63/bobsled.git\", {tag, \"v1.0.0\"}}}\n]}.\n```\n\n## Quick Start\n\n```erlang\n% Open a database\n{ok, Db} = bobsled:open(\u003c\u003c\"/tmp/mydb\"\u003e\u003e, [\n    {mode, fast},\n    {cache_capacity, 100_000_000}  % 100MB cache\n]),\n\n% Basic operations\nok = bobsled:put(Db, \u003c\u003c\"user:123\"\u003e\u003e, \u003c\u003c\"Alice\"\u003e\u003e),\n{ok, \u003c\u003c\"Alice\"\u003e\u003e} = bobsled:get(Db, \u003c\u003c\"user:123\"\u003e\u003e),\nok = bobsled:delete(Db, \u003c\u003c\"user:123\"\u003e\u003e),\n\n% Batch operations\nok = bobsled:batch_put(Db, [\n    {\u003c\u003c\"user:1\"\u003e\u003e, \u003c\u003c\"Bob\"\u003e\u003e},\n    {\u003c\u003c\"user:2\"\u003e\u003e, \u003c\u003c\"Carol\"\u003e\u003e},\n    {\u003c\u003c\"user:3\"\u003e\u003e, \u003c\u003c\"Dave\"\u003e\u003e}\n]),\n\n% Compare-and-swap\nok = bobsled:put(Db, \u003c\u003c\"counter\"\u003e\u003e, \u003c\u003c\"0\"\u003e\u003e),\nok = bobsled:compare_and_swap(Db, \u003c\u003c\"counter\"\u003e\u003e, \u003c\u003c\"0\"\u003e\u003e, \u003c\u003c\"1\"\u003e\u003e),\n{error, cas_failed} = bobsled:compare_and_swap(Db, \u003c\u003c\"counter\"\u003e\u003e, \u003c\u003c\"0\"\u003e\u003e, \u003c\u003c\"2\"\u003e\u003e),\n\n% Hierarchical keys and listing\nok = bobsled:put(Db, \u003c\u003c\"config/app/name\"\u003e\u003e, \u003c\u003c\"MyApp\"\u003e\u003e),\nok = bobsled:put(Db, \u003c\u003c\"config/app/version\"\u003e\u003e, \u003c\u003c\"1.0\"\u003e\u003e),\nok = bobsled:put(Db, \u003c\u003c\"config/db/host\"\u003e\u003e, \u003c\u003c\"localhost\"\u003e\u003e),\n{ok, Keys} = bobsled:list(Db, \u003c\u003c\"config/app/\"\u003e\u003e),\n% Keys = [\u003c\u003c\"config/app/name\"\u003e\u003e, \u003c\u003c\"config/app/version\"\u003e\u003e]\n\n% Close database\nok = bobsled:close(Db).\n```\n\n## API Reference\n\n### Database Management\n\n#### `open/2`\n```erlang\nopen(Path :: binary(), Options :: proplists:proplist()) -\u003e \n    {ok, db_handle()} | {error, term()}.\n```\nOpens or creates a database at the specified path.\n\n**Options:**\n- `{mode, fast | safe}` - Performance vs durability tradeoff (default: `fast`)\n- `{cache_capacity, pos_integer()}` - Cache size in bytes (default: 1GB)\n- `{compression_factor, 0..22}` - Zstd compression level (default: 3)\n- `{flush_every_ms, pos_integer()}` - Background flush interval (default: 500ms)\n\n#### `close/1`\n```erlang\nclose(DbHandle :: db_handle()) -\u003e ok | {error, term()}.\n```\nCloses the database and flushes all pending writes.\n\n### Basic Operations\n\n#### `put/3`\n```erlang\nput(DbHandle :: db_handle(), Key :: binary(), Value :: binary()) -\u003e \n    ok | {error, term()}.\n```\nStores a key-value pair in the database.\n\n#### `get/2`\n```erlang\nget(DbHandle :: db_handle(), Key :: binary()) -\u003e \n    {ok, binary()} | not_found | {error, term()}.\n```\nRetrieves a value by key.\n\n#### `delete/2`\n```erlang\ndelete(DbHandle :: db_handle(), Key :: binary()) -\u003e \n    ok | {error, term()}.\n```\nDeletes a key-value pair.\n\n### Advanced Operations\n\n#### `compare_and_swap/4`\n```erlang\ncompare_and_swap(DbHandle :: db_handle(), Key :: binary(), \n                 OldValue :: binary() | not_found, NewValue :: binary()) -\u003e \n    ok | {error, cas_failed} | {error, term()}.\n```\nAtomically updates a value only if it matches the expected old value.\n\n#### `batch_put/2`\n```erlang\nbatch_put(DbHandle :: db_handle(), KVPairs :: [{binary(), binary()}]) -\u003e \n    ok | {error, term()}.\n```\nAtomically writes multiple key-value pairs.\n\n#### `list/2`\n```erlang\nlist(DbHandle :: db_handle(), Prefix :: binary()) -\u003e \n    {ok, [binary()]} | {error, term()}.\n```\nLists all keys with the given prefix.\n\n#### `fold/4`\n```erlang\nfold(DbHandle :: db_handle(), \n     Fun :: fun((Key :: binary(), Value :: binary(), Acc) -\u003e Acc),\n     InitAcc :: term(), Prefix :: binary()) -\u003e \n    {ok, Acc} | {error, term()}.\n```\nFolds over all key-value pairs with the given prefix.\n\n### Utility Functions\n\n#### `flush/1`\n```erlang\nflush(DbHandle :: db_handle()) -\u003e ok | {error, term()}.\n```\nManually flushes all pending writes to disk.\n\n#### `size_on_disk/1`\n```erlang\nsize_on_disk(DbHandle :: db_handle()) -\u003e {ok, pos_integer()} | {error, term()}.\n```\nReturns the database size on disk in bytes.\n\n## OTP Integration\n\n### Application Example\n\n```erlang\n-module(myapp_db).\n-behaviour(gen_server).\n\n-export([start_link/0, get/1, put/2]).\n-export([init/1, handle_call/3, handle_cast/2, terminate/2]).\n\nstart_link() -\u003e\n    gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).\n\nget(Key) -\u003e\n    gen_server:call(?MODULE, {get, Key}).\n\nput(Key, Value) -\u003e\n    gen_server:call(?MODULE, {put, Key, Value}).\n\ninit([]) -\u003e\n    {ok, Db} = bobsled:open(\u003c\u003c\"/var/db/myapp\"\u003e\u003e, [{mode, fast}]),\n    {ok, #{db =\u003e Db}}.\n\nhandle_call({get, Key}, _From, #{db := Db} = State) -\u003e\n    Reply = bobsled:get(Db, Key),\n    {reply, Reply, State};\nhandle_call({put, Key, Value}, _From, #{db := Db} = State) -\u003e\n    Reply = bobsled:put(Db, Key, Value),\n    {reply, Reply, State}.\n\nhandle_cast(_Msg, State) -\u003e\n    {noreply, State}.\n\nterminate(_Reason, #{db := Db}) -\u003e\n    bobsled:close(Db),\n    ok.\n```\n\n### Supervisor Setup\n\n```erlang\n-module(myapp_sup).\n-behaviour(supervisor).\n\n-export([start_link/0, init/1]).\n\nstart_link() -\u003e\n    supervisor:start_link({local, ?MODULE}, ?MODULE, []).\n\ninit([]) -\u003e\n    Children = [\n        {myapp_db, {myapp_db, start_link, []},\n         permanent, 5000, worker, [myapp_db]}\n    ],\n    {ok, {{one_for_one, 5, 10}, Children}}.\n```\n\n## Benchmarks\n\nRun the comprehensive benchmark suite:\n\n```bash\n# Run all benchmarks\n./run_benchmarks.escript\n\n# Run specific benchmark\n./run_benchmarks.escript --test writes --count 1000000\n\n# Run hierarchical list benchmarks\n./run_list_bench.escript\n```\n\n## Testing\n\nRun the test suite:\n\n```bash\n# Run EUnit tests\n./run_eunit_tests.escript\n\n# Run with rebar3\nrebar3 eunit\n```\n\n## Architecture\n\nBobsled uses Rust's [Sled](https://github.com/spacejam/sled) embedded database, which provides:\n\n- **Lock-free B+ tree**: Concurrent operations without blocking\n- **Log-structured storage**: Fast writes with compression\n- **MVCC**: Multi-version concurrency control for snapshots\n- **Crash safety**: Atomic writes with automatic recovery\n\nThe NIF wrapper ensures:\n- **Panic safety**: Rust panics are caught and converted to Erlang errors\n- **Resource management**: Proper cleanup of database handles\n- **Type safety**: Binary-only keys and values for consistency\n- **Error translation**: Rust errors mapped to Erlang terms\n\n## Contributing\n\nWe welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## Acknowledgments\n\n- The [Sled](https://github.com/spacejam/sled) team for the excellent embedded database\n- The [Rustler](https://github.com/rusterlium/rustler) team for Erlang-Rust interop\n- The Erlang/OTP community for the amazing BEAM platform","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftwilson63%2Fbobsled","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftwilson63%2Fbobsled","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftwilson63%2Fbobsled/lists"}