{"id":42043825,"url":"https://github.com/cffls/triedb-go","last_synced_at":"2026-01-26T06:04:44.323Z","repository":{"id":324591473,"uuid":"1095385981","full_name":"cffls/triedb-go","owner":"cffls","description":"TrieDB Go Bindings via FFI","archived":false,"fork":false,"pushed_at":"2025-12-03T18:34:36.000Z","size":62,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-12-28T21:29:53.624Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cffls.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-13T01:29:13.000Z","updated_at":"2025-12-03T07:31:27.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/cffls/triedb-go","commit_stats":null,"previous_names":["cffls/triedb-go"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/cffls/triedb-go","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cffls%2Ftriedb-go","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cffls%2Ftriedb-go/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cffls%2Ftriedb-go/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cffls%2Ftriedb-go/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cffls","download_url":"https://codeload.github.com/cffls/triedb-go/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cffls%2Ftriedb-go/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28768033,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-26T03:54:34.369Z","status":"ssl_error","status_checked_at":"2026-01-26T03:54:33.031Z","response_time":59,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-26T06:04:44.267Z","updated_at":"2026-01-26T06:04:44.318Z","avatar_url":"https://github.com/cffls.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TrieDB Go Bindings via FFI\n\nComplete guide for building and using TrieDB from Go through C FFI bindings.\n\n## Table of Contents\n\n- [Quick Start](#quick-start)\n- [Architecture](#architecture)\n- [Building](#building)\n- [API Reference](#api-reference)\n- [Benchmarks](#benchmarks)\n- [Troubleshooting](#troubleshooting)\n\n---\n\n## Quick Start\n\n### Run Benchmarks\n\n```bash\n# Quick benchmarks (1s each, ~30s total)\nmake bench-go-quick\n\n# Full benchmarks (5s each, ~2min total)\nmake bench-go\n```\n\n### Key Features\n\n- 🔗 **Static Linking**: Rust code embedded in Go binaries (no runtime dependencies)\n- 🚀 **Fast**: ~2µs reads, ~750µs writes, 50x speedup with batching\n- 🔒 **Safe**: Zero-copy FFI with proper memory management\n- 📦 **Simple**: Just build Rust library once, then use like any Go package\n\n---\n\n## Architecture\n\nThe FFI implementation consists of three layers:\n\n```\n┌─────────────────────────────────┐\n│     Go Application              │\n│  (Your blockchain/EVM code)     │\n└────────────┬────────────────────┘\n             │\n┌────────────▼────────────────────┐\n│   Go Bindings (triedb.go)       │\n│  - Type conversions             │\n│  - Error mapping                │\n│  - Memory management            │\n└────────────┬────────────────────┘\n             │ CGO\n┌────────────▼────────────────────┐\n│   Rust FFI (triedb-ffi/lib.rs) │\n│  - C-compatible API             │\n│  - Opaque pointers              │\n│  - Safe memory handling         │\n└────────────┬────────────────────┘\n             │\n┌────────────▼────────────────────┐\n│   TrieDB Core (Rust)            │\n│  - Page-based storage           │\n│  - MVCC snapshots               │\n│  - Merkle trie operations       │\n└─────────────────────────────────┘\n```\n\n### Components\n\n- **`triedb-ffi/`** - Rust FFI layer exposing C-compatible API\n- **`triedb-go/`** - Go wrapper with idiomatic API\n- **`triedb-go/example/`** - Working example application\n\n---\n\n## Building\n\n### Prerequisites\n\n- Rust toolchain (1.70+)\n- Go toolchain (1.21+)\n- C compiler (gcc/clang)\n- `cbindgen` for header generation: `cargo install cbindgen`\n\n### Build Commands\n\n```bash\n# Build everything\nmake build-ffi build-go\n\n# Just the FFI library\nmake build-ffi\n\n# Run example\nmake example\n\n# Run tests\nmake test-go-unit\n```\n\n### Installation\n\nThe library uses **static linking** by default, which means the Rust code is compiled directly into your Go binary. No runtime dependencies or environment variables needed!\n\n**Build Steps:**\n\n```bash\n# 1. Build the Rust FFI library (creates libtriedb_ffi.a)\ncargo build --release\n\n# 2. Build your Go application\ncd triedb-go\ngo build\n```\n\n**What happens behind the scenes:**\n- The static library (`libtriedb_ffi.a`) is linked into your Go binary\n- No `.so` or `.dylib` files needed at runtime\n- Self-contained, portable executables\n- No `LD_LIBRARY_PATH` configuration required\n\n**Current cgo configuration (in `triedb.go`):**\n```go\n// #cgo CFLAGS: -I${SRCDIR}/../triedb-ffi\n// #cgo LDFLAGS: ${SRCDIR}/../target/release/libtriedb_ffi.a -ldl -lm -lpthread\n// #include \"triedb.h\"\n```\n\n**For library users:**\n```bash\ngo get github.com/yourname/triedb-go/triedb-go\n# Build the Rust library first\ncd $GOPATH/pkg/mod/github.com/yourname/triedb-go@version\ncargo build --release\n# Now use in your Go code\n```\n\n---\n\n## API Reference\n\n### Database Operations\n\n```go\nimport triedb \"github.com/base/triedb-go\"\n\n// Open existing database\ndb, err := triedb.Open(\"state.db\")\n\n// Create new database (fails if exists)\ndb, err := triedb.CreateNew(\"state.db\")\n\n// Get state root\nroot, err := db.StateRoot()  // Returns Hash (32 bytes)\n\n// Get size in pages\nsize, err := db.Size()\n\n// Close database\nerr = db.Close()\n```\n\n### Read-Only Transactions\n\n```go\n// Begin RO transaction (multiple concurrent readers allowed)\ntx, err := db.BeginRO()\n\n// Get account\naccount, err := tx.GetAccount(address)  // nil if not found\n\n// Get storage slot\nvalue, err := tx.GetStorage(address, slot)  // nil if not found\n\n// Commit (releases snapshot)\nerr = tx.Commit()\n```\n\n### Read-Write Transactions\n\n```go\n// Begin RW transaction (single writer, blocks other writers)\ntx, err := db.BeginRW()\n\n// Set account\nerr = tx.SetAccount(address, account)\n\n// Delete account\nerr = tx.SetAccount(address, nil)\n\n// Set storage\nerr = tx.SetStorage(address, slot, \u0026value)\n\n// Delete storage\nerr = tx.SetStorage(address, slot, nil)\n\n// Commit changes\nerr = tx.Commit()\n\n// Or rollback\nerr = tx.Rollback()\n```\n\n### Overlay State (State Root Computation)\n\nThe overlay feature allows you to compute what the state root would be after applying a set of changes, without actually modifying the database. This is useful for transaction simulation, gas estimation, and optimistic execution.\n\n```go\n// Create an overlay to accumulate changes\noverlay, err := triedb.NewOverlayState()\ndefer overlay.Close()\n\n// Or with pre-allocated capacity for better performance\noverlay, err := triedb.NewOverlayStateWithCapacity(1000)\n\n// Insert account changes\nerr = overlay.InsertAccount(addr1, account1)\nerr = overlay.InsertAccount(addr2, account2)\n\n// Insert storage changes\nerr = overlay.InsertStorage(addr1, slot1, value1)\nerr = overlay.InsertStorage(addr1, slot2, value2)\n\n// Insert deletions (tombstones)\nerr = overlay.InsertAccount(addr3, nil)  // Delete account\nerr = overlay.InsertStorage(addr1, slot3, nil)  // Delete storage slot\n\n// Check overlay state\nlength, err := overlay.Len()\nisEmpty, err := overlay.IsEmpty()\n\n// Compute the new state root with a read-only transaction\n// The overlay is automatically frozen when needed\ntx, err := db.BeginRO()\nnewRoot, err := tx.ComputeRootWithOverlay(overlay)\n\n// The database remains unchanged - changes are only in the overlay\n```\n\n**Use Cases:**\n- **Transaction Simulation**: Compute the resulting state root without committing\n- **Gas Estimation**: Test multiple transaction orderings efficiently\n- **Optimistic Execution**: Speculatively compute state while previous blocks finalize\n- **State Prediction**: Preview state changes for UI/API responses\n\n### Types\n\n```go\ntype Address [20]byte\ntype Hash [32]byte\n\ntype Account struct {\n    Nonce       uint64\n    Balance     *uint256.Int\n    StorageRoot Hash\n    CodeHash    []byte\n}\n\n// Overlay types\ntype OverlayState struct { ... }  // Overlay for accumulating and computing changes\n```\n\n### Helper Functions\n\n```go\n// Parse hex strings\naddr, err := triedb.AddressFromHex(\"0xd8da6bf26964af9d7eed9e03e53415d37aa96045\")\nhash, err := triedb.HashFromHex(\"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421\")\n\n// Convert to hex\nhexStr := addr.Hex()  // Returns \"0x...\"\nhexStr := hash.Hex()\n```\n\n### Complete Example\n\n```go\npackage main\n\nimport (\n    \"log\"\n\n    \"github.com/holiman/uint256\"\n    triedb \"github.com/base/triedb-go\"\n)\n\nfunc main() {\n    // Create database\n    db, err := triedb.CreateNew(\"state.db\")\n    if err != nil {\n        log.Fatal(err)\n    }\n    defer db.Close()\n\n    // Parse address\n    addr, _ := triedb.AddressFromHex(\"0xd8da6bf26964af9d7eed9e03e53415d37aa96045\")\n\n    // Create account\n    emptyRoot, _ := triedb.HashFromHex(\"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421\")\n    emptyCode, _ := triedb.HashFromHex(\"0xc5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470\")\n\n    account := \u0026triedb.Account{\n        Nonce:       1,\n        Balance:     uint256.NewInt(1_000_000_000_000_000_000), // 1 ETH\n        StorageRoot: emptyRoot,\n        CodeHash:    emptyCode[:],\n    }\n\n    // Write transaction\n    tx, _ := db.BeginRW()\n    tx.SetAccount(addr, account)\n    tx.Commit()\n\n    // Read transaction\n    roTx, _ := db.BeginRO()\n    readAccount, _ := roTx.GetAccount(addr)\n    roTx.Commit()\n\n    log.Printf(\"Balance: %s wei\", readAccount.Balance.String())\n}\n```\n\n---\n\n## Benchmarks\n\n### Quick Benchmarks\n\n```bash\n# All benchmarks, 1s each (~30s total)\nmake bench-go-quick\n\n# Full benchmarks, 5s each (~2min)\nmake bench-go\n\n# Compare TrieDB vs in-memory map\nmake bench-go-compare\n\n# Test concurrent scaling (1/2/4/8 cores)\nmake bench-go-concurrent\n```\n\n### Results on AMD Ryzen 7 5800H + NVMe SSD\n\n```\nBenchmarkAccountWrite/OpsPerTx_1-16           746087 ns/op    1.000 ops/tx    424 B/op   9 allocs/op\nBenchmarkAccountWrite/OpsPerTx_16-16           83678 ns/op     16.00 ops/tx    409 B/op   7 allocs/op\nBenchmarkAccountWrite/OpsPerTx_256-16          19456 ns/op     256.0 ops/tx    408 B/op   7 allocs/op\nBenchmarkAccountWrite/OpsPerTx_4096-16         11701 ns/op    4096 ops/tx     408 B/op   7 allocs/op\nBenchmarkAccountWrite/OpsPerTx_65536-16         6102 ns/op   65536 ops/tx     408 B/op   7 allocs/op\nBenchmarkAccountRead/OpsPerTx_1-16              2186 ns/op    1.000 ops/tx    368 B/op  10 allocs/op\nBenchmarkAccountRead/OpsPerTx_16-16             1941 ns/op     16.00 ops/tx    346 B/op   8 allocs/op\nBenchmarkAccountRead/OpsPerTx_256-16            1916 ns/op     256.0 ops/tx    345 B/op   8 allocs/op\nBenchmarkAccountRead/OpsPerTx_4096-16           1912 ns/op    4096 ops/tx     345 B/op   8 allocs/op\nBenchmarkAccountRead/OpsPerTx_65536-16          1908 ns/op   65536 ops/tx     345 B/op   8 allocs/op\nBenchmarkStorageWrite/OpsPerTx_1-16            760010 ns/op   1.000 ops/tx    104 B/op   5 allocs/op\nBenchmarkStorageWrite/OpsPerTx_16-16            82904 ns/op     16.00 ops/tx     89 B/op   3 allocs/op\nBenchmarkStorageWrite/OpsPerTx_256-16           19723 ns/op     256.0 ops/tx     88 B/op   3 allocs/op\nBenchmarkStorageWrite/OpsPerTx_4096-16          10646 ns/op    4096 ops/tx     88 B/op   3 allocs/op\nBenchmarkStorageWrite/OpsPerTx_65536-16          6676 ns/op   65536 ops/tx     88 B/op   3 allocs/op\nBenchmarkStorageRead/OpsPerTx_1-16              2190 ns/op    1.000 ops/tx    176 B/op   8 allocs/op\nBenchmarkStorageRead/OpsPerTx_16-16             1974 ns/op     16.00 ops/tx    154 B/op   6 allocs/op\nBenchmarkStorageRead/OpsPerTx_256-16            1945 ns/op     256.0 ops/tx    153 B/op   6 allocs/op\nBenchmarkStorageRead/OpsPerTx_4096-16           1955 ns/op    4096 ops/tx     153 B/op   6 allocs/op\nBenchmarkStorageRead/OpsPerTx_65536-16          1957 ns/op   65536 ops/tx     153 B/op   6 allocs/op\nBenchmarkConcurrentReads-16                      315.4 ns/op                  346 B/op   8 allocs/op\nBenchmarkStateRootComputation-16                 135.1 ns/op                   64 B/op   2 allocs/op\nBenchmarkMixedWorkload-16                        8942 ns/op                    365 B/op   8 allocs/op\n```\n\n### Batch Operations\n\n| Ops per Tx | Per Account | Speedup vs Single |\n| ---------- | ----------- | ----------------- |\n| 1          | 746 µs      | 1x                |\n| 16         | 84 µs       | 8.9x              |\n| 256        | 19 µs       | 38x               |\n| 4096       | 12 µs       | 63x               |\n| 65536      | 6 µs        | 122x              |\n\n**Key Insight:** Batching provides massive speedup by amortizing transaction overhead.\n\n\n### Profiling\n\n```bash\n# CPU profiling\nmake bench-go-profile\ncd triedb-go \u0026\u0026 go tool pprof -http=:8080 cpu.prof\n\n# Memory profiling\nmake bench-go-mem-profile\ncd triedb-go \u0026\u0026 go tool pprof -http=:8080 mem.prof\n```\n\n### Performance Targets\n\n| Operation     | Excellent | Good    | Acceptable |\n| ------------- | --------- | ------- | ---------- |\n| Account Write | \u003c 20 µs   | \u003c 50 µs | \u003c 100 µs   |\n| Account Read  | \u003c 10 µs   | \u003c 20 µs | \u003c 50 µs    |\n| Storage Write | \u003c 40 µs   | \u003c 80 µs | \u003c 150 µs   |\n| Storage Read  | \u003c 15 µs   | \u003c 30 µs | \u003c 80 µs    |\n| State Root    | \u003c 5 µs    | \u003c 10 µs | \u003c 50 µs    |\n\n### Optimization Tips\n\n**1. Batch Writes**\n\n❌ Bad (752 µs per account):\n```go\nfor _, account := range accounts {\n    tx, _ := db.BeginRW()\n    tx.SetAccount(account.Address, account.Data)\n    tx.Commit()\n}\n```\n\n✅ Good (15 µs per account):\n```go\ntx, _ := db.BeginRW()\nfor _, account := range accounts {\n    tx.SetAccount(account.Address, account.Data)\n}\ntx.Commit()\n```\n\n**2. Reuse Read Transactions**\n\n❌ Bad:\n```go\nfor _, addr := range addresses {\n    tx, _ := db.BeginRO()\n    account, _ := tx.GetAccount(addr)\n    tx.Commit()\n}\n```\n\n✅ Good:\n```go\ntx, _ := db.BeginRO()\nfor _, addr := range addresses {\n    account, _ := tx.GetAccount(addr)\n}\ntx.Commit()\n```\n\n**3. Concurrent Reads**\n\n```go\nvar wg sync.WaitGroup\nfor i := 0; i \u003c runtime.NumCPU(); i++ {\n    wg.Add(1)\n    go func(batch []Address) {\n        defer wg.Done()\n        tx, _ := db.BeginRO()\n        defer tx.Commit()\n        for _, addr := range batch {\n            tx.GetAccount(addr)\n        }\n    }(addressBatches[i])\n}\nwg.Wait()\n```\n\n### Continuous Benchmarking\n\n```bash\n# Save baseline\nmake bench-go \u003e baseline.txt\n\n# After changes\nmake bench-go \u003e new.txt\n\n# Compare\nbenchstat baseline.txt new.txt\n```\n\nInstall `benchstat`:\n```bash\ngo install golang.org/x/perf/cmd/benchstat@latest\n```\n\n---\n\n## Makefile Commands\n\n### Building\n```bash\nmake build-ffi          # Build Rust library\nmake build-go           # Build Go bindings\nmake install            # Install system-wide\nmake clean              # Clean artifacts\n```\n\n### Testing\n```bash\nmake test-go-unit       # Run unit tests\nmake example            # Run example\n```\n\n### Benchmarking\n```bash\nmake bench-go-quick           # Quick (1s each)\nmake bench-go                 # Full (5s each)\nmake bench-go-compare         # vs in-memory\nmake bench-go-concurrent      # Scalability\nmake bench-go-profile         # CPU profile\nmake bench-go-mem-profile     # Memory profile\n```\n\n### Code Quality\n```bash\nmake format             # Format all code\nmake lint               # Lint all code\n```\n\n---\n\n## Concurrency Model\n\n- **Multiple concurrent readers**: Safe across goroutines\n- **Single writer**: Only one RW transaction at a time (blocks)\n- **Snapshot isolation**: Readers see consistent state from transaction start\n- **MVCC**: Readers never block writers, writers never block readers\n\n```go\n// Multiple readers work concurrently\ngo func() {\n    tx, _ := db.BeginRO()\n    defer tx.Commit()\n    // Read operations...\n}()\n\ngo func() {\n    tx, _ := db.BeginRO()\n    defer tx.Commit()\n    // Read operations...\n}()\n\n// Only one writer at a time\ntx, _ := db.BeginRW()\ntx.SetAccount(addr, account)\ntx.Commit()\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcffls%2Ftriedb-go","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcffls%2Ftriedb-go","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcffls%2Ftriedb-go/lists"}