{"id":31758416,"url":"https://github.com/sxwebdev/chaindb","last_synced_at":"2025-10-09T20:25:12.357Z","repository":{"id":296734689,"uuid":"994284538","full_name":"sxwebdev/chaindb","owner":"sxwebdev","description":"ChainDB is a blazing-fast key-value database library for Go, built on top of Pebble DB. It delivers exceptional performance for modern hardware while maintaining a simple and intuitive API. Perfect for high-throughput applications that require reliable and fast data storage.","archived":false,"fork":false,"pushed_at":"2025-06-11T11:39:07.000Z","size":118,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-07-01T10:03:24.140Z","etag":null,"topics":["database","databases","golang","key-value","key-value-store","library"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sxwebdev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-01T16:06:34.000Z","updated_at":"2025-06-11T11:39:11.000Z","dependencies_parsed_at":"2025-06-02T03:08:36.591Z","dependency_job_id":"d95ad5ff-d676-4f4f-9aef-d1a89d05ddbd","html_url":"https://github.com/sxwebdev/chaindb","commit_stats":null,"previous_names":["sxwebdev/chaindb"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sxwebdev/chaindb","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sxwebdev%2Fchaindb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sxwebdev%2Fchaindb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sxwebdev%2Fchaindb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sxwebdev%2Fchaindb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sxwebdev","download_url":"https://codeload.github.com/sxwebdev/chaindb/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sxwebdev%2Fchaindb/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279002014,"owners_count":26083258,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-09T02:00:07.460Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","databases","golang","key-value","key-value-store","library"],"created_at":"2025-10-09T20:25:10.218Z","updated_at":"2025-10-09T20:25:12.351Z","avatar_url":"https://github.com/sxwebdev.png","language":"Go","readme":"# ChainDB\n\nChainDB is a high-performance key-value database library for Go, built on top of Pebble DB. It provides a simple and efficient interface for storing and retrieving data, with support for atomic batch operations, range queries, and more.\n\n## Features\n\n- 🔥 High performance: Built on top of Pebble DB, which is optimized for modern hardware\n- 🔒 ACID transactions: Support for atomic batch operations\n- 📊 Range queries: Efficient iteration over key ranges\n- 🔄 Compaction: Built-in support for database optimization\n- 🛠️ Simple API: Easy to use interface for common database operations\n- 📑 Table support: Namespace your data with prefixed tables\n- 🔄 Thread-safety: All batch operations are thread-safe by default\n- ⛓️ Blockchain ready: Optimized for blockchain applications with atomic operations and efficient state management\n\n## Installation\n\n```bash\ngo get github.com/sxwebdev/chaindb\n```\n\n## Quick Start\n\nHere's a simple example of how to use ChainDB:\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"log\"\n    \"sync\"\n\n    \"github.com/cockroachdb/pebble/v2\"\n    \"github.com/sxwebdev/chaindb\"\n)\n\nfunc main() {\n    db, err := chaindb.NewDatabase(\"./data\")\n    if err != nil {\n        log.Fatal(err)\n    }\n    defer db.Close()\n\n    // Create a table with a prefix\n    usersTable := chaindb.NewTable(db, []byte(\"users:\"))\n    settingsTable := chaindb.NewTable(db, []byte(\"settings:\"))\n\n    // Basic operations with tables\n    usersTable.Put([]byte(\"john\"), []byte(\"John Doe\"))\n    settingsTable.Put([]byte(\"theme\"), []byte(\"dark\"))\n\n    // Read from tables\n    userData, _ := usersTable.Get([]byte(\"john\"))\n    theme, _ := settingsTable.Get([]byte(\"theme\"))\n\n    // Batch operations with tables\n    batch := usersTable.NewBatch()\n    defer batch.Close()\n\n    // Safe concurrent usage of batch operations\n    var wg sync.WaitGroup\n    wg.Add(2)\n\n    go func() {\n        defer wg.Done()\n        batch.Put([]byte(\"alice\"), []byte(\"Alice Smith\"))\n    }()\n\n    go func() {\n        defer wg.Done()\n        batch.Put([]byte(\"bob\"), []byte(\"Bob Johnson\"))\n    }()\n\n    wg.Wait()\n    batch.Write()\n\n    // Range operations on tables\n    iter, err := usersTable.NewIterator(context.Background(), nil)\n    if err != nil {\n        log.Fatal(err)\n    }\n    defer iter.Release()\n\n    // Properly initialize iterator with First() call\n    for valid := iter.First(); valid \u0026\u0026 iter.Error() == nil; valid = iter.Next() {\n        key := iter.Key()\n        value := iter.Value()\n        // Process user data\n    }\n}\n```\n\n## Advanced Usage\n\n### Table Operations\n\nTables provide a way to namespace your data within the database. Each table automatically prefixes all keys with a specified string:\n\n```go\n// Create tables for different data types\nusersTable := chaindb.NewTable(db, []byte(\"users:\"))\nsettingsTable := chaindb.NewTable(db, []byte(\"settings:\"))\nlogsTable := chaindb.NewTable(db, []byte(\"logs:\"))\n\n// Operations on tables work the same as on the main database\nusersTable.Put([]byte(\"user1\"), []byte(\"data\"))\nsettingsTable.Put([]byte(\"config\"), []byte(\"value\"))\n\n// Tables support all database operations\nbatch := usersTable.NewBatch()\ndefer batch.Close()\n\nbatch.Put([]byte(\"user2\"), []byte(\"data\"))\nbatch.Write()\n\n// Iterate over table contents\niter := usersTable.NewIterator(nil, nil)\ndefer iter.Release()\n```\n\n### Batch Operations\n\nChainDB supports two types of batch operations with tables, and all batch operations are thread-safe by default:\n\n#### 1. Separate batches for each table\n\nEach table creates its own batch. Such batches are committed independently.\n\n```go\nusersBatch := usersTable.NewBatch()\ndefer usersBatch.Close()\n\nsettingsBatch := settingsTable.NewBatch()\ndefer settingsBatch.Close()\n\nusersBatch.Put([]byte(\"user1\"), []byte(\"John\"))\nsettingsBatch.Put([]byte(\"user1:theme\"), []byte(\"dark\"))\n\n// Safe for concurrent use from multiple goroutines\ngo func() {\n    usersBatch.Put([]byte(\"user2\"), []byte(\"Alice\"))\n}()\n\nif err := usersBatch.Write(); err != nil {\n    log.Fatal(err)\n}\nif err := settingsBatch.Write(); err != nil {\n    log.Fatal(err)\n}\n```\n\n#### 2. One common batch for all tables (atomically)\n\nYou can create one batch at the database level and use it for all tables through `NewBatchFrom`. All changes will be written atomically in one operation.\n\n```go\nimport (\n    \"log\"\n    \"time\"\n\n    \"github.com/sxwebdev/chaindb\"\n)\n\n// ...\n\nbatch := db.NewBatch()\ndefer batch.Close()\n\nusersBatch := usersTable.NewBatchFrom(batch)\nsettingsBatch := settingsTable.NewBatchFrom(batch)\n\nusersBatch.Put([]byte(\"user1\"), []byte(\"John\"))\nsettingsBatch.Put([]byte(\"user1:theme\"), []byte(\"dark\"))\n\n// Multiple goroutines can safely use the batches concurrently\ngo func() {\n    usersBatch.Put([]byte(\"user2\"), []byte(\"Alice\"))\n}()\ngo func() {\n    settingsBatch.Put([]byte(\"user2:theme\"), []byte(\"light\"))\n}()\n\n// Allow time for concurrent operations to complete\ntime.Sleep(100 * time.Millisecond)\n\n// All changes will be applied atomically\nif err := batch.Write(); err != nil {\n    log.Fatal(err)\n}\n```\n\n**Difference:**\n\n- Variant 1 — independent batches, committed separately.\n- Variant 2 — all changes from different tables go into one batch and are committed atomically.\n\nBoth variants are fully thread-safe and can be used concurrently from multiple goroutines.\n\n### Thread-safe Batch Operations\n\nChainDB provides thread-safe batch operations that can be safely used from multiple goroutines:\n\n```go\npackage main\n\nimport (\n    \"fmt\"\n    \"log\"\n    \"sync\"\n\n    \"github.com/sxwebdev/chaindb\"\n)\n\nfunc main() {\n    db, err := chaindb.NewDatabase(\"./data\")\n    if err != nil {\n        log.Fatal(err)\n    }\n    defer db.Close()\n\n    userTable := chaindb.NewTable(db, []byte(\"users:\"))\n\n    // Create a shared batch\n    batch := userTable.NewBatch()\n    defer batch.Close()\n\n    // Use the batch concurrently from multiple goroutines\n    var wg sync.WaitGroup\n    wg.Add(10)\n\n    for i := 0; i \u003c 10; i++ {\n        id := i // Capture loop variable\n        go func() {\n            defer wg.Done()\n            key := []byte(fmt.Sprintf(\"user_%d\", id))\n            val := []byte(fmt.Sprintf(\"User #%d\", id))\n            batch.Put(key, val)\n        }()\n    }\n\n    // Wait for all operations to complete\n    wg.Wait()\n\n    // Write all changes atomically\n    if err := batch.Write(); err != nil {\n        log.Fatal(err)\n    }\n\n    // Verify the data was written\n    count := 0\n    iter := userTable.NewIterator(context.Background(), nil)\n    defer iter.Release()\n    for valid := iter.First(); valid \u0026\u0026 iter.Error() == nil; valid = iter.Next() {\n        count++\n        log.Printf(\"Found: %s = %s\", iter.Key(), iter.Value())\n    }\n    log.Printf(\"Total users: %d\", count)\n}\n```\n\n### Advantages of Thread-Safe Batches\n\nThe thread-safety features of ChainDB provide several advantages:\n\n1. **Simplified concurrent programming**: No need to manually synchronize access to batch operations.\n2. **Reduced boilerplate code**: Avoid writing additional synchronization code with mutexes or channels.\n3. **Better performance**: Internal locking is optimized for the specific operations.\n4. **Prevention of race conditions**: The batch implementation uses fine-grained locking to prevent data corruption.\n5. **Easier parallel processing**: Process data in parallel and write it safely to the database.\n\nHere's an example showing how to process data in parallel and write it atomically:\n\n```go\nfunc processDataConcurrently(db chaindb.Database, data []DataItem) error {\n    table := chaindb.NewTable(db, []byte(\"processed:\"))\n    batch := table.NewBatch()\n    defer batch.Close()\n\n    var wg sync.WaitGroup\n    errCh := make(chan error, len(data))\n\n    // Process items in parallel\n    for _, item := range data {\n        wg.Add(1)\n        go func(item DataItem) {\n            defer wg.Done()\n\n            // Process the item\n            processedData, err := processItem(item)\n            if err != nil {\n                errCh \u003c- err\n                return\n            }\n\n            // Add to batch - thread-safe operation\n            key := []byte(item.ID)\n            if err := batch.Put(key, processedData); err != nil {\n                errCh \u003c- err\n                return\n            }\n        }(item)\n    }\n\n    // Wait for all goroutines to complete\n    wg.Wait()\n    close(errCh)\n\n    // Check for errors\n    for err := range errCh {\n        if err != nil {\n            return err\n        }\n    }\n\n    // Write all processed data atomically\n    return batch.Write()\n}\n```\n\nThis pattern enables efficient parallel processing while maintaining data consistency.\n\n### Range Queries\n\nYou can iterate over a range of keys:\n\n```go\n// Iterate over all keys\niter, err := db.NewIterator(context.Background(), nil)\nif err != nil {\n    log.Fatal(err)\n}\ndefer iter.Release()\n\n// Correctly iterate using a First()-based loop\nfor valid := iter.First(); valid \u0026\u0026 iter.Error() == nil; valid = iter.Next() {\n    key := iter.Key()\n    value := iter.Value()\n    // Process key-value pair\n}\n\n// Iterate over keys with a specific prefix\niterOptions := \u0026pebble.IterOptions{\n    LowerBound: []byte(\"user:\"),\n    UpperBound: []byte(\"user;\"), // Next byte after : is ; in ASCII\n}\nprefixIter, err := db.NewIterator(context.Background(), iterOptions)\nif err != nil {\n    log.Fatal(err)\n}\ndefer prefixIter.Release()\n\n// Properly initialize the iterator and check for errors\nfor valid := prefixIter.First(); valid \u0026\u0026 prefixIter.Error() == nil; valid = prefixIter.Next() {\n    key := prefixIter.Key()\n    value := prefixIter.Value()\n    // Process key-value pair\n}\n```\n\n### Database Maintenance\n\nChainDB provides methods for database maintenance:\n\n```go\n// Compact the database\ndb.Compact(nil, nil)\n\n// Get database statistics\nstats, _ := db.Stat()\n\n// Sync data to disk\ndb.SyncKeyValue()\n```\n\n## Performance Considerations\n\n- Use tables to organize and namespace your data\n- Use batch operations for multiple writes\n- Take advantage of built-in thread-safety for concurrent batch operations\n- Implement proper error handling\n- Close iterators after use\n- Use appropriate cache sizes for your workload\n- Consider using compression for large values\n\n## Concurrency Support\n\nChainDB provides safe concurrent access to its APIs:\n\n- **Thread-safe batches**: All batch operations are protected by internal locks and can be safely used from multiple goroutines without external synchronization.\n- **Safe iterator usage**: While iterators themselves are not thread-safe and should not be shared between goroutines, multiple iterators can be created and used concurrently.\n- **Atomic batch operations**: Use batch operations for atomicity when working with multiple keys.\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## License\n\nThis project is licensed under the MIT License - see the LICENSE file for details.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsxwebdev%2Fchaindb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsxwebdev%2Fchaindb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsxwebdev%2Fchaindb/lists"}