{"id":39749715,"url":"https://github.com/sochdb/sochdb-go","last_synced_at":"2026-02-16T10:20:13.975Z","repository":{"id":331980612,"uuid":"1127468191","full_name":"sochdb/sochdb-go","owner":"sochdb","description":"SochDB is a high-performance embedded, ACID-compliant database purpose-built for AI agents - golang","archived":false,"fork":false,"pushed_at":"2026-01-26T06:09:14.000Z","size":37874,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-01-26T12:27:43.013Z","etag":null,"topics":["database","golang","golang-age","golang-tools","golang-vector-database","vector-database"],"latest_commit_sha":null,"homepage":"https://sochdb.dev","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sochdb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-01-04T00:12:23.000Z","updated_at":"2026-01-26T06:06:30.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/sochdb/sochdb-go","commit_stats":null,"previous_names":["toondb/toondb-go","sochdb/toondb-go","sochdb/sochdb-go"],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/sochdb/sochdb-go","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sochdb%2Fsochdb-go","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sochdb%2Fsochdb-go/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sochdb%2Fsochdb-go/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sochdb%2Fsochdb-go/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sochdb","download_url":"https://codeload.github.com/sochdb/sochdb-go/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sochdb%2Fsochdb-go/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29331589,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-11T06:13:03.264Z","status":"ssl_error","status_checked_at":"2026-02-11T06:12:55.843Z","response_time":97,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","golang","golang-age","golang-tools","golang-vector-database","vector-database"],"created_at":"2026-01-18T11:24:38.779Z","updated_at":"2026-02-16T10:20:13.953Z","avatar_url":"https://github.com/sochdb.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SochDB Go SDK\n\n**LLM-Optimized Embedded Database with Native Vector Search**\n\n---\n\n## Installation\n\n### Step 1: Install the Native Library\n\n```bash\n# Automatic installation (installs to /usr/local/lib)\nSOCHDB_ROOT=/path/to/sochdb ./install-lib.sh\n```\n\n**If automatic installation doesn't work**, see manual installation steps in [Troubleshooting](#library-not-found-error).\n\n### Step 2: Install the Go SDK\n\n```bash\ngo get github.com/sochdb/sochdb-go\n```\n\n### Step 3: Install pkg-config (if not already installed)\n\n**macOS:**\n```bash\nbrew install pkg-config\n```\n\n**Linux:**\n```bash\n# Ubuntu/Debian\nsudo apt-get install pkg-config\n\n# Fedora/RHEL\nsudo yum install pkgconfig\n```\n\n### Verify Installation\n\n```bash\n# Check if library is found\npkg-config --libs libsochdb_storage\n\n# Expected output: -L/usr/local/lib -lsochdb_storage\n```\n\n---\n\n## Architecture: Flexible Deployment\n\n**Dual-mode architecture: Embedded (FFI) + Concurrent + Server (gRPC/IPC)**  \nChoose the deployment mode that fits your needs.\n\n---\n\n# SochDB Go SDK Documentation\n\n\u003e **Version 0.4.5** — LLM-Optimized Embedded Database with Native Vector Search\n\n---\n\n## Table of Contents\n\n1. [Quick Start](#1-quick-start)\n2. [Features](#features)\n   - [Semantic Cache](#semantic-cache---llm-response-caching)\n   - [Context Query Builder](#context-query-builder---token-aware-llm-context)\n   - [Namespace API](#namespace-api---multi-tenant-isolation)\n   - [Priority Queue API](#priority-queue-api---task-processing)\n3. [Architecture](#architecture-flexible-deployment)\n4. [System Requirements](#system-requirements)\n5. [Troubleshooting](#troubleshooting)\n6. [API Reference](#api-reference)\n   - [Core Key-Value Operations](#core-key-value-operations)\n   - [Transactions (ACID with SSI)](#transactions-acid-with-ssi)\n   - [Prefix Scanning](#prefix-scanning)\n   - [Namespaces \u0026 Collections](#namespaces--collections)\n   - [Priority Queues](#priority-queues)\n   - [Semantic Cache](#semantic-cache)\n   - [Graph Operations](#graph-operations-ipc--grpc)\n   - [Context Query Builder](#context-query-builder)\n   - [Memory System (LLM-Native)](#memory-system-llm-native)\n   - [Data Formats](#data-formats-toonjsoncolumnar)\n   - [Server Mode (IPC / gRPC)](#server-mode-ipc--grpc)\n   - [Checkpoints \u0026 Statistics](#checkpoints--statistics)\n   - [Index Policies](#index-policies)\n   - [Error Handling](#error-handling)\n   - [Performance](#performance)\n\n---\n\n## 1. Quick Start\n\n### Concurrent Embedded Mode\n\nFor web applications with multiple workers/processes:\n\n```go\nimport \"github.com/sochdb/sochdb-go/embedded\"\n\n// Open in concurrent mode - multiple processes can access simultaneously\ndb, err := embedded.OpenConcurrent(\"./web_db\")\nif err != nil {\n    log.Fatal(err)\n}\ndefer db.Close()\n\n// Reads are lock-free and parallel (~100ns)\nvalue, err := db.Get([]byte(\"user:123\"))\n\n// Writes are automatically coordinated\nerr = db.Put([]byte(\"user:123\"), []byte(`{\"name\": \"Alice\"}`))\n\n// Check if concurrent mode is active\nfmt.Printf(\"Concurrent mode: %v\\n\", db.IsConcurrent())  // true\n```\n\n### Gin/Echo Example (Multiple Workers)\n\n```go\npackage main\n\nimport (\n    \"github.com/gin-gonic/gin\"\n    \"github.com/sochdb/sochdb-go/embedded\"\n    \"log\"\n)\n\nfunc main() {\n    // Open database in concurrent mode\n    db, err := embedded.OpenConcurrent(\"./gin_db\")\n    if err != nil {\n        log.Fatal(err)\n    }\n    defer db.Close()\n    \n    r := gin.Default()\n    \n    r.GET(\"/user/:id\", func(c *gin.Context) {\n        // Multiple concurrent requests can read simultaneously\n        key := []byte(\"user:\" + c.Param(\"id\"))\n        data, err := db.Get(key)\n        if err != nil || data == nil {\n            c.JSON(404, gin.H{\"error\": \"not found\"})\n            return\n        }\n        c.Data(200, \"application/json\", data)\n    })\n    \n    r.POST(\"/user/:id\", func(c *gin.Context) {\n        // Writes are serialized automatically\n        key := []byte(\"user:\" + c.Param(\"id\"))\n        body, _ := c.GetRawData()\n        if err := db.Put(key, body); err != nil {\n            c.JSON(500, gin.H{\"error\": err.Error()})\n            return\n        }\n        c.JSON(200, gin.H{\"status\": \"ok\"})\n    })\n    \n    // Start with multiple workers (e.g., via systemd or Docker)\n    // Each worker process can access the database concurrently\n    r.Run(\":8080\")\n}\n```\n\n### Performance\n\n| Operation | Standard Mode | Concurrent Mode |\n|-----------|---------------|-----------------|\n| Read (single process) | ~100ns | ~100ns |\n| Read (multi-process) | **Blocked** ❌ | ~100ns ✅ |\n| Write | ~5ms (fsync) | ~60µs (amortized) |\n| Max concurrent readers | 1 | 1024 |\n\n### Deployment with Systemd (Multiple Workers)\n\n```bash\n# /etc/systemd/system/myapp@.service\n[Unit]\nDescription=MyApp Worker %i\nAfter=network.target\n\n[Service]\nType=simple\nUser=appuser\nWorkingDirectory=/opt/myapp\nExecStart=/opt/myapp/server\nRestart=always\n\n[Install]\nWantedBy=multi-user.target\n```\n\n```bash\n# Start 4 worker instances (all can access same DB concurrently)\nsudo systemctl start myapp@1\nsudo systemctl start myapp@2\nsudo systemctl start myapp@3\nsudo systemctl start myapp@4\n\n# Enable on boot\nsudo systemctl enable myapp@{1,2,3,4}\n```\n\n### Docker Compose Example\n\n```yaml\nversion: '3.8'\nservices:\n  app:\n    build: .\n    deploy:\n      replicas: 4  # 4 workers share the same database\n    volumes:\n      - ./data:/app/data  # Shared database volume\n    ports:\n      - \"8080-8083:8080\"\n```\n\n---\n\n## Engine Status\n\n| Component | Status |\n|-----------|--------|\n| **Cost-based optimizer** | ✅ Production-ready — full cost model, cardinality estimation, plan caching |\n| **Adaptive group commit** | ✅ Implemented — Little's Law-based batch sizing |\n| **WAL compaction** | ⚠️ Partial — manual `Checkpoint()` + `TruncateWAL()` available |\n| **HNSW vector index** | ✅ Production-ready — CGo FFI bindings |\n\n---\n\n## Features\n\n### Semantic Cache - LLM Response Caching\nVector similarity-based caching for LLM responses to reduce costs and latency:\n\n```go\nimport (\n    sochdb \"github.com/sochdb/sochdb-go\"\n    \"github.com/sochdb/sochdb-go/embedded\"\n)\n\ndb, _ := embedded.Open(\"./mydb\")\ndefer db.Close()\n\ncache := sochdb.NewSemanticCache(db, \"llm_responses\")\n\n// Store LLM response with embedding\ncache.Put(\n    \"What is machine learning?\",\n    \"Machine learning is a subset of AI...\",\n    []float32{0.1, 0.2, ...},  // 384-dim vector\n    3600,  // TTL in seconds\n    map[string]interface{}{\"model\": \"gpt-4\", \"tokens\": 150},\n)\n\n// Check cache before calling LLM\nhit, _ := cache.Get(queryEmbedding, 0.85)\nif hit != nil {\n    fmt.Printf(\"Cache HIT! Similarity: %.4f\\n\", hit.Score)\n    fmt.Printf(\"Response: %s\\n\", hit.Value)\n}\n\n// Get statistics\nstats, _ := cache.Stats()\nfmt.Printf(\"Hit rate: %.1f%%\\n\", stats.HitRate*100)\n```\n\n**Key Benefits:**\n- ✅ Cosine similarity matching (0-1 threshold)\n- ✅ TTL-based expiration\n- ✅ Hit/miss statistics tracking\n- ✅ Memory usage monitoring\n- ✅ Automatic expired entry purging\n\n### Context Query Builder - Token-Aware LLM Context\nAssemble LLM context with priority-based truncation and token budgeting:\n\n```go\nimport sochdb \"github.com/sochdb/sochdb-go\"\n\nbuilder := sochdb.NewContextQueryBuilder().\n    WithBudget(4096).  // Token limit\n    SetFormat(sochdb.FormatTOON).\n    SetTruncation(sochdb.TailDrop)\n\nbuilder.\n    Literal(\"SYSTEM\", 0, \"You are a helpful AI assistant.\").\n    Literal(\"USER_PROFILE\", 1, \"User: Alice, Role: Engineer\").\n    Literal(\"HISTORY\", 2, \"Recent conversation context...\").\n    Literal(\"KNOWLEDGE\", 3, \"Retrieved documents...\")\n\nresult, _ := builder.Execute()\nfmt.Printf(\"Tokens: %d/%d\\n\", result.TokenCount, 4096)\nfmt.Printf(\"Context:\\n%s\\n\", result.Text)\n```\n\n**Key Benefits:**\n- ✅ Priority-based section ordering (lower = higher priority)\n- ✅ Token budget enforcement\n- ✅ Multiple truncation strategies (tail drop, head drop, proportional)\n- ✅ Multiple output formats (TOON, JSON, Markdown)\n- ✅ Token count estimation\n\n### Namespace API - Multi-Tenant Isolation\nFirst-class namespace handles for secure multi-tenancy and data isolation:\n\n```go\nimport (\n    sochdb \"github.com/sochdb/sochdb-go\"\n    \"github.com/sochdb/sochdb-go/embedded\"\n)\n\ndb, _ := embedded.Open(\"./mydb\")\ndefer db.Close()\n\n// Create isolated namespace for each tenant\nnamespace := \u0026sochdb.Namespace{}\n\n// Create vector collection\ncollection, _ := namespace.CreateCollection(sochdb.CollectionConfig{\n    Name:      \"documents\",\n    Dimension: 384,\n    Metric:    sochdb.DistanceMetricCosine,\n    Indexed:   true,\n})\n\n// Insert and search vectors\ncollection.Insert([]float32{1.0, 2.0, ...}, map[string]interface{}{\"title\": \"Doc 1\"}, \"\")\nresults, _ := collection.Search(sochdb.SearchRequest{\n    QueryVector: []float32{...},\n    K:          10,\n})\n```\n\n**[→ See Full Example](./examples/namespace/main.go)**\n\n### Priority Queue API - Task Processing\nEfficient priority queue with ordered-key storage (no O(N) blob rewrites):\n\n```go\nimport (\n    sochdb \"github.com/sochdb/sochdb-go\"\n    \"github.com/sochdb/sochdb-go/embedded\"\n)\n\ndb, _ := embedded.Open(\"./queue_db\")\ndefer db.Close()\n\nqueue := sochdb.NewPriorityQueue(db, \"tasks\", nil)\n\n// Enqueue with priority (lower = higher urgency)\ntaskID, _ := queue.Enqueue(1, []byte(\"urgent task\"), map[string]interface{}{\"type\": \"payment\"})\n\n// Worker processes tasks\ntask, _ := queue.Dequeue(\"worker-1\")\nif task != nil {\n    // Process task...\n    queue.Ack(task.TaskID)\n}\n\n// Get statistics\nstats, _ := queue.Stats()\nfmt.Printf(\"Pending: %d, Completed: %d\\n\", stats.Pending, stats.Completed)\n```\n\n**[→ See Full Example](./examples/queue/main.go)**\n\n**Key Benefits:**\n- ✅ O(log N) enqueue/dequeue with ordered scans\n- ✅ Atomic claim protocol for concurrent workers\n- ✅ Visibility timeout for crash recovery\n- ✅ Dead letter queue for failed tasks\n- ✅ Multiple queues per database\n\n---\n\n## Architecture: Flexible Deployment\n\n```\n┌─────────────────────────────────────────────────────────────┐\n│                    DEPLOYMENT OPTIONS                        │\n├─────────────────────────────────────────────────────────────┤\n│                                                               │\n│  1. EMBEDDED MODE (FFI)          2. SERVER MODE (gRPC)      │\n│  ┌─────────────────────┐         ┌─────────────────────┐   │\n│  │   Go App        │         │   Go App        │   │\n│  │   ├─ Database.open()│         │   ├─ SochDBClient() │   │\n│  │   └─ Direct FFI     │         │   └─ gRPC calls     │   │\n│  │         │           │         │         │           │   │\n│  │         ▼           │         │         ▼           │   │\n│  │   libsochdb_storage │         │   sochdb-grpc       │   │\n│  │   (Rust native)     │         │   (Rust server)     │   │\n│  └─────────────────────┘         └─────────────────────┘   │\n│                                                               │\n│  ✅ No server needed               ✅ Multi-language          │\n│  ✅ Local files                    ✅ Centralized logic      │\n│  ✅ Simple deployment              ✅ Production scale       │\n└─────────────────────────────────────────────────────────────┘\n```\n\n### When to Use Each Mode\n\n**Embedded Mode (FFI):**\n- ✅ Local development and testing\n- ✅ Jupyter notebooks and data science\n- ✅ Single-process applications\n- ✅ Edge deployments without network\n- ✅ No server setup required\n\n**Server Mode (gRPC):**\n- ✅ Production deployments\n- ✅ Multi-language teams (Python, Node.js, Go)\n- ✅ Distributed systems\n- ✅ Centralized business logic\n- ✅ Horizontal scaling\n\n---\n\n---\n\n## System Requirements\n\n### For Concurrent Mode\n\n- **SochDB Core**: Latest version\n- **Go Version**: 1.18+\n- **CGO**: Required (uses C bindings)\n- **Native Library**: `libsochdb_storage.{dylib,so}`\n\n**Operating Systems:**\n- ✅ Linux (Ubuntu 20.04+, RHEL 8+)\n- ✅ macOS (10.15+, both Intel and Apple Silicon)\n- ⚠️  Windows (requires WSL2 or native builds with MinGW)\n\n**File Descriptors:**\n- Default limit: 1024 (sufficient for most workloads)\n- For high concurrency: Increase with `ulimit -n 4096`\n\n**Memory:**\n- Standard mode: ~50MB base + data\n- Concurrent mode: +4KB per concurrent reader slot (1024 slots = ~4MB overhead)\n\n---\n\n## Troubleshooting\n\n### \"Database is locked\" Error (Standard Mode)\n\n```\nError: database is locked by another process\n```\n\n**Solution**: Use concurrent mode for multi-process access:\n\n```go\n// ❌ Standard mode - only one process allowed\ndb, _ := embedded.Open(\"./data.db\")\n\n// ✅ Concurrent mode - unlimited processes\ndb, _ := embedded.OpenConcurrent(\"./data.db\")\n```\n\n### Library Not Found Error\n\n```\nld: library not found for -lsochdb_storage\n```\n\n**Solution 1** - Manual installation:\n```bash\n# Build the native library\ncd /path/to/sochdb\ncargo build --release\n\n# Copy to system location (macOS/Linux)\nsudo cp target/release/libsochdb_storage.{dylib,so} /usr/local/lib/\nsudo ldconfig  # Linux only\n\n# Create pkg-config file\ncat \u003e /tmp/libsochdb_storage.pc \u003c\u003cEOF\nprefix=/usr/local\nexec_prefix=\\${prefix}\nlibdir=\\${exec_prefix}/lib\nincludedir=\\${prefix}/include\n\nName: libsochdb_storage\nDescription: SochDB Native Storage Library\nLibs: -L\\${libdir} -lsochdb_storage\nCflags: -I\\${includedir}\nEOF\n\nsudo mv /tmp/libsochdb_storage.pc /usr/local/lib/pkgconfig/\n```\n\n**Solution 2** - Development mode (no installation):\n```bash\nexport DYLD_LIBRARY_PATH=/path/to/sochdb/target/release  # macOS\nexport LD_LIBRARY_PATH=/path/to/sochdb/target/release    # Linux\nexport CGO_LDFLAGS=\"-L/path/to/sochdb/target/release -lsochdb_storage\"\n```\n\n### CGO Not Found\n\n```\ngo: C compiler \"gcc\" not found\n```\n\n**macOS**:\n```bash\nxcode-select --install\n```\n\n**Linux**:\n```bash\n# Ubuntu/Debian\nsudo apt-get install build-essential\n\n# RHEL/Fedora\nsudo yum groupinstall \"Development Tools\"\n```\n\n### Performance Issues\n\n**Symptom**: Concurrent reads slower than expected\n\n**Check 1** - Verify concurrent mode is active:\n```go\nif !db.IsConcurrent() {\n    log.Fatal(\"Database opened in standard mode!\")\n}\n```\n\n**Check 2** - Monitor reader slot usage:\n```bash\n# Enable debug logging in SochDB core\nexport RUST_LOG=sochdb_storage=debug\n```\n\n**Check 3** - Tune write batching:\n```go\n// Batch writes for better throughput\ntx, _ := db.BeginTxn()\nfor i := 0; i \u003c 1000; i++ {\n    tx.Put([]byte(fmt.Sprintf(\"key%d\", i)), value)\n}\ntx.Commit()  // Single fsync for entire batch\n```\n\n---\n\n---\n\n## API Reference\n\n\u003e **Version 0.4.5** — Complete API documentation with Go examples.\n\nAll core logic runs in the Rust engine via CGo FFI. The SDK is a thin client.\n\n---\n\n### Core Key-Value Operations\n\n```go\npackage main\n\nimport (\n    \"fmt\"\n    \"log\"\n    \"github.com/sochdb/sochdb-go/embedded\"\n)\n\nfunc main() {\n    db, err := embedded.Open(\"./mydb\")\n    if err != nil {\n        log.Fatal(err)\n    }\n    defer db.Close()\n\n    // Put / Get / Delete\n    db.Put([]byte(\"user:1\"), []byte(`{\"name\":\"Alice\"}`))\n    value, _ := db.Get([]byte(\"user:1\"))\n    fmt.Println(string(value)) // {\"name\":\"Alice\"}\n    db.Delete([]byte(\"user:1\"))\n\n    // Path-based keys (hierarchical)\n    db.PutPath(\"/users/alice/profile\", []byte(`{\"age\":30}`))\n    profile, _ := db.GetPath(\"/users/alice/profile\")\n    fmt.Println(string(profile))\n}\n```\n\n---\n\n### Transactions (ACID with SSI)\n\nSochDB uses Serializable Snapshot Isolation for full ACID transactions:\n\n```go\ndb, _ := embedded.Open(\"./mydb\")\ndefer db.Close()\n\n// Auto-managed transaction\nerr := db.WithTransaction(func(txn *embedded.Transaction) error {\n    txn.Put([]byte(\"key1\"), []byte(\"val1\"))\n    txn.Put([]byte(\"key2\"), []byte(\"val2\"))\n    val, _ := txn.Get([]byte(\"key1\"))\n    fmt.Println(string(val))\n    return nil // auto-commits; return error to abort\n})\n\n// Manual transaction control\ntxn := db.Begin()\ntxn.Put([]byte(\"balance:alice\"), []byte(\"100\"))\ntxn.Put([]byte(\"balance:bob\"), []byte(\"200\"))\nif err := txn.Commit(); err != nil {\n    // SSI conflict — retry\n    fmt.Println(\"Conflict:\", err)\n}\n```\n\n---\n\n### Prefix Scanning\n\n```go\ndb, _ := embedded.Open(\"./mydb\")\ndefer db.Close()\n\n// Insert test data\ndb.Put([]byte(\"user:1\"), []byte(\"Alice\"))\ndb.Put([]byte(\"user:2\"), []byte(\"Bob\"))\ndb.Put([]byte(\"user:3\"), []byte(\"Charlie\"))\n\n// Scan all keys with prefix (auto-transaction)\niter := db.ScanPrefix([]byte(\"user:\"))\ndefer iter.Close()\n\nfor {\n    key, value, ok := iter.Next()\n    if !ok {\n        break\n    }\n    fmt.Printf(\"%s = %s\\n\", key, value)\n}\n\n// Transaction-scoped scan\ntxn := db.Begin()\ndefer txn.Abort()\nit := txn.ScanPrefix([]byte(\"order:\"))\ndefer it.Close()\nfor {\n    k, v, ok := it.Next()\n    if !ok { break }\n    fmt.Printf(\"%s = %s\\n\", k, v)\n}\ntxn.Commit()\n```\n\n---\n\n### Namespaces \u0026 Collections\n\nMulti-tenant isolation with vector-enabled collections:\n\n```go\nimport (\n    sochdb \"github.com/sochdb/sochdb-go\"\n    \"github.com/sochdb/sochdb-go/embedded\"\n)\n\ndb, _ := embedded.Open(\"./mydb\")\ndefer db.Close()\n\n// Create a namespace\nns := sochdb.NewNamespace(db, \"tenant_1\", sochdb.NamespaceConfig{\n    Name:        \"tenant_1\",\n    DisplayName: \"Tenant One\",\n    Labels:      map[string]string{\"tier\": \"premium\"},\n})\n\n// Create a collection with vector search\ndocs, _ := ns.CreateCollection(sochdb.CollectionConfig{\n    Name:      \"documents\",\n    Dimension: 384,\n    Metric:    sochdb.DistanceMetricCosine,\n    Indexed:   true,\n})\n\n// Insert vectors with metadata\nid, _ := docs.Insert(\n    []float32{0.1, 0.2, 0.3 /* ...384 dims */},\n    map[string]interface{}{\"title\": \"Introduction to AI\", \"author\": \"Alice\"},\n    \"\", // auto-generate ID\n)\n\n// Batch insert\nids, _ := docs.InsertMany(\n    [][]float32{{0.1, 0.2}, {0.3, 0.4}},\n    []map[string]interface{}{{\"title\": \"Doc 1\"}, {\"title\": \"Doc 2\"}},\n    nil, // auto-generate IDs\n)\n\n// Search with filter\nresults, _ := docs.Search(sochdb.SearchRequest{\n    QueryVector:     []float32{0.1, 0.2, 0.3 /* ... */},\n    K:               5,\n    Filter:          map[string]interface{}{\"author\": \"Alice\"},\n    IncludeMetadata: true,\n})\nfor _, r := range results {\n    fmt.Printf(\"%s: score=%.4f\\n\", r.ID, r.Score)\n}\n\n// Collection management\ncount, _ := docs.Count()\ncollections, _ := ns.ListCollections()\nns.DeleteCollection(\"documents\")\n```\n\n---\n\n### Priority Queues\n\nOrdered task queues with priority-based dequeue, ack/nack, and dead-letter support:\n\n```go\nimport sochdb \"github.com/sochdb/sochdb-go\"\n\ndb, _ := embedded.Open(\"./mydb\")\ndefer db.Close()\n\n// Create a queue\nqueue := sochdb.CreateQueue(db, \"background-jobs\", \u0026sochdb.QueueConfig{\n    VisibilityTimeout: 30,\n    MaxRetries:        3,\n    DeadLetterQueue:   \"failed-jobs\",\n})\n\n// Enqueue tasks (lower priority = higher urgency)\ntaskID, _ := queue.Enqueue(\n    1,\n    []byte(`{\"action\":\"send_email\",\"to\":\"alice@example.com\"}`),\n    map[string]interface{}{\"source\": \"api\"},\n)\n\nqueue.Enqueue(10, []byte(\"low priority\"), nil)\nqueue.Enqueue(1, []byte(\"high priority\"), nil)\n\n// Dequeue highest priority task\ntask, _ := queue.Dequeue(\"worker-1\")\nif task != nil {\n    fmt.Printf(\"Processing: %s (priority: %d)\\n\", task.TaskID, task.Priority)\n    fmt.Printf(\"Payload: %s\\n\", task.Payload)\n\n    // Process task...\n    if err := processTask(task); err != nil {\n        queue.Nack(task.TaskID) // Re-queue for retry\n    } else {\n        queue.Ack(task.TaskID) // Mark completed\n    }\n}\n\n// Queue statistics\nstats, _ := queue.Stats()\nfmt.Printf(\"Pending: %d, Claimed: %d, Completed: %d\\n\",\n    stats.Pending, stats.Claimed, stats.Completed)\n\n// Purge completed/dead-lettered tasks\npurged, _ := queue.Purge()\nfmt.Printf(\"Purged %d tasks\\n\", purged)\n```\n\n---\n\n### Semantic Cache\n\nCache LLM responses with vector-similarity retrieval:\n\n```go\nimport sochdb \"github.com/sochdb/sochdb-go\"\n\ndb, _ := embedded.Open(\"./cache_db\")\ndefer db.Close()\n\ncache := sochdb.NewSemanticCache(db, \"llm_responses\")\n\n// Store response with embedding\ncache.Put(\n    \"What is machine learning?\",\n    \"Machine learning is a subset of AI...\",\n    []float32{0.1, 0.2, 0.3 /* ... */}, // embedding\n    3600,  // TTL seconds\n    map[string]interface{}{\"model\": \"gpt-4\", \"tokens\": 42},\n)\n\n// Check cache before calling LLM\nhit, _ := cache.Get(queryEmbedding, 0.85)\nif hit != nil {\n    fmt.Printf(\"Cache HIT (score: %.4f)\\n\", hit.Score)\n    fmt.Printf(\"Response: %s\\n\", hit.Value)\n}\n\n// Cache management\nstats, _ := cache.Stats()\nfmt.Printf(\"Hit rate: %.1f%%\\n\", stats.HitRate*100)\ncache.PurgeExpired()\ncache.Clear()\n```\n\n---\n\n### Graph Operations (IPC / gRPC)\n\nGraph overlay via IPC or gRPC client:\n\n```go\n// IPC Client\nclient, _ := sochdb.Connect(\"/tmp/sochdb.sock\")\ndefer client.Close()\n\n// Add nodes\nclient.AddNode(\"social\", \"alice\", \"person\", map[string]string{\"name\": \"Alice\"})\nclient.AddNode(\"social\", \"bob\", \"person\", map[string]string{\"name\": \"Bob\"})\n\n// Add edges\nclient.AddEdge(\"social\", \"alice\", \"works_at\", \"acme\", nil)\nclient.AddEdge(\"social\", \"alice\", \"knows\", \"bob\", nil)\n\n// Graph traversal (BFS/DFS)\nresult, _ := client.Traverse(\"social\", \"alice\", 2, \"bfs\")\nfmt.Printf(\"Nodes: %d, Edges: %d\\n\", len(result.Nodes), len(result.Edges))\n\n// gRPC Client\ngrpc, _ := sochdb.GrpcConnect(\"localhost:50051\")\ndefer grpc.Close()\n\ngrpc.AddGraphNode(\"alice\", \"person\", map[string]string{\"name\": \"Alice\"}, \"social\")\ngrpc.AddGraphEdge(\"alice\", \"knows\", \"bob\", nil, \"social\")\nnodes, edges, _ := grpc.TraverseGraph(\"alice\", 2, \"bfs\", \"social\")\n```\n\n---\n\n### Context Query Builder\n\nToken-budget-aware context assembly for LLM prompts:\n\n```go\nbuilder := sochdb.NewContextQueryBuilder()\n\nresult, _ := builder.\n    ForSession(\"session_123\").\n    WithBudget(4096).\n    SetFormat(sochdb.FormatMarkdown).\n    SetTruncation(sochdb.Proportional).\n    Literal(\"system\", 100, \"You are a helpful assistant.\").\n    Literal(\"user_query\", 90, \"Tell me about SochDB\").\n    Execute()\n\nfmt.Printf(\"Tokens used: %d\\n\", result.TokenCount)\nfmt.Println(result.Text)\n```\n\n---\n\n### Memory System (LLM-Native)\n\nComplete memory for AI agents — extraction, consolidation, hybrid retrieval:\n\n```go\nimport sochdb \"github.com/sochdb/sochdb-go\"\n\ndb, _ := embedded.Open(\"./memory_db\")\ndefer db.Close()\n\n// 1. Extract entities and relations\npipeline := sochdb.NewExtractionPipeline(db, \"user_123\", \u0026sochdb.ExtractionSchema{\n    EntityTypes:   []string{\"person\", \"organization\"},\n    MinConfidence: 0.7,\n})\n\nextracted, _ := pipeline.ExtractAndCommit(\n    \"Alice joined Acme Corp\",\n    func(text string) (map[string]interface{}, error) {\n        return map[string]interface{}{\n            \"entities\": []map[string]interface{}{\n                {\"name\": \"Alice\", \"entity_type\": \"person\", \"confidence\": 0.95},\n                {\"name\": \"Acme Corp\", \"entity_type\": \"organization\", \"confidence\": 0.9},\n            },\n        }, nil\n    },\n)\nfmt.Printf(\"Extracted %d entities\\n\", len(extracted.Entities))\n\n// 2. Consolidate facts\nconsolidator := sochdb.NewConsolidator(db, \"user_123\", nil)\nconsolidator.Add(\u0026sochdb.RawAssertion{\n    Fact:       map[string]interface{}{\"subject\": \"Alice\", \"predicate\": \"lives_in\", \"object\": \"SF\"},\n    Source:     \"conversation_1\",\n    Confidence: 0.9,\n})\nconsolidator.Consolidate()\nfacts, _ := consolidator.GetCanonicalFacts()\n\n// 3. Hybrid retrieval (vector + BM25 with RRF fusion)\nretriever := sochdb.NewHybridRetriever(db, \"user_123\", nil)\nretriever.IndexDocuments(map[string]map[string]interface{}{\n    \"doc1\": {\"content\": \"Alice is a software engineer\", \"embedding\": []float32{0.1, 0.2}},\n})\nresults, _ := retriever.Retrieve(\"Who is Alice?\", sochdb.NewAllAllowedSet())\n```\n\n---\n\n### Data Formats (TOON/JSON/Columnar)\n\n```go\n// Wire format types\nwire, _ := sochdb.ParseWireFormat(\"toon\")\nctx, _ := sochdb.ParseContextFormat(\"json\")\n\ncaps := sochdb.NewFormatCapabilities()\nroundTrip := caps.SupportsRoundTrip(wire)\nfmt.Printf(\"TOON round-trip: %v\\n\", roundTrip) // true\n```\n\n---\n\n### Server Mode (IPC / gRPC)\n\n```go\n// IPC Client (Unix socket)\nclient, _ := sochdb.Connect(\"/tmp/sochdb.sock\")\ndefer client.Close()\n\nclient.Put([]byte(\"key\"), []byte(\"value\"))\nvalue, _ := client.Get([]byte(\"key\"))\nclient.Delete([]byte(\"key\"))\n\n// Scan with prefix\nkvs, _ := client.Scan(\"user:\")\nfor _, kv := range kvs {\n    fmt.Printf(\"%s = %s\\n\", kv.Key, kv.Value)\n}\n\n// Transactions\ntxnID, _ := client.BeginTransaction()\n// ... operations ...\nclient.CommitTransaction(txnID)\n\n// Statistics\nstats, _ := client.Stats()\nfmt.Printf(\"Memtable: %d bytes, WAL: %d bytes\\n\",\n    stats.MemtableSizeBytes, stats.WALSizeBytes)\n\n// gRPC Client\ngrpc, _ := sochdb.GrpcConnect(\"localhost:50051\")\ndefer grpc.Close()\n\ngrpc.CreateIndex(\"embeddings\", 384, \"cosine\")\ngrpc.InsertVectors(\"embeddings\", []uint64{1, 2}, vectors)\nresults, _ := grpc.GrpcSearch(\"embeddings\", queryVector, 5)\n```\n\n---\n\n### Checkpoints \u0026 Statistics\n\n```go\ndb, _ := embedded.Open(\"./mydb\")\ndefer db.Close()\n\n// Force checkpoint (flush memtable → disk)\nlsn, _ := db.Checkpoint()\nfmt.Printf(\"Checkpoint LSN: %d\\n\", lsn)\n\n// Database statistics\nstats, _ := db.Stats()\nfmt.Printf(\"Memtable: %d bytes\\n\", stats.MemtableSizeBytes)\nfmt.Printf(\"WAL: %d bytes\\n\", stats.WalSizeBytes)\nfmt.Printf(\"Active txns: %d\\n\", stats.ActiveTransactions)\n```\n\n---\n\n### Index Policies\n\n```go\ndb, _ := embedded.Open(\"./mydb\")\ndefer db.Close()\n\n// Set index policy for a table\ndb.SetTableIndexPolicy(\"events\", embedded.IndexAppendOnly)\n\n// Get current policy\npolicy, _ := db.GetTableIndexPolicy(\"events\")\n// IndexWriteOptimized(0), IndexBalanced(1), IndexScanOptimized(2), IndexAppendOnly(3)\n```\n\n---\n\n### Error Handling\n\n```go\nimport sochdb \"github.com/sochdb/sochdb-go\"\n\ndb, err := embedded.Open(\"./mydb\")\nif err != nil {\n    var lockErr *sochdb.DatabaseLockedError\n    if errors.As(err, \u0026lockErr) {\n        fmt.Printf(\"Database locked by PID %d\\n\", lockErr.HolderPID)\n    }\n    log.Fatal(err)\n}\n\nerr = db.WithTransaction(func(txn *embedded.Transaction) error {\n    return txn.Put([]byte(\"key\"), []byte(\"value\"))\n})\nif err != nil {\n    // SSI conflict error: retry\n    fmt.Println(\"Transaction error:\", err)\n}\n```\n\n**Error types:**\n- `ConnectionError` — socket/network failures\n- `ProtocolError` — wire protocol issues\n- `TransactionError` — SSI conflicts\n- `DatabaseLockedError` — database locked by another process\n- `LockTimeoutError` — lock acquisition timeout\n- `EpochMismatchError` — stale epoch\n- `SplitBrainError` — concurrent mode conflict\n- `FormatConversionError` — format conversion failure\n\n---\n\n### Performance\n\n| Operation | Latency |\n|-----------|---------|\n| KV Read | ~100ns |\n| KV Write (fsync) | ~5ms |\n| KV Write (concurrent, amortized) | ~60µs |\n| Vector Search (HNSW, 1M vectors) | \u003c5ms |\n| Prefix Scan (per item) | ~200ns |\n| Max Concurrent Readers | 1024 |\n\n---\n\n## Contributing\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) for development setup, building from source, and pull request guidelines.\n\n---\n\n## License\n\nApache License 2.0\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsochdb%2Fsochdb-go","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsochdb%2Fsochdb-go","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsochdb%2Fsochdb-go/lists"}