{"id":31542513,"url":"https://github.com/databendlabs/sub-cache","last_synced_at":"2025-10-04T11:58:54.630Z","repository":{"id":298740222,"uuid":"1000901810","full_name":"databendlabs/sub-cache","owner":"databendlabs","description":"A client side cache by subscribe remote changes","archived":false,"fork":false,"pushed_at":"2025-06-15T04:23:07.000Z","size":64,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-02T04:31:03.134Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/databendlabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-12T13:50:01.000Z","updated_at":"2025-06-18T06:56:21.000Z","dependencies_parsed_at":"2025-06-12T16:44:20.938Z","dependency_job_id":"5477f4f5-2a29-413c-ad84-4ecabe26734d","html_url":"https://github.com/databendlabs/sub-cache","commit_stats":null,"previous_names":["databendlabs/sub-cache"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/databendlabs/sub-cache","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databendlabs%2Fsub-cache","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databendlabs%2Fsub-cache/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databendlabs%2Fsub-cache/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databendlabs%2Fsub-cache/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/databendlabs","download_url":"https://codeload.github.com/databendlabs/sub-cache/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databendlabs%2Fsub-cache/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278308622,"owners_count":25965654,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-04T02:00:05.491Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-04T11:58:52.256Z","updated_at":"2025-10-04T11:58:54.623Z","avatar_url":"https://github.com/databendlabs.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Distributed Cache\n\nA distributed cache implementation providing reliable resource management and data synchronization across distributed systems.\n\n## Features\n\n- **Automatic Synchronization**: Background watcher task keeps local cache in sync with remote data store\n- **Concurrency Control**: Two-level concurrency control mechanism for safe access\n- **Event-based Updates**: Real-time updates through watch API\n- **Safe Reconnection**: Automatic recovery from connection failures with state consistency\n\n## Key Components\n\n### Cache Structure\n\n```text\n\u003cprefix\u003e/foo\n\u003cprefix\u003e/..\n\u003cprefix\u003e/..\n```\n\n- `\u003cprefix\u003e`: User-defined string to identify a cache instance\n\n### Main Types\n\n- `Cache`: The main entry point for cache operations\n  - Provides safe access to cached data\n- `CacheData`: Internal data structure holding the cached values\n- `EventWatcher`: Background task that watches for changes in the remote data store\n  - Handles synchronization with remote data store\n\n## Usage\n\n```rust\nlet client = RemoteClient::try_create(/*..*/);\nlet cache = Cache::new(\n        client,\n        \"your/cache/key/space\",\n        \"your-app-name-for-logging\",\n).await;\n\n// Access cached data\ncache.try_access(|c: \u0026CacheData| {\n    println!(\"last-seq:{}\", c.last_seq);\n    println!(\"all data: {:?}\", c.data);\n}).await?;\n\n// Get a specific value\nlet value = cache.try_get(\"key\").await?;\n\n// List all entries under a prefix\nlet entries = cache.try_list_dir(\"prefix\").await?;\n```\n\n## Concurrency Control\n\nThe cache employs a two-level concurrency control mechanism:\n\n1. **Internal Lock (Mutex)**: Protects concurrent access between user operations and the background cache updater. This lock is held briefly during each operation.\n\n2. **External Lock (Method Design)**: Public methods require `\u0026mut self` even for read-only operations. This prevents concurrent access to the cache instance from multiple call sites. External synchronization should be implemented by the caller if needed.\n\nThis design intentionally separates concerns:\n- The internal lock handles short-term, fine-grained synchronization with the updater\n- The external lock requirement (`\u0026mut self`) enables longer-duration access patterns without blocking the background updater unnecessarily\n\nNote that despite requiring `\u0026mut self`, all operations are logically read-only with respect to the cache's public API.\n\n## Initialization Process\n\nWhen a `Cache` is created, it goes through the following steps:\n\n1. Creates a new instance with specified prefix and context\n2. Spawns a background task to watch for key-value changes\n3. Establishes a watch stream to the remote data store\n4. Fetches and processes initial data\n5. Waits for the cache to be fully initialized before returning\n6. Maintains continuous synchronization\n\nThe initialization is complete only when the cache has received a full copy of the data from the remote data store, ensuring users see a consistent view of the data.\n\n## Error Handling\n\nThe cache implements robust error handling:\n\n- Connection failures are automatically retried in the background\n- Background watcher task automatically recovers from errors\n- Users are shielded from transient errors through the abstraction\n- The cache ensures data consistency by tracking sequence numbers\n\n## License\n\nThis project is licensed under the Apache License 2.0 - see the LICENSE file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatabendlabs%2Fsub-cache","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatabendlabs%2Fsub-cache","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatabendlabs%2Fsub-cache/lists"}