{"id":15067738,"url":"https://github.com/orlovevgeny/go-mcache","last_synced_at":"2026-04-10T17:33:56.051Z","repository":{"id":57482585,"uuid":"129562054","full_name":"OrlovEvgeny/go-mcache","owner":"OrlovEvgeny","description":"Fast in-memory key:value store/cache with TTL","archived":false,"fork":false,"pushed_at":"2020-01-21T12:43:35.000Z","size":51,"stargazers_count":97,"open_issues_count":1,"forks_count":16,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-10-20T07:49:40.162Z","etag":null,"topics":["bigcache","cache","fast-cache","go-cache","golang","key-value","memcached","storage"],"latest_commit_sha":null,"homepage":"https://pkg.go.dev/github.com/OrlovEvgeny/go-mcache?tab=doc","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OrlovEvgeny.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-04-14T23:31:21.000Z","updated_at":"2024-09-10T07:32:20.000Z","dependencies_parsed_at":"2022-09-02T04:30:19.406Z","dependency_job_id":null,"html_url":"https://github.com/OrlovEvgeny/go-mcache","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OrlovEvgeny%2Fgo-mcache","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OrlovEvgeny%2Fgo-mcache/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OrlovEvgeny%2Fgo-mcache/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OrlovEvgeny%2Fgo-mcache/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OrlovEvgeny","download_url":"https://codeload.github.com/OrlovEvgeny/go-mcache/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248251223,"owners_count":21072686,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bigcache","cache","fast-cache","go-cache","golang","key-value","memcached","storage"],"created_at":"2024-09-25T01:26:53.361Z","updated_at":"2026-04-10T17:33:56.031Z","avatar_url":"https://github.com/OrlovEvgeny.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# go-mcache\n\n[![Go Report Card](https://goreportcard.com/badge/github.com/OrlovEvgeny/go-mcache)](https://goreportcard.com/report/github.com/OrlovEvgeny/go-mcache)\n[![GoDoc](https://pkg.go.dev/badge/github.com/OrlovEvgeny/go-mcache)](https://pkg.go.dev/github.com/OrlovEvgeny/go-mcache)\n[![Tests](https://github.com/OrlovEvgeny/go-mcache/actions/workflows/test.yml/badge.svg)](https://github.com/OrlovEvgeny/go-mcache/actions/workflows/test.yml)\n[![Release](https://img.shields.io/github/v/release/OrlovEvgeny/go-mcache)](https://github.com/OrlovEvgeny/go-mcache/releases/latest)\n\nGeneric in-memory cache for Go with TinyLFU admission, sharded storage, batched read tracking, and a timing-wheel expiration path.\n\n## Design\n\nmcache combines two ideas to achieve both high hit ratios and high throughput:\n\n**Admission control via TinyLFU.** Not every new item gets into the cache. On insertion, a Count-Min Sketch estimates the access frequency of the incoming item and compares it against a random sample of existing entries (SampledLFU). If the new item has lower frequency, it is rejected. This prevents one-time keys from evicting frequently accessed data — a problem that LRU caches have by design. The frequency sketch uses 4-bit counters (8 bytes per ~16 tracked keys) and resets periodically to adapt to changing access patterns.\n\n**Cheap read path.** Reads always go through a sharded map lookup. Frequency tracking is updated in a best-effort batched buffer once the cache becomes meaningfully occupied, which avoids paying TinyLFU bookkeeping on every hot read while the cache is still far from capacity.\n\n### Architecture\n\n```\nCache[K, V]\n├── ShardedStore         1024 shards, cache-line padded, per-shard RWMutex\n│   └── map[K]*Entry     standard Go map per shard\n├── Policy[K]            generic over key type (no hash-collision ambiguity)\n│   ├── TinyLFU          doorkeeper (Bloom filter) + Count-Min Sketch\n│   └── SampledLFU[K]    dense array + map for O(1) random sampling\n├── ExpiryWheel          coarse timing wheel for best-effort background expiry\n├── RadixTree            opt-in, for prefix search on string keys\n├── WriteBuffer          lock-free ring buffer for async batching\n├── ReadBuffer           lossy batched policy-access replay\n└── Metrics              optional atomic counters\n```\n\n### Why exact-key policy matters\n\nThe policy tracks entries by exact key `K`, not by hash. This means:\n- No hash collision ambiguity — two keys with the same hash are tracked independently\n- Eviction victims are returned as `{Key, KeyHash}` pairs — no reverse lookup needed\n- The `SampledLFU` uses a dense array with swap-delete, so random sampling is `O(sampleSize)` instead of `O(n)` map iteration\n\n### Expiration\n\nEntries with TTL are scheduled into a coarse timing wheel for best-effort background cleanup. Exact TTL enforcement still happens on `Get`/`Has` by checking the entry's `ExpireAt`, while the background worker lazily removes entries whose scheduled expiration still matches the live entry.\n\n## Install\n\n```\ngo get github.com/OrlovEvgeny/go-mcache\n```\n\nRequires Go 1.23+\n\n## Usage\n\n```go\ncache := mcache.NewCache[string, int](\n    mcache.WithMaxEntries[string, int](100_000),\n)\ndefer cache.Close()\n\ncache.Set(\"key\", 42, 5*time.Minute)\n\nval, ok := cache.Get(\"key\")  // type-safe, no assertion\n```\n\n### Cost-based eviction\n\n```go\ncache := mcache.NewCache[string, []byte](\n    mcache.WithMaxCost[string, []byte](100 \u003c\u003c 20), // 100 MB\n    mcache.WithCostFunc[string, []byte](func(v []byte) int64 {\n        return int64(len(v))\n    }),\n)\n\n// A 10 MB value may evict multiple smaller entries\ncache.Set(\"large\", make([]byte, 10\u003c\u003c20), 0)\n```\n\n### Batch reads\n\n```go\nbatch := cache.GetBatchOptimized(keys)\n// Keys are sorted by shard index before lookup\n// for sequential memory access and reduced lock contention\nfor i, key := range batch.Keys {\n    if batch.Found[i] {\n        process(key, batch.Values[i])\n    }\n}\n```\n\n### Prefix search (string keys, opt-in)\n\n```go\ncache := mcache.NewCache[string, int](\n    mcache.WithPrefixSearch[string, int](true),\n)\n\ncache.Set(\"user:1:name\", 1, 0)\ncache.Set(\"user:1:email\", 2, 0)\ncache.Set(\"user:2:name\", 3, 0)\n\niter := cache.ScanPrefix(\"user:1:\", 0, 100)\nfor iter.Next() {\n    fmt.Println(iter.Key()) // user:1:name, user:1:email\n}\n```\n\n### Async writes\n\n```go\ncache := mcache.NewCache[string, int](\n    mcache.WithBufferItems[string, int](64),\n)\n\ncache.Set(\"key\", 1, 0)    // buffered, returns immediately\ncache.Wait()               // blocks until buffer is flushed\nval, _ := cache.Get(\"key\") // guaranteed to see the value\n```\n\nWhen the write buffer is full, the operation falls back to synchronous execution instead of dropping the entry.\n\n## Configuration\n\n| Option | Description | Default |\n|--------|-------------|---------|\n| `WithMaxEntries` | Maximum number of entries | unlimited |\n| `WithMaxCost` | Maximum total cost | unlimited |\n| `WithNumCounters` | TinyLFU counters (recommend 10x max entries) | auto |\n| `WithShardCount` | Number of shards (power of 2) | 1024 |\n| `WithBufferItems` | Async write buffer size (0 = sync) | 0 |\n| `WithMetrics` | Enable cache metrics collection | true |\n| `WithExpirationResolution` | Background expiration tick resolution | 100ms |\n| `WithDefaultTTL` | Default TTL for entries without explicit TTL | 0 (no expiry) |\n| `WithCostFunc` | Custom cost calculator | cost = 1 |\n| `WithKeyHasher` | Custom key hash function | auto (FNV-1a) |\n| `WithLockFreePolicy` | Use lock-free TinyLFU for reads | true |\n| `WithPrefixSearch` | Enable radix tree for ScanPrefix | false |\n| `WithOnEvict` | Callback on eviction | nil |\n| `WithOnExpire` | Callback on TTL expiration | nil |\n| `WithOnReject` | Callback when TinyLFU rejects entry | nil |\n\n### Supported key types\n\nBuilt-in zero-allocation hashing for: `string`, `int`, `int8`–`int64`, `uint`–`uint64`, `float32`, `float64`, `bool`, `uintptr`. Other `comparable` types use `fmt.Sprintf` fallback — provide `WithKeyHasher` for production use with custom types.\n\n## API\n\n```go\n// Core\ncache.Set(key K, value V, ttl time.Duration) bool\ncache.SetWithCost(key K, value V, cost int64, ttl time.Duration) bool\ncache.Get(key K) (V, bool)\ncache.Has(key K) bool\ncache.Delete(key K) bool\ncache.Len() int\ncache.Clear()\ncache.Close()\ncache.Wait()\n\n// Batch\ncache.GetMany(keys []K) map[K]V\ncache.GetBatch(keys []K) *BatchResult[K, V]\ncache.GetBatchOptimized(keys []K) *BatchResult[K, V]\ncache.SetMany(items []Item[K, V]) int\ncache.DeleteMany(keys []K) int\n\n// Iterators (cursor-based, Redis-style)\ncache.Scan(cursor uint64, count int) *Iterator[K, V]\ncache.ScanPrefix(prefix string, cursor uint64, count int) *Iterator[K, V]\ncache.ScanMatch(pattern string, cursor uint64, count int) *Iterator[K, V]\n\n// Iterator methods\niter.Next() bool\niter.Key() K\niter.Value() V\niter.All() []Item[K, V]\niter.Keys() []K\niter.ForEach(func(K, V) bool)\niter.Count() int\niter.Cursor() uint64\niter.Close()\n\n// Metrics\ncache.Metrics() MetricsSnapshot\n// Fields: Hits, Misses, HitRatio, Sets, Deletes, Evictions,\n//         Expirations, Rejections, CostAdded, CostEvicted, BufferDrops\n```\n\n## Benchmarks\n\nThe repo includes a cross-library comparison suite for local in-process caches:\n`go-mcache`, `ristretto`, `bigcache`, `freecache`, `go-cache`, `ttlcache`,\n`golang-lru/expirable`, `otter`, and `theine`.\n\nRun the same suite used for the numbers below:\n\n```bash\nenv GOCACHE=/tmp/go-build-cache GOTOOLCHAIN=auto \\\ngo test -run '^$' -bench '^BenchmarkCompare/' -benchmem -benchtime=1s -count=3 .\n```\n\nCurrent reference run:\n- machine: `Apple M4 Pro`\n- metric shown in tables: median `ns/op` across `count=3` runs (`lower is better`)\n\nThe suite separates:\n- core throughput (`ReadParallelHot`, `WriteParallelOverwrite`, `MixedParallel80_20`, `DeleteCycle`)\n- TTL overhead (`SetWithTTLParallel`, `ExpiredRead` for precise TTL implementations)\n- bounded-cache pressure (`MixedParallelZipf95_5`, `MissThenSetZipf`)\n\nThese numbers are useful as a comparative signal for this repository, but they\nare still microbenchmarks on one machine. They should not be read as a universal\n\"best cache for every workload\" claim.\n\nCore scenarios:\n\n| Scenario | `go-mcache` | `ristretto` | `bigcache` | `freecache` | `go-cache` | `ttlcache` | `golang-lru-expirable` | `otter` | `theine` |\n|---|---|---|---|---|---|---|---|---|---|\n| Core/ReadParallelHot | 8.598 | 14.640 | 51.220 | 52.180 | 120.700 | 408.300 | 319.300 | 4.919 | 8.412 |\n| Core/WriteParallelOverwrite | 21.600 | 238.000 | 50.130 | 52.840 | 227.900 | 365.100 | 320.500 | 393.600 | 284.500 |\n| Core/MixedParallel80_20 | 15.040 | 75.300 | 38.090 | 51.160 | 49.070 | 401.000 | 340.900 | 85.120 | 92.620 |\n| Core/DeleteCycle | 148.500 | 241.700 | 62.730 | 39.670 | 36.910 | 78.370 | 85.140 | 129.500 | 197.400 |\n\nTTL and bounded scenarios:\n\n| Scenario | `go-mcache` | `ristretto` | `bigcache` | `freecache` | `go-cache` | `ttlcache` | `golang-lru-expirable` | `otter` | `theine` |\n|---|---|---|---|---|---|---|---|---|---|\n| TTL/SetWithTTLParallel | 237.000 | 507.300 | 59.800 | 53.430 | 271.900 | 261.900 | 298.600 | 401.600 | 339.300 |\n| TTL/ExpiredRead | 7.032 | 20.850 | — | — | 158.100 | 350.100 | 100.900 | 3.635 | 11.010 |\n| Bounded/MixedParallelZipf95_5 | 37.430 | 64.920 | 86.130 | 122.200 | — | 320.700 | 263.100 | 21.790 | 49.720 |\n| Bounded/MissThenSetZipf | 22.440 | 26.600 | 54.650 | 120.700 | — | 324.400 | 262.100 | 6.258 | 9.766 |\n\n`—` means the scenario is not part of that library's comparison set in this suite\n(for example, `ExpiredRead` is only run for caches with precise TTL semantics).\n\n## Thread safety\n\nAll operations are safe for concurrent use. The concurrency model:\n\n- **Reads**: shard `RLock` + optional best-effort read buffering for policy replay\n- **Writes**: overwrite fast path updates entries in-place when possible; inserts go through admission/eviction\n- **Expiration**: timing-wheel scheduling on writes, lazy delete on background ticks\n- **Metrics**: atomic counters when enabled\n- **Shards**: cache-line padded to prevent false sharing between cores\n\n## Legacy API\n\nThe `mcache.New()` / `CacheDriver` API from v1 still works. It uses `safeMap` + GC-based expiration without TinyLFU. See `mcache.go` and `gcmap/` for details. For new code, use the generic `NewCache[K, V]` API.\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Forlovevgeny%2Fgo-mcache","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Forlovevgeny%2Fgo-mcache","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Forlovevgeny%2Fgo-mcache/lists"}