{"id":13411831,"url":"https://github.com/Yiling-J/theine-go","last_synced_at":"2025-03-14T17:31:09.176Z","repository":{"id":151188299,"uuid":"623797735","full_name":"Yiling-J/theine-go","owner":"Yiling-J","description":"high performance in-memory cache","archived":false,"fork":false,"pushed_at":"2025-03-02T12:47:45.000Z","size":3903,"stargazers_count":289,"open_issues_count":5,"forks_count":18,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-11T07:09:02.569Z","etag":null,"topics":["cache","concurrency","performance"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Yiling-J.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-05T05:33:28.000Z","updated_at":"2025-03-10T14:25:45.000Z","dependencies_parsed_at":"2024-08-19T13:39:28.946Z","dependency_job_id":"e2a0cb6b-5191-4783-ba59-93a514663d9a","html_url":"https://github.com/Yiling-J/theine-go","commit_stats":null,"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Yiling-J%2Ftheine-go","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Yiling-J%2Ftheine-go/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Yiling-J%2Ftheine-go/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Yiling-J%2Ftheine-go/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Yiling-J","download_url":"https://codeload.github.com/Yiling-J/theine-go/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243618672,"owners_count":20320274,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cache","concurrency","performance"],"created_at":"2024-07-30T20:01:17.345Z","updated_at":"2025-03-14T17:31:09.170Z","avatar_url":"https://github.com/Yiling-J.png","language":"Go","funding_links":[],"categories":["Data Integration Frameworks","Database","数据库"],"sub_categories":["Caches","缓存"],"readme":"# Theine\n[![Go Reference](https://pkg.go.dev/badge/github.com/Yiling-J/theine-go.svg)](https://pkg.go.dev/github.com/Yiling-J/theine-go)\n[![codecov](https://codecov.io/gh/Yiling-J/theine-go/branch/main/graph/badge.svg?token=E1HJLJH07V)](https://codecov.io/gh/Yiling-J/theine-go)\n\nHigh performance in-memory \u0026 hybrid cache inspired by [Caffeine](https://github.com/ben-manes/caffeine).\n\n\n- Good performance\n- Support for Generics\n- High hit ratio with adaptive [W-TinyLFU](https://arxiv.org/pdf/1512.00727.pdf) eviction policy\n- Expired data are removed automatically using [hierarchical timer wheel](http://www.cs.columbia.edu/~nahum/w6998/papers/ton97-timing-wheels.pdf)\n- Simple API\n\n## Table of Contents\n\n- [Requirements](#requirements)\n- [Installation](#installation)\n- [API](#api)\n- [Cache Persistence](#cache-persistence)\n- [Benchmarks](#benchmarks)\n  * [throughput](#throughput)\n  * [hit ratios](#hit-ratios)\n- [Secondary Cache(Experimental)](#secondary-cacheexperimental)\n- [Support](#support)\n\n## Requirements\nGo 1.19+\n\n## Installation\n```\ngo get github.com/Yiling-J/theine-go\n```\n\n## API\n\n**Builder API**\n\nTheine provides two types of client, simple cache and loading cache. Both of them are initialized from a builder. The difference between simple cache and loading cache is: loading cache's Get method will compute the value using loader function when there is a miss, while simple cache client only return false and do nothing.\n\nLoading cache uses [singleflight](https://pkg.go.dev/golang.org/x/sync/singleflight) to prevent concurrent loading to same key(thundering herd).\n\n**Entry Pool**\n\nTheine stores `*Entry` as the value in the hashmap, where each entry contains key, value, and metadata related to the policy. Before v0.6.0, Theine used a `sync pool` to automatically reuse evicted entries. This approach was beneficial for scenarios with heavy concurrent writes. If your system is already optimized for allocation, it should significantly reduce memory allocations. However, if cache writes are rare and GC is triggered often by other parts of your system, the sync pool becomes nearly useless.\n\nAnd sync pool had a potential drawback: **race conditions within the policy**. Theine sends events to the policy asynchronously via channels/buffers, so there was a small chance that an event could be applied to the wrong entry if the entry was evicted and then reused by the pool.\n\nTo mitigate this, Theine rechecks the key first when updating the policy, but this behavior might be flagged by the race detector. Starting from v0.6.0, Theine introduced a new option called `UseEntryPool`, which defaults to `false`. If you are dealing with heavy concurrent writes and minimize allocations is crucial, you can enable this option.\n\n**API Details**\n\nsimple cache:\n\n```GO\nimport \"github.com/Yiling-J/theine-go\"\n\n// key type string, value type string, max size 1000\n// max size is the only required configuration to build a client\nclient, err := theine.NewBuilder[string, string](1000).Build()\nif err != nil {\n\tpanic(err)\n}\n\n// builder also provide several optional configurations\n// you can chain them together and call build once\n// client, err := theine.NewBuilder[string, string](1000).Cost(...).Doorkeeper(...).Build()\n\n// or create builder first\nbuilder := theine.NewBuilder[string, string](1000)\n\n// dynamic cost function based on value\n// use 0 in Set will call this function to evaluate cost at runtime\nbuilder.Cost(func(v string) int64 {\n\t\treturn int64(len(v))\n})\n\n// enable entryPool (default false)\nbuilder.UseEntryPool(true)\n\n// doorkeeper (default false)\n// doorkeeper will drop Set if they are not in bloomfilter yet\n// this can improve write performance, but may lower hit ratio\nbuilder.Doorkeeper(true)\n\n// removal listener, this function will be called when entry is removed\n// RemoveReason could be REMOVED/EVICTED/EXPIRED\n// REMOVED: remove by API\n// EVICTED: evicted by Window-TinyLFU policy\n// EXPIRED: expired by timing wheel\nbuilder.RemovalListener(func(key K, value V, reason theine.RemoveReason) {})\n\n```\nloading cache:\n\n```go\nimport \"github.com/Yiling-J/theine-go\"\n\n// loader function: func(ctx context.Context, key K) (theine.Loaded[V], error)\n// Loaded struct should include cache value, cost and ttl, which required by Set method\nclient, err := theine.NewBuilder[string, string](1000).Loading(\n\tfunc(ctx context.Context, key string) (theine.Loaded[string], error) {\n\t\treturn theine.Loaded[string]{Value: key, Cost: 1, TTL: 0}, nil\n\t},\n).Build()\nif err != nil {\n\tpanic(err)\n}\n\n```\nOther builder options are same as simple cache(cost, doorkeeper, removal listener).\n\n\n**Client API**\n\n```Go\n// set, key foo, value bar, cost 1\n// success will be false if cost \u003e max size\nsuccess := client.Set(\"foo\", \"bar\", 1)\n// cost 0 means using dynamic cost function\n// success := client.Set(\"foo\", \"bar\", 0)\n\n// set with ttl\nsuccess = client.SetWithTTL(\"foo\", \"bar\", 1, 1*time.Second)\n\n// get(simple cache version)\nvalue, ok := client.Get(\"foo\")\n\n// get(loading cache version)\nvalue, err := client.Get(ctx, \"foo\")\n\n// remove\nclient.Delete(\"foo\")\n\n// iterate key/value in cache and apply custom function\n// if function returns false, range stops the iteration\nclient.Range(func(key, value int) bool {\n\treturn true\n})\n\n// returns an estimation of the cache size usage\nclient.EstimatedSize()\n\n// get cache stats(in-memory cache only), include hits, misses and hit ratio\nclient.Stats()\n\n// close client, set hashmaps in shard to nil and close all goroutines\nclient.Close()\n\n```\n\n## Cache Persistence\nTheine supports persisting the cache into `io.Writer` and restoring from `io.Reader`. [Gob](https://pkg.go.dev/encoding/gob) is used to encode/decode data, so **make sure your key/value can be encoded by gob correctly first** before using this feature.\n\n#### API\n```go\nfunc (c *Cache[K, V]) SaveCache(version uint64, writer io.Writer) error\nfunc (c *Cache[K, V]) LoadCache(version uint64, reader io.Reader) error\n```\n**- Important:** please `LoadCache` immediately after client created, or existing entries' TTL might be affected.\n\n#### Example:\n```go\n// save\nf, err := os.Create(\"test\")\nerr := client.SaveCache(0, f)\nf.Close()\n\n// load\nf, err = os.Open(\"test\")\nrequire.Nil(t, err)\nnewClient, err := theine.NewBuilder[int, int](100).Build()\n// load immediately after client created\nerr = newClient.LoadCache(0, f)\nf.Close()\n```\nVersion number must be same when saving and loading, or `LoadCache` will return `theine.VersionMismatch` error. You can change the version number when you want to ignore persisted cache.\n```go\nerr := newClient.LoadCache(1, f)\n// VersionMismatch is a global variable\nif err == theine.VersionMismatch {\n\t// ignore and skip loading\n} else if err != nil {\n\t// panic error\n}\n```\n\nTheine does not guarantee that caches from previous versions will be compatible after an upgrade, so it is highly recommended not to use the old persistence file when upgrading Theine.\n\n#### Details\nWhen persisting cache, Theine roughly do:\n- Store version number.\n- Store clock(used in TTL).\n- Store frequency sketch.\n- Store entries one by one in protected LRU in most-recently:least-recently order.\n- Store entries one by one in probation LRU in most-recently:least-recently order.\n- Loop shards and store entries one by one in each shard deque.\n\nWhen loading cache, Theine roughly do:\n- Load version number, compare to current version number.\n- Load clock.\n- Load frequency sketch.\n- Load protected LRU and insert entries back to new protected LRU and shards/timingwheel, expired entries will be ignored. Because cache capacity may change, this step will stop if max protected LRU size reached.\n- Load probation LRU and insert entries back to new probation LRU and shards/timingwheel, expired entries will be ignored, Because cache capacity may change, this step will stop if max probation LRU size reached.\n- Load deque entries and insert back to shards, expired entries will be ignored.\n\nTheine will save checksum when persisting cache and verify checksum first when loading.\n\n## Benchmarks\n\nSource: https://github.com/maypok86/benchmarks\n\n### throughput\n\n100% read (cpu 8/16/32)\n\n```\ngoos: linux\ngoarch: amd64\npkg: github.com/maypok86/benchmarks/throughput\ncpu: Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz\n\nBenchmarkCache/zipf_otter_reads=100%,writes=0%-8                88954334                14.78 ns/op       67648151 ops/s\nBenchmarkCache/zipf_theine_reads=100%,writes=0%-8               51908306                21.87 ns/op       45729075 ops/s\nBenchmarkCache/zipf_ristretto_reads=100%,writes=0%-8            27217994                42.36 ns/op       23606992 ops/s\n\nBenchmarkCache/zipf_otter_reads=100%,writes=0%-16               132372591                8.397 ns/op     119086508 ops/s\nBenchmarkCache/zipf_theine_reads=100%,writes=0%-16              85420364                13.78 ns/op       72549558 ops/s\nBenchmarkCache/zipf_ristretto_reads=100%,writes=0%-16           47790158                25.17 ns/op       39734070 ops/s\n\nBenchmarkCache/zipf_otter_reads=100%,writes=0%-32               174121321                7.078 ns/op     141273879 ops/s\nBenchmarkCache/zipf_theine_reads=100%,writes=0%-32              118185849               10.45 ns/op       95703790 ops/s\nBenchmarkCache/zipf_ristretto_reads=100%,writes=0%-32           66458452                18.85 ns/op       53055079 ops/s\n\n```\n\n75% read (cpu 8/16/32)\n```\ngoos: linux\ngoarch: amd64\npkg: github.com/maypok86/benchmarks/throughput\ncpu: Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz\n\nBenchmarkCache/zipf_otter_reads=75%,writes=25%-8                49907841                32.67 ns/op       30609572 ops/s\nBenchmarkCache/zipf_theine_reads=75%,writes=25%-8               21484245                48.89 ns/op       20453469 ops/s\nBenchmarkCache/zipf_ristretto_reads=75%,writes=25%-8             8651056               130.5 ns/op         7664450 ops/s\n\nBenchmarkCache/zipf_otter_reads=75%,writes=25%-16               50226466                21.85 ns/op       45764160 ops/s\nBenchmarkCache/zipf_theine_reads=75%,writes=25%-16              46674459                24.68 ns/op       40523215 ops/s\nBenchmarkCache/zipf_ristretto_reads=75%,writes=25%-16           10233784               108.0 ns/op         9262524 ops/s\n\nBenchmarkCache/zipf_otter_reads=75%,writes=25%-32               89651678                11.96 ns/op       83606257 ops/s\nBenchmarkCache/zipf_theine_reads=75%,writes=25%-32              75969892                15.53 ns/op       64394679 ops/s\nBenchmarkCache/zipf_ristretto_reads=75%,writes=25%-32           15766912                76.37 ns/op       13093551 ops/s\n\n```\n\n\n100% write (cpu 8/16/32)\n\n```\ngoos: linux\ngoarch: amd64\npkg: github.com/maypok86/benchmarks/throughput\ncpu: Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz\n\nBenchmarkCache/zipf_otter_reads=0%,writes=100%-8                 1567917               723.0 ns/op         1383080 ops/s\nBenchmarkCache/zipf_theine_reads=0%,writes=100%-8                2194747               542.4 ns/op         1843615 ops/s\nBenchmarkCache/zipf_ristretto_reads=0%,writes=100%-8             1839237               642.5 ns/op         1556503 ops/s\n\nBenchmarkCache/zipf_otter_reads=0%,writes=100%-16                1384345               846.0 ns/op         1181980 ops/s\nBenchmarkCache/zipf_theine_reads=0%,writes=100%-16               1915946               528.8 ns/op         1891008 ops/s\nBenchmarkCache/zipf_ristretto_reads=0%,writes=100%-16            1765465               697.3 ns/op         1434089 ops/s\n\nBenchmarkCache/zipf_otter_reads=0%,writes=100%-32                1265883               979.8 ns/op         1020607 ops/s\nBenchmarkCache/zipf_theine_reads=0%,writes=100%-32               1953358               526.1 ns/op         1900935 ops/s\nBenchmarkCache/zipf_ristretto_reads=0%,writes=100%-32            1618098               696.1 ns/op         1436625 ops/s\n```\n\nbenchmem 100% write (cpu 32)\n```\ngoos: linux\ngoarch: amd64\npkg: github.com/maypok86/benchmarks/throughput\ncpu: Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz\n\nBenchmarkCache/zipf_otter_reads=0%,writes=100%-32                80 B/op          1 allocs/op\nBenchmarkCache/zipf_theine_reads=0%,writes=100%-32               0 B/op           0 allocs/op\nBenchmarkCache/zipf_ristretto_reads=0%,writes=100%-32            112 B/op         3 allocs/op\n\n```\n\n### hit ratios\n\n**zipf**\n\n![hit ratios](benchmarks/results/zipf.png)\n**s3**\n\n![hit ratios](benchmarks/results/s3.png)\n**ds1**\n\n![hit ratios](benchmarks/results/ds1.png)\n**oltp**\n\n![hit ratios](benchmarks/results/oltp.png)\n**wiki CDN**\n\n![hit ratios](benchmarks/results/wikicdn.png)\n**Twitter Cache**\n\n![hit ratios](benchmarks/results/twitter-c52s10.png)\n\n\n## Secondary Cache(Experimental)\n\nSecondaryCache is the interface for caching data on a secondary tier, which can be a non-volatile media or alternate forms of caching such as sqlite. The purpose of the secondary cache is to support other ways of caching the object. It can be viewed as an extension of Theine’s current in-memory cache.\n\nCurrently, the SecondaryCache interface has one implementation inspired by CacheLib's Hybrid Cache.\n\n```go\ntype SecondaryCache[K comparable, V any] interface {\n\tGet(key K) (value V, cost int64, expire int64, ok bool, err error)\n\tSet(key K, value V, cost int64, expire int64) error\n\tDelete(key K) error\n\tHandleAsyncError(err error)\n}\n```\n\nIf you plan to use a remote cache or database, such as Redis, as a secondary cache, keep in mind that the in-memory cache remains the primary source of truth. Evicted entries from memory are sent to the secondary cache. This approach differs from most tiered cache systems, where the remote cache is treated as the primary source of truth and is written to first.\n\n#### Secondary Cache Implementations\nNVM: https://github.com/Yiling-J/theine-nvm\n\n#### Limitations\n- Cache Persistence is not currently supported, but it may be added in the future. You can still use the Persistence API in a hybrid-enabled cache, but only the DRAM part of the cache will be saved or loaded.\n- The removal listener will only receive REMOVED events, which are generated when an entry is explicitly removed by calling the Delete API.\n- No Range/Len API.\n\n\n## Support\nFeel free to open an issue or ask question in discussions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FYiling-J%2Ftheine-go","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FYiling-J%2Ftheine-go","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FYiling-J%2Ftheine-go/lists"}