{"id":19288523,"url":"https://github.com/samber/hot","last_synced_at":"2025-04-09T18:22:08.666Z","repository":{"id":231066351,"uuid":"780817705","full_name":"samber/hot","owner":"samber","description":"🌶️ In-memory caching library for read-intensive Go applications","archived":false,"fork":false,"pushed_at":"2025-03-25T02:03:06.000Z","size":137,"stargazers_count":78,"open_issues_count":5,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-02T13:54:36.141Z","etag":null,"topics":["2q","arc","cache","chain","concurrency","expiration","generics","go","golang","in-memory","jitter","lfu","loader","lru","performance","sharding","ttl"],"latest_commit_sha":null,"homepage":"https://pkg.go.dev/github.com/samber/hot","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/samber.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["samber"]}},"created_at":"2024-04-02T08:07:07.000Z","updated_at":"2025-03-27T03:53:06.000Z","dependencies_parsed_at":"2024-04-02T09:32:49.368Z","dependency_job_id":"ada4cd0b-cfc0-4112-a830-56d59510cb36","html_url":"https://github.com/samber/hot","commit_stats":{"total_commits":43,"total_committers":2,"mean_commits":21.5,"dds":0.2325581395348837,"last_synced_commit":"5f3eb3366d5d026ad3a9f4213bdffc62496cbe65"},"previous_names":["samber/hot"],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samber%2Fhot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samber%2Fhot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samber%2Fhot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samber%2Fhot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/samber","download_url":"https://codeload.github.com/samber/hot/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248085674,"owners_count":21045194,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["2q","arc","cache","chain","concurrency","expiration","generics","go","golang","in-memory","jitter","lfu","loader","lru","performance","sharding","ttl"],"created_at":"2024-11-09T22:09:15.681Z","updated_at":"2025-04-09T18:22:08.636Z","avatar_url":"https://github.com/samber.png","language":"Go","funding_links":["https://github.com/sponsors/samber"],"categories":["Database","Go"],"sub_categories":["Caches"],"readme":"\n# HOT - In-memory caching\n\n[![tag](https://img.shields.io/github/tag/samber/hot.svg)](https://github.com/samber/hot/releases)\n![Go Version](https://img.shields.io/badge/Go-%3E%3D%201.22-%23007d9c)\n[![GoDoc](https://godoc.org/github.com/samber/hot?status.svg)](https://pkg.go.dev/github.com/samber/hot)\n![Build Status](https://github.com/samber/hot/actions/workflows/test.yml/badge.svg)\n[![Go report](https://goreportcard.com/badge/github.com/samber/hot)](https://goreportcard.com/report/github.com/samber/hot)\n[![Coverage](https://img.shields.io/codecov/c/github/samber/hot)](https://codecov.io/gh/samber/hot)\n[![Contributors](https://img.shields.io/github/contributors/samber/hot)](https://github.com/samber/hot/graphs/contributors)\n[![License](https://img.shields.io/github/license/samber/hot)](./LICENSE)\n\n**HOT** stands for **H**ot **O**bject **T**racker.\n\nA feature-complete and [blazing-fast](#🏎️-benchmark) caching library for Go.\n\n## 💡 Features\n\n- 🚀 Fast, concurrent\n- 💫 Generics\n- 🗑️ Eviction policies: LRU, LFU, 2Q\n- ⏰ TTL with jitter\n- 🔄 Stale while revalidation\n- ❌ Missing key caching\n- 🍕 Sharded cache\n- 🔒 Optional locking\n- 🔗 Chain of data loaders with in-flight deduplication\n- 🌶️ Cache warmup\n- 📦 Batching all the way\n- 🧩 Composable caching strategy\n- 📝 Optional copy on read and/or write\n- 📊 Stat collection\n\n## 🚀 Install\n\n```sh\ngo get github.com/samber/hot\n```\n\nThis library is v0 and follows SemVer strictly.\n\nSome breaking changes might be made to exported APIs before v1.0.0.\n\n## 🤠 Getting started\n\n[GoDoc: https://godoc.org/github.com/samber/hot](https://godoc.org/github.com/samber/hot)\n\n### Simple LRU cache\n\n```go\nimport \"github.com/samber/hot\"\n\n// Available eviction policies: hot.LRU, hot.LFU, hot.TwoQueue, hot.ARC\n// Capacity: 100k keys/values\ncache := hot.NewHotCache[string, int](hot.LRU, 100_000).\n    Build()\n\ncache.Set(\"hello\", 42)\ncache.SetMany(map[string]int{\"foo\": 1, \"bar\": 2})\n\nvalues, missing := cache.GetMany([]string{\"bar\", \"baz\", \"hello\"})\n// values: {\"bar\": 2, \"hello\": 42}\n// missing: [\"baz\"]\n\nvalue, found, _ := cache.Get(\"foo\")\n// value: 1\n// found: true\n```\n\n### Cache with remote data source\n\nIf a value is not available in the in-memory cache, it will be fetched from a database or any data source.\n\nConcurrent calls to loaders are deduplicated by key.\n\n```go\nimport \"github.com/samber/hot\"\n\ncache := hot.NewHotCache[string, *User](hot.LRU, 100_000).\n    WithLoaders(func(keys []string) (found map[string]*User, err error) {\n        rows, err := db.Query(\"SELECT * FROM users WHERE id IN (?)\", keys)\n        // ...\n        return users, err\n    }).\n    Build()\n\nuser, found, err := cache.Get(\"user-123\")\n// might fail if \"user-123\" is not in cache and loader returns error\n\n// get or create\nuser, found, err := cache.GetWithLoaders(\n    \"user-123\",\n    func(keys []string) (found map[string]*User, err error) {\n        rows, err := db.Query(\"SELECT * FROM users WHERE id IN (?)\", keys)\n        // ...\n        return users, err\n    },\n    func(keys []string) (found map[string]*User, err error) {\n        rows, err := db.Query(\"INSERT INTO users (id, email) VALUES (?, ?)\", id, email)\n        // ...\n        return users, err\n    },\n)\n// either `err` is not nil, or `found` is true\n\n// missing value vs nil value\nuser, found, err := cache.GetWithLoaders(\n    \"user-123\",\n    func(keys []string) (found map[string]*User, err error) {\n        // value could not be found\n        return map[string]*User{}, nil\n\n       // or\n\n        // value exists but is nil\n        return map[string]*User{\"user-123\": nil}, nil\n    },\n)\n```\n\n### Cache with expiration\n\n```go\nimport \"github.com/samber/hot\"\n\ncache := hot.NewHotCache[string, int](hot.LRU, 100_000).\n    WithTTL(1 * time.Minute).      // items will expire after 1 minute\n    WithJitter(2, 30*time.Second). // optional: randomizes the TTL with an exponential distribution in the range [0, +30s)\n    WithJanitor(1 * time.Minute).  // optional: background job will purge expired keys every minutes\n    Build()\n\ncache.SetWithTTL(\"foo\", 42, 10*time.Second) // shorter TTL for \"foo\" key\n```\n\nWith cache revalidation:\n\n```go\nloader := func(keys []string) (found map[string]*User, err error) {\n    rows, err := db.Query(\"SELECT * FROM users WHERE id IN (?)\", keys)\n    // ...\n    return users, err\n}\n\ncache := hot.NewHotCache[string, *User](hot.LRU, 100_000).\n    WithTTL(1 * time.Minute).\n    // Keep delivering cache 5 more second, but refresh value in background.\n    // Keys that are not fetched during the interval will be dropped anyway.\n    // A timeout or error in loader will drop keys.\n    WithRevalidation(5 * time.Second, loader).\n    // On revalidation error, the cache entries are either kept or dropped.\n    // Optional (default: drop)\n    WithRevalidationErrorPolicy(hot.KeepOnError).\n    Build()\n```\n\nIf WithRevalidation is used without loaders, the one provided in `WithRevalidation()` or `GetWithLoaders()` is used.\n\n## 🍱 Spec\n\n```go\nhot.NewHotCache[K, V](algorithm hot.EvictionAlgorithm, capacity int).\n    // Enables cache of missing keys. The missing cache is shared with the main cache.\n    WithMissingSharedCache().\n    // Enables cache of missing keys. The missing keys are stored in a separate cache.\n    WithMissingCache(algorithm hot.EvictionAlgorithm, capacity int).\n    // Sets the time-to-live for cache entries\n    WithTTL(ttl time.Duration).\n    // Sets the time after which the cache entry is considered stale and needs to be revalidated\n    // * keys that are not fetched during the interval will be dropped anyway\n    // * a timeout or error in loader will drop keys.\n    // If no revalidation loader is added, the default loaders or the one used in GetWithLoaders() are used.\n    WithRevalidation(stale time.Duration, loaders ...hot.Loader[K, V]).\n    // Sets the policy to apply when a revalidation loader returns an error.\n    // By default, the key is dropped from the cache.\n    WithRevalidationErrorPolicy(policy revalidationErrorPolicy).\n    // Randomizes the TTL with an exponential distribution in the range [0, +upperBoundDuration).\n    WithJitter(lambda float64, upperBoundDuration time.Duration).\n    // Enables cache sharding.\n    WithSharding(nbr uint64, fn sharded.Hasher[K]).\n    // Preloads the cache with the provided data.\n    WithWarmUp(fn func() (map[K]V, []K, error)).\n    // Preloads the cache with the provided data. Useful when the inner callback does not have timeout strategy.\n    WithWarmUpWithTimeout(timeout time.Duration, fn func() (map[K]V, []K, error)).\n    // Disables mutex for the cache and improves internal performances.\n    WithoutLocking().\n    // Enables the cache janitor.\n    WithJanitor().\n    // Sets the chain of loaders to use for cache misses.\n    WithLoaders(loaders ...hot.Loader[K, V]).\n    // Sets the callback to be called when an entry is evicted from the cache.\n    // The callback is called synchronously and might block the cache operations if it is slow.\n    // This implementation choice is subject to change. Please open an issue to discuss.\n    WithEvictionCallback(hook func(key K, value V)).\n    // Sets the function to copy the value on read.\n    WithCopyOnRead(copyOnRead func(V) V).\n    // Sets the function to copy the value on write.\n    WithCopyOnWrite(copyOnWrite func(V) V).\n    // Returns a HotCache[K, V].\n    Build()\n```\n\nAvailable eviction algorithm:\n\n```go\nhot.LRU\nhot.LFU\nhot.TwoQueue\nhot.ARC\n```\n\nLoaders:\n\n```go\nfunc loader(keys []K) (found map[K]V, err error) {\n    // ...\n}\n```\n\nShard partitioner:\n\n```go\nfunc hash(key K) uint64 {\n    // ...\n}\n```\n\n## 🏛️ Architecture\n\nThis project has been split into multiple layers to respect the separation of concern.\n\nEach cache layer implements the `pkg/base.InMemoryCache[K, V]` interface. Combining multiple encapsulation has a small cost (~1ns per call), but offers great customization.\n\nWe highly recommend using `hot.HotCache[K, V]` instead of lower layers.\n\n### Eviction policies\n\nThis project provides multiple eviction policies. Each implements the `pkg/base.InMemoryCache[K, V]` interface.\n\nThey are not protected against concurrent access. If safety is required, encapsulate it into `pkg/safe.SafeInMemoryCache[K comparable, V any]`.\n\nPackages:\n- `pkg/lru`\n- `pkg/lfu`\n- `pkg/twoqueue`\n- `pkg/arc`\n\nExample:\n\n```go\ncache := lru.NewLRUCache[string, *User](100_000)\n```\n\n### Concurrent access\n\nThe `hot.HotCache[K, V]` offers protection against concurrent access by default. But in some cases, unnecessary locking might just slow down a program.\n\nLow-level cache layers are not protected by default. Use the following encapsulation to bring safety:\n\n```go\nimport (\n\t\"github.com/samber/hot/pkg/lfu\"\n\t\"github.com/samber/hot/pkg/safe\"\n)\n\ncache := safe.NewSafeInMemoryCache(\n    lru.NewLRUCache[string, *User](100_000),\n)\n```\n\n### Sharded cache\n\nA sharded cache might be useful in two scenarios:\n\n* highly concurrent application slowed down by cache locking -\u003e 1 lock per shard instead of 1 global lock\n* highly parallel application with no concurrency -\u003e no lock\n\nThe sharding key must not be too costly to compute and must offer a nice balance between shards. The hashing function must have `func(k K) uint64` signature.\n\nA sharded cache can be created via `hot.HotCache[K, V]` or using a low-level layer:\n\n```go\nimport (\n    \"hash/fnv\"\n    \"github.com/samber/hot/pkg/lfu\"\n    \"github.com/samber/hot/pkg/safe\"\n    \"github.com/samber/hot/pkg/sharded\"\n)\n\ncache := sharded.NewShardedInMemoryCache(\n    1_000, // number of shards\n    func() base.InMemoryCache[K, *item[V]] {\n        return safe.NewSafeInMemoryCache(\n            lru.NewLRUCache[string, *User](100_000),\n        )\n    },\n    func(key string) uint64 {\n        h := fnv.New64a()\n        h.Write([]byte(key))\n        return h.Sum64()\n    },\n)\n```\n\n### Missing key caching\n\nInstead of calling the loader chain every time an invalid key is requested, a \"missing cache\" can be enabled. Note that it won't protect your app against a DDoS attack with high cardinality keys.\n\nIf the missing keys are infrequent, sharing the missing cache with the main cache might be reasonable:\n\n```go\nimport \"github.com/samber/hot\"\n\ncache := hot.NewHotCache[string, int](hot.LRU, 100_000).\n    WithMissingSharedCache().\n    Build()\n```\n\nIf the missing keys are frequent, use a dedicated cache to prevent pollution of the main cache:\n\n```go\nimport \"github.com/samber/hot\"\n\ncache := hot.NewHotCache[string, int](hot.LRU, 100_000).\n    WithMissingCache(hot.LFU, 50_000).\n    Build()\n```\n\n## 🏎️ Benchmark\n\n// TODO: copy here the benchmarks of bench/ directory\n\n// - compare libraries\n\n// - measure encapsulation cost\n\n// - measure lock cost\n\n// - measure ttl cost\n\n// - measure size.Of cost\n\n// - measure stats collection cost\n\n## 🤝 Contributing\n\n- Ping me on Twitter [@samuelberthe](https://twitter.com/samuelberthe) (DMs, mentions, whatever :))\n- Fork the [project](https://github.com/samber/hot)\n- Fix [open issues](https://github.com/samber/hot/issues) or request new features\n\nDon't hesitate ;)\n\n```bash\n# Install some dev dependencies\nmake tools\n\n# Run tests\nmake test\n# or\nmake watch-test\n```\n\n## 👤 Contributors\n\n![Contributors](https://contrib.rocks/image?repo=samber/hot)\n\n## 💫 Show your support\n\nGive a ⭐️ if this project helped you!\n\n[![GitHub Sponsors](https://img.shields.io/github/sponsors/samber?style=for-the-badge)](https://github.com/sponsors/samber)\n\n## 📝 License\n\nCopyright © 2024 [Samuel Berthe](https://github.com/samber).\n\nThis project is [MIT](./LICENSE) licensed.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsamber%2Fhot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsamber%2Fhot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsamber%2Fhot/lists"}