https://github.com/motoki317/sc
A simple generic in-memory caching layer
https://github.com/motoki317/sc
cache caching generics go golang memoization memoize
Last synced: 9 months ago
JSON representation
A simple generic in-memory caching layer
- Host: GitHub
- URL: https://github.com/motoki317/sc
- Owner: motoki317
- License: mit
- Created: 2022-04-02T12:54:38.000Z (almost 4 years ago)
- Default Branch: master
- Last Pushed: 2025-03-14T14:35:42.000Z (10 months ago)
- Last Synced: 2025-04-09T21:51:17.669Z (9 months ago)
- Topics: cache, caching, generics, go, golang, memoization, memoize
- Language: Go
- Homepage:
- Size: 135 KB
- Stars: 25
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# sc
[](https://github.com/motoki317/sc/releases/)
[](https://codecov.io/gh/motoki317/sc)
[](https://pkg.go.dev/github.com/motoki317/sc)
sc is a simple in-memory caching layer for golang.
[Introduction slide](https://speakerdeck.com/motoki317/effective-generic-cache-in-golang) (Japanese)
## Usage
Wrap your function with sc - it will automatically cache the values for specified amount of time, with minimal overhead.
```go
type HeavyData struct {
Data string
// and all the gazillion fields you may have in your data
}
func retrieveHeavyData(_ context.Context, name string) (*HeavyData, error) {
// Query to database or something...
return &HeavyData{
Data: "my-data-" + name,
}, nil
}
func main() {
// Wrap your data retrieval function.
cache, _ := sc.New[string, *HeavyData](retrieveHeavyData, 1*time.Minute, 2*time.Minute, sc.WithLRUBackend(500))
// It will automatically call the given function if value is missing.
foo, _ := cache.Get(context.Background(), "foo")
}
```
For a more detailed guide, see [reference](https://pkg.go.dev/github.com/motoki317/sc).
## Notable Features
- Simple to use: wrap your function with `New()` and just call `Get()`.
- There is no `Set()` method. Calling `Get()` will automatically retrieve the value for you.
- This prevents [cache stampede](https://en.wikipedia.org/wiki/Cache_stampede) problem idiomatically (see below).
- Supports 1.18 generics - both key and value are generic.
- No `interface{}` or `any` used other than in type parameters, even in internal implementations.
- All methods are safe to be called from multiple goroutines.
- Ensures only a single goroutine is launched per key to retrieve value.
- Allows 'graceful cache replacement' (if `freshFor` < `ttl`) - a single goroutine is launched in the background to
re-fetch a fresh value while serving stale value to readers.
- Allows strict request coalescing (`EnableStrictCoalescing()` option) - ensures that all returned values are fresh (a
niche use-case).
## Supported cache backends (cache replacement policy)
- Built-in map (default)
- Note: This backend cannot have max number of items configured. It holds all values in memory until expiration. For more, see the [documentation](https://pkg.go.dev/github.com/motoki317/sc#WithMapBackend).
- LRU (Least Recently Used)
- 2Q (Two Queue Cache)
## The design
### Why no Set() method? / Why cannot I dynamically provide load function to Get() method?
Short answer: sc is designed as a foolproof 'cache layer', not an overly complicated 'cache library'.
Long answer:
sc is designed as a simple, foolproof 'cache layer'.
Users of sc simply wrap data-retrieving functions and retrieve values via the cache.
By doing so, sc automatically reuses retrieved values and minimizes load on your data-store.
Now, let's imagine how users would use a more standard cache library with `Set()` method.
One could use `Get()` and `Set()` method to build the following logic:
1. `Get()` from the cache.
2. If the value is not in the cache, retrieve it from the source.
3. `Set()` the value.
This is probably the most common use-case, and it is fine for most applications.
But if you do not write it properly, the following problems may occur:
- If data flow is large, cache stampede might occur.
- Accidentally using different keys for `Get()` and `Set()`.
- Over-caching or under-caching by using inappropriate keys.
sc solves the problems mentioned above by acting as a 'cache layer'.
- sc will manage the requests for you - no risk of accidentally writing a bad caching logic and overloading your data-store with cache stampede.
- No manual `Set()` needed - no risk of accidentally using different keys.
- Only the cache key is passed to the pre-provided replacement function - no risk of over-caching or under-caching.
This is why sc does not have a `Set()` method, and forces you to provide replacement function on setup.
In this way, there is no risk of cache stampede and possible bugs described above -
sc will handle it for you.
### But I still want to manually `Set()` value on update!
By the nature of the design, sc is a no-write-allocate type cache.
You update the value on the data-store, and then call `Forget()` to clear the value on the cache.
sc will automatically load the value next time `Get()` is called.
One could design another cache layer library with `Set()` method which automatically calls the pre-provided
update function which updates the data-store, then updates the value on the cache.
But that would add whole another level of complexity - sc aims to be a simple cache layer.
## Inspirations from
I would like to thank the following libraries for giving me ideas:
- [go-chi/stampede: Function and HTTP request coalescer](https://github.com/go-chi/stampede)
- For "request coalescing" and "cache layer" idea
- [singleflight package - golang.org/x/sync/singleflight - pkg.go.dev](https://pkg.go.dev/golang.org/x/sync/singleflight)
- For internal implementation
- [Songmu/smartcache](https://github.com/Songmu/smartcache)
- For "graceful replacement" idea
- The term "graceful" comes from the [varnish](https://varnish-cache.org/) configuration.
- [methane/zerotimecache](https://github.com/methane/zerotimecache)
- For "zero-time cache" idea