https://github.com/logocomune/echocache
https://github.com/logocomune/echocache
Last synced: 3 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/logocomune/echocache
- Owner: logocomune
- License: mit
- Created: 2025-02-08T15:19:47.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-02-21T21:03:37.000Z (over 1 year ago)
- Last Synced: 2025-02-21T22:19:13.393Z (over 1 year ago)
- Language: Go
- Size: 60.5 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# EchoCache
EchoCache is a generic, concurrency-safe Go caching library that prevents redundant computations under high load by combining a pluggable storage backend with [singleflight](https://pkg.go.dev/golang.org/x/sync/singleflight) deduplication: when many goroutines request the same missing key at the same time, only **one** refresh function is executed and the result is shared with all waiting callers.
Two caching modes are available:
| Mode | API | Description |
|------|-----|-------------|
| Synchronous | `EchoCache.FetchWithCache` | Callers block until the value is computed and cached. |
| Lazy / Stale-While-Revalidate | `EchoCacheLazy.FetchWithLazyRefresh` | A stale value is returned immediately; a background goroutine silently refreshes the entry. |
## Features
- **Generic** – works with any Go type (`string`, structs, slices, …).
- **Singleflight deduplication** – collapses concurrent misses for the same key into a single computation.
- **Stale-while-revalidate** – serve stale data instantly and refresh in the background.
- **Pluggable backends** – swap storage without changing application code:
- In-memory **LRU** (`NewLRUCache`) – bounded, no expiry.
- In-memory **Expirable LRU** (`NewLRUExpirableCache`) – bounded + per-entry TTL.
- In-memory **Single-entry** (`NewSingleCache`) – single slot with TTL, zero allocations on hits.
- **Redis** (`NewRedisCache`) – persistent, distributed, JSON serialisation.
- **NATS JetStream** (`NewNatsCache`) – distributed KeyValue-backed cache.
## Installation
```sh
go get github.com/logocomune/echocache
```
Requires **Go 1.21+**.
## Quick Start
### Synchronous cache (`EchoCache`)
```go
package main
import (
"context"
"fmt"
"time"
"github.com/logocomune/echocache"
"github.com/logocomune/echocache/store"
)
func main() {
// Create a cache backed by a 128-entry in-memory LRU.
cache := echocache.NewEchoCache[string](store.NewLRUCache[string](128))
ctx := context.Background()
key := "greeting"
// First call: cache miss – the refresh function is executed.
value, cached, err := cache.FetchWithCache(ctx, key, func(ctx context.Context) (string, error) {
time.Sleep(50 * time.Millisecond) // simulate slow work
return "Hello, World!", nil
})
fmt.Printf("value=%q cached=%v err=%v\n", value, cached, err)
// Output: value="Hello, World!" cached=true err=
// Second call: cache hit – refresh function is NOT executed.
value, cached, err = cache.FetchWithCache(ctx, key, func(ctx context.Context) (string, error) {
panic("should not be called")
})
fmt.Printf("value=%q cached=%v err=%v\n", value, cached, err)
// Output: value="Hello, World!" cached=true err=
}
```
### Concurrent deduplication
When many goroutines request the same key simultaneously, only one refresh is performed. All others wait and receive the same result.
```go
package main
import (
"context"
"fmt"
"sync"
"time"
"github.com/logocomune/echocache"
"github.com/logocomune/echocache/store"
)
func main() {
cache := echocache.NewEchoCache[string](store.NewLRUCache[string](128))
ctx := context.Background()
var wg sync.WaitGroup
for i := range 10 {
wg.Add(1)
go func(id int) {
defer wg.Done()
start := time.Now()
val, _, _ := cache.FetchWithCache(ctx, "shared-key", func(ctx context.Context) (string, error) {
time.Sleep(200 * time.Millisecond)
return "computed once", nil
})
fmt.Printf("goroutine %d: %q elapsed=%v\n", id, val, time.Since(start).Round(time.Millisecond))
}(i)
}
wg.Wait()
// All goroutines complete in ~200 ms, not 2 000 ms.
}
```
### Stale-while-revalidate (`EchoCacheLazy`)
Return the cached value immediately (even if stale) and refresh silently in the background.
```go
package main
import (
"context"
"fmt"
"time"
"github.com/logocomune/echocache"
"github.com/logocomune/echocache/store"
)
type Weather struct {
City string
Temperature float64
}
func main() {
// LRU backend – no TTL; the lazy refresh interval controls staleness.
backend := store.NewStaleWhileRevalidateLRUCache[Weather](128)
cache := echocache.NewLazyEchoCache[Weather](backend, 5*time.Second)
defer cache.ShutdownLazyRefresh()
ctx := context.Background()
refresh := func(ctx context.Context) (Weather, error) {
// Simulate a slow external API call.
time.Sleep(300 * time.Millisecond)
return Weather{City: "Rome", Temperature: 22.5}, nil
}
// First call: cold cache – blocks until the value is fetched (~300 ms).
w, _, _ := cache.FetchWithLazyRefresh(ctx, "weather:rome", refresh, 10*time.Second)
fmt.Printf("%s: %.1f°C\n", w.City, w.Temperature)
time.Sleep(11 * time.Second) // wait until the 10-second refresh interval expires
// Second call: returns the stale value INSTANTLY and queues a background refresh.
w, _, _ = cache.FetchWithLazyRefresh(ctx, "weather:rome", refresh, 10*time.Second)
fmt.Printf("%s: %.1f°C (stale – refresh enqueued)\n", w.City, w.Temperature)
}
```
### Using a Redis backend
```go
import (
"github.com/logocomune/echocache"
"github.com/logocomune/echocache/store"
"github.com/redis/go-redis/v9"
"time"
)
rdb := redis.NewClient(&redis.Options{Addr: "localhost:6379"})
// Simple cache
cache := echocache.NewEchoCache[string](
store.NewRedisCache[string](rdb, "myapp", 5*time.Minute),
)
// Lazy (stale-while-revalidate) cache
lazyBackend := store.NewStaleWhileRevalidateRedisCache[string](rdb, "myapp", 5*time.Minute)
lazyCache := echocache.NewLazyEchoCache[string](lazyBackend, 10*time.Second)
defer lazyCache.ShutdownLazyRefresh()
```
### Using a NATS JetStream backend
```go
import (
"github.com/logocomune/echocache"
"github.com/logocomune/echocache/store"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
)
nc, _ := nats.Connect(nats.DefaultURL)
js, _ := jetstream.New(nc)
kv, _ := js.CreateKeyValue(context.Background(), jetstream.KeyValueConfig{Bucket: "mycache"})
cache := echocache.NewEchoCache[string](store.NewNatsCache[string](kv, "prefix"))
```
## Storage backends reference
| Constructor | Interface | Description |
|-------------|-----------|-------------|
| `store.NewLRUCache[T](size)` | `Cacher[T]` | Bounded in-memory LRU, no expiry |
| `store.NewLRUExpirableCache[T](size, ttl)` | `Cacher[T]` | Bounded in-memory LRU with per-entry TTL |
| `store.NewSingleCache[T](ttl)` | `Cacher[T]` | Single-entry in-memory cache with TTL |
| `store.NewRedisCache[T](client, prefix, ttl)` | `Cacher[T]` | Redis-backed, JSON serialisation |
| `store.NewNatsCache[T](kv, prefix)` | `Cacher[T]` | NATS JetStream KeyValue-backed |
| `store.NewStaleWhileRevalidateLRUCache[T](size)` | `StaleWhileRevalidateCache[T]` | LRU variant for lazy caching |
| `store.NewStaleWhileRevalidateExpiringLRUCache[T](size, ttl)` | `StaleWhileRevalidateCache[T]` | Expirable LRU variant for lazy caching |
| `store.NewStaleWhileRevalidateSingleCache[T](ttl)` | `StaleWhileRevalidateCache[T]` | Single-entry variant for lazy caching |
| `store.NewStaleWhileRevalidateRedisCache[T](client, prefix, ttl)` | `StaleWhileRevalidateCache[T]` | Redis variant for lazy caching |
| `store.NewStaleWhileRevalidateNatsCache[T](kv, prefix)` | `StaleWhileRevalidateCache[T]` | NATS variant for lazy caching |
## Benchmarks
Run with:
```sh
go test ./... -bench=. -benchmem
```
Representative results on an AMD Ryzen 7 5800H (16 threads):
```
BenchmarkFetchWithCache_CacheHit-16 43965662 53.5 ns/op 0 B/op 0 allocs/op
BenchmarkFetchWithCache_CacheMiss-16 2335598 958.4 ns/op 319 B/op 5 allocs/op
BenchmarkFetchWithCache_Concurrent-16 46195622 56.7 ns/op 0 B/op 0 allocs/op
BenchmarkFetchWithLazyRefresh_CacheHit-16 15728220 159.3 ns/op 0 B/op 0 allocs/op
BenchmarkFetchWithLazyRefresh_CacheHitStale-16 24293737 99.0 ns/op 0 B/op 0 allocs/op
BenchmarkLRUCache_Get-16 49176892 48.9 ns/op 0 B/op 0 allocs/op
BenchmarkLRUCache_Set-16 44030736 56.8 ns/op 0 B/op 0 allocs/op
BenchmarkLRUExpirableCache_Get-16 26374305 86.9 ns/op 0 B/op 0 allocs/op
BenchmarkLRUExpirableCache_Set-16 20437982 117.7 ns/op 0 B/op 0 allocs/op
BenchmarkSingleCache_Get-16 87536472 28.2 ns/op 0 B/op 0 allocs/op
BenchmarkSingleCache_Set-16 26237246 93.0 ns/op 0 B/op 0 allocs/op
```
## License
Distributed under the MIT license.