Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/borislavv/adv-cache

HighLoad cache service / reverse proxy. Benches and battle tests: 40ns read, 50ns write, 200K RPS (CPU=6).
https://github.com/borislavv/adv-cache

caddy doorkeeper dump-to-file fasthttp highload http-cache https im-memory k8s low-latency low-level-programming lru-eviction prometheus-metrics refreshing-data reverse-proxy tiny-lfu traefik zero-allocation

Last synced: 4 months ago
JSON representation

HighLoad cache service / reverse proxy. Benches and battle tests: 40ns read, 50ns write, 200K RPS (CPU=6).

Awesome Lists containing this project

README 

          
# Advanced Cache (advCache)

[![Go Version](https://img.shields.io/static/v1?label=Go&message=1.23%2B&logo=go&color=00ADD8)](https://go.dev/dl/) [![Coverage](https://img.shields.io/codecov/c/github/Borislavv/adv-cache?label=coverage)](https://codecov.io/gh/Borislavv/adv-cache) [![License](https://img.shields.io/badge/License-Apache--2.0-green.svg)](./LICENSE)



High‑performance **in‑memory HTTP cache & reverse proxy** for latency‑sensitive workloads. Built in Go on top of `fasthttp`, with sharded storage, TinyLFU admission, background refresh, upstream controls, and minimal‑overhead observability (Prometheus + OpenTelemetry).



---



## Why advCache?

- **Throughput**: 160–170k RPS locally; ~250k RPS sustained on 24‑core bare‑metal with a 50GB cache.

- **Memory safety**: 1.5–3GB overhead at 50GB (no traces); ~7GB at 100% OTEL sampling.

- **Hot path discipline**: zero allocations, sharded counters, per‑shard LRU, TinyLFU admission.

- **Control plane**: runtime API for toggles (admission, eviction, refresh, compression, observability).

- **Observability**: Prometheus/VictoriaMetrics metrics + OpenTelemetry tracing.

- **Kubernetes‑friendly**: health probes, config via ConfigMap, Docker image.



---



## Quick start (production‑capable starter config)

**Edit the CHANGEME fields and run.** This is a complete config based on `advcache.cfg.yaml`, trimmed for a fast start but **fully runnable**.



```yaml

cache:

  env: prod

  enabled: true

  logs:

    level: debug

  runtime:

    gomaxprocs: 0

  api:

    name: advCache.local

    port: '8020             # <-- CHANGEME: API port to listen on'

  upstream:

    backend:

      id: example-ams-web

      enabled: true

      policy: deny

      host: service-example:8080

      scheme: http

      rate: 15000

      concurrency: 4096

      timeout: 10s

      max_timeout: 1m

      use_max_timeout_header: ''

      healthcheck: /healthcheck

      addr: http://127.0.0.1:8081  # <-- CHANGEME: your upstream origin URL

      health_path: /health

  compression:

    enabled: true

    level: 1

  data:

    dump:

      enabled: true

      dump_dir: public/dump

      dump_name: cache.dump

      crc32_control_sum: true

      max_versions: 3

      gzip: false

    mock:

      enabled: false

      length: 1000000

  storage:

    mode: listing

    size: 53687091200

  admission:

    enabled: true

    capacity: 2000000

    sample_multiplier: 4

    shards: 256

    min_table_len_per_shard: 65536

    door_bits_per_counter: 12

  eviction:

    enabled: true

    replicas: 32

    soft_limit: 0.8

    hard_limit: 0.99

    check_interval: 100ms

  lifetime:

    enabled: true

    ttl: 2h

    on_ttl: refresh

    beta: 0.35

    rate: 1000

    replicas: 32

    coefficient: 0.25

  observability:

    enabled: true

    service_name: advCache.local

    service_version: dev

    service_tenant: star

    exporter: http

    endpoint: 127.0.0.1:4318     # <-- CHANGEME: your OTEL Collector (http/4318 or grpc/4317)

    insecure: true

    sampling_mode: ratio

    sampling_rate: 0.1

    export_batch_size: 512

    export_batch_timeout: 3s

    export_max_queue: 1024

  forceGC:

    enabled: true

    interval: 6m

  metrics:

    enabled: true

  k8s:

    probe:

      timeout: 5s

  rules:

    /api/v2/pagedata:

      cache_key:

        query:

          - project

          - language

          - timezone

        headers:

          - Accept-Encoding

      cache_value:

        headers:

          - Vary

          - Server

          - Content-Type

          - Content-Length

          - Content-Encoding

          - Cache-Control



```



**What to change first:**

- `cache.api.port` — the port **advCache** listens on.

- `cache.upstream.backend.addr` — point to your origin.

- `cache.compression.enabled` — enable if latency budget allows (runtime‑toggle also available).

- `cache.observability.*` — set `enabled: true` and `endpoint` of your OTEL Collector; adjust sampling.

- `cache.admission.enabled` — **true** to protect hot set; details of TinyLFU/Doorkeeper in the main config comments.

- `cache.upstream.policy` — **both `deny` and `await` are production‑ready**; choose behavior:

    - `deny` → fail‑fast under pressure (good for synthetic load / when back‑pressure is handled elsewhere).

    - `await` → apply back‑pressure (preferred default in many prod setups).



> Full field descriptions and advanced knobs are documented inline in the canonical `advcache.cfg.yaml`.



---



## Endpoints (runtime control plane)

- Main: `GET /{any:*}`

- Health: `GET /k8s/probe`

- Metrics: `GET /metrics` (Prometheus/VictoriaMetrics)

- Bypass: `/cache/bypass`, `/on`, `/off`

- Compression: `/cache/http/compression`, `/on`, `/off`

- Config dump: `/cache/config`

- Entry by key: `/cache/entry?key=`

- Clear (two‑step): `/cache/clear` → then `/cache/clear?token=<...>`

- Invalidate: `/cache/invalidate` (supports `X-Entries-Remove` for remove entries and `_path` + any queries)

- Upstream policy: `/cache/upstream/policy`, `/await`, `/deny`

- Evictor: `/cache/eviction`, `/on`, `/off`, `/scale?to=`

- Lifetime manager: `/cache/lifetime-manager`, `/on`, `/off`, `/rate?to=`, `/scale?to=`, `/policy`, `/policy/remove`, `/policy/refresh`

- Force GC: `/cache/force-gc`, `/on`, `/off`, `/call`

- Admission: `/cache/admission`, `/on`, `/off`

- Tracing: `/cache/observability`, `/on`, `/off`



---



## Observability: traces with minimal overhead

- Spans: `ingress` (server), `upstream` (client on miss/proxy), `refresh` (background).

- When disabled: **fast no‑op** provider (atomic toggle only).

- When enabled: stdout exporter → sync; OTLP (`grpc`/`http`) → **batch** exporter.



**Enable quickly:** set in YAML and/or toggle at runtime:

```bash

GET /cache/observability/on    # enable tracing now

GET /cache/observability/off   # disable tracing

```



---



## Build & Run

```bash

go build -o advCache ./cmd/main.go

./advCache -cfg ./advcache.starter.yaml

# Docker

docker build -t advcache .

docker run --rm -p 8020:8020 -v "$PWD/public/dump:/app/public/dump" advcache -cfg /app/advcache.starter.yaml

```



---



## Notes on performance

Remember that this depends largely on the specifics of your load.

- Local (4–6 CPU, 1–16KB docs, 20–25GB store): **160–170k RPS** steady.

- Bare‑metal (24 CPU, 50GB store, prod traffic): **~250k RPS** sustained.

- Memory overhead at 50GB: **1.5–3GB** (no traces) • **~7GB** (100% sampling).



---



## Testing

```bash

go test ./...

```



---



## Benching

```bash

go test -bench . -benchmem ./...

```



---



## License & Maintainer

Apache‑2.0 — see [LICENSE](./LICENSE).  

Maintainer: **Borislav Glazunov** —  · Telegram `@gl_c137`