{"id":35176409,"url":"https://github.com/bugzmanov/rediscache","last_synced_at":"2026-05-17T21:35:31.310Z","repository":{"id":47768165,"uuid":"255057626","full_name":"bugzmanov/rediscache","owner":"bugzmanov","description":"LRU Cache Service  with Monitoring \u0026 Alerting (Scala \u0026 Akka \u0026 Redis) ","archived":false,"fork":false,"pushed_at":"2021-08-13T19:28:43.000Z","size":65,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-04-24T11:54:31.779Z","etag":null,"topics":["cache-simulator","demo-app"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bugzmanov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-04-12T10:23:05.000Z","updated_at":"2024-04-24T11:54:31.780Z","dependencies_parsed_at":"2022-09-08T13:03:04.300Z","dependency_job_id":null,"html_url":"https://github.com/bugzmanov/rediscache","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/bugzmanov/rediscache","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bugzmanov%2Frediscache","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bugzmanov%2Frediscache/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bugzmanov%2Frediscache/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bugzmanov%2Frediscache/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bugzmanov","download_url":"https://codeload.github.com/bugzmanov/rediscache/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bugzmanov%2Frediscache/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33155809,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-17T09:28:26.183Z","status":"ssl_error","status_checked_at":"2026-05-17T09:27:52.702Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cache-simulator","demo-app"],"created_at":"2025-12-28T22:21:43.968Z","updated_at":"2026-05-17T21:35:31.301Z","avatar_url":"https://github.com/bugzmanov.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Requirements\n\nTo build a transparent Redis proxy service. This proxy is implemented as an HTTP web service which allows the ability to add additional features on top of Redis (e.g. caching and sharding)\n\n\n## Problem analysis\n\nRedis itself is an in-memory cache, potential reasons for putting another in-mem cache in-front of it:\n\n* To reduce the load on the Redis server  \n* To reduce the latency for fetching data if the server has limited bandwidth or if it's geographically distant from clients\n\nFor the second use case, caching larger objects would give more visible results\n\nImportant constraints:\n* A key size of a record in Redis is limited at 512 MiB\n* A record size in Redis is limited at 512 MiB\n* HTTP URL length is not limited by RFC, but in practice URLs longer than 2K can experience issues with processing software (like firewalls and proxies)  \n\nGiven that keys \u0026 values can potentially be relatively large, it's important to keep control of memory usage in addition to keys set size.\n\n## Protocol specification\n\n`GET /v1/\u003ckey\u003e`\n\nResponse content type: `application/octet-stream`\n\n| Status Code   | Description           | \n| ------------- |:-------------:| \n| 200           | Successfull operation | \n| 404           | No data for that key      | \n| 500           | Service error      | \n\n\n## Running instructions\n\n```bash\n\ngit clone https://github.com/bugzmanov/rediscache.git\ncd rediscache \nmake test\n\n```\n\nThen you can open grafana dashboard: `http://\u003chost\u003e:3000/d/dw2aBiqkz/mydashboard?refresh=5s\u0026orgId=1` (admin/admin)\n\nTo run the longer demo:\n\n```\n    make demo\n```\n\nConfiguration file for the docker-compose environment:\nhttps://github.com/bugzmanov/rediscache/blob/master/config/application-docker.conf\n\n## Overall description\n\n* Runtime \u0026 Language: scala on JVM \n* Webserver: akka-http (https://doc.akka.io/docs/akka-http/current/index.html)\n* Caching library: guava cache (https://github.com/google/guava/wiki/CachesExplained)\n* Redis client: jedis (https://github.com/xetorthio/jedis)\n* Monitoring: micrometer (https://micrometer.io/), graphite (https://graphiteapp.org/), grafana (https://grafana.com/grafana/)\n* Performance tests: locust (https://locust.io/)\n\n## Cache design \n\nGuava caching library provides support for main requirements: \n- eviction after TTL\n- LRU eviction after hitting a predefined capacity limit\n\nWebcache is not caching nil results from Redis, as those requests are inexpensive.\n\nGuava doesn't support weight and size limits simultaneously.\nTo avoid running out of memory situation, the cache is configured to wrap values in soft-references\n(https://en.wikipedia.org/wiki/Soft_reference).\n\nThis provides more stable behavior in case of memory overcommitment at the expense of less predictable latency.\n\nDownsides of soft-references:\n* stop-the-world GC pause become noticeably longer\n* Have to keep track of object size - because they might disappear from the cache (https://github.com/bugzmanov/rediscache/blob/master/src/main/scala/com/bugzmanov/cache/Cache.scala#L20)\n\n\nAlternatives to explore: \n* caffeine - https://github.com/ben-manes/caffeine\nSimilar API, better performance, worse memory footprint\n\n* ehcache - https://www.ehcache.org/\nThe main difference: it provides API for off-heap based cache. \nCan be used to create multi-layer cache: on-heap for small objects, off-heap for large objects\n\n\n## Redis client\n\nAlmost all clients of Redis for JVM languages suffer from the same downside - they fully accumulate response from Redis in a buffer\nbefore returning it to the requesting side. \nGiven that a record in Redis can be up to 512 MiB this puts significant constraints on how many clients can the  webcache serve \nsimultaneously.\n\nThe approach I've implemented so far:\n* a configuration parameter on how many active connections can web cache handle. This puts the burden of making webcache stable on\nDevOps. A user of the webcache has a better understanding of how data size distribution looks like and can decide on how\nmany concurrent clients webcache should handle.\n\nAn alternative approach:\n* provide configuration parameter allowing webcache to avoid caching objects larger specified size. For those objects webcache can serve\nas a proxy between a client and Redis instance, using relatively small buffers.\n\nRedis protocol specifies the size of the payload in RESP Bulk Strings. This can be used to make an early decision to cache or proxy.\nI didn't have enough time to play around with this approach. It doesn't look hard.\n\nThis approach makes sense only if large objects population is small and is accessed infrequently.\n\n## Monitoring \u0026 Alerting\n\nWebcache generates metrics for all layers: web, cache, Redis client.\nMetrics are generated using \"micrometer\" library, and can be adapted to report to any metrics collection system.\n\nCurrently metrics are reported to graphite and can be observed using the grafana dashboard.\n\n![image](https://user-images.githubusercontent.com/502482/79080375-517fbb00-7ce2-11ea-8fa8-b0262a141e21.png)\n\n\n### Alerting conditions:\n\n* Running out of memory: when free JVM memory size becomes less than 300 MiB. Indicate service overload. \n\u003cbr/\u003eMitigation strategy: \n    - reduce cache capacity and/or decrease the number of allowed active connections.\n\n* Cache is down: graphite hasn't received any heartbeats from the webcache for the last minute. \n\u003cbr/\u003eMitigation strategy: \n    - make sure cache is up and running\n    - make sure it can connect to graphite instance\n\n* Redis access errors: webcache is getting errors while reading data from Redis.\n\u003cbr/\u003eMitigation strategy:\n    - make sure backing Redis instance is up and running\n    - make sure webcache can connect to the Redis instance\n\n## Load testing\n\nhttps://github.com/bugzmanov/rediscache/blob/master/locust/locustfile.py\n\n## Implementation plan\n\n1. Redis-client       [90%]\n   - opt: a custom client that provides access to underlying InputStream\n2. Cache              [90%]\n   - opt: skip large objects\n   - opt: ehcache with off-heap\n3. Memory control     [Done]\n4. Webserver          [Done]\n5. Monitoring         [Done]\n6. Docker             [Done]\n7. Tests              [90%]\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbugzmanov%2Frediscache","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbugzmanov%2Frediscache","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbugzmanov%2Frediscache/lists"}