{"id":15056418,"url":"https://github.com/tanguilp/http_cache_store_memory","last_synced_at":"2025-04-10T04:08:04.713Z","repository":{"id":176720786,"uuid":"654148380","full_name":"tanguilp/http_cache_store_memory","owner":"tanguilp","description":"Memory store for http_cache","archived":false,"fork":false,"pushed_at":"2024-11-25T18:21:49.000Z","size":95,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-10T04:07:51.182Z","etag":null,"topics":["elixir","erlang","http-caching"],"latest_commit_sha":null,"homepage":"https://hexdocs.pm/http_cache_store_memory/","language":"Erlang","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tanguilp.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-15T13:39:49.000Z","updated_at":"2024-11-25T18:21:40.000Z","dependencies_parsed_at":null,"dependency_job_id":"6b7c78b2-b744-4764-b1b7-e13c45eeea83","html_url":"https://github.com/tanguilp/http_cache_store_memory","commit_stats":null,"previous_names":["tanguilp/http_cache_store_memory"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tanguilp%2Fhttp_cache_store_memory","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tanguilp%2Fhttp_cache_store_memory/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tanguilp%2Fhttp_cache_store_memory/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tanguilp%2Fhttp_cache_store_memory/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tanguilp","download_url":"https://codeload.github.com/tanguilp/http_cache_store_memory/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248154986,"owners_count":21056543,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elixir","erlang","http-caching"],"created_at":"2024-09-24T21:51:08.648Z","updated_at":"2025-04-10T04:08:04.704Z","avatar_url":"https://github.com/tanguilp.png","language":"Erlang","funding_links":[],"categories":[],"sub_categories":[],"readme":"http_cache_store_memory\n=====\n\n`http_cache_store_memory` is an in-memory LRU cache that can be used as a backend for `http_cache`.\nIt implements the `http_cache_store` behaviour.\n\nIt supports:\n- in-memory caching, with fixed limit or limit in % of used system memory\n- clustering, using BEAM distribution. The following events are broadcast:\n  - newly cached HTTP responses (in an efficient manner)\n  - invalidation requests\n  - warmup: already present nodes send their most recently used cached HTTP responses to joining nodes\n- telemetry events (see [Telemetry](#telemetry))\n- Backpressure mechanisms to avoid overloading the whole system with caching operations\n- the optional `http_cache_store:invalidate_by_alternate_key/1` callback\n\nIt uses Erlang's capabilities and doesn't require external stores (Redis, memcached, DB...).\nUnder the hood, it uses ETS tables, system monitor and Erlang's distribution.\n\nFor on-disk caching, see: [`http_cache_store_disk`](https://github.com/tanguilp/http_cache_store_disk).\n\n## `http_cache*` pro versions\n\nAdvanced HTTP caching features are available as licensed packages:\n- [`http_cache_pro`](https://hex.codecodeship.com/package/http_cache_pro): brings HTTP request\ncollapsing, brotli compression support and more\n- [`http_cache_store_disk_pro`](https://hex.codecodeship.com/package/http_cache_store_disk_pro):\noffers SSD protection and persistence support to your disk backend\n\nRefer to these links for technical insights and license information.\n\n## Support\n\nOTP25+\n\n## Usage\n\nThis is an OTP application, and automatically starts.\n\n### Configuration parameters\n\n- `cluster_enabled`: exchange of information between nodes of the Erlang cluster is enabled.\nDefaults to `false`\n- `memory_limit`: how much memory is allocated for caching. If this is an integer, then it's the\nnumber of bytes allocated to store the cached responses. If it is a float, it's the system memory\nthreshold that triggers nuking older entries. Defaults to `0.9`, that is, as soon as 90% of the\nsystem memory is used, objects are deleted until system memory use no longer exceeds this threshold\n- `max_concurrency`: how many workers are to be working at the same time for adding new cache\nentries (including from remote nodes). Defaults to `32`\n- `pull_table_stats_interval`: how often stats are retrieved and associated telemetry event emitted,\nin milliseconds. Defaults to `1000`\n- `warmup_nb_objects`: how many objects are sent to joining nodes when they request warm-up.\nDefault to `5000`\n- `warmup_timeout`: how long the warmup process is active, that is it tries to get objects from\njoining nodes, in milliseconds. Default to `20000`\n- `limit_check_interval`: how often to check for limits, and trigger LRU nuking when exceeded, in\nmilliseconds. Defaults to `200`\n- `expired_resp_sweep_interval`: how often expired responses are purged, in milliseconds.\nDefaults to `3000`\n- `outdated_lru_sweep_interval`: how often outdated LRU entries are purged, in milliseconds.\nDefaults to `2000`\n\nAll are environment parameters.\n\nAll are read at runtime (and can be changed dynamically) except `cluster_enabled`.\n\n## Installation\n\nErlang (rebar3):\n\n```erlang\n{deps, [{http_cache_store_memory, \"~\u003e 0.3.0\"}]}.\n```\n\nElixir:\n\n```elixir\n{:http_cache_store_memory, \"~\u003e 0.3.0\"}\n```\n\n## Telemetry\n\n- `[http_cache_store_memory, object_deleted]` is emitted whenever an object is deleted\n  - Measurements: none\n  - Metadata:\n    - `reason`: one of `lru_nuked`, `expired`, `url_invalidation`, `alternate_key_invalidation`\n- `[http_cache_store_memory, memory]` is emitted regularly by the stats service\n  - Measurements:\n    - `total_mem`: total memory used by `http_cache_store_memory` subsystems\n    - `objects_mem`: memory used by `http_cache_store_memory` to store HTTP responses\n    - `lru_mem`: memory used by `http_cache_store_memory` to store LRU data\n    - `objects_count`: number of HTTP responses cached\n  - Metadata: none\n- `[http_cache_store_memory, lru_nuker]`: events triggered by the LRU nuker process\n(uses `telemetry:span/3`)\n- `[http_cache_store_memory, expired_lru_entry_sweeper]`: events triggered by the LRU sweeper process\n(uses `telemetry:span/3`)\n- `[http_cache_store_memory, expired_resp_sweeper]`: events triggered by the outdated response\nsweeper process (uses `telemetry:span/3`)\n\n## Architecture\n\nThis application uses 3 ETS tables:\n- the object table, that stores the cached HTTP responses\n- the LRU table, that stores the last time a response was used (returned to a client)\n- the configuration table\n\nThe configuration table is used to store when the limit is reached, so as to discard immediately\nrequests to cache HTTP responses.\n\nThe object and LRU tables store the following tuples:\n\nObject table: `{ObjectKey, VaryHeaders, UrlDigest, Response, RespMetadata, Expires, SeqNumber}`\n\nLRU table: `{{LastUsedTime, ObjectKey, SeqNumber}}`. Note there's only one 3-tuple entry.\n\nBoth are `ordered_set` tables, so that:\n- they can be traversed even though they're modified at the same time. This is useful for cleanup\nprocesses (see below)\n- oldest entries of the LRU table can be found very efficiently (since the timestamp is the primary\nkey, the entry returned by `ets:first/1` is the oldest)\n\nAlso note that the `ObjectKey` of the object table is a `{RequestKey, VaryHeadersHash}` tuple.\nIndeed, the same URL and HTTP verb can have different HTTP responses, depending on the `vary` header\nand we need to return all of them to `http_cache` to select the correct one. Fortunately, matching\non the first term of a compound key in an `ordered_set` table is very efficient.\n\nWhenever `http_cache:notify_response_used/2` is called, a new entry is added in the LRU table\nwith the current time, object key and a `SeqNumber`. This sequence number is a random integer and\nis also updated in the object table along with the cached HTTP response. When nuking older entries,\nwe then can use this sequence number to determine if the cached response was used since\n(the `SeqNumber` between the 2 tables don't match) or not (they do match).\n\nAt startup, some processes are launched:\n\n![Screenshot of the supervision tree](https://raw.githubusercontent.com/tanguilp/http_cache_store_memory/master/supervision_tree.png)\n\n`http_cache_store_memory_table_holder` holds the ETS tables.\n\n`http_cache_store_memory_worker_sup` is a supervisor responsible for spawning workers that\ninsert or update new HTTP responses into the ETS object table, invalidate entries, and process\ncluster work. It is used for backpressure, as there's a maximum number of tasks that can be run at\nthe same time (except for invalidation requests).\n\n`http_cache_store_memory_stats` collects stats about memory, emits telemetry events related to\nit and provides with helper function to calculate how much allocated memory is used, depending on\nthe configuration.\n\n`http_cache_store_memory_cluster_mon` handles communication with the other members of the cluster,\nby:\n- broadcasting and handling invalidation requests\n- listening for new cached objects available from other members, and requesting them if they're\nmissing locally\n- sending warmup request on startup.\n\nThen we have 3 sweeper processes:\n- `http_cache_store_memory_expired_resp_sweeper` removes HTTP responses than can no longer be used,\nthat is those whose grace period have expired\n- `http_cache_store_memory_lru_nuker` starts nuking least recently used HTTP responses as soon\nas 99% of the allocated memory for caching is used. It does it more and more aggressively by first\nnuking 200 objects, then 400, then 800... until this threshold is no longer exceeded. It also blocks\nany new insert of HTTP responses as soon as the threshold exceeds 100%\n- `http_cache_store_memory_outdated_lru_sweeper` is responsible for sweeping LRU entries that no\nlonger have a corresponding HTTP response in the object table. That might happen for 2 reasons:\n  - a newer LRU entry was inserted since, that is the response was returned to a client and\n  `http_cache:notify_response_used/2` was called\n  - the HTTP response was replaced with a freshest one\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftanguilp%2Fhttp_cache_store_memory","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftanguilp%2Fhttp_cache_store_memory","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftanguilp%2Fhttp_cache_store_memory/lists"}