{"id":13778239,"url":"https://github.com/thibaultcha/lua-resty-mlcache","last_synced_at":"2025-04-05T20:06:21.424Z","repository":{"id":20360236,"uuid":"89051243","full_name":"thibaultcha/lua-resty-mlcache","owner":"thibaultcha","description":"Layered caching library for OpenResty","archived":false,"fork":false,"pushed_at":"2024-02-09T20:14:55.000Z","size":442,"stargazers_count":389,"open_issues_count":7,"forks_count":82,"subscribers_count":23,"default_branch":"main","last_synced_at":"2024-04-14T20:01:54.792Z","etag":null,"topics":["lua-resty","luajit","ngx-lua","openresty"],"latest_commit_sha":null,"homepage":"","language":"Perl","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thibaultcha.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-04-22T07:05:33.000Z","updated_at":"2024-08-03T18:11:17.390Z","dependencies_parsed_at":"2024-01-13T11:13:51.105Z","dependency_job_id":"bf370f51-f743-4c65-97a7-5152d29e77e9","html_url":"https://github.com/thibaultcha/lua-resty-mlcache","commit_stats":null,"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thibaultcha%2Flua-resty-mlcache","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thibaultcha%2Flua-resty-mlcache/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thibaultcha%2Flua-resty-mlcache/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thibaultcha%2Flua-resty-mlcache/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thibaultcha","download_url":"https://codeload.github.com/thibaultcha/lua-resty-mlcache/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247393569,"owners_count":20931812,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["lua-resty","luajit","ngx-lua","openresty"],"created_at":"2024-08-03T18:00:52.319Z","updated_at":"2025-04-05T20:06:21.398Z","avatar_url":"https://github.com/thibaultcha.png","language":"Perl","readme":"# lua-resty-mlcache\n\n[![CI](https://github.com/thibaultcha/lua-resty-mlcache/actions/workflows/ci.yml/badge.svg)](https://github.com/thibaultcha/lua-resty-mlcache/actions/workflows/ci.yml)\n\nFast and automated layered caching for OpenResty.\n\nThis library can be manipulated as a key/value store caching scalar Lua types\nand tables, combining the power of the [lua_shared_dict] API and\n[lua-resty-lrucache], which results in an extremely performant and flexible\ncaching solution.\n\nFeatures:\n\n- Caching and negative caching with TTLs.\n- Built-in mutex via [lua-resty-lock] to prevent dog-pile effects to your\n  database/backend on cache misses.\n- Built-in inter-worker communication to propagate cache invalidations\n  and allow workers to update their L1 (lua-resty-lrucache) caches upon changes\n  (`set()`, `delete()`).\n- Support for split hits and misses caching queues.\n- Multiple isolated instances can be created to hold various types of data\n  while relying on the *same* `lua_shared_dict` L2 cache.\n\nIllustration of the various caching levels built into this library:\n\n```\n┌─────────────────────────────────────────────────┐\n│ Nginx                                           │\n│       ┌───────────┐ ┌───────────┐ ┌───────────┐ │\n│       │worker     │ │worker     │ │worker     │ │\n│ L1    │           │ │           │ │           │ │\n│       │ Lua cache │ │ Lua cache │ │ Lua cache │ │\n│       └───────────┘ └───────────┘ └───────────┘ │\n│             │             │             │       │\n│             ▼             ▼             ▼       │\n│       ┌───────────────────────────────────────┐ │\n│       │                                       │ │\n│ L2    │           lua_shared_dict             │ │\n│       │                                       │ │\n│       └───────────────────────────────────────┘ │\n│                           │ mutex               │\n│                           ▼                     │\n│                  ┌──────────────────┐           │\n│                  │     callback     │           │\n│                  └────────┬─────────┘           │\n└───────────────────────────┼─────────────────────┘\n                            │\n  L3                        │   I/O fetch\n                            ▼\n\n                   Database, API, DNS, Disk, any I/O...\n```\n\nThe cache level hierarchy is:\n- **L1**: Least-Recently-Used Lua VM cache using [lua-resty-lrucache].\n   Provides the fastest lookup if populated, and avoids exhausting the workers'\n   Lua VM memory.\n- **L2**: `lua_shared_dict` memory zone shared by all workers. This level\n   is only accessed if L1 was a miss, and prevents workers from requesting the\n   L3 cache.\n- **L3**: a custom function that will only be run by a single worker\n   to avoid the dog-pile effect on your database/backend\n   (via [lua-resty-lock]). Values fetched via L3 will be set to the L2 cache\n   for other workers to retrieve.\n\nThis library has been presented at **OpenResty Con 2018**.  See the\n[Resources](#resources) section for a recording of the talk.\n\n# Table of Contents\n\n- [Synopsis](#synopsis)\n- [Requirements](#requirements)\n- [Installation](#installation)\n- [Methods](#methods)\n    - [new](#new)\n    - [get](#get)\n    - [get_bulk](#get_bulk)\n    - [new_bulk](#new_bulk)\n    - [each_bulk_res](#each_bulk_res)\n    - [peek](#peek)\n    - [set](#set)\n    - [delete](#delete)\n    - [purge](#purge)\n    - [update](#update)\n- [Resources](#resources)\n- [Changelog](#changelog)\n- [License](#license)\n\n# Synopsis\n\n```\n# nginx.conf\n\nhttp {\n    # you do not need to configure the following line when you\n    # use LuaRocks or opm.\n    lua_package_path \"/path/to/lua-resty-mlcache/lib/?.lua;;\";\n\n    # 'on' already is the default for this directive. If 'off', the L1 cache\n    # will be inefective since the Lua VM will be re-created for every\n    # request. This is fine during development, but ensure production is 'on'.\n    lua_code_cache on;\n\n    lua_shared_dict cache_dict 1m;\n\n    init_by_lua_block {\n        local mlcache = require \"resty.mlcache\"\n\n        local cache, err = mlcache.new(\"my_cache\", \"cache_dict\", {\n            lru_size = 500,    -- size of the L1 (Lua VM) cache\n            ttl      = 3600,   -- 1h ttl for hits\n            neg_ttl  = 30,     -- 30s ttl for misses\n        })\n        if err then\n\n        end\n\n        -- we put our instance in the global table for brevity in\n        -- this example, but prefer an upvalue to one of your modules\n        -- as recommended by ngx_lua\n        _G.cache = cache\n    }\n\n    server {\n        listen 8080;\n\n        location / {\n            content_by_lua_block {\n                local function callback(username)\n                    -- this only runs *once* until the key expires, so\n                    -- do expensive operations like connecting to a remote\n                    -- backend here. i.e: call a MySQL server in this callback\n                    return db:get_user(username) -- { name = \"John Doe\", email = \"john@example.com\" }\n                end\n\n                -- this call will try L1 and L2 before running the callback (L3)\n                -- the returned value will then be stored in L2 and L1\n                -- for the next request.\n                local user, err = cache:get(\"my_key\", nil, callback, \"jdoe\")\n\n                ngx.say(user.name) -- \"John Doe\"\n            }\n        }\n    }\n}\n```\n\n[Back to TOC](#table-of-contents)\n\n# Requirements\n\n* OpenResty \u003e= `1.11.2.2`\n    * ngx_lua\n    * lua-resty-lrucache\n    * lua-resty-lock\n\nTests matrix results:\n\n| OpenResty   | Compatibility\n|------------:|:--------------------|\n| \u003c           | not tested\n| `1.11.2.x`  | :heavy_check_mark:\n| `1.13.6.x`  | :heavy_check_mark:\n| `1.15.8.x`  | :heavy_check_mark:\n| `1.17.8.x`  | :heavy_check_mark:\n| `1.19.3.x`  | :heavy_check_mark:\n| `1.19.9.x`  | :heavy_check_mark:\n| `1.21.4.x`  | :heavy_check_mark:\n| `1.25.3.x`  | :heavy_check_mark:\n| \u003e           | not tested\n\n[Back to TOC](#table-of-contents)\n\n# Installation\n\nWith [LuaRocks](https://luarocks.org/):\n\n```\n$ luarocks install lua-resty-mlcache\n```\n\nOr via [opm](https://github.com/openresty/opm):\n\n```\n$ opm get thibaultcha/lua-resty-mlcache\n```\n\nOr manually:\n\nOnce you have a local copy of this module's `lib/` directory, add it to your\n`LUA_PATH` (or `lua_package_path` directive for OpenResty):\n\n```\n/path/to/lib/?.lua;\n```\n\n[Back to TOC](#table-of-contents)\n\n# Methods\n\nnew\n---\n**syntax:** `cache, err = mlcache.new(name, shm, opts?)`\n\nCreate a new mlcache instance. If failed, returns `nil` and a string\ndescribing the error.\n\nThe first argument `name` is an arbitrary name of your choosing for this cache,\nand must be a string. Each mlcache instance namespaces the values it holds\naccording to its name, so several instances with the same name will\nshare the same data.\n\nThe second argument `shm` is the name of the `lua_shared_dict` shared memory\nzone. Several instances of mlcache can use the same shm (values will be\nnamespaced).\n\nThe third argument `opts` is optional. If provided, it must be a table\nholding the desired options for this instance. The possible options are:\n\n- `lru_size`: a number defining the size of the underlying L1 cache\n  (lua-resty-lrucache instance). This size is the maximal number of items\n  that the L1 cache can hold.\n  **Default:** `100`.\n- `ttl`: a number specifying the expiration time period of the cached\n  values. The unit is seconds, but accepts fractional number parts, like\n  `0.3`. A `ttl` of `0` means the cached values will never expire.\n  **Default:** `30`.\n- `neg_ttl`: a number specifying the expiration time period of the cached\n  misses (when the L3 callback returns `nil`). The unit is seconds, but\n  accepts fractional number parts, like `0.3`. A `neg_ttl` of `0` means the\n  cached misses will never expire.\n  **Default:** `5`.\n- `resurrect_ttl`: _optional_ number. When specified, the mlcache instance will\n  attempt to resurrect stale values when the L3 callback returns `nil, err`\n  (soft errors). More details are available for this option in the\n  [get()](#get) section. The unit is seconds, but accepts fractional number\n  parts, like `0.3`.\n- `lru`: _optional_. A lua-resty-lrucache instance of your choosing. If\n  specified, mlcache will not instantiate an LRU. One can use this value to use\n  the `resty.lrucache.pureffi` implementation of lua-resty-lrucache if desired.\n- `shm_set_tries`: the number of tries for the lua_shared_dict `set()`\n  operation. When the `lua_shared_dict` is full, it attempts to free up to 30\n  items from its queue. When the value being set is much larger than the freed\n  space, this option allows mlcache to retry the operation (and free more slots)\n  until the maximum number of tries is reached or enough memory was freed for\n  the value to fit.\n  **Default**: `3`.\n- `shm_miss`: _optional_ string. The name of a `lua_shared_dict`. When\n  specified, misses (callbacks returning `nil`) will be cached in this separate\n  `lua_shared_dict`. This is useful to ensure that a large number of cache\n  misses (e.g. triggered by malicious clients) does not evict too many cached\n  items (hits) from the `lua_shared_dict` specified in `shm`.\n- `shm_locks`: _optional_ string. The name of a `lua_shared_dict`. When\n  specified, lua-resty-lock will use this shared dict to store its locks. This\n  option can help reducing cache churning: when the L2 cache (shm) is full,\n  every insertion (such as locks created by concurrent accesses triggering L3\n  callbacks) purges the oldest 30 accessed items. These purged items are most\n  likely to be previously (and valuable) cached values. By isolating locks in a\n  separate shared dict, workloads experiencing cache churning can mitigate this\n  effect.\n- `resty_lock_opts`: _optional_ table. Options for [lua-resty-lock] instances.\n  When mlcache runs the L3 callback, it uses lua-resty-lock to ensure that a\n  single worker runs the provided callback.\n- `ipc_shm`: _optional_ string. If you wish to use [set()](#set),\n  [delete()](#delete), or [purge()](#purge), you must provide an IPC\n  (Inter-Process Communication) mechanism for workers to synchronize and\n  invalidate their L1 caches. This module bundles an \"off-the-shelf\" IPC\n  library, and you can enable it by specifying a dedicated `lua_shared_dict` in\n  this option. Several mlcache instances can use the same shared dict (events\n  will be namespaced), but no other actor than mlcache should tamper with it.\n- `ipc`: _optional_ table. Like the above `ipc_shm` option, but lets you use\n  the IPC library of your choice to propagate inter-worker events.\n- `l1_serializer`: _optional_ function. Its signature and accepted values are\n  documented under the [get()](#get) method, along with an example. If\n  specified, this function will be called each time a value is promoted from the\n  L2 cache into the L1 (worker Lua VM). This function can perform arbitrary\n  serialization of the cached item to transform it into any Lua object _before_\n  storing it into the L1 cache. It can thus avoid your application from\n  having to repeat such transformations on every request, such as creating\n  tables, cdata objects, loading new Lua code, etc...\n\nExample:\n\n```lua\nlocal mlcache = require \"resty.mlcache\"\n\nlocal cache, err = mlcache.new(\"my_cache\", \"cache_shared_dict\", {\n    lru_size = 1000, -- hold up to 1000 items in the L1 cache (Lua VM)\n    ttl      = 3600, -- caches scalar types and tables for 1h\n    neg_ttl  = 60    -- caches nil values for 60s\n})\nif not cache then\n    error(\"could not create mlcache: \" .. err)\nend\n```\n\nYou can create several mlcache instances relying on the same underlying\n`lua_shared_dict` shared memory zone:\n\n```lua\nlocal mlcache = require \"mlcache\"\n\nlocal cache_1 = mlcache.new(\"cache_1\", \"cache_shared_dict\", { lru_size = 100 })\nlocal cache_2 = mlcache.new(\"cache_2\", \"cache_shared_dict\", { lru_size = 1e5 })\n```\n\nIn the above example, `cache_1` is ideal for holding a few, very large values.\n`cache_2` can be used to hold a large number of small values. Both instances\nwill rely on the same shm: `lua_shared_dict cache_shared_dict 2048m;`. Even if\nyou use identical keys in both caches, they will not conflict with each other\nsince they each have a different namespace.\n\nThis other example instantiates an mlcache using the bundled IPC module for\ninter-worker invalidation events (so we can use [set()](#set),\n[delete()](#delete), and [purge()](#purge)):\n\n```lua\nlocal mlcache = require \"resty.mlcache\"\n\nlocal cache, err = mlcache.new(\"my_cache_with_ipc\", \"cache_shared_dict\", {\n    lru_size = 1000,\n    ipc_shm = \"ipc_shared_dict\"\n})\n```\n\n**Note:** for the L1 cache to be effective, ensure that\n[lua_code_cache](https://github.com/openresty/lua-nginx-module#lua_code_cache)\nis enabled (which is the default). If you turn off this directive during\ndevelopment, mlcache will work, but L1 caching will be ineffective since a new\nLua VM will be created for every request.\n\n[Back to TOC](#table-of-contents)\n\nget\n---\n**syntax:** `value, err, hit_level = cache:get(key, opts?, callback?, ...)`\n\nPerform a cache lookup. This is the primary and most efficient method of this\nmodule. A typical pattern is to *not* call [set()](#set), and let [get()](#get)\nperform all the work.\n\nWhen this method succeeds, it returns `value` and `err` is set to `nil`.\n**Because `nil` values from the L3 callback can be cached (i.e. \"negative\ncaching\"), `value` can be `nil` albeit already cached. Hence, one must note to\ncheck the second return value `err` to determine if this method succeeded or\nnot**.\n\nThe third return value is a number which is set if no error was encountered.\nIt indicates the level at which the value was fetched: `1` for L1, `2` for L2,\nand `3` for L3.\n\nIf, however, an error is encountered, then this method returns `nil` in `value`\nand a string describing the error in `err`.\n\nThe first argument `key` is a string. Each value must be stored under a unique\nkey.\n\nThe second argument `opts` is optional. If provided, it must be a table holding\nthe desired options for this key. These options will supersede the instance's\noptions:\n\n- `ttl`: a number specifying the expiration time period of the cached\n  values. The unit is seconds, but accepts fractional number parts, like\n  `0.3`. A `ttl` of `0` means the cached values will never expire.\n  **Default:** inherited from the instance.\n- `neg_ttl`: a number specifying the expiration time period of the cached\n  misses (when the L3 callback returns `nil`). The unit is seconds, but\n  accepts fractional number parts, like `0.3`. A `neg_ttl` of `0` means the\n  cached misses will never expire.\n  **Default:** inherited from the instance.\n- `resurrect_ttl`: _optional_ number. When specified, `get()` will attempt to\n  resurrect stale values when errors are encountered. Errors returned by the L3\n  callback (`nil, err`) are considered to be failures to fetch/refresh a value.\n  When such return values from the callback are seen by `get()`, and if the\n  stale value is still in memory, then `get()` will resurrect the stale value\n  for `resurrect_ttl` seconds. The error returned by `get()` will be logged at\n  the WARN level, but _not_ returned to the caller. Finally, the `hit_level`\n  return value will be `4` to signify that the served item is stale. When\n  `resurrect_ttl` is reached, `get()` will once again attempt to run the\n  callback. If by then, the callback returns an error again, the value is\n  resurrected once again, and so on. If the callback succeeds, the value is\n  refreshed and not marked as stale anymore. Due to current limitations within\n  the LRU cache module, `hit_level` will be `1` when stale values are promoted\n  to the L1 cache and retrieved from there. Lua errors thrown by the\n  callback _do not_ trigger a resurrect, and are returned by `get()` as usual\n  (`nil, err`). When several workers time out while waiting for the worker\n  running the callback (e.g. because the datastore is timing out), then users\n  of this option will see a slight difference compared to the traditional\n  behavior of `get()`. Instead of returning `nil, err` (indicating a lock\n  timeout), `get()` will return the stale value (if available), no error, and\n  `hit_level` will be `4`. However, the value will not be resurrected (since\n  another worker is still running the callback). The unit for this option is\n  seconds, but it accepts fractional number parts, like `0.3`. This option\n  **must** be greater than `0`, to prevent stale values from being cached\n  indefinitely.\n  **Default:** inherited from the instance.\n- `shm_set_tries`: the number of tries for the lua_shared_dict `set()`\n  operation. When the `lua_shared_dict` is full, it attempts to free up to 30\n  items from its queue. When the value being set is much larger than the freed\n  space, this option allows mlcache to retry the operation (and free more slots)\n  until the maximum number of tries is reached or enough memory was freed for\n  the value to fit.\n  **Default:** inherited from the instance.\n- `l1_serializer`: _optional_ function. Its signature and accepted values are\n  documented under the [get()](#get) method, along with an example. If\n  specified, this function will be called each time a value is promoted from the\n  L2 cache into the L1 (worker Lua VM). This function can perform arbitrary\n  serialization of the cached item to transform it into any Lua object _before_\n  storing it into the L1 cache. It can thus avoid your application from\n  having to repeat such transformations on every request, such as creating\n  tables, cdata objects, loading new Lua code, etc...\n  **Default:** inherited from the instance.\n- `resty_lock_opts`: _optional_ table. If specified, override the instance\n  `resty_lock_opts` for the current `get()` lookup.\n  **Default:** inherited from the instance.\n\nThe third argument `callback` is optional. If provided, it must be a function\nwhose signature and return values are documented in the following example:\n\n```lua\n-- arg1, arg2, and arg3 are arguments forwarded to the callback from the\n-- `get()` variadic arguments, like so:\n-- cache:get(key, opts, callback, arg1, arg2, arg3)\n\nlocal function callback(arg1, arg2, arg3)\n    -- I/O lookup logic\n    -- ...\n\n    -- value: the value to cache (Lua scalar or table)\n    -- err: if not `nil`, will abort get(), which will return `value` and `err`\n    -- ttl: override ttl for this value\n    --      If returned as `ttl \u003e= 0`, it will override the instance\n    --      (or option) `ttl` or `neg_ttl`.\n    --      If returned as `ttl \u003c 0`, `value` will be returned by get(),\n    --      but not cached. This return value will be ignored if not a number.\n    return value, err, ttl\nend\n```\n\nThe provided `callback` function is allowed to throw Lua errors as it runs in\nprotected mode. Such errors thrown from the callback will be returned as strings\nin the second return value `err`.\n\nIf `callback` is not provided, `get()` will still lookup the requested key in\nthe L1 and L2 caches and return it if found. In the case when no value is found\nin the cache **and** no callback is provided, `get()` will return `nil, nil,\n-1`, where -1 signifies a **cache miss** (no value). This is not to be confused\nwith return values such as `nil, nil, 1`, where 1 signifies a **negative cached\nitem** found in L1 (cached `nil`).\n\nNot providing a `callback` function allows implementing cache lookup patterns\nthat are guaranteed to be on-cpu for a more constant, smoother latency tail end\n(e.g. with values refreshed in background timers via `set()`).\n\n```lua\nlocal value, err, hit_lvl = cache:get(\"key\")\nif value == nil then\n    if err ~= nil then\n        -- error\n    elseif hit_lvl == -1 then\n        -- miss (no value)\n    else\n        -- negative hit (cached `nil` value)\n    end\nend\n```\n\nWhen provided a callback, `get()` follows the below logic:\n\n1. query the L1 cache (lua-resty-lrucache instance). This cache lives in the\n   Lua VM, and as such, it is the most efficient one to query.\n    1. if the L1 cache has the value, return it.\n    2. if the L1 cache does not have the value (L1 miss), continue.\n2. query the L2 cache (`lua_shared_dict` memory zone). This cache is\n   shared by all workers, and is almost as efficient as the L1 cache. It\n   however requires serialization of stored Lua tables.\n    1. if the L2 cache has the value, return it.\n        1. if `l1_serializer` is set, run it, and promote the resulting value\n           in the L1 cache.\n        2. if not, directly promote the value as-is in the L1 cache.\n    2. if the L2 cache does not have the value (L2 miss), continue.\n3. create a [lua-resty-lock], and ensures that a single worker will run the\n   callback (other workers trying to access the same value will wait).\n4. a single worker runs the L3 callback (e.g. performs a database query)\n   1. the callback succeeds and returns a value: the value is set in the\n      L2 cache, and then in the L1 cache (as-is by default, or as returned by\n      `l1_serializer` if specified).\n   2. the callback failed and returned `nil, err`:\n      a. if `resurrect_ttl` is specified, and if the stale value is still\n         available, resurrect it in the L2 cache and promote it to the L1.\n      b. otherwise, `get()` returns `nil, err`.\n5. other workers that were trying to access the same value but were waiting\n   are unlocked and read the value from the L2 cache (they do not run the L3\n   callback) and return it.\n\nWhen not provided a callback, `get()` will only execute steps 1. and 2.\n\nHere is a complete example usage:\n\n```lua\nlocal mlcache = require \"mlcache\"\n\nlocal cache, err = mlcache.new(\"my_cache\", \"cache_shared_dict\", {\n    lru_size = 1000,\n    ttl      = 3600,\n    neg_ttl  = 60\n})\n\nlocal function fetch_user(user_id)\n    local user, err = db:query_user(user_id)\n    if err then\n        -- in this case, get() will return `nil` + `err`\n        return nil, err\n    end\n\n    return user -- table or nil\nend\n\nlocal user_id = 3\n\nlocal user, err = cache:get(\"users:\" .. user_id, nil, fetch_user, user_id)\nif err then\n    ngx.log(ngx.ERR, \"could not retrieve user: \", err)\n    return\nend\n\n-- `user` could be a table, but could also be `nil` (does not exist)\n-- regardless, it will be cached and subsequent calls to get() will\n-- return the cached value, for up to `ttl` or `neg_ttl`.\nif user then\n    ngx.say(\"user exists: \", user.name)\nelse\n    ngx.say(\"user does not exists\")\nend\n```\n\nThis second example is similar to the one above, but here we apply some\ntransformation to the retrieved `user` record before caching it via the\n`l1_serializer` callback:\n\n```lua\n-- Our l1_serializer, called when a value is promoted from L2 to L1\n--\n-- Its signature receives a single argument: the item as returned from\n-- an L2 hit. Therefore, this argument can never be `nil`. The result will be\n-- kept in the L1 cache, but it cannot be `nil`.\n--\n-- This function can return `nil` and a string describing an error, which\n-- will bubble up to the caller of `get()`. It also runs in protected mode\n-- and will report any Lua error.\nlocal function load_code(user_row)\n    if user_row.custom_code ~= nil then\n        local f, err = loadstring(user_row.raw_lua_code)\n        if not f then\n            -- in this case, nothing will be stored in the cache (as if the L3\n            -- callback failed)\n            return nil, \"failed to compile custom code: \" .. err\n        end\n\n        user_row.f = f\n    end\n\n    return user_row\nend\n\nlocal user, err = cache:get(\"users:\" .. user_id,\n                            { l1_serializer = load_code },\n                            fetch_user, user_id)\nif err then\n     ngx.log(ngx.ERR, \"could not retrieve user: \", err)\n     return\nend\n\n-- now we can call a function that was already loaded once, upon entering\n-- the L1 cache (Lua VM)\nuser.f()\n```\n\n[Back to TOC](#table-of-contents)\n\nget_bulk\n--------\n**syntax**: `res, err = cache:get_bulk(bulk, opts?)`\n\nPerforms several [get()](#get) lookups at once (in bulk). Any of these lookups\nrequiring an L3 callback call will be executed concurrently, in a pool of\n[ngx.thread](https://github.com/openresty/lua-nginx-module#ngxthreadspawn).\n\nThe first argument `bulk` is a table containing `n` operations.\n\nThe second argument `opts` is optional. If provided, it must be a table holding\nthe options for this bulk lookup. The possible options are:\n\n- `concurrency`: a number greater than `0`. Specifies the number of threads\n  that will concurrently execute the L3 callbacks for this bulk lookup. A\n  concurrency of `3` with 6 callbacks to run means than each thread will\n  execute 2 callbacks. A concurrency of `1` with 6 callbacks means than a\n  single thread will execute all 6 callbacks. With a concurrency of `6` and 1\n  callback, a single thread will run the callback.\n  **Default**: `3`.\n\nUpon success, this method returns `res`, a table containing the results of\neach lookup, and no error.\n\nUpon failure, this method returns `nil` plus a string describing the error.\n\nAll lookup operations performed by this method will fully integrate into other\noperations being concurrently performed by other methods and Nginx workers\n(e.g. L1/L2 hits/misses storage, L3 callback mutex, etc...).\n\n\nThe `bulk` argument is a table that must have a particular layout (documented\nin the below example). It can be built manually, or via the\n[new_bulk()](#new_bulk) helper method.\n\nSimilarly, the `res` table also has a particular layout of its own. It can be\niterated upon manually, or via the [each_bulk_res](#each_bulk_res) iterator\nhelper.\n\nExample:\n\n```lua\nlocal mlcache = require \"mlcache\"\n\nlocal cache, err = mlcache.new(\"my_cache\", \"cache_shared_dict\")\n\ncache:get(\"key_c\", nil, function() return nil end)\n\nlocal res, err = cache:get_bulk({\n  -- bulk layout:\n  -- key     opts          L3 callback                    callback argument\n\n    \"key_a\", { ttl = 60 }, function() return \"hello\" end, nil,\n    \"key_b\", nil,          function() return \"world\" end, nil,\n    \"key_c\", nil,          function() return \"bye\" end,   nil,\n    n = 3 -- specify the number of operations\n}, { concurrency = 3 })\nif err then\n     ngx.log(ngx.ERR, \"could not execute bulk lookup: \", err)\n     return\nend\n\n-- res layout:\n-- data, \"err\", hit_lvl }\n\nfor i = 1, res.n, 3 do\n    local data = res[i]\n    local err = res[i + 1]\n    local hit_lvl = res[i + 2]\n\n    if not err then\n        ngx.say(\"data: \", data, \", hit_lvl: \", hit_lvl)\n    end\nend\n```\n\nThe above example would produce the following output:\n\n```\ndata: hello, hit_lvl: 3\ndata: world, hit_lvl: 3\ndata: nil, hit_lvl: 1\n```\n\nNote that since `key_c` was already in the cache, the callback returning\n`\"bye\"` was never run, since `get_bulk()` retrieved the value from L1, as\nindicated by the `hit_lvl` value.\n\n**Note:** unlike [get()](#get), this method only allows specifying a single\nargument to each lookup's callback.\n\n[Back to TOC](#table-of-contents)\n\nnew_bulk\n--------\n**syntax**: `bulk = mlcache.new_bulk(n_lookups?)`\n\nCreates a table holding lookup operations for the [get_bulk()](#get_bulk)\nfunction. It is not required to use this function to construct a bulk lookup\ntable, but it provides a nice abstraction.\n\nThe first and only argument `n_lookups` is optional, and if specified, is a\nnumber hinting the amount of lookups this bulk will eventually contain so that\nthe underlying table is pre-allocated for optimization purposes.\n\nThis function returns a table `bulk`, which contains no lookup operations yet.\nLookups are added to a `bulk` table by invoking `bulk:add(key, opts?, cb,\narg?)`:\n\n```lua\nlocal mlcache = require \"mlcache\"\n\nlocal cache, err = mlcache.new(\"my_cache\", \"cache_shared_dict\")\n\nlocal bulk = mlcache.new_bulk(3)\n\nbulk:add(\"key_a\", { ttl = 60 }, function(n) return n * n, 42)\nbulk:add(\"key_b\", nil, function(str) return str end, \"hello\")\nbulk:add(\"key_c\", nil, function() return nil end)\n\nlocal res, err = cache:get_bulk(bulk)\n```\n\n[Back to TOC](#table-of-contents)\n\neach_bulk_res\n-------------\n**syntax**: `iter, res, i = mlcache.each_bulk_res(res)`\n\nProvides an abstraction to iterate over a [get_bulk()](#get_bulk) `res` return\ntable. It is not required to use this method to iterate over a `res` table, but\nit provides a nice abstraction.\n\nThis method can be invoked as a Lua iterator:\n\n```lua\nlocal mlcache = require \"mlcache\"\n\nlocal cache, err = mlcache.new(\"my_cache\", \"cache_shared_dict\")\n\nlocal res, err = cache:get_bulk(bulk)\n\nfor i, data, err, hit_lvl in mlcache.each_bulk_res(res) do\n    if not err then\n        ngx.say(\"lookup \", i, \": \", data)\n    end\nend\n```\n\n[Back to TOC](#table-of-contents)\n\npeek\n----\n**syntax:** `ttl, err, value = cache:peek(key, stale?)`\n\nPeek into the L2 (`lua_shared_dict`) cache.\n\nThe first argument `key` is a string which is the key to lookup in the cache.\n\nThe second argument `stale` is optional. If `true`, then `peek()` will consider\nstale values as cached values. If not provided, `peek()` will consider stale\nvalues, as if they were not in the cache\n\nThis method returns `nil` and a string describing the error upon failure.\n\nIf there is no value for the queried `key`, it returns `nil` and no error.\n\nIf there is a value for the queried `key`, it returns a number indicating the\nremaining TTL of the cached value (in seconds) and no error. If the value for\n`key` has expired but is still in the L2 cache, returned TTL value will be\nnegative. The remaining TTL return value will only be `0` if the queried `key`\nhas an indefinite ttl (`ttl=0`). Otherwise, this return value may be positive\n(`key` still valid), or negative (`key` is stale).\n\nThe third returned value will be the cached value as stored in the L2 cache, if\nstill available.\n\nThis method is useful when you want to determine if a value is cached. A value\nstored in the L2 cache is considered cached regardless of whether or not it is\nalso set in the L1 cache of the worker. That is because the L1 cache is\nconsidered volatile (as its size unit is a number of slots), and the L2 cache is\nstill several orders of magnitude faster than the L3 callback anyway.\n\nAs its only intent is to take a \"peek\" into the cache to determine its warmth\nfor a given value, `peek()` does not count as a query like [get()](#get), and\ndoes not promote the value to the L1 cache.\n\nExample:\n\n```lua\nlocal mlcache = require \"mlcache\"\n\nlocal cache = mlcache.new(\"my_cache\", \"cache_shared_dict\")\n\nlocal ttl, err, value = cache:peek(\"key\")\nif err then\n    ngx.log(ngx.ERR, \"could not peek cache: \", err)\n    return\nend\n\nngx.say(ttl)   -- nil because `key` has no value yet\nngx.say(value) -- nil\n\n-- cache the value\n\ncache:get(\"key\", { ttl = 5 }, function() return \"some value\" end)\n\n-- wait 2 seconds\n\nngx.sleep(2)\n\nlocal ttl, err, value = cache:peek(\"key\")\nif err then\n    ngx.log(ngx.ERR, \"could not peek cache: \", err)\n    return\nend\n\nngx.say(ttl)   -- 3\nngx.say(value) -- \"some value\"\n```\n\n**Note:** since mlcache `2.5.0`, it is also possible to call [get()](#get)\nwithout a callback function in order to \"query\" the cache. Unlike `peek()`, a\n`get()` call with no callback *will* promote the value to the L1 cache, and\n*will not* return its TTL.\n\n[Back to TOC](#table-of-contents)\n\nset\n---\n**syntax:** `ok, err = cache:set(key, opts?, value)`\n\nUnconditionally set a value in the L2 cache and broadcasts an event to other\nworkers so they can refresh the value from their L1 cache.\n\nThe first argument `key` is a string, and is the key under which to store the\nvalue.\n\nThe second argument `opts` is optional, and if provided, is identical to the\none of [get()](#get).\n\nThe third argument `value` is the value to cache, similar to the return value\nof the L3 callback. Just like the callback's return value, it must be a Lua\nscalar, a table, or `nil`. If a `l1_serializer` is provided either from the\nconstructor or in the `opts` argument, it will be called with `value` if\n`value` is not `nil`.\n\nOn success, the first return value will be `true`.\n\nOn failure, this method returns `nil` and a string describing the error.\n\n**Note:** by its nature, `set()` requires that other instances of mlcache (from\nother workers) refresh their L1 cache. If `set()` is called from a single\nworker, other workers' mlcache instances bearing the same `name` must call\n[update()](#update) before their cache be requested during the next request, to\nmake sure they refreshed their L1 cache.\n\n**Note bis:** It is generally considered inefficient to call `set()` on a hot\ncode path (such as in a request being served by OpenResty). Instead, one should\nrely on [get()](#get) and its built-in mutex in the L3 callback. `set()` is\nbetter suited when called occasionally from a single worker, for example upon a\nparticular event that triggers a cached value to be updated. Once `set()`\nupdates the L2 cache with the fresh value, other workers will rely on\n[update()](#update) to poll the invalidation event and invalidate their L1\ncache, which will make them fetch the (fresh) value in L2.\n\n**See:** [update()](#update)\n\n[Back to TOC](#table-of-contents)\n\ndelete\n------\n**syntax:** `ok, err = cache:delete(key)`\n\nDelete a value in the L2 cache and publish an event to other workers so they\ncan evict the value from their L1 cache.\n\nThe first and only argument `key` is the string at which the value is stored.\n\nOn success, the first return value will be `true`.\n\nOn failure, this method returns `nil` and a string describing the error.\n\n**Note:** by its nature, `delete()` requires that other instances of mlcache\n(from other workers) refresh their L1 cache. If `delete()` is called from a\nsingle worker, other workers' mlcache instances bearing the same `name` must\ncall [update()](#update) before their cache be requested during the next\nrequest, to make sure they refreshed their L1 cache.\n\n**See:** [update()](#update)\n\n[Back to TOC](#table-of-contents)\n\npurge\n-----\n**syntax:** `ok, err = cache:purge(flush_expired?)`\n\nPurge the content of the cache, in both the L1 and L2 levels. Then publishes\nan event to other workers so they can purge their L1 cache as well.\n\nThis method recycles the lua-resty-lrucache instance, and calls\n[ngx.shared.DICT:flush_all](https://github.com/openresty/lua-nginx-module#ngxshareddictflush_all)\n, so it can be rather expensive.\n\nThe first and only argument `flush_expired` is optional, but if given `true`,\nthis method will also call\n[ngx.shared.DICT:flush_expired](https://github.com/openresty/lua-nginx-module#ngxshareddictflush_expired)\n(with no arguments). This is useful to release memory claimed by the L2 (shm)\ncache if needed.\n\nOn success, the first return value will be `true`.\n\nOn failure, this method returns `nil` and a string describing the error.\n\n**Note:** it is not possible to call `purge()` when using a custom LRU cache in\nOpenResty 1.13.6.1 and below. This limitation does not apply for OpenResty\n1.13.6.2 and above.\n\n**Note:** by its nature, `purge()` requires that other instances of mlcache\n(from other workers) refresh their L1 cache. If `purge()` is called from a\nsingle worker, other workers' mlcache instances bearing the same `name` must\ncall [update()](#update) before their cache be requested during the next\nrequest, to make sure they refreshed their L1 cache.\n\n**See:** [update()](#update)\n\n[Back to TOC](#table-of-contents)\n\nupdate\n------\n**syntax:** `ok, err = cache:update(timeout?)`\n\nPoll and execute pending cache invalidation events published by other workers.\n\nThe [set()](#set), [delete()](#delete), and [purge()](#purge) methods require\nthat other instances of mlcache (from other workers) refresh their L1 cache.\nSince OpenResty currently has no built-in mechanism for inter-worker\ncommunication, this module bundles an \"off-the-shelf\" IPC library to propagate\ninter-worker events. If the bundled IPC library is used, the `lua_shared_dict`\nspecified in the `ipc_shm` option **must not** be used by other actors than\nmlcache itself.\n\nThis method allows a worker to update its L1 cache (by purging values\nconsidered stale due to an other worker calling `set()`, `delete()`, or\n`purge()`) before processing a request.\n\nThis method accepts a `timeout` argument whose unit is seconds and which\ndefaults to `0.3` (300ms). The update operation will timeout if it isn't done\nwhen this threshold in reached. This avoids `update()` from staying on the CPU\ntoo long in case there are too many events to process. In an eventually\nconsistent system, additional events can wait for the next call to be processed.\n\nA typical design pattern is to call `update()` **only once** before each\nrequest processing. This allows your hot code paths to perform a single shm\naccess in the best case scenario: no invalidation events were received, all\n`get()` calls will hit in the L1 cache. Only on a worst case scenario (`n`\nvalues were evicted by another worker) will `get()` access the L2 or L3 cache\n`n` times. Subsequent requests will then hit the best case scenario again,\nbecause `get()` populated the L1 cache.\n\nFor example, if your workers make use of [set()](#set), [delete()](#delete), or\n[purge()](#purge) anywhere in your application, call `update()` at the entrance\nof your hot code path, before using `get()`:\n\n```\nhttp {\n    listen 9000;\n\n    location / {\n        content_by_lua_block {\n            local cache = ... -- retrieve mlcache instance\n\n            -- make sure L1 cache is evicted of stale values\n            -- before calling get()\n            local ok, err = cache:update()\n            if not ok then\n                ngx.log(ngx.ERR, \"failed to poll eviction events: \", err)\n                -- /!\\ we might get stale data from get()\n            end\n\n            -- L1/L2/L3 lookup (best case: L1)\n            local value, err = cache:get(\"key_1\", nil, cb1)\n\n            -- L1/L2/L3 lookup (best case: L1)\n            local other_value, err = cache:get(key_2\", nil, cb2)\n\n            -- value and other_value are up-to-date because:\n            -- either they were not stale and directly came from L1 (best case scenario)\n            -- either they were stale and evicted from L1, and came from L2\n            -- either they were not in L1 nor L2, and came from L3 (worst case scenario)\n        }\n    }\n\n    location /delete {\n        content_by_lua_block {\n            local cache = ... -- retrieve mlcache instance\n\n            -- delete some value\n            local ok, err = cache:delete(\"key_1\")\n            if not ok then\n                ngx.log(ngx.ERR, \"failed to delete value from cache: \", err)\n                return ngx.exit(500)\n            end\n\n            ngx.exit(204)\n        }\n    }\n\n    location /set {\n        content_by_lua_block {\n            local cache = ... -- retrieve mlcache instance\n\n            -- update some value\n            local ok, err = cache:set(\"key_1\", nil, 123)\n            if not ok then\n                ngx.log(ngx.ERR, \"failed to set value in cache: \", err)\n                return ngx.exit(500)\n            end\n\n            ngx.exit(200)\n        }\n    }\n}\n```\n\n**Note:** you **do not** need to call `update()` to refresh your workers if\nthey never call `set()`,  `delete()`, or `purge()`. When workers only rely on\n`get()`, values expire naturally from the L1/L2 caches according to their TTL.\n\n**Note bis:** this library was built with the intent to use a better solution\nfor inter-worker communication as soon as one emerges. In future versions of\nthis library, if an IPC library can avoid the polling approach, so will this\nlibrary. `update()` is only a necessary evil due to today's Nginx/OpenResty\n\"limitations\". You can however use your own IPC library by use of the\n`opts.ipc` option when creating your mlcache instance.\n\n[Back to TOC](#table-of-contents)\n\n# Resources\n\nIn November 2018, this library was presented at OpenResty Con in Hangzhou,\nChina.\n\nThe slides and a recording of the talk (about 40 min long) can be viewed\n[here][talk].\n\n[Back to TOC](#table-of-contents)\n\n# Changelog\n\nSee [CHANGELOG.md](CHANGELOG.md).\n\n[Back to TOC](#table-of-contents)\n\n# License\n\nWork licensed under the MIT License.\n\n[Back to TOC](#table-of-contents)\n\n[lua-resty-lock]: https://github.com/openresty/lua-resty-lock\n[lua-resty-lrucache]: https://github.com/openresty/lua-resty-lrucache\n[lua_shared_dict]: https://github.com/openresty/lua-nginx-module#lua_shared_dict\n[talk]: https://www.slideshare.net/ThibaultCharbonnier/layered-caching-in-openresty-openresty-con-2018\n","funding_links":[],"categories":["Libraries","Perl"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthibaultcha%2Flua-resty-mlcache","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthibaultcha%2Flua-resty-mlcache","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthibaultcha%2Flua-resty-mlcache/lists"}