{"id":22230848,"url":"https://github.com/bigeasy/magazine","last_synced_at":"2025-07-27T20:31:44.458Z","repository":{"id":9833781,"uuid":"11822388","full_name":"bigeasy/magazine","owner":"bigeasy","description":"A LRU cache for memory paging and content caching.","archived":false,"fork":false,"pushed_at":"2022-02-07T06:50:16.000Z","size":516,"stargazers_count":4,"open_issues_count":4,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-11-14T10:41:28.711Z","etag":null,"topics":["cache","eviction","javascript","lru-cache","memory"],"latest_commit_sha":null,"homepage":"https://bigeasy.github.io/magazine/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bigeasy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-08-01T16:54:40.000Z","updated_at":"2023-09-08T16:41:08.000Z","dependencies_parsed_at":"2022-09-06T01:24:23.855Z","dependency_job_id":null,"html_url":"https://github.com/bigeasy/magazine","commit_stats":null,"previous_names":[],"tags_count":45,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bigeasy%2Fmagazine","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bigeasy%2Fmagazine/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bigeasy%2Fmagazine/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bigeasy%2Fmagazine/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bigeasy","download_url":"https://codeload.github.com/bigeasy/magazine/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":227831082,"owners_count":17826155,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cache","eviction","javascript","lru-cache","memory"],"created_at":"2024-12-03T01:16:22.306Z","updated_at":"2024-12-03T01:16:28.843Z","avatar_url":"https://github.com/bigeasy.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Actions Status](https://github.com/bigeasy/magazine/workflows/Node%20CI/badge.svg)](https://github.com/bigeasy/magazine/actions)\n[![codecov](https://codecov.io/gh/bigeasy/magazine/branch/master/graph/badge.svg)](https://codecov.io/gh/bigeasy/magazine)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\nA LRU cache for memory paging and content caching.\n\n| What          | Where                                         |\n| --- | --- |\n| Discussion    | https://github.com/bigeasy/magazine/issues/1  |\n| Documentation | https://bigeasy.github.io/magazine            |\n| Source        | https://github.com/bigeasy/magazine           |\n| Issues        | https://github.com/bigeasy/magazine/issues    |\n| CI            | https://travis-ci.org/bigeasy/magazine        |\n| Coverage:     | https://codecov.io/gh/bigeasy/magazine        |\n| License:      | MIT                                           |\n\n\n```\nnpm install magazine\n```\n\n# Magazine\n\nMagazine is a least-recently used cache.\n\nMagazine is designed for use in database implementations where the cached data\nis authorative, representing the latest version of a database page and not\nfleeting, cursory nor readily disposable. This is why Magazine exists and why\nother LRU caches where not fit for this purpose.\n\nMagazine can also be used as generated content cache and will do a fine job.\n\nMagazine implements a reference counted cache that controls eviction, provides\nmechanisms for eviction based on object size instead of object count, allows for\nthe sub-division of a primary cache into sub-caches. The documentation will\noffer suggestions for concurrent file system programming and file locking using\nMagazine which you might find interesting regardless of whether or not you adopt\nMagazine.\n\nThe nicest thing anyone ever said about me on the Internet was this excellent\nartible about\n[Magazine](http://francescomari.github.io/2014/06/26/the-magazine-cache.html),\nhowever it is sadly out of date now with Magazine 6.0. I've shamelessly borrowed\nfrom this article in this documentation.\n\n## Magazine as Page Cache\n\nThere is a [step-by-step tutorial](x) of Magazine. This is a feature overview\nfor evaluation.\n\n```javascript\nconst Cache = require('magazine')\n\n// Create a cache.\nconst cache = new Cache\n\n// Get a value or set it with a default if it doesn't exist.\nconst entry = cache.hold('one', { number: 1, initialized: false })\n\n// Magazine returns an entry, not a value. The value is a property of the entry.\nif (! entry.value.initialized) {\n    entry.value.initialized = true\n}\n\n// Entries are reference counted and cannot be evicted when references are held.\nentry.release()\n\n// Ask the cache to make a best effort to evict entries down to zero.\ncache.shrink(0)\n```\n\nEvery entry in a Magazine **cache** is wrapped in an **entry** object. An\nentry represents a piece of data stored in the cache. The entires are added and\nretreived using by a key.\n\nThe `hold()` method is used to both and retrieve data. It does double duty\nbecause we're not just stashing generated content the way we do in a content\ncache. We're storing an authoriative object, one that co-ordinates changes\naccross different asynchronous paths of execution.\n\nBut, if you really want to use Magazine as a generated content cache, here you\ngo.\n\n```\nconst Cache = require('magazine')\n\n// Create a cache.\nconst cache = new Cache\n\n// Create a get/put/remove wrapper around the cache.\nconst map = new Cache.Map(cache)\n\n// Hmm… Looks like the other LRU caches have expiration methods, so maybe this\n// ought to have one too. Blah. Blah. Okay it will. Let me finish documenting\n// the other usage first.\nasync function webGen () {\n    let html = map.get(url)\n    if (html == null) {\n        html = await genreateWebPage()\n        cache.put(url, html)\n    }\n    send(html)\n}\n```\n\nWhen we call `hold()` we pass an initial empty object. You should provide an\nobject that is relatively cheap to construct since it will be discarded if you\nget a cache hit. If you get a cache miss, you can then complete the construction\nof the object.\n\n```\nconst cache = new Magazine\n\nasync function getPage (path) {\n    const entry = cache.hold(path, { path: path, nodes: null })\n    if (entry.value.nodes == null) {\n        entry.value.nodes = await load(entry.value.path)\n    }\n    return entry\n}\n\nconst entry = await getPage('./tree/root')\n\naddKey(entry.value.nodes, 'some key')\n\nentry.release()\n```\n\nIn the example above we've implemented a page loading subsystem. A page in a\ndatabase is a file that contains a range of records. `async load()` reads a page\nfrom the file system and `addKey()` that adds a key to the page.\n\nWe try to `hold()` an existing page object. If the `nodes` property of the page\nobject is `null`, then we have a newly constructed, uninitialized page. We then\nload the page from the file system into the object.\n\nOur function returns the `entry` and not the entry value because we do not want\nMagazine to evict the page from memory while we're using it. We'll be\nresponsible for the entry and release it when we're done using it and it is safe\nto evict it.\n\nOf course, the astute reader will notice a race condition in the example above.\nIf `getPage()` is called simultaneously there will be two calls to `load()` that\nare racing to assign the `nodes` property. This is a problem. We are adding a\nkey to the `nodes` array, but that array might be reassigned and our node\naddition lost.\n\nHere is a cannonical `getPage()`.\n\n```javascript\nasync function getPage (path) {\n    const entry = cache.hold(path)\n    if (entry == null) {\n        const nodes = await load(path)\n        return cache.hold(path, { nodes })\n    }\n    return entry\n}\n```\n\nA key-only call to `hold` returns `null` on a cache miss. If the entry is null\nwe load the page. We then add it to the cache using `hold()` which will get an\nexisting entry or set it with the initializer. In a race between two simulateous\ncalls, both will load the page from file, but only one will be added to the\ncache. Everyone will receive the same entry containing the same nodes array.\nEveryone will see the changes to the nodes array.\n\nMagazine is essentially a key/value store, but it does not operate like `Map`\nwith `get` and `put` operations.\n\nMagazine does not provide `get` and `put` methods like a `Map`. Instead it\nprovides a `hold()` method that\n\n```javascript\n// create a cache\nvar Cache = require('magazine')\nvar cache = new Cache\n\n// create a magazine that stores to the cache\nvar magazine = cache.createMagazine()\n\n// hold and release to key/value pairs to get them into the magazine\nmagazine.hold('one', { number: 1 }).release()\nmagazine.hold('two', { number: 2 }).release()\n\n// wait a second\nsetTimeout(afterOneSecond, 1000)\n\nfunction afterOneSecond () {\n    // after a second hold and release one of the keys to refresh it\n    magazine.hold('one', null).release()\n\n    // wait another second\n    setTimeout(afterTwoSeconds, 1000)\n}\n\nfunction afterTwoSeconds () {\n    // after two seconds, purge anything that is older than a second and a half\n    cache.expire(Date.now() - 1500)\n\n    // hold the one cartidge, the given value is the default value to use if the\n    // value does not exist in the cache.\n    var cartridge = magazine.hold('one', { number: null })\n\n    // check for a cache hit on one\n    if (cartridge.value.number == null) {\n        console.log('one is not in cache')\n        cartridge.remove()\n    } else {\n        console.log('one is in cache')\n        cartridge.release()\n    }\n\n    cartridge = magazine.hold('two', { number: null })\n\n    // check for a cache hit on two\n    if (cartridge.value.number == null) {\n        console.log('two is not in cache')\n        cartridge.remove()\n    } else {\n        console.log('two is in cache')\n        cartridge.release()\n    }\n}\n```\n\nI created Magazine for use wtih [Strata](https://github.com/bigeasy/strata), a\nb-tree implementation in pure JavaScript. The b-tree implementation needs a\ncache for pages read from disk. Most applications that use Strata are going to\nwant to use more than one b-tree, but why make the application developer have to\nthink about tuning the page cache for each b-tree? So I created Magazine, a\ncommon cache has multiple collections.\n\n\n#### `new Cache`\n\nCreate a new cache.\n\n#### `cache.expire(before)`.\n\nPurge the cache removing items before the given date.\n\n#### `iterator = cache.purge()`\n\nCreate an iterator over the cache in least recently used order.\n\n#### `magazine = cache.createMagazine()`\n\nCreate a magazine using the cache.\n\n#### `magazine.expire(before)`\n\nExpire only the items in the current magazine.\n\n#### `cartridge = magazine.hold(key, value)`\n\nHold a cartridge for the given key creating a cartridge if one does not exist.\n\n#### `cartridge.release()`\n\nRelease the hold on the cartridge.\n\n#### `cartridge.remove()`\n\nRemove the hold the cartridge. You must be the only one holding it to remove it.\n\n#### `cartridge.adjustHeft(value)`\n\nAdjust the heft of the cartridge. Heft is some arbitrary measure of the weight\nof cartridge. For cached managed by count, the actual count of items might not\nbe a the cache entry itself. The cache entry could contain an array of items,\nfor example, and user wants to purge the cache when it has more than a total\nnumber of items.\n\n#### `cartridge.heft`\n\nAn arbitrary measure of the weight of the cartridge.\n\n#### `purge.next()`\n\nMove to the next oldest entry in the queue.\n\n#### `purge.release()`\n\nReleases a cartridge if one is held, a safe way to finalize a purge iteration.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbigeasy%2Fmagazine","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbigeasy%2Fmagazine","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbigeasy%2Fmagazine/lists"}