{"id":23280991,"url":"https://github.com/nicholaswmin/automap","last_synced_at":"2026-04-26T23:31:26.942Z","repository":{"id":245859689,"uuid":"819232549","full_name":"nicholaswmin/automap","owner":"nicholaswmin","description":"objects in Redis [WIP]","archived":false,"fork":false,"pushed_at":"2024-08-10T13:58:27.000Z","size":275,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-03-29T08:34:02.758Z","etag":null,"topics":["oop","orm","redis"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit-0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nicholaswmin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-24T05:29:31.000Z","updated_at":"2024-08-10T13:58:28.000Z","dependencies_parsed_at":null,"dependency_job_id":"69dd1fb2-9fea-48bb-90a2-6b34a2383846","html_url":"https://github.com/nicholaswmin/automap","commit_stats":null,"previous_names":["nicholaswmin/automap"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nicholaswmin/automap","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nicholaswmin%2Fautomap","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nicholaswmin%2Fautomap/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nicholaswmin%2Fautomap/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nicholaswmin%2Fautomap/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nicholaswmin","download_url":"https://codeload.github.com/nicholaswmin/automap/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nicholaswmin%2Fautomap/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32317162,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-26T23:26:28.701Z","status":"ssl_error","status_checked_at":"2026-04-26T23:26:25.802Z","response_time":129,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["oop","orm","redis"],"created_at":"2024-12-19T23:39:59.075Z","updated_at":"2026-04-26T23:31:26.924Z","avatar_url":"https://github.com/nicholaswmin.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![test-workflow][test-badge]][test-workflow] [![integration-workflow][integration-badge]][integration-workflow] [![performance-workflow][performance-badge]][performance-workflow] [![coverage-workflow][coverage-badge]][coverage-report] [![codeql-workflow][codeql-badge]][codeql-workflow]\n\n# automap\n\nStore [OOP][oop] object-graphs in [Redis][redis]\n\n- [Install](#install)\n- [Usage](#usage)\n  * [Model definition](#model-definition)\n  * [`List`, `LazyList` \u0026 `AppendList`](#the-list-types)\n  * [Lazy-loading with `LazyList`](#lazy-loading)\n  * [Infinite lists with `AppendList`](#infinite-lists-with-appendlist)\n  * [Runnable example](#runnable-example)\n- [Redis data structure](#redis-data-structure)\n- [Performance](#performance)\n  * [Benchmarks](#benchmarks)\n  * [Atomicity](#atomicity)\n  * [Time complexity](#time-complexity)\n    + [Flat lists](#flat-lists)\n    + [Nested lists](#nested-lists)\n- [Alternatives](#alternatives)\n- [Minimum Redis and `ioredis` versions](#minimum-redis-and-ioredis)\n- [Tests](#tests)\n  + [Unit tests](#tests)\n  + [Integration tests](#tests)\n  + [Performance tests](#tests)\n  + [Test Coverage](#tests)\n- [Contributing](#contributing)\n- [Authors](#authors)\n- [License](#license)\n\n## Install\n\n```bash\nnpm i https://github.com/nicholaswmin/automap\n```\n\n## Usage\n\nexports a `repository` and an array-like type, called `List`.\n\n- `repository.save()` saves an object\n- `repository.fetch()` gets it back\n\nList-like data is detached \u0026 saved as a [`Hash`][redis-hash]\nor [`List`][redis-list] rather than jam everything into a\nsingle [`Key`][redis-string].\n\n\u003e Example: A `Building` with `Flats`:\n\n```js\nconst building = new Building({\n  id: 'foo',\n  flats: new List({\n    type: Flat,\n    from: [\n      new Flat({ id: 101 }),\n      new Flat({ id: 102 })\n    ]\n  })\n})\n```\n\n... save it:\n\n```js\nimport { Repository } from 'automap'\n\nconst repo = new Repository(Building, new ioredis())\n\nconst building = new Building({\n  id: 'foo',\n  flats: new List({\n    type: Flat,\n    from: [\n      new Flat({ id: 101 }),\n      new Flat({ id: 102 })\n    ]\n  })\n})\n\nawait repo.save(building)\n```\n\n... saved like so:\n\n```js\n           ┌──────────────────┐\n           │ Building         |\n           |                  |\n           │ id: foo          │\n           │ flats:           │\n           │  - Flat          │\n           │  - Flat          │\n           │  - Flat          │\n           │  - Flat          │\n           └─────────┬────────┘\n┌──────────────────┐ │ ┌─────────────────┐\n│ Redis String     │◄┴►│ Redis Hash      │\n│                  │   │                 │\n│ id: foo          │   │  - foo:flats:1  │\n│ flats: foo:flats |   |  - foo:flats:2  │\n│                  │   │  - foo:flats:3  │\n│                  │   │  - foo:flats:4  │\n└──────────────────┘   └─────────────────┘\n```\n\n\u003e The item *order* is preserved; despite using a Hash, by storing the\n\u003e `index` of an `item` alongside it's JSON.\n\n... fetch it back:\n\n```js\nconst building = await repo.fetch('foo')\n\nfor (let flat of building.flats)\n  console.log(flat instanceof Flat, flat)\n  // true { id: '101' }, true { id: '102' },...\n```\n\n... reassembles it, with the correct types:\n\n```js\n┌──────────────────┐   ┌─────────────────┐\n│ Redis String     │   │ Redis Hash      │\n│                  │   │                 │\n│ id: foo          │   │  - foo:flats:1  │\n│ flats: foo:flats |   |  - foo:flats:2  │\n│                  │   │  - foo:flats:3  │\n│                  │   │  - foo:flats:4  │\n└──────────────────┘◄|►└─────────────────┘\n                     |\n            ┌───────────────────┐\n            │ Building          |\n            |                   |\n            │ id: \"foo\"         │\n            │ flats:            │\n            │  - Flat           │\n            │  - Flat           │\n            │  - Flat           │\n            │  - Flat           │\n            └───────────────────┘\n```\n\nit rebuilds the entire object graph including *nested* types.\n\ni.e: calling a `Flat.method()` works.\n\n```js\nconst building = await repo.fetch('foo')\n\nbuilding.flats[0].doorbell()\n// 🔔 at flat: 101 !\n```\n\n## Model definition\n\nYou can use any object as long as it:\n\n1. has an `id` property set to a unique value\n2. can be reconstructed by calling `new` and passing it's JSON\n\nA working example:\n\n```js\nimport { List } from 'automap'\n\nclass Building {\n  constructor({ id, flats = [] }) {\n    this.id = id\n    this.flats = new List({\n      type: Flat,\n      from: flats\n    })\n  }\n}\n\nclass Flat {\n  constructor({ id }) {\n    this.id = id\n  }\n\n  doorbell() {\n    console.log(`🔔 at flat: ${this.id}`)\n  }\n}\n```\n\nThe above works because:\n\n- ✅ The `Building` has an `id` set to a unique value\n- ✅ The `Building` can be entirely reconstructed by calling\n    `new Building(json)` and  passing it's JSON\n- ✅ The `flats` array is replaced with a `List` type\n\nbut this example does not work:\n\n```js\nclass Building {\n  // `flats = []` is missing in the\n  // constructor arguments\n  constructor({ id }) {\n    this.id = id\n    this.flats = new List({\n      type: Flat,\n      from: flats\n    })\n  }\n}\n```\n\n❌ the `Building` root will be constructed OK, but its nested `flats` will not.\n\nthe following won't work either:\n\n```js\nclass Building {\n  // no `id` property\n  constructor({ name, flats = [] }) {\n    this.name = name\n    this.flats = new List({\n      type: Flat,\n      from: flats\n    })\n  }\n}\n\nconst building = new Building({ name: 'bar' })\n\nawait repository.save(building)\n// throws \"error: no id present\"\n```\n\n❌ the root object is missing an `id` property.\n\n## The `List` types\n\nList-like data must use one of the `List` types instead of an [`Array`][array].\nYou can still use a regular `Array` but it won't be decomposed from the\nmain object-graph.\n\n`List`\n\n- fetched with all items loaded\n- [linear-time O\u003csup\u003en\u003c/sup\u003e][linear] additions\n- saved as a [`Hash`][redis-hash]\n\nused for lists that must always be loaded to do any work with the object.\n\n[`LazyList`](#lazy-loading)\n\n- fetched empty\n- can be loaded with `list.load()`\n- [linear-time O\u003csup\u003en\u003c/sup\u003e][linear] additions\n- saved as a [`Hash`][redis-hash]\n\nused for lists that can become \"large-ish\", yet not always required.\n\n[`AppendList`](#infinite-lists-with-appendlist)\n\n- fetched empty\n- can be loaded with `list.load()`\n- [constant-time O\u003csup\u003e1\u003c/sup\u003e][const] additions\n- saved as a [`List`][redis-list]\n\nused for lists that are way too big to carry around and don't need to be\nloaded to do work in most cases.\n\n\u003e Example:\n\n```js\nclass Building {\n  constructor({ id, flats = [] }) {\n    this.id = id\n\n    // ! List instead of Array\n    this.flats = new List({\n      type: Flat,\n      from: flats\n    })\n  }\n}\n```\n\nAll `List` types are subtypes of the native [`Array`][array] and\nbehave *exactly* the same:\n\n```js\n\nconst list = new List({\n from: [1, 2, 3]\n})\n\nfor (const item of list)\n  console.log(item.constructor.name, item)\n\n// Number 1, Number 2 ...\n\nconsole.log(Array.isArray(list)) // true\n```\n\n... you can also specify a `type` parameter to cast to a type:\n\n```js\nconst list = new List({\n type: String,\n from: [1, 2, 3]\n})\n\nfor (const item of list)\n  console.log(item.constructor.name, item)\n\n// String '1', String '2' ...\n```\n\n... and use it like a regular `Array`:\n\n```js\nconst array = new List(1, 2, 3)\n\nconst two = array.find(num =\u003e num === 2)\n\nconsole.log(two)\n// 2\n```\n\n### Lazy loading\n\nSometimes you won't need to load the contents of a list initially.\nYou might want to load it's contents after you fetch it, or even none at all.\n\nIn that case, use a `LazyList` instead of a `List`.\n\n```js\nimport { LazyList } from 'automap'\n\nclass Building {\n  constructor({ id, flats = [] }) {\n    this.id = id\n    this.flats = new LazyList({\n      type: Flat,\n      from: flats\n    })\n  }\n}\n```\n\n... and load its contents by calling `list.load()`:\n\n```js\nconst building = await repo.fetch('foo')\n\nconsole.log(building.flats)\n// [] (empty)\n\nawait building.flats.load(repo)\n\nconsole.log(building.flats)\n// [ Flat { id: '101' }, Flat { id: '102' }, ...]\n```\n\n### Infinite lists with `AppendList`\n\nLists with millions of items should use an `AppendList`.\n\n- not loaded on `repository.fetch`\n- saves items in a [`List`][redis-list] instead of [`Hash`][redis-hash]\n\nThe `repository.save` time of an `AppendList` does not increase in\nproportion to the number of items in the list.\n\nCaveats:\n\n- No notion of item deletion. It functions as an [append-only log][append-only],\n  hence the name.\n- No constant O\u003csup\u003e1\u003c/sup\u003e time lookups for individual list items in Redis.\n\nAn example:\n\n\u003e Each `Flat` has a list of `Mail` items, which can reach millions of items[^2].\n\n```js\nimport { LazyList } from 'automap'\n\nclass Building {\n  constructor({ id, flats = [] }) {\n    this.id = id\n    this.flats = new LazyList({\n      type: Flat,\n      from: flats\n    })\n  }\n}\n\nclass Flat {\n  constructor({ id, mail = [] }) {\n    this.id = id\n    this.mails = new AppendList({\n      type: Mail,\n      from: mail\n    })\n  }\n\n  addMail({ id, text }) {\n    this.mails.push(new Mail({ id, text }))\n  }\n}\n\nclass Mail {\n  constructor({ id, text }) {\n    this.id = id\n    this.text = text\n  }\n}\n```\n\n### Runnable example\n\nThe `Building` example demonstrated above can be [found here][runnable-example].\n\nRun it with:\n\n```bash\nnpm run example\n```\n\nThe [model][example-model] itself can be found [here].\n\n## Redis data structure\n\nAll keys saved in Redis follow a canonical and *human-readable* format.\n\nAssuming the above example, the flats are saved under this Redis key:\n\n```\nbuilding:foo:flats\n```\n\nwhich is a [Hash][redis-hash] with the following shape:\n\n\n| Field \t| Value                       \t  |\n|-------\t|-----------------------------\t  |\n| 101   \t| `{\"i\":0,\"json\":{\"id\":\"101\"}}` \t|\n| 102   \t| `{\"i\":1,\"json\":{\"id\":\"102\"}}` \t|\n| 103   \t| `{\"i\":2,\"json\":{\"id\":\"103\"}}` \t|\n\n\nIf you need to access an individual flat directly from Redis,\nyou can simply run:\n\n```\nHGET building:foo:flats 101\n```\n\nor fetch all the flats:\n\n```\nHGETALL building:foo:flats\n```\n\nThe `Building` itself is saved as:\n\n```\nbuilding:foo\n```\n\nwhich you can easily get by:\n\n```\nGET building:foo\n```\n\n### List items without `id`\n\nList items without an `id` property will use the `index`; their current\nposition in the list, as the separator.\n\nIf the flats didn't have an `id` and they contained a list of `Persons`,\nthe persons of the 1st flat would be saved under key:\n\n```\nbuilding:foo:flats:0:persons\n```\n\n## Performance\n\n### Benchmarks\n\nThe closest thing to a benchmark is a concurrent load test, available\n[here][paper-benchmark].\n\nAs a rule of thumb, the `Building` example with `100 Flats` takes about:\n\n- ~ `1.5 ms` to `fetch`\n- ~ `3 ms` to `save`\n\nand can handle ~ `300` x `fetch-edit-save` cycles per-second, without creating\na backlog, on a 10 minute sustained-load test.\n\nThese results were gathered with the benchmark mentioned above on a popular\ncloud-provider with native Redis add-ons and about `20x` concurrency.\n\nFor the internal-use that this module is designed for, these are satisfactory\nresults.\n\n### Atomicity\n\n#### Save\n\n- Each list needs a single Redis `HSET` command.\n- All `HSET`s are assembled into a single [pipelined][pipe] transaction\n  then they are sent down the wire.\n\nAdditionally, there's a Lua script which allows something akin to\na [`mget`][mget], but for hashes.\n\nThese methods ensure updates are both performant and [atomic][atomic][^1].\n\n#### Fetch\n\nFetching an object is not entirely atomic.\n\n### Nested Lists\n\nAn arbitrary amount of nesting-level of lists is allowed.\nYou can have a list, inside another list, inside another list and so on...\n\nBut you should note the following ...\n\n### Time-complexity\n\n\u003e This section briefly describes the [time-complexity][time] of possible input\n\u003e configurations.\n\u003e Note that in these time-complexity speculations are solely in the context\n\u003e of network roundtrips since they are by far the biggest bottleneck in\n\u003e most cases.\n\n#### Flat lists\n\nObject graphs which don't have lists nested inside other lists, are fetched\nin a process that exhibits an almost\n[constant-time complexity O(1)][const].\n\nThere's no network roundtrip involved for each list, or even separate requests.\n\n#### Nested lists\n\nIn contrast, fetching object graphs which have nested lists is a process which\nperforms in [quadratic-time O(n\u003csup\u003e2\u003c/sup\u003e)][qtc], at a minimum.\n\nEvery nesting level increases the exponent by `1` so you can easily jump from\nO(n) to O(n\u003csup\u003e2\u003c/sup\u003e) then O(n\u003csup\u003e3\u003c/sup\u003e) and so on.\n\nIn short, don't do it.\n\nThat being said, this problem only concerns a `List` nested in another `List`.\nNone of the 2 other types exhibit this issue, simply because they are not\nautomatically fetched.\n\n## Alternatives\n\n### Saving encoded JSONs\n\nA small enough object-graph can easily get away with:\n\n- `JSON.stringify(object)`\n- `SET building:foo json`\n- `GET building:foo`\n- `JSON.parse(json)`\n- done\n\nThis is a simple, efficient and inherently atomic operation.\n\nThe obvious caveat is that you cannot fetch individual list items directly\nfrom Redis since you would always need to fetch and parse the entire graph,\nbut for (probably most) use-cases that's simply just a non-problem.\n\n### Redis JSON\n\nIf [Redis JSON][redis-json] is available then you should use that instead.\n\nHalf the issues this module attempts to solve are solved out-the-box\nby using Redis JSON directly.\n\n## Minimum redis and ioredis\n\nTested on:\n\n- [Redis 6+][r]\n- [ioredis 5+][ioredis]\n\n## Tests\n\ninstall deps:\n\n```bash\nnpm ci\n```\n\nrun unit tests:\n\n```bash\nnpm test\n```\n\nrun integration tests:\n\n\u003e integration \u0026 performance tests need a [redis server][r] running on `:6379`\n\n```bash\nnpm run test:integration\n```\n\nrun performance tests:\n\n\u003e these tests are slow:\n\n```bash\nnpm run test:performance\n```\n\nproduce a test coverage report:\n\n```bash\nnpm run test:coverage\n```\n\ncheck standards:\n\n\u003e `eslint`, `npm audit` etc ...\n\n```bash\nnpm run checks\n```\n\n## Contributing\n\n[Contribution Guidelines][contributing].\n\n## Authors\n\nNicholas Kyriakides, [@nicholaswmin][nicholaswmin]\n\n## License\n\n[MIT \"No Attribution\" License][license]\n\n## Footnotes\n\n[^1]: Redis transactions do not assume the same atomicity and isolation\n      context that a relational database might assume.\n      By every definition they are atomic since they don't allow for\n      partial updates, however the entire transaction can fail if client B\n      updates a value while it's in the process of being modified by client A\n      as part of a transaction.\n      Retries are not currently implemented.\n\n[^2]: The singer \"Sting\" lives here and gets lots of fan-mail.\n      Obviously, this isn't your run-of-the-mill apartment building.\n\n\u003c!--- Badges --\u003e\n\n[test-badge]: https://github.com/nicholaswmin/automap/actions/workflows/test:unit.yml/badge.svg\n[test-workflow]: https://github.com/nicholaswmin/automap/actions/workflows/test:unit.yml\n\n[integration-badge]: https://github.com/nicholaswmin/automap/actions/workflows/test:integration.yml/badge.svg\n[integration-workflow]: https://github.com/nicholaswmin/automap/actions/workflows/test:integration.yml\n\n[performance-badge]: https://github.com/nicholaswmin/automap/actions/workflows/test:performance.yml/badge.svg\n[performance-workflow]: https://github.com/nicholaswmin/automap/actions/workflows/test:performance.yml\n\n[coverage-badge]: https://coveralls.io/repos/github/nicholaswmin/automap/badge.svg?branch=main\n[coverage-report]: https://coveralls.io/github/nicholaswmin/automap?branch=main\n\n[codeql-badge]: https://github.com/nicholaswmin/automap/actions/workflows/codeql.yml/badge.svg\n[codeql-workflow]: https://github.com/nicholaswmin/automap/actions/workflows/codeql.yml\n\n\n\u003c!--- /Badges --\u003e\n\n[oop]: https://en.wikipedia.org/wiki/Object-oriented_programming\n[redis]: https://redis.io/\n[array]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array\n[atomic]: https://en.wikipedia.org/wiki/Atomicity_(database_systems)\n[pipe]: https://en.wikipedia.org/wiki/HTTP_pipelining\n[redis-hash]: https://redis.io/docs/latest/develop/data-types/hashes/\n[redis-string]: https://redis.io/docs/latest/develop/data-types/strings/\n[redis-list]: https://redis.io/docs/latest/develop/data-types/lists/\n[append-only]: https://en.wikipedia.org/wiki/Append-only\n[lpush]: https://redis.io/docs/latest/commands/lpush/\n[const]: https://en.wikipedia.org/wiki/Time_complexity#Constant_time\n[qtc]: https://en.wikipedia.org/wiki/Time_complexity#Sub-quadratic_time\n[linear]: https://en.wikipedia.org/wiki/Linear_search\n[mget]: https://redis.io/docs/latest/commands/mget/\n[redisom]: https://github.com/redis/redis-om-node\n[redis-json]: https://redis.io/docs/latest/develop/data-types/json/\n[qs]: https://en.wikipedia.org/wiki/Quicksort\n[time]: https://en.wikipedia.org/wiki/Time_complexity\n[bench]: https://redis.io/docs/latest/develop/data-types/json/performance/\n[nicholaswmin]: https://github.com/nicholaswmin\n[contributing]: .github/CONTRIBUTING.md\n[runnable-example]: .github/example/index.js\n[example-model]: ./test/util/model/index.js\n[paper-benchmark]: .github/benchmark/README.md\n[r]: https://redis.io/docs/latest/operate/oss_and_stack/install/install-redis/\n[ioredis]: https://github.com/redis/ioredis\n[non-func]: https://en.wikipedia.org/wiki/Non-functional_requirement\n[perf-tests]: ./test/performance\n[license]: ./LICENSE\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnicholaswmin%2Fautomap","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnicholaswmin%2Fautomap","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnicholaswmin%2Fautomap/lists"}