{"id":17149547,"url":"https://github.com/mohd-akram/jify","last_synced_at":"2025-04-13T11:43:37.955Z","repository":{"id":45967189,"uuid":"148985893","full_name":"mohd-akram/jify","owner":"mohd-akram","description":"JSON indexed file database/querying library/tool","archived":false,"fork":false,"pushed_at":"2024-01-13T10:17:14.000Z","size":258,"stargazers_count":26,"open_issues_count":1,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-27T02:51:16.117Z","etag":null,"topics":["database","index","json","node","nodejs","nosql","npm","query"],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/jify","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mohd-akram.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-09-16T10:41:47.000Z","updated_at":"2025-03-02T09:16:49.000Z","dependencies_parsed_at":"2024-01-13T11:23:11.546Z","dependency_job_id":"bf3d4840-b0c3-459c-9ae4-d46fdac2ce67","html_url":"https://github.com/mohd-akram/jify","commit_stats":null,"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mohd-akram%2Fjify","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mohd-akram%2Fjify/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mohd-akram%2Fjify/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mohd-akram%2Fjify/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mohd-akram","download_url":"https://codeload.github.com/mohd-akram/jify/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248317733,"owners_count":21083528,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","index","json","node","nodejs","nosql","npm","query"],"created_at":"2024-10-14T21:33:25.256Z","updated_at":"2025-04-13T11:43:37.911Z","avatar_url":"https://github.com/mohd-akram.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# jify\n\njify is an experimental library/tool for querying large (GBs) JSON files. It\ndoes this by first indexing the required fields. It can also be used as an\nappend-only database.\n\nWhen a JSON file is indexed (eg. `data.json`) an index file is created in the\nsame directory with a `.index.json` extension (eg. `data.index.json`).\n\n## Install\n\n    npm install jify\n\n## Usage\n\n```javascript\nimport { Database, predicate as p } from \"jify\";\n\nconst db = new Database(\"books.json\");\n\n// Create\nawait db.create();\n\n// Insert - Single\nawait db.insert({\n  title: \"Robinson Crusoe\",\n  year: 1719,\n  author: { name: \"Daniel Defoe\" },\n});\n\n// Insert - Batch\nawait db.insert([\n  {\n    title: \"Great Expectations\",\n    year: 1861,\n    author: { name: \"Charles Dickens\" },\n  },\n  {\n    title: \"Oliver Twist\",\n    year: 1838,\n    author: { name: \"Charles Dickens\" },\n  },\n  {\n    title: \"Pride and Prejudice\",\n    year: 1813,\n    author: { name: \"Jane Austen\" },\n  },\n  {\n    title: \"Nineteen Eighty-Four\",\n    year: 1949,\n    author: { name: \"George Orwell\" },\n  },\n]);\n\n// Index - creates books.index.json file\nawait db.index(\"title\", \"year\", \"author.name\");\n\n// Query\nconsole.log(\"author.name = Charles Dickens, year \u003e 1840\");\nconst query = { \"author.name\": \"Charles Dickens\", year: p`\u003e ${1840}` };\nfor await (const record of db.find(query)) console.log(record);\n\nlet records;\n\n// Range query\nconsole.log(\"1800 \u003c= year \u003c 1900\");\nrecords = await db.find({ year: p`\u003e= ${1800} \u003c ${1900}` }).toArray();\nconsole.log(records);\n\n// Multiple queries\nconsole.log(\"year \u003c 1800 or year \u003e 1900\");\nrecords = await db\n  .find({ year: p`\u003c ${1800}` }, { year: p`\u003e ${1900}` })\n  .toArray();\nconsole.log(records);\n```\n\n### CLI\n\n```terminal\n$ jify index --field title --field author.name --field year books.json\n$ jify find --query \"author.name=Charles Dickens,year\u003e1840\" books.json\n$ jify find --query \"year\u003e=1800\u003c1900\" books.json\n$ jify find --query \"year\u003c1800\" --query \"year\u003e1900\" books.json\n```\n\n## Implementation\n\nThe index is implemented as a JSON array of skip list entries. The entries are\nencoded as strings and all numbers embedded in the string are encoded using\n[Z85](https://rfc.zeromq.org/spec:32/Z85/). This implementation was chosen for\nits simplicity and to allow for using a single JSON file as an index. Better\nperformance might be achieved by using a different data structure, a binary\nformat, or multiple index files.\n\n## Performance\n\njify is reasonably fast. It can index about 1M records (~700 MB) per minute and\nsupports parallel indexing of fields. Inserting (with indexes) has similar\nperformance. Query time is \u003c 5ms for the first result + (0.1ms find + 0.1ms\nfetch) per subsequent result. All tests on a MBP 2016 base model.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmohd-akram%2Fjify","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmohd-akram%2Fjify","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmohd-akram%2Fjify/lists"}