{"id":16901062,"url":"https://github.com/ryanramage/reduce-group","last_synced_at":"2026-07-02T18:03:35.792Z","repository":{"id":27467492,"uuid":"30946680","full_name":"ryanramage/reduce-group","owner":"ryanramage","description":"Reduce ldjson by group levels, much like couchdb","archived":false,"fork":false,"pushed_at":"2017-02-01T20:10:05.000Z","size":14,"stargazers_count":3,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-14T20:49:15.880Z","etag":null,"topics":["ndjson","stats"],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ryanramage.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-02-18T01:12:02.000Z","updated_at":"2017-02-01T20:10:07.000Z","dependencies_parsed_at":"2022-09-16T02:11:42.446Z","dependency_job_id":null,"html_url":"https://github.com/ryanramage/reduce-group","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryanramage%2Freduce-group","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryanramage%2Freduce-group/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryanramage%2Freduce-group/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryanramage%2Freduce-group/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ryanramage","download_url":"https://codeload.github.com/ryanramage/reduce-group/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247339135,"owners_count":20923009,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ndjson","stats"],"created_at":"2024-10-13T17:56:57.998Z","updated_at":"2025-10-28T18:14:14.554Z","avatar_url":"https://github.com/ryanramage.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"reduce-group\n===================\n\nReduce objects by group levels, much like couchdb. Use built in functions like _sum, _count, and _stats.\n\n\nModule Usage\n------------\n\n    npm install reduce-group\n\nand then\n\n    var db = require('levelup')('/in/memory', { db: require('memdown') });\n    var reducer = require('reduce-group')(db);\n\n\n    reducer.pipe(concat(function(objs){\n      var expected = [\n        { key: [ 'c3rrcv0' ], value: 14 },\n        { key: [ 'c3rrcv1' ], value: 4.4 },\n        { key: [ 'c3rrcv4' ], value: 5 },\n        { key: [ 'c3rrcv5' ], value: 4 },\n        { key: [ 'c3rrcyz' ], value: 3 }\n      ];\n      t.deepEqual(objs, expected)\n      t.end();\n    }))\n    reducer.write({ \"key\": [\"c3rrcv5\",\"Robbery\"], \"value\": 3});\n    reducer.write({ \"key\": [\"c3rrcv4\",\"Robbery\"], \"value\": 2});\n    reducer.write({ \"key\": [\"c3rrcv1\",\"Robbery\"], \"value\": 2});\n    reducer.write({ \"key\": [\"c3rrcv0\",\"Robbery\"], \"value\": 1});\n    reducer.write({ \"key\": [\"c3rrcyz\",\"Robbery\"], \"value\": 3});\n    reducer.write({ \"key\": [\"c3rrcv5\",\"Assualt\"], \"value\": 1});\n    reducer.write({ \"key\": [\"c3rrcv4\",\"Assualt\"], \"value\": 1});\n    reducer.write({ \"key\": [\"c3rrcv1\",\"Assualt\"], \"value\": 1});\n    reducer.write({ \"key\": [\"c3rrcv0\",\"Robbery\"], \"value\": 1});\n    reducer.write({ \"key\": [\"c3rrcv5\",\"Robbery\"], \"value\": 0});\n    reducer.write({ \"key\": [\"c3rrcv4\",\"Robbery\"], \"value\": 2});\n    reducer.write({ \"key\": [\"c3rrcv1\",\"Robbery\"], \"value\" :1.4});\n    reducer.write({ \"key\": [\"c3rrcv0\",\"Robbery\"], \"value\": 12});\n    reducer.end()\n\n\nSee it run here: https://runkit.com/58923e6f56d1720014b8b4d1/58923f648417ac00141964d7\n\nCommand Line Usage\n------------------\n\n    npm install reduce-group -g\n\nfirst, find some data in ndjson format, with objects with keys and values on each line. Something like\n\n\n```\ncat data.json\n\n{ \"key\": [\"c3rrcv5\",\"Robbery\"], \"value\": 3}\n{ \"key\": [\"c3rrcv4\",\"Robbery\"], \"value\": 2}\n{ \"key\": [\"c3rrcv1\",\"Robbery\"], \"value\": 2}\n{ \"key\": [\"c3rrcv0\",\"Robbery\"], \"value\": 1}\n{ \"key\": [\"c3rrcyz\",\"Robbery\"], \"value\": 3}\n{ \"key\": [\"c3rrcv5\",\"Assualt\"], \"value\": 1}\n{ \"key\": [\"c3rrcv4\",\"Assualt\"], \"value\": 1}\n{ \"key\": [\"c3rrcv1\",\"Assualt\"], \"value\": 1}\n{ \"key\": [\"c3rrcv0\",\"Robbery\"], \"value\": 1}\n{ \"key\": [\"c3rrcv5\",\"Robbery\"], \"value\": 0}\n{ \"key\": [\"c3rrcv4\",\"Robbery\"], \"value\": 2}\n{ \"key\": [\"c3rrcv1\",\"Robbery\"], \"value\" :1.4}\n{ \"key\": [\"c3rrcv0\",\"Robbery\"], \"value\": 12}\n\n```\n\n\nso to reduce to the sums of all crime types in a geohash, do the following:\n\n```\ncat mapped.json | reduce-group\n\n{ key: [ 'c3rrcv0' ], value: 14 }\n{ key: [ 'c3rrcv1' ], value: 4.4 }\n{ key: [ 'c3rrcv4' ], value: 5 }\n{ key: [ 'c3rrcv5' ], value: 4 }\n{ key: [ 'c3rrcyz' ], value: 3 }\n```\n\nby default this is run with a --group_level=1. You can change the it to a level 2, like so:\n\n\n```\ncat mapped.json | reduce-group  --group_level=2\n\n{ key: [ 'c3rrcv0', 'Robbery' ], value: 14 }\n{ key: [ 'c3rrcv1', 'Assualt' ], value: 1 }\n{ key: [ 'c3rrcv1', 'Robbery' ], value: 3.4 }\n{ key: [ 'c3rrcv4', 'Assualt' ], value: 1 }\n{ key: [ 'c3rrcv4', 'Robbery' ], value: 4 }\n{ key: [ 'c3rrcv5', 'Assualt' ], value: 1 }\n{ key: [ 'c3rrcv5', 'Robbery' ], value: 3 }\n{ key: [ 'c3rrcyz', 'Robbery' ], value: 3 }\n```\n\nThe ndjson output will be ordered by key. It is using _sum to reduce the value, grouped by the first two columns of the key.\n\nThere are other built in functions, eg _count\n\n```\ncat mapped.json | reduce-group --reduce=_count\n\n{ key: [ 'c3rrcv0' ], value: 3 }\n{ key: [ 'c3rrcv1' ], value: 3 }\n{ key: [ 'c3rrcv4' ], value: 3 }\n{ key: [ 'c3rrcv5' ], value: 3 }\n{ key: [ 'c3rrcyz' ], value: 1 }\n```\n\n```\nexample.json | reduce-group --group_level=2  --reduce=_count\n{\"key\":[\"c3rrcv0\",\"Robbery\"],\"value\":3}\n{\"key\":[\"c3rrcv1\",\"Assualt\"],\"value\":1}\n{\"key\":[\"c3rrcv1\",\"Robbery\"],\"value\":2}\n{\"key\":[\"c3rrcv4\",\"Assualt\"],\"value\":1}\n{\"key\":[\"c3rrcv4\",\"Robbery\"],\"value\":2}\n{\"key\":[\"c3rrcv5\",\"Assualt\"],\"value\":1}\n{\"key\":[\"c3rrcv5\",\"Robbery\"],\"value\":2}\n{\"key\":[\"c3rrcyz\",\"Robbery\"],\"value\":1}\n```\n\n\nand _stats\n\n```\ncat mapped.json | reduce-group --reduce=_stats\n\n{\"key\":[\"c3rrcv0\"],\"value\":{\"min\":1,\"max\":12,\"sum\":14,\"count\":3,\"sumsqr\":146}}\n{\"key\":[\"c3rrcv1\"],\"value\":{\"min\":1,\"max\":2,\"sum\":4.4,\"count\":3,\"sumsqr\":6.96}}\n{\"key\":[\"c3rrcv4\"],\"value\":{\"min\":1,\"max\":2,\"sum\":5,\"count\":3,\"sumsqr\":9}}\n{\"key\":[\"c3rrcv5\"],\"value\":{\"min\":0,\"max\":3,\"sum\":4,\"count\":3,\"sumsqr\":10}}\n{\"key\":[\"c3rrcyz\"],\"value\":{\"min\":3,\"max\":3,\"sum\":3,\"count\":1,\"sumsqr\":9}}\n```\n\n\n\n\nAll Options\n-----------\n\n - --group_level=1  What level to group the data at,\n - --reduce=_sum  How to reduce. Can be _sum, _stats, _count or a custom reduce function\n - --file=reduce.js Define a reduce in an external js file.\n - --db_module=memdown What backing to store the reduce in. Larger datasets might want 'leveldown'\n - --db_path=./data Where to store a persistent level store like leveldown.\n - --separator='\\xff' The default separator in the underlying leveldb\n\n\n\nFuture\n------\n\nThis is the first release. I plan to have reduce options such as _sum, _count and _stats in the future.\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fryanramage%2Freduce-group","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fryanramage%2Freduce-group","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fryanramage%2Freduce-group/lists"}