{"id":16399464,"url":"https://github.com/mtrunkat/jsonlash","last_synced_at":"2025-08-06T15:31:53.515Z","repository":{"id":57285771,"uuid":"151153035","full_name":"mtrunkat/jsonlash","owner":"mtrunkat","description":"CLI utility for filtering and aggregation of JSONL streams. ","archived":false,"fork":false,"pushed_at":"2019-02-03T21:18:24.000Z","size":15,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-12-05T21:40:13.198Z","etag":null,"topics":["aggregation","cli","filtering","logs","stream"],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/jsonlash","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mtrunkat.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-10-01T20:15:37.000Z","updated_at":"2023-10-31T11:20:16.000Z","dependencies_parsed_at":"2022-09-08T21:42:53.892Z","dependency_job_id":null,"html_url":"https://github.com/mtrunkat/jsonlash","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mtrunkat%2Fjsonlash","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mtrunkat%2Fjsonlash/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mtrunkat%2Fjsonlash/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mtrunkat%2Fjsonlash/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mtrunkat","download_url":"https://codeload.github.com/mtrunkat/jsonlash/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228920546,"owners_count":17992005,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aggregation","cli","filtering","logs","stream"],"created_at":"2024-10-11T05:24:23.631Z","updated_at":"2024-12-09T16:12:34.491Z","avatar_url":"https://github.com/mtrunkat.png","language":"JavaScript","readme":"jsonlash\n========\n\n[![oclif](https://img.shields.io/badge/cli-oclif-brightgreen.svg)](https://oclif.io)\n[![Version](https://img.shields.io/npm/v/jsonlash.svg)](https://npmjs.org/package/jsonlash)\n\nCLI utility for filtering and aggregation of [JSONL](http://jsonlines.org/) streams. No matter which service for logging you use (LogDNA, Papertrail, Loggly, etc.) simply pipe log into **jsonlash** set up filters and aggregators and see aggregated data in realtime.\n\n## Usage\n\n### Installation\n\nInstall from [NPM](http://npmjs.com) globally:\n\n```bash\nnpm install -g jsonlash\n```\n\nAfter installation you can simply run jsonlash from your terminal with `-h` parameter to display help page:\n\n```bash\njsonlash -h\n```\n\n### Basic usage with filtering\n\nWe currently use [Log DNA](https://logdna.com/) as logging service so I am going to use it in examples but it's going to work with any [JSONL](http://jsonlines.org/) stream. So pipe your log stream to jsonlash:\n\n```bash\nlogdna tail | jsonlash\n```\n\nNow it will simply print out the log as it comes. So let's filter the API logs that are in the form:\n\n```\n{\n    \"msg\": \"API call\",\n    \"req\": {\n        \"duration\": 590,\n        \"method\": \"GET\",\n        \"route\": \"V2.datasets.items\",\n        ...\n    }\n    ...\n}\n```\n\nFiltering is done using `-f [FILTER]` parameter:\n\n```bash\nlogdna tail | jsonlash -f 'msg=API call'\n```\n\nWe can add more filters to filter out only requests with POST method and duration over 1000ms. And also add parameter `-e` to expand printed JSONs to be more readable:\n\n```bash\nlogdna tail | jsonlash -f 'msg=API call' -f 'req.method=POST' -f 'req.duration\u003e1000' -e\n```\n\n### Aggregations\n\nLet's continue with API logs example. To group log lines by request method and compute average and maximal duration call:\n\n```bash\nlogdna tail | jsonlash -f 'msg=API call' -a req.method --max req.duration --avg req.duration\n```\n\nand output will be a table with data aggregated in realtime:\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"https://trunkat.eu/external-assets/jsonlash.gif\" /\u003e\n\u003c/div\u003e\n\n## Examples\n\n### 1.\n\nAggregate logs by two fields `req.method` and `req.routeName` and compute average duration and the maximum duration\n\n```bash\n\n... | jsonlash -a req.method -a req.routeName --max req.duration --avg req.duration\n\n```\n\n### 2.\n\nFilter out requests taking more than a 10s, grouped them by `req.routeName` and compute how many users requested each of them:\n\n```bash\n... | jsonlash -f 'req.duration\u003e10000' -a req.routeName --uni req.userId\n```\n\n## Command reference\n\n```\nThis is a simple command line tool to filter and aggregate JSONL (json-lines) streams.\n\nUSAGE\n  $ jsonlash\n\nOPTIONS\n  -a, --aggregate=[FIELD]    aggregate JSONL items\n  -d, --debug                debug mode, shows JSON parsing errors\n  -e, --expand               expand outputted JSON\n  -f, --filter=[CONDITION]   filter JSONL items\n  -h, --help                 show CLI help\n\n  -v, --version              show CLI version\n  --avg=avg                  aggregate average value over all occurrences of given field\n  --max=max                  aggregate maximum value over all occurrences of given field\n  --min=min                  aggregate minimum value over all occurrences of given field\n  --sum=sum                  aggregate sum over all occurrences of given field\n  --uni=uni                  aggregate number of unique occurrences of given field\n\nDESCRIPTION\nSimply pipe in any JSONL stream and with filter and/or aggregation flags.\n\nIf you use only --filter flag then jsonlash outputs filtered jsonl stream.\n\nIf you also use --aggregate flag then it renders a table with aggregated data.\nAdditionally you may add one or more --min|--max|--sum|---avg|--uni flags to\ncompute aggregated values of given fields.\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmtrunkat%2Fjsonlash","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmtrunkat%2Fjsonlash","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmtrunkat%2Fjsonlash/lists"}