{"id":13564555,"url":"https://github.com/rs/jaggr","last_synced_at":"2025-04-05T01:09:35.351Z","repository":{"id":57481178,"uuid":"124478010","full_name":"rs/jaggr","owner":"rs","description":"JSON Aggregation CLI","archived":false,"fork":false,"pushed_at":"2024-08-29T11:04:00.000Z","size":2749,"stargazers_count":473,"open_issues_count":2,"forks_count":15,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-03-29T00:12:32.609Z","etag":null,"topics":["cli","golang","jplot","json","monitoring","statistics"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-03-09T02:45:20.000Z","updated_at":"2025-03-24T14:21:48.000Z","dependencies_parsed_at":"2024-06-18T22:45:54.288Z","dependency_job_id":"0f957e85-fc64-4699-8707-b2425329e1e3","html_url":"https://github.com/rs/jaggr","commit_stats":{"total_commits":15,"total_committers":3,"mean_commits":5.0,"dds":0.1333333333333333,"last_synced_commit":"0f5ada6eeda811f62a4b8e913ccbf6c2804cd273"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rs%2Fjaggr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rs%2Fjaggr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rs%2Fjaggr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rs%2Fjaggr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rs","download_url":"https://codeload.github.com/rs/jaggr/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247271532,"owners_count":20911587,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","golang","jplot","json","monitoring","statistics"],"created_at":"2024-08-01T13:01:32.908Z","updated_at":"2025-04-05T01:09:35.332Z","avatar_url":"https://github.com/rs.png","language":"Go","readme":"# jaggr: JSON Aggregation CLI\n[![license](http://img.shields.io/badge/license-MIT-red.svg?style=flat)](https://raw.githubusercontent.com/rs/jaggr/master/LICENSE) [![Build Status](https://travis-ci.org/rs/jaggr.svg?branch=master)](https://travis-ci.org/rs/jaggr)\n\nJaggr is a command line tool to aggregate in real time a series of JSON logs. The main goal of this tool is to prepare data for plotting with [jplot](https://github.com/rs/jplot).\n\n## Install\n\nDirect downloads are available through the [releases page](https://github.com/rs/jaggr/releases/latest).\n\nUsing [homebrew](http://brew.sh/) on macOS (Go not required):\n\n```\nbrew install rs/tap/jaggr\n```\n\nFrom source:\n\n```\ngo install github.com/rs/jaggr@latest\n```\n\n## Usage\n\nGiven the input below, generate one line per second with mean, min, max:\n\n```\n{\"code\": 200, \"latency\": 4788000, \"error\": \"\"}\n{\"code\": 200, \"latency\": 5785000, \"error\": \"\"}\n{\"code\": 200, \"latency\": 4162000, \"error\": \"\"}\n{\"code\": 502, \"latency\": 4461000, \"error\": \"i/o error\"}\n{\"code\": 200, \"latency\": 5884000, \"error\": \"\"}\n{\"code\": 200, \"latency\": 4702000, \"error\": \"\"}\n...\n```\n\n```\ntail -f log.json | jaggr @count=rps hist[200,300,400,500]:code min,max,mean:latency\n```\n\nOutput will be on line per second as follow:\n\n```\n{\"rps\":123, \"code\": {\"hist\": {\"200\": 100, \"300\": 0, \"400\": 0, \"500\": 13}}, \"latency\":{\"min\": 4461000, \"max\": 5884000, \"mean\": 4483000}}\n```\n\nSo here we give a stream of real-time requests to jaggr standard input and request the aggregation of the `code` and `latency` fields. For the `code` we request an histogram with some known error codes with an \"other\" bucket defined by `*`. The `latency` field is aggregated using minimum, maximum and mean. In addition, `@count` adds an extra field indicating the total number of lines aggregated. The `=` sign can be used on any field to rename it, here we use it to say that the count is an `rps` as we are using the default aggregation time of 1 second.\n\nNote that any field not specified in the argument list are removed from the output (i.e. `error` field).\n\n### Field Syntax\n\nA fields are JSON path prefixed with a list of aggregators. You can rename a field by suffixing it with `=\u003cname\u003e`. Here are some example of valid field declarations:\n\n* `median:latency`: Median computed for the latency field.\n* `median:latency=lat`: Same as above but the field is renamed `lat`.\n* `min,max,mean:latency`: Several aggregators applied to the `latency` field.\n* `median:timing.latency=latency`: Median of the sub-field latency of the `timing` JSON object renamed as top level `latency`.\n* `[100,200,300,400,500]hist:code`: Code counted into bucket of 100s.\n\n### Aggregators\n\nAvailable aggregators:\n\n* `min`, `max`, `mean`: Computes the min, max, mean of the field's values during the sample interval.\n* `median`, `p#`: The p1 to p99 compute the percentile of the field's values during the sample interval.\n* `sum`: Sum all values for the field.\n* `[bucket1,bucketN]hist`: Count number of values between bucket and bucket+1.\n* `[bucket1,bucketN]cat`: Count number of values equal to the define buckets (can be non-number values). The special `*` matches values that fit in none of the defined buckets.\n\n## Recipes\n\n### Vegeta\n\nJaggr can be used to integrate [vegeta](https://github.com/tsenart/vegeta) with [jplot](https://github.com/rs/jplot) as follow:\n\n```\necho 'GET http://localhost:8080' | \\\n    vegeta attack -rate 5000 -duration 10m | vegeta dump | \\\n    jaggr @count=rps \\\n          hist\\[100,200,300,400,500\\]:code \\\n          p25,p50,p95:latency \\\n          sum:bytes_in \\\n          sum:bytes_out | \\\n    jplot rps+code.hist.100+code.hist.200+code.hist.300+code.hist.400+code.hist.500 \\\n          latency.p95+latency.p50+latency.p25 \\\n          bytes_in.sum+bytes_out.sum\n```\n\n![](doc/vegeta.gif)","funding_links":[],"categories":["Go"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frs%2Fjaggr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frs%2Fjaggr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frs%2Fjaggr/lists"}