{"id":13475291,"url":"https://github.com/JFryy/qq","last_synced_at":"2025-03-26T23:30:42.810Z","repository":{"id":245761464,"uuid":"819154949","full_name":"JFryy/qq","owner":"JFryy","description":"jq, but with many interoperable configuration format transcodings and interactive querying.","archived":false,"fork":false,"pushed_at":"2025-02-21T04:51:10.000Z","size":2096,"stargazers_count":598,"open_issues_count":6,"forks_count":4,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-24T22:36:57.225Z","etag":null,"topics":["config","csv","go","golang","hcl","ini","jq","json","terraform","toml","transcoding","yaml"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JFryy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-23T23:47:53.000Z","updated_at":"2025-03-22T07:42:55.000Z","dependencies_parsed_at":"2024-07-18T05:06:28.427Z","dependency_job_id":"c03a2ade-047a-426e-bbf9-4fe032fc8d8d","html_url":"https://github.com/JFryy/qq","commit_stats":{"total_commits":54,"total_committers":2,"mean_commits":27.0,"dds":0.01851851851851849,"last_synced_commit":"9849050ffc76149a3ba1f7353d5a8a0d44ef7bd1"},"previous_names":["jfryy/qq"],"tags_count":16,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JFryy%2Fqq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JFryy%2Fqq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JFryy%2Fqq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JFryy%2Fqq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JFryy","download_url":"https://codeload.github.com/JFryy/qq/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245753836,"owners_count":20666822,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["config","csv","go","golang","hcl","ini","jq","json","terraform","toml","transcoding","yaml"],"created_at":"2024-07-31T16:01:19.125Z","updated_at":"2025-03-26T23:30:42.803Z","avatar_url":"https://github.com/JFryy.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# qq\n\n`qq` is a interoperable configuration format transcoder with `jq` query syntax powered by `gojq`. `qq` is multi modal, and can be used as a replacement for `jq` or be interacted with via a repl with autocomplete and realtime rendering preview for building queries.\n\n## Usage\nHere's some example usage, this emphasizes the interactive mode for demonstrantion, but `qq` is designed for usage in shell scripts.\n![Demo GIF](docs/demo.gif)\n\n\n\n```sh\n# JSON is default in and output.\ncat file.${ext} | qq -i ${ext}\n\n# Extension is parsed, no need for input flag\nqq '.' file.xml\n\n# random example: query xml, grep with gron using qq io and output as json\nqq file.xml -o gron | grep -vE \"sweet.potatoes\" | qq -i gron\n\n# get some content from a site with html input\ncurl motherfuckingwebsite.com | bin/qq -i html '.html.body.ul.li[0]'\n\n# interactive query builder mode on target file\nqq . file.json --interactive\n```\n\n## Installation\nFrom brew:\n```shell\nbrew install jfryy/tap/qq \n```\n\nFrom [AUR](https://aur.archlinux.org/packages/qq-git) (ArchLinux):\n```shell\nyay qq-git\n```\n\nFrom source (requires `go` `\u003e=1.22.4`)\n```shell\nmake install\n```\n\nDownload at releases [here](https://github.com/JFryy/qq/releases).\n\nDocker quickstart:\n\n```shell\n# install the image\ndocker pull jfryy/qq\n\n# run an example\necho '{\"foo\":\"bar\"}' | docker run -i jfryy/qq '.foo = \"bazz\"' -o tf\n```\n## Background\n\n`qq` is inspired by `fq` and `jq`. `jq` is a powerful and succinct query tool, sometimes I would find myself needing to use another bespoke tool for another format than json, whether its something dedicated with json query built in or a simple converter from one configuration format to json to pipe into jq. `qq` aims to be a handly utility on the terminal or in shell scripts that can be used for most interaction with structured formats in the terminal. It can transcode configuration formats interchangeably between one-another with the power of `jq` and it has an `an interactive repl (with automcomplete)` to boot so you can have an interactive experience when building queries optionally. Many thanks to the authors of the libraries used in this project, especially `jq`, `gojq`, `gron` and `fq` for direct usage and/or inspiration for the project.\n\n\n## Features\n* support a wide range of configuration formats and transform them interchangeably between eachother.\n* quick and comprehensive querying of configuration formats without needing a pipeline of dedicated tools.\n* provide an interactive mode for building queries with autocomplete and realtime rendering preview.\n* `qq` is broad, but performant encodings are still a priority, execution is quite fast despite covering a broad range of codecs. `qq` performs comparitively with dedicated tools for a given format.\n\n### Rough Benchmarks\nnote: these improvements generally only occur on large files and are miniscule otherwise. qq may be slower than dedicated tools for a given format, but it is pretty fast for a broad range of formats.\n\n```shell\n$ du -h large-file.json\n25M     large-file.json\n```\n\n```shell\n# gron large file bench\n\n$ time gron large-file.json --no-sort | rg -v '[1-4]' | gron --ungron --no-sort \u003e /dev/null 2\u003e\u00261\ngron large-file.json --no-sort  2.58s user 0.48s system 153% cpu 1.990 total\nrg -v '[1-4]'  0.18s user 0.24s system 21% cpu 1.991 total\ngron --ungron --no-sort \u003e /dev/null 2\u003e\u00261  7.68s user 1.15s system 197% cpu 4.475 total\n\n$ time qq -o gron large-file.json | rg -v '[1-4]' | qq -i gron \u003e /dev/null 2\u003e\u00261\nqq -o gron large-file.json  0.81s user 0.09s system 128% cpu 0.706 total\nrg -v '[1-4]'  0.02s user 0.01s system 5% cpu 0.706 total\nqq -i gron \u003e /dev/null 2\u003e\u00261  0.07s user 0.01s system 11% cpu 0.741 total\n\n# yq large file bench\n\n$ time yq large-file.json -M -o yaml \u003e /dev/null 2\u003e\u00261\nyq large-file.json -M -o yaml \u003e /dev/null 2\u003e\u00261  4.02s user 0.31s system 208% cpu 2.081 total\n\n$ time qq large-file.json -o yaml \u003e /dev/null 2\u003e\u00261\nqq large-file.json -o yaml \u003e /dev/null 2\u003e\u00261  2.72s user 0.16s system 190% cpu 1.519 total\n```\n\n## Supported Formats\n| Format      | Input          | Output         |\n|-------------|----------------|----------------|\n| JSON        | ✅ Supported   | ✅ Supported   |\n| YAML        | ✅ Supported   | ✅ Supported   |\n| TOML        | ✅ Supported   | ✅ Supported   |\n| XML         | ✅ Supported   | ✅ Supported   |\n| INI         | ✅ Supported   | ✅ Supported   |\n| HCL         | ✅ Supported   | ✅ Supported   |\n| TF          | ✅ Supported   | ✅ Supported   |\n| GRON        | ✅ Supported   | ✅ Supported   |\n| CSV         | ✅ Supported   | ✅ Supported |\n| Proto (.proto)    | ✅ Supported | ❌ Not Supported |\n| HTML        | ✅ Supported   | ✅ Supported |\n| TXT (newline)| ✅ Supported   | ❌ Not Supported |\n\n\n## Caveats\n1. `qq` is not a full `jq` replacement, some flags may or may not be supported. \n3. `qq` is under active development, more codecs in the future may be supported along with improvements to `interactive mode`.\n\n\n## Contributions\nAll contributions are welcome to `qq`, especially for upkeep/optimization/addition of new encodings.\n\n## Thanks and Acknowledgements / Related Projects\nThis tool would not be possible without the following projects, this project is arguably more of a composition of these projects than a truly original work, with glue code, some dedicated encoders/decoders, and the interactive mode being original work.\nNevertheless, I hope this project can be useful to others, and I hope to contribute back to the community with this project.\n\n* [gojq](https://github.com/itchyny/gojq): `gojq` is a pure Go implementation of jq. It is used to power the query engine of qq.\n* [fq](https://github.com/wader/fq) : fq is a `jq` like tool for querying a wide array of binary formats.\n* [jq](https://github.com/jqlang/jq): `jq` is a lightweight and flexible command-line JSON processor.\n* [gron](https://github.com/tomnomnom/gron): gron transforms JSON into discrete assignments that are easy to grep.\n* [yq](https://github.com/mikefarah/yq): yq is a lightweight and flexible command-line YAML (and much more) processor.\n* [goccy](https://github.com/goccy/go-json): goccy has quite a few encoders and decoders for various formats, and is used in the project for some encodings.\n* [go-toml](https://github.com/BurntSushi/toml): go-toml is a TOML parser for Golang with reflection.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FJFryy%2Fqq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FJFryy%2Fqq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FJFryy%2Fqq/lists"}