{"id":13416809,"url":"https://github.com/wader/fq","last_synced_at":"2025-05-14T22:05:35.136Z","repository":{"id":37064994,"uuid":"336895436","full_name":"wader/fq","owner":"wader","description":"jq for binary formats - tool, language and decoders for working with binary and text formats","archived":false,"fork":false,"pushed_at":"2025-05-07T18:58:10.000Z","size":52720,"stargazers_count":10109,"open_issues_count":53,"forks_count":232,"subscribers_count":53,"default_branch":"master","last_synced_at":"2025-05-14T22:04:32.751Z","etag":null,"topics":["binary","cli","decode","go","golang","hexdump","jq","json","pcap","query","repl","tls","xml"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wader.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-02-07T21:37:53.000Z","updated_at":"2025-05-14T11:22:56.000Z","dependencies_parsed_at":"2023-09-23T04:11:36.060Z","dependency_job_id":"691f37de-cd4e-4d28-8518-e9234661b110","html_url":"https://github.com/wader/fq","commit_stats":{"total_commits":1842,"total_committers":38,"mean_commits":"48.473684210526315","dds":0.4315960912052117,"last_synced_commit":"de43e358e68e063624ec16e096e210e270a0bfd3"},"previous_names":[],"tags_count":25,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wader%2Ffq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wader%2Ffq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wader%2Ffq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wader%2Ffq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wader","download_url":"https://codeload.github.com/wader/fq/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254235687,"owners_count":22036962,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["binary","cli","decode","go","golang","hexdump","jq","json","pcap","query","repl","tls","xml"],"created_at":"2024-07-30T22:00:22.565Z","updated_at":"2025-05-14T22:05:35.076Z","avatar_url":"https://github.com/wader.png","language":"Go","readme":"# fq\n\nTool, language and decoders for working with binary data.\n\nTLDR: it aims to be jq, hexdump, dd and gdb for files combined into one.\n\n![fq demo](doc/demo.svg)\n\nBasic usage is `fq . file`, `fq d file` or `fq 'some query' file ...`.\n\nFor details see [usage.md](doc/usage.md).\n\n### Background\n\nfq is inspired by the [jq](https://jqlang.github.io/jq/) tool and language and allows you to work with binary formats in the same way. In addition to using jq expressions it can also present decoded tree structures, transform, slice and concatenate binary data. It also supports nested formats and features an interactive REPL with auto-completion of functions and names.\n\nIt was originally designed to query, inspect and debug media codecs and containers like MP4, FLAC and JPEG but has since been extended to support a variety of formats like executables, packet captures (with TCP reassembly) and serialization formats like JSON, YAML, XML, CBOR, protobuf. In addition it also has functions to work with URLs, convert to/from hex, number bases, search for patterns etc.\n\n### Goals\n\n- Make binaries more accessible, queryable and sliceable.\n- Nested formats and bit-oriented decoding.\n- Quick and comfortable CLI tool.\n- Bits and bytes transformations.\n\n### Hopes\n\n- Make it useful enough that people want to help improve it.\n- Inspire people to create similar tools.\n\n### Supported formats\n\n[fq -rn -L doc 'include \"formats\"; formats_list']: sh-start\n\n[aac_frame](doc/formats.md#aac_frame),\nadts,\nadts_frame,\naiff,\namf0,\napev2,\n[apple_bookmark](doc/formats.md#apple_bookmark),\nar,\n[asn1_ber](doc/formats.md#asn1_ber),\nav1_ccr,\nav1_frame,\nav1_obu,\navc_annexb,\n[avc_au](doc/formats.md#avc_au),\navc_dcr,\navc_nalu,\navc_pps,\navc_sei,\navc_sps,\n[avi](doc/formats.md#avi),\n[avro_ocf](doc/formats.md#avro_ocf),\n[bencode](doc/formats.md#bencode),\nbitcoin_blkdat,\n[bitcoin_block](doc/formats.md#bitcoin_block),\nbitcoin_script,\nbitcoin_transaction,\n[bits](doc/formats.md#bits),\n[bplist](doc/formats.md#bplist),\nbsd_loopback_frame,\n[bson](doc/formats.md#bson),\n[bytes](doc/formats.md#bytes),\nbzip2,\n[caff](doc/formats.md#caff),\n[cbor](doc/formats.md#cbor),\n[csv](doc/formats.md#csv),\ndns,\ndns_tcp,\nelf,\nether8023_frame,\nexif,\nfairplay_spc,\n[fit](doc/formats.md#fit),\nflac,\n[flac_frame](doc/formats.md#flac_frame),\nflac_metadatablock,\nflac_metadatablocks,\nflac_picture,\nflac_streaminfo,\ngif,\ngzip,\nhevc_annexb,\n[hevc_au](doc/formats.md#hevc_au),\nhevc_dcr,\nhevc_nalu,\nhevc_pps,\nhevc_sps,\nhevc_vps,\n[html](doc/formats.md#html),\nicc_profile,\nicmp,\nicmpv6,\nid3v1,\nid3v11,\nid3v2,\nipv4_packet,\nipv6_packet,\njp2c,\njpeg,\njson,\njsonl,\n[leveldb_descriptor](doc/formats.md#leveldb_descriptor),\n[leveldb_log](doc/formats.md#leveldb_log),\n[leveldb_table](doc/formats.md#leveldb_table),\n[luajit](doc/formats.md#luajit),\n[macho](doc/formats.md#macho),\nmacho_fat,\n[markdown](doc/formats.md#markdown),\n[matroska](doc/formats.md#matroska),\n[midi](doc/formats.md#midi),\n[moc3](doc/formats.md#moc3),\n[mp3](doc/formats.md#mp3),\nmp3_frame,\nmp3_frame_vbri,\nmp3_frame_xing,\n[mp4](doc/formats.md#mp4),\nmpeg_asc,\nmpeg_es,\nmpeg_pes,\nmpeg_pes_packet,\nmpeg_spu,\nmpeg_ts,\n[msgpack](doc/formats.md#msgpack),\n[negentropy](doc/formats.md#negentropy),\n[nes](doc/formats.md#nes),\nogg,\nogg_page,\n[opentimestamps](doc/formats.md#opentimestamps),\nopus_packet,\n[pcap](doc/formats.md#pcap),\npcapng,\n[pg_btree](doc/formats.md#pg_btree),\n[pg_control](doc/formats.md#pg_control),\n[pg_heap](doc/formats.md#pg_heap),\npng,\nprores_frame,\n[protobuf](doc/formats.md#protobuf),\nprotobuf_widevine,\npssh_playready,\n[rtmp](doc/formats.md#rtmp),\nsll2_packet,\nsll_packet,\n[tap](doc/formats.md#tap),\ntar,\ntcp_segment,\ntiff,\n[tls](doc/formats.md#tls),\ntoml,\n[tzif](doc/formats.md#tzif),\n[tzx](doc/formats.md#tzx),\nudp_datagram,\nvorbis_comment,\nvorbis_packet,\nvp8_frame,\nvp9_cfm,\nvp9_frame,\nvpx_ccr,\n[wasm](doc/formats.md#wasm),\nwav,\nwebp,\n[xml](doc/formats.md#xml),\nyaml,\n[zip](doc/formats.md#zip)\n\n[#]: sh-end\n\nIt can also work with some common text formats like URLs, hex, base64, PEM etc and for some serialization formats like XML, YAML, etc. it can transform both from and to jq values.\n\nFor details see [formats.md](doc/formats.md) and [usage.md](doc/usage.md).\n\n## Presentations and media\n\n- [PBS Tidbit 8 of Y: Interview with jq Maintainer Mattias Wadman](https://pbs.bartificer.net/tidbit8) - English podcast episode about jq and some fq.\n- [Kodsnack 585 - Polymorfisk JSON](https://kodsnack.se/585/) - Swedish podcast episode about jq and fq\n- \"fq - jq for binary formats\" at [FOSDEM 2023](https://fosdem.org/2023/) - [video \u0026 slides](https://fosdem.org/2023/schedule/event/bintools_fq/)\n- \"fq - jq for binary formats\" at [No time to wait 6](https://mediaarea.net/NoTimeToWait6) - [video](https://www.youtube.com/watch?v=-Pwt5KL-xRs\u0026t=1450s) - [slides](doc/presentations/nttw6/fq-nttw6-slides.pdf)\n- \"fq - jq for binary formats\" at [Binary Tools Summit 2022](https://binary-tools.net/summit.html) - [video](https://www.youtube.com/watch?v=GJOq_b0eb-s\u0026list=PLTj8twuHdQz-JcX7k6eOwyVPDB8CyfZc8\u0026index=1) - [slides](doc/presentations/bts2022/fq-bts2022-v1.pdf)\n\n## Install\n\nUse one of the methods listed below or download a pre-built [release](https://github.com/wader/fq/releases) for macOS, Linux or Windows. Unarchive it and move the executable to `PATH` etc.\n\nOn macOS if you don't install using one of the method below then you might have to manually allow the binary to run. This can be done by trying to run the binary, ignore the warning and then go into security preference and allow it. Same can be done with these commands:\n\n```sh\nxattr -d com.apple.quarantine fq\n# for macOS version before Sequoia you might need this also\nspctl --add fq\n```\n\n### Homebrew (macOS)\n\n```sh\nbrew install wader/tap/fq\n```\n\n### MacPorts\n\nOn macOS, `fq` can also be installed via [MacPorts](https://www.macports.org).  More details [here](https://ports.macports.org/port/fq/).\n\n```sh\nsudo port install fq\n```\n\n### Windows\n\n`fq` can be installed via [scoop](https://scoop.sh/).\n\n```powershell\nscoop install fq\n```\n\n### Arch Linux\n\n`fq` can be installed from the [extra repository](https://archlinux.org/packages/extra/x86_64/fq/) using [pacman](https://wiki.archlinux.org/title/Pacman):\n\n```sh\npacman -S fq\n```\n\nYou can also build and install the development (VCS) package using an [AUR helper](https://wiki.archlinux.org/index.php/AUR_helpers):\n\n```sh\nparu -S fq-git\n```\n\n### Nix\n\n```sh\nnix-shell -p fq\n```\n\n### FreeBSD\n\nUse the [fq](https://cgit.freebsd.org/ports/tree/misc/fq) port.\n\n### Alpine\n\nCurrently in edge testing but should work fine in stable also.\n\n```\napk add -X http://dl-cdn.alpinelinux.org/alpine/edge/testing fq\n```\n\n### Build from source\n\nMake sure you have [go](https://go.dev) 1.22 or later installed.\n\nTo install directly from git repository (no git clone needed):\n```sh\n# build and install latest release\ngo install github.com/wader/fq@latest\n\n# build and install latest master\ngo install github.com/wader/fq@master\n\n# copy binary to $PATH if needed\ncp \"$(go env GOPATH)/bin/fq\" /usr/local/bin\n```\n\nTo build, run and test from source:\n```sh\n# build and run\ngo run .\n# build and run with arguments\ngo run . -d mp3 . file.mp3\n# just build\ngo build -o fq .\n# run all tests and build binary\nmake test fq\n```\n\n## Development and adding a new decoder\n\nSee [dev.md](doc/dev.md)\n\n## Thanks and related projects\n\nThis project would not have been possible without [itchyny](https://github.com/itchyny)'s\njq implementation [gojq](https://github.com/itchyny/gojq). I also want to thank\n[HexFiend](https://github.com/HexFiend/HexFiend) for inspiration and ideas and [stedolan](https://github.com/stedolan)\nfor inventing the [jq](https://github.com/stedolan/jq) language.\n\n### Similar or related works\n\n#### Tools\n\n- [HexFiend](https://github.com/HexFiend/HexFiend) - Hex editor for macOS with format template support.\n- [ImHex](https://github.com/WerWolv/ImHex) - A Hex Editor for Reverse Engineers.\n- [binspector](https://github.com/binspector/binspector) - Binary format analysis tool with query language and REPL.\n- [kaitai](https://kaitai.io) - Declarative binary format parsing.\n- [Wireshark](https://www.wireshark.org) - Decodes network traffic (tip: `tshark -T json`).\n- [MediaInfo](https://mediaarea.net/en/MediaInfo) - Analyze media files  (tip `mediainfo --Output=JSON` and `mediainfo --Details=1`).\n- [GNU poke](https://www.jemarch.net/poke) - The extensible editor for structured binary data.\n- [ffmpeg/ffprobe](https://ffmpeg.org) - Powerful media libraries and tools.\n- [hexdump](https://git.kernel.org/pub/scm/utils/util-linux/util-linux.git/tree/text-utils/hexdump.c) - Hex viewer tool.\n- [hex](https://git.janouch.name/p/hex) - Interactive hex viewer with format support via lua.\n- [hachoir](https://github.com/vstinner/hachoir) - General python library for working binary data.\n- [scapy](https://scapy.net) - Decode/Encode formats, focus on network protocols.\n\n#### Projects and Standards\n\n- [Let's Solve the File Format Problem](http://fileformats.archiveteam.org).\n- [PRONOM](https://www.nationalarchives.gov.uk/PRONOM/) file format registry.\n- [Sustainability of Digital Formats](https://www.loc.gov/preservation/digital/formats/) at Library of Congress.\n- [Data Format Description Language (DFDL)](https://en.wikipedia.org/wiki/Data_Format_Description_Language).\n\n## TODO and ideas\n\nSee [TODO.md](doc/TODO.md)\n\n## License\n\n`fq` is distributed under the terms of the MIT License.\n\nSee the [LICENSE](LICENSE) file for license details.\n\nLicenses of direct dependencies:\n\n- Forked version of gojq - https://github.com/itchyny/gojq/blob/main/LICENSE (MIT)\n- github.com/ergochat/readline - https://github.com/ergochat/readline/blob/master/LICENSE (MIT)\n- github.com/BurntSushi/toml - https://github.com/BurntSushi/toml/blob/master/COPYING (MIT)\n- github.com/creasty/defaults - https://github.com/creasty/defaults/blob/master/LICENSE (MIT)\n- github.com/gomarkdown/markdown - https://github.com/gomarkdown/markdown/blob/master/LICENSE.txt (BSD)\n- github.com/gopacket/gopacket - https://github.com/gopacket/gopacket/blob/master/LICENSE (BSD)\n- github.com/mitchellh/copystructure - https://github.com/mitchellh/copystructure/blob/master/LICENSE (MIT)\n- github.com/mitchellh/mapstructure - https://github.com/mitchellh/mapstructure/blob/master/LICENSE (MIT)\n- github.com/pmezard/go-difflib - https://github.com/pmezard/go-difflib/blob/master/LICENSE (BSD)\n- golang/snappy - https://github.com/golang/snappy/blob/master/LICENSE (BSD)\n- golang/x/* - https://github.com/golang/text/blob/master/LICENSE (BSD)\n- gopkg.in/yaml.v3 - https://github.com/go-yaml/yaml/blob/v3/LICENSE (MIT)\n- Parts of go crypto/tls and github.com/zmap/zcrypto - https://github.com/zmap/zcrypto/blob/master/LICENSE (Apache)\n","funding_links":[],"categories":["Popular","HarmonyOS","Go","开源类库","Awesome Repositories","Open source library","\u003ca name=\"data\"\u003e\u003c/a\u003edata","Repositories"],"sub_categories":["Windows Manager","未归类","ELF binary format","Not Categorized"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwader%2Ffq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwader%2Ffq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwader%2Ffq/lists"}