{"id":29020307,"url":"https://github.com/jonathan-gramain/bitpunch","last_synced_at":"2025-06-26T01:05:18.095Z","repository":{"id":222175637,"uuid":"92790670","full_name":"jonathan-gramain/bitpunch","owner":"jonathan-gramain","description":"Swiss Army knife for arbitrarily structured binary files","archived":false,"fork":false,"pushed_at":"2019-06-12T23:15:33.000Z","size":2955,"stargazers_count":9,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-02-12T19:39:06.481Z","etag":null,"topics":["binaryformat","c","development-utility","parsing-engine","python","recovery"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jonathan-gramain.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING.BSD","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null}},"created_at":"2017-05-30T02:13:53.000Z","updated_at":"2024-02-12T19:39:10.960Z","dependencies_parsed_at":"2024-02-12T19:51:40.287Z","dependency_job_id":null,"html_url":"https://github.com/jonathan-gramain/bitpunch","commit_stats":null,"previous_names":["jonathan-gramain/bitpunch"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jonathan-gramain/bitpunch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonathan-gramain%2Fbitpunch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonathan-gramain%2Fbitpunch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonathan-gramain%2Fbitpunch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonathan-gramain%2Fbitpunch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jonathan-gramain","download_url":"https://codeload.github.com/jonathan-gramain/bitpunch/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonathan-gramain%2Fbitpunch/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261978910,"owners_count":23239417,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["binaryformat","c","development-utility","parsing-engine","python","recovery"],"created_at":"2025-06-26T01:05:16.977Z","updated_at":"2025-06-26T01:05:18.084Z","avatar_url":"https://github.com/jonathan-gramain.png","language":"C","readme":"# BitPunch\n\n*A Swiss Army knife for binary data files*\n\nBitPunch is a core library and a set of tools that allow a user to\naccess meaningful data from arbitrary binary files.\n\nBitPunch schemas form a descriptive, declarative language used to\ndescribe the structure of the data stored in the files, and schemas\ncan express a wide range of binary structures.\n\nOnce a schema is defined, APIs in various programming languages can be\nused to access data in an object-oriented way using path expressions\nor iteration. Currently APIs in C and Python2 are available.\n\n## What for?\n\n- to convert binary data coming from custom binary formats into\n  human-friendly representations like JSON with minimal effort\n  (e.g. for interoperable data exchange);\n\n- to check sanity of files and be able to recover sane data from\n  corrupt files that cannot be read with standard tools;\n\n- to implement a reliable and safe parser for some binary format\n  instead of writing it directly in a low-level language like C;\n\n- for reverse-engineering of uncommonly used or proprietary file\n  formats, and extract data from them or translate to a better data\n  interchange format like JSON;\n\n- for writing a variety of applications that can benefit from knowing\n  the structure of the files they manipulate. For example, a\n  structure-aware hex editor that can highlight or collapse sections\n  in a tree structure that maps to the binary schema;\n\n- because it's cool to find out about all the crap that is in your\n  favorite MP4 videos using the CLI! :)\n\n## Development status\n\nThe current code is to be considered in alpha stage:\n\n- heavy development ongoing\n\n- syntax of BP files likely to evolve\n\n\n## Overview\n\nRight now bitpunch is made of:\n\n- a C library (libbitpunch) that contains low-level C routines to:\n\n  - parse schema specifications found in .bp files\n\n  - open binary files to expose their contents to filters defined in\n    .bp files\n\n- a Python module that allows accessing raw data through an object\n  model\n\n- a CLI with analysis commands to access binary files in an\n  interactive fashion, with powerful autocompletion support and inline\n  help. The CLI relies on the Python library.\n\n\nIt understands a custom specialized syntax (let's call it the BitPunch\nsyntax, or BP syntax) that describes precisely the layout of the\nbinary format. Once loaded, binary files can be opened and mapped to\nthe described layout, so that their meaningful content becomes readily\navailable through iteration and path expressions.\n\nIt's meant to *be able to* cover as many as possible of the numerous\nbinary formats in existence, which is possible if a conformant .bp\nfile *can be* written for those formats. This is a key design goal\nbecause esoteric or legacy formats are notoriously hard to exploit by\nthe lack of available tools to do it, or lack of support on modern\nOSes and languages. With some effort, even undocumented proprietary\nformats should be exploitable with some reverse-engineering and help\nfrom bitpunch tooling.\n\nIt shall support very large binary files without trouble, since mmap()\nis used to map the file contents into memory, and browsing specific\nelements does not require to browse the entire file first, information\nis gathered in a lazy fashion. (The other side of the coin is that you\nmay be notified of a format error only when you attempt to access\nitems affected by the error).\n\n## More\n\n- Install instructions and basic usage in [INSTALL.md](INSTALL.md)\n\n- For details about the BP file format, see [BP.md](doc/BP.md)\n\n- A list of planned features is in [TODO.md](TODO.md)\n\n## History\n\nThe project started in Boston, Massachusetts in May 2015.\n\nThe original motivation was to investigate a compatibility issue with\nMP4 files on an embedded device, and the tool was meant to help in\nunderstanding the format and comparing files in their detailed\nlayout. It then evolved in scope towards a general-purpose tool for\nany kind of structured binary file.\n\n## License\n\n- The libbitpunch C library and the bitpunch Python package (the\n  Python bindings) are distributed under a [3-Clause BSD\n  License](COPYING.BSD).\n\n- The bitpunch Command-Line tool and its helper Python package\n  \"nestedcmd\" are distributed under a [GPL v3 license](COPYING.GPLv3).\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonathan-gramain%2Fbitpunch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjonathan-gramain%2Fbitpunch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonathan-gramain%2Fbitpunch/lists"}