{"id":28195889,"url":"https://github.com/matrixeditor/caterpillar","last_synced_at":"2026-02-01T12:11:25.390Z","repository":{"id":290210413,"uuid":"733602499","full_name":"MatrixEditor/caterpillar","owner":"MatrixEditor","description":"A Python 3.12+ library to pack and unpack structured binary data.","archived":false,"fork":false,"pushed_at":"2026-01-30T22:44:18.000Z","size":7609,"stargazers_count":34,"open_issues_count":5,"forks_count":3,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-01-31T14:30:59.668Z","etag":null,"topics":["binary-format","binary-parsing","file-parsing","pystruct","python","python-struct","reverse-engineering","struct","unpacking"],"latest_commit_sha":null,"homepage":"https://matrixeditor.github.io/caterpillar/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MatrixEditor.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-12-19T17:41:07.000Z","updated_at":"2026-01-09T14:42:49.000Z","dependencies_parsed_at":null,"dependency_job_id":"39cdcf55-16f5-471f-ae86-464d1bd96468","html_url":"https://github.com/MatrixEditor/caterpillar","commit_stats":null,"previous_names":["matrixeditor/caterpillar"],"tags_count":14,"template":false,"template_full_name":null,"purl":"pkg:github/MatrixEditor/caterpillar","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MatrixEditor%2Fcaterpillar","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MatrixEditor%2Fcaterpillar/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MatrixEditor%2Fcaterpillar/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MatrixEditor%2Fcaterpillar/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MatrixEditor","download_url":"https://codeload.github.com/MatrixEditor/caterpillar/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MatrixEditor%2Fcaterpillar/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28977806,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-01T11:31:13.034Z","status":"ssl_error","status_checked_at":"2026-02-01T11:30:25.558Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["binary-format","binary-parsing","file-parsing","pystruct","python","python-struct","reverse-engineering","struct","unpacking"],"created_at":"2025-05-16T14:14:50.912Z","updated_at":"2026-02-01T12:11:25.384Z","avatar_url":"https://github.com/MatrixEditor.png","language":"Python","readme":"# Caterpillar - 🐛\n\n[![python](https://img.shields.io/badge/Python-3.12+-blue?logo=python\u0026logoColor=yellow)](https://www.python.org/downloads/)\n![![Latest Version](https://pypi.org/project/caterpillar-py/)](https://img.shields.io/github/v/release/MatrixEditor/caterpillar.svg?logo=github\u0026label=Latest+Version)\n[![Build and Deploy Docs](https://github.com/MatrixEditor/caterpillar/actions/workflows/python-sphinx.yml/badge.svg)](https://github.com/MatrixEditor/caterpillar/actions/workflows/python-sphinx.yml)\n[![Run Tests](https://github.com/MatrixEditor/caterpillar/actions/workflows/python-test.yml/badge.svg)](https://github.com/MatrixEditor/caterpillar/actions/workflows/python-test.yml)\n![GitHub issues](https://img.shields.io/github/issues/MatrixEditor/caterpillar?logo=github)\n![GitHub License](https://img.shields.io/github/license/MatrixEditor/caterpillar?logo=github)\n\n\nCaterpillar is a Python 3.12+ library to pack and unpack structurized binary\ndata (with support for 3.10+). It enhances the capabilities of [Python Struct](https://docs.python.org/3/library/struct.html)\nby enabling direct class declaration. More information about the different configuration\noptions will be added in the future. Documentation is [here \u003e](https://matrixeditor.github.io/caterpillar/).\n\n*Caterpillar* is able to:\n\n* Pack and unpack data just from processing Python class definitions (including support for powerful bitfields, c++-like templates and c-like unions!),\n* apply a wide range of data types (with endianess and architecture configuration),\n* dynamically adapt structs based on their inheritance layout,\n* reduce the used memory space using `__slots__`,\n* allowing you to place conditional statements into class definitions,\n* insert proper types into the class definition to support documentation and\n* it helps you to create cleaner and more compact code.\n* There is also a feature that lets you dynamically change the endian within a struct!\n* You can even extend Caterpillar and write your parsing logic in C or C++\n* All struct definitions can be typing compliant!!! (tested with pyright)\n\n## Give me some code!\n\n*The following code is typing compliant, meaning your static type checker won't*\n*scream at you when developing with this code*.\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003ci\u003eIf you want to check out the default syntax, open this block.\u003c/i\u003e\u003c/summary\u003e\n\n```python\nfrom caterpillar.py import *\nfrom caterpillar.types import *\n\n@bitfield(order=LittleEndian)\nclass Header:\n    version : 4                   # 4bit integer\n    valid   : 1                   # 1bit flag (boolean)\n    ident   : (8, CharFactory)    # 8bit char\n    # automatic alignment to 16bits\n\nTHE_KEY = b\"ITS MAGIC\"\n\n@struct(order=LittleEndian, kw_only=True)\nclass Format:\n    magic  : THE_KEY                      # Supports string and byte constants directly\n    header : Header\n    a      : uint8                        # Primitive data types\n    b      : Dynamic + int32              # dynamic endian based on global config\n    length : uint8                        # String fields with computed lengths\n    name   : String(this.length)          #  -\u003e you can also use Prefixed(uint8)\n\n    # custom actions, e.g. for hashes\n    _hash_begin : DigestField.begin(\"hash\", Md5_Algo)\n    # Sequences with prefixed, computed lengths    -+ part of the MD5 hash\n    names       : CString[uint8::]               #  |\n    #                                              -+\n    # automatic hash creation and verification + default value\n    hash        : Md5_Field(\"hash\", verify=True)\n\n# Creation, packing and unpacking remains the same\n```\n\n\u003c/details\u003e\n\n```python\nfrom caterpillar.py import *\nfrom caterpillar.types import *\n\n@bitfield(order=LittleEndian)\nclass Header:\n    version : int4_t                   # 4bit integer\n    valid   : int1_t                   # 1bit flag (boolean)\n    ident   : f[str, (8, CharFactory)] # 8bit char\n    # automatic alignment to 16bits\n\nTHE_KEY = b\"ITS MAGIC\"\n\n@struct(order=LittleEndian, kw_only=True)\nclass Format:\n    magic  : f[bytes, THE_KEY] = THE_KEY  # Supports string and byte constants directly\n    header : Header\n    a      : uint8_t                      # Primitive data types\n    b      : f[int, Dynamic + int32]      # dynamic endian based on global config\n    length : uint8_t                      # String fields with computed lengths\n    name   : f[str, String(this.length)]  #  -\u003e you can also use Prefixed(uint8)\n\n    # custom actions, e.g. for hashes\n    _hash_begin : f[None, DigestField.begin(\"hash\", Md5_Algo)] = None\n    # Sequences with prefixed, computed lengths    -+ part of the MD5 hash\n    names       : f[list[str], CString[uint8::]] #  |\n    #                                              -+\n    # automatic hash creation and verification + default value\n    hash        : f[bytes, Md5_Field(\"hash\", verify=True)] = b\"\"\n\n# Creation (keyword-only arguments, magic is auto-inferred):\nobj = Format(\n    header=Header(version=2, valid=True, ident=\"F\"),\n    a=1,\n    b=2,\n    length=3,\n    name=\"foo\",\n    names=[\"a\", \"b\"]\n)\n\n# Packing the object; reads as 'PACK obj FROM Format'\n# objects of struct classes can be packed right away\ndata_le = pack(obj, Format)\n# results in: b'ITS MAGIC0*\\x01\\x02\\x00\\x00\\x00\\x03foo\\x02a\\x00b\\x00)\\x9a...'\n\n# Unpacking the binary data, reads as 'UNPACK Format FROM blob'\nobj2 = unpack(Format, data_le)\nassert obj2.names == obj.names\n\n# to pack with a different endian for fields 'a' and 'b', use 'order'\ndata_be = pack(obj, Format, order=BigEndian)\nassert data_le != data_be\n```\n\n\u003e [!NOTE]\n\u003e Python 3.14 breaks `with` statements in class definitions since `__annotations__` are added at the end\n\u003e of a class definition. Therefore, `Digest` and conditional statements **ARE NOT SUPPORTED** using the `with` syntax in Python 3.14+.\n\u003e As of version `2.4.5` the `Digest` class has a counterpart (`DigestField`), which can be used to manually specify a digest without\n\u003e the need of a `ẁith` statement.\n\nThis library offers extensive functionality beyond basic struct definitions. For further details\non its powerful features, explore the official [documentation](https://matrixeditor.github.io/caterpillar/),\n[examples](./examples/), and [test cases](./test/).\n\n\n## Installation\n\n\u003e [!NOTE]\n\u003e As of Caterpillar v2.1.2 it is possible to install the library without the need of\n\u003e compiling the C extension.\n\n### PIP installation (Python-only)\n\n```bash\npip install caterpillar-py\n```\n\n### Python-only installation\n\n```bash\npip install \"caterpillar[all]@git+https://github.com/MatrixEditor/caterpillar\"\n```\n\n### Installation + C-extension\n\n```bash\npip install \"caterpillar[all]@git+https://github.com/MatrixEditor/caterpillar/#subdirectory=src/ccaterpillar\"\n```\n\n\n## Starting Point\n\nPlease visit the [Documentation](https://matrixeditor.github.io/caterpillar/), it contains a complete tutorial on how to use this library.\n\n## Other Approaches\n\nA list of similar approaches to parsing structured binary data with Python can be taken from below:\n\n* [construct](https://github.com/construct/construct)\n* [kaitai_struct](https://github.com/kaitai-io/kaitai_struct)\n* [hachoir](https://hachoir.readthedocs.io/en/latest/)\n* [mrcrowbar](https://github.com/moralrecordings/mrcrowbar)\n\nThe documentation also provides a [Comparison](https://matrixeditor.github.io/caterpillar/reference/introduction.html#comparison)\nto these approaches.\n\n## License\n\nDistributed under the GNU General Public License (V3). See [License](LICENSE) for more information.","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatrixeditor%2Fcaterpillar","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmatrixeditor%2Fcaterpillar","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatrixeditor%2Fcaterpillar/lists"}