{"id":13494082,"url":"https://github.com/waszil/pyembc","last_synced_at":"2025-03-28T13:32:06.029Z","repository":{"id":62580115,"uuid":"237062461","full_name":"waszil/pyembc","owner":"waszil","description":"Declarative library for for describing embedded C data types in python","archived":false,"fork":false,"pushed_at":"2020-02-04T11:25:21.000Z","size":75,"stargazers_count":9,"open_issues_count":1,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-09-18T01:10:16.728Z","etag":null,"topics":["binary-parsing","codegeneration","ctypes","ctypes-wrapper","declarative","embedded-c","python3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/waszil.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-01-29T19:16:10.000Z","updated_at":"2023-10-03T10:05:15.000Z","dependencies_parsed_at":"2022-11-03T21:01:05.374Z","dependency_job_id":null,"html_url":"https://github.com/waszil/pyembc","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waszil%2Fpyembc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waszil%2Fpyembc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waszil%2Fpyembc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waszil%2Fpyembc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/waszil","download_url":"https://codeload.github.com/waszil/pyembc/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":222382408,"owners_count":16975377,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["binary-parsing","codegeneration","ctypes","ctypes-wrapper","declarative","embedded-c","python3"],"created_at":"2024-07-31T19:01:21.675Z","updated_at":"2024-10-31T08:31:44.428Z","avatar_url":"https://github.com/waszil.png","language":"Python","readme":"# pyembc\n\nDeclarative library for for describing embedded C data types in python\n\n## Motivation\n\n`pyembc` is a wrapper above the `ctypes` library providing a more simple and declarative syntax,\nwhat I find more easy to use.\n\nThe motivation behind creating this library is to be able to write down a c structure/union very\nsimilarly as its written down in a c file, and then to be able to get the memory contents of\nsuch a data structure, that can be transferred to an embedded device with a direct memory\naccess method (like XCP) and in the other way as well, and also to be able to generate the actual\nc code that describes the same data structure, from the python description.\n\nI have used several similar libraries, such as [construct](https://construct.readthedocs.io/en/latest/),\n[structures](https://github.com/malinoff/structures), or even Protocol Buffers, however one was too slow,\nthe other could not generate code, the third is waaay too big, and actually what I needed was very\nvery limited, and that's exactly the scope of this library. Also, it was fun to write, and beware,\nI use `exec` under the hood as well! Yes, I know.\n\n## Examples\n\n### Declaring structures / unions\n\nstructures and unions can be declared in a similar way as with `dataclasses`:\n\n```python\nfrom pyembc import pyembc_struct, pyembc_union\nfrom ctypes import *\n\n@pyembc_struct\nclass Inner:\n    a: c_uint8\n    b: c_uint8\n\n@pyembc_struct\nclass Outer:\n    first: Inner\n    second: c_uint8\n    third: c_uint8\n    \n@pyembc_union\nclass MyUnion:\n    as_struct: Outer\n    as_int: c_uint32\n```\n\n### Instantiating\n\nThe instances of the above declared classes can be created as shown below.\n\n```python\n# empty constructor\ninner = Inner()\nprint(inner)\n\u003e\u003e\u003e Inner(a:u8=0x0, b:u8=0x0)\n\n# constructor with default values\ninner = Inner(a=1, b=2)\nprint(inner)\n\u003e\u003e\u003e Inner(a:u8=0x1, b:u8=0x2)\n\n# value checking\ninner = Inner(a=256, b=300)\n\u003e\u003e\u003e ValueError: 256 cannot be set for c_ubyte (error('ubyte format requires 0 \u003c= number \u003c= 255'))!\n\n# embedded structures\nouter = Outer()\n\u003e\u003e\u003e Outer(first=Inner(a:u8=0x0, b:u8=0x0), second:u8=0x0, third:u8=0x0)\n\nouter = Outer(Inner(42, 43), 1, 2)\n\u003e\u003e\u003e Outer(first=Inner(a:u8=0x2A, b:u8=0x2B), second:u8=0x1, third:u8=0x2)\n\n# creating a union instance\nmy_union = MyUnion()\n\u003e\u003e\u003e MyUnion(as_struct=Outer(first=Inner(a:u8=0x0, b:u8=0x0), second:u8=0x0, third:u8=0x0), as_int:u32=0x0)\n\n# with defaults\nmy_union = MyUnion(as_struct=Outer(Inner(1,2), 3,4))\n\u003e\u003e\u003e MyUnion(as_struct=Outer(first=Inner(a:u8=0x1, b:u8=0x2), second:u8=0x3, third:u8=0x4), as_int:u32=0x4030201)\n```\n\n### Setting field/member values\n\nValue setting is protected for fields:\n\n```python\nouter.second = 0x1234\n\u003e\u003e\u003e ValueError: 4660 cannot be set for c_ubyte (error('ubyte format requires 0 \u003c= number \u003c= 255'))!\n```\n\n### Parsing from binary data\n\n```python\nouter = Outer(first=Inner(a=1, b=2), second=3)\nprint(outer)\n\u003e\u003e\u003e Outer(first=Inner(a:u8=0x1, b:u8=0x2), second:u8=0x3)\n\nouter.parse(b'\\x11\\x22\\x33')\nprint(outer)\n\u003e\u003e\u003e Outer(first=Inner(a:u8=0x11, b:u8=0x22), second:u8=0x33)\n```\n\n### Packing of structures\n\nThe structures are by default packed to 4 bytes. This means, that empty fill bytes are added\nbetween struct members to align them to the 4 byte boundaries.\nThis packing can be modified by the user.\n\n```python\n@pyembc_struct\nclass Inner:\n    a: c_uint8\n    \n@pyembc_struct\nclass Outer:\n    first: Inner\n    second: c_uint32\n    \nouter = Outer(first=Inner(1), second = 0xFFEEDDCC)\nouter.stream()\n\u003e\u003e\u003e b'\\x01\\x00\\x00\\x00\\xcc\\xdd\\xee\\xff'\n```\n\nIf we define the outer structure as below, the fill bytes will disappear\n\n```python\n@pyembc_struct(pack=1)\nclass Outer:\n    first: Inner\n    second: c_uint32\n    \nouter = Outer(first=Inner(1), second = 0xFFEEDDCC)\nouter.stream()\n\u003e\u003e\u003e b'\\x01\\xcc\\xdd\\xee\\xff'\n```\n\nNote: this is true for the parse() method as well! \n\n### Endianness\n\nThe default endianness / byteorder is the one of the system's. (`sys.byteorder`).\nHowever, it can be adjusted in the decorators.\n\n```python\n@pyembc_struct(endian=\"little\")\nclass Little:\n    a: c_uint16\n\nlittle = Little(a=0xFF00)\nlittle.stream()\n\u003e\u003e\u003e b'\\x00\\xff'\n\n@pyembc_struct(endian=\"big\")\nclass Big:\n    a: c_uint16\n\nbig = Big(a=0xFF00)\nbig.stream()\n\u003e\u003e\u003e b'\\xff\\x00'\n```\n\nHowever, for now, unions only support native byteorder, as the ctypes module does not\nimplement the appropriate BigEndianUnion and LittleEndianUnion types.\nSee details:\n\nhttps://stackoverflow.com/questions/49524952/bigendianunion-is-not-part-of-pythons-ctypes\nhttps://bugs.python.org/issue33178\n\n### Bitfields\n\nBitfields can be defined with the following syntax:\n\n```python\n@pyembc_struct\nclass S:\n    a: (c_uint8, 2)\n    b: (c_uint8, 6)\n\ns = S()\nlen(s)\n\u003e\u003e\u003e 1\n\nprint(s)\n\u003e\u003e\u003e S(a:u8@2=0x0, b:u8@6=0x0)\n\ns.parse(b'\\xAA')\nprint(s)\n\u003e\u003e\u003e S(a:u8@2=0x2, b:u8@6=0x2A)\n```\n\nThe parsing and streaming works for them just like for normal structures.\n\n#### Bitfield definition order\n\nNote, that just as in c, the definition order of the bitfields in one byte\ndepends on the byteorder of the containing structure.\n\nThis means, that for a little-endian structure, the bitfields inside a byte shall be\ndefined from top-down as LSB to MSB, however, for big-endian structures, the top-down\norder means MSB to LSB. See the example below, these two bitfield structures describe\nthe same thing, note the change in the order of the bitfields!\n\n```python\n@pyembc_struct(endian=\"little\")\nclass BF_LE:\n    # byte 0\n    a: (c_uint8, 3)     # LSB\n    b: (c_uint8, 5)     # MSB\n    # byte 1\n    c: c_uint8\n    \n@pyembc_struct(endian=\"big\")\nclass BF_BE:\n    # byte 0\n    b: (c_uint8, 5)  # MSB\n    a: (c_uint8, 3)  # LSB\n    # byte 1\n    c: c_uint8\n```\n\n### Generating c code\n\nThe ANSI c representation of a structure/union can be created from the class itself\nor from its instance. The `ccode()` static method returns a list of lines.\n\n```python\nprint('\\n'.join(Outer.ccode()))\n# or\nprint('\\n'.join(outer.ccode()))\n# or\nouter.print_ccode()\n```\n\n```c\ntypedef struct _tag_Inner {\n    unsigned char a;\n    unsigned char b;\n} Inner;\ntypedef struct _tag_Outer {\n    Inner first;\n    unsigned char second;\n} Outer;\n```\n\n#### Generating c code for bitfields\n\n```python\nBF_LE.print_ccode()\n```\n\n```c\ntypedef struct _tag_BF_LE {\n    unsigned char a : 3;\n    unsigned char b : 5;\n    unsigned char c;\n} BF_LE;\n```\n","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwaszil%2Fpyembc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwaszil%2Fpyembc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwaszil%2Fpyembc/lists"}