{"id":13399555,"url":"https://github.com/sealmove/nimitai","last_synced_at":"2025-10-09T04:04:55.216Z","repository":{"id":176260378,"uuid":"222257494","full_name":"sealmove/nimitai","owner":"sealmove","description":"A parser generator for binary data","archived":false,"fork":false,"pushed_at":"2020-11-18T03:45:45.000Z","size":601,"stargazers_count":21,"open_issues_count":0,"forks_count":1,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-07-31T19:21:38.576Z","etag":null,"topics":["forensics","parser","reverse-engineering"],"latest_commit_sha":null,"homepage":"","language":"Nim","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sealmove.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-11-17T14:06:01.000Z","updated_at":"2023-09-20T03:28:11.000Z","dependencies_parsed_at":"2024-01-18T11:09:52.813Z","dependency_job_id":"345d8430-07ad-462c-83c0-91550f650867","html_url":"https://github.com/sealmove/nimitai","commit_stats":null,"previous_names":["sealmove/nimitai"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sealmove%2Fnimitai","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sealmove%2Fnimitai/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sealmove%2Fnimitai/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sealmove%2Fnimitai/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sealmove","download_url":"https://codeload.github.com/sealmove/nimitai/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240018687,"owners_count":19734872,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["forensics","parser","reverse-engineering"],"created_at":"2024-07-30T19:00:39.397Z","updated_at":"2025-10-09T04:04:50.179Z","avatar_url":"https://github.com/sealmove.png","language":"Nim","funding_links":[],"categories":["Compiler"],"sub_categories":[],"readme":"# \u003cp align=\"center\"\u003enimitai (STALLED)\u003c/p\u003e\n\n## Introduction\nNimitai is a compile-time parser generator for binary data. It transforms JSON objects into parsing procedures.  \nEach input object should describe a binary format according to [Kaitai Struct](https://kaitai.io/) conventions.\n\n| Exported symbol | Production |\n|-----------------|------------|\n| `macro injectParser(spec: static[JsonNode])` | static library (compile time code embedding) |\n| `proc createDynlib(spec: JsonNode, path: string)` | dynamic library |\n| `proc outputModule(spec: JsonNode): string` | nim module (source code) |\n\n## Status\nThe project is stalled indefinitely because Kaitai Struct semantics are not idiomatic for Nim and therefore it is not worth the effort (there are better native solutions; see [binaryparse](https://github.com/PMunch/binaryparse/))\n\n## Writing specs\nBeing ubiquitous and easily parsable, JSON is a great base format, and there is even a CT parser [in Nim stdlib](https://nim-lang.org/docs/json.html). However, writing in JSON is not appealing at all. Luckily there are a couple of nice JSON superset, and nimitai can work in conjuction with any CT parser that outputs `JsonNode`. The end-goal is to use nimitai alongside a YAML compiler in order to leverage the excellent [Kaitai Struct IDE](https://ide.kaitai.io/). Sadly, we don't have one that works at compile-time yet. The plan is to refactor [NimYAML](https://github.com/flyx/NimYAML). Nevertheless, nimitai can progress independently.\n\n![Data flow](flow.svg)\n\n## Motivation\n**Pluggable spec-as-compiler**: Instead of being an external compiler, nimitai is a CT library which means you don't have to mess with makefiles or similar mechanisms - everything is done within the language. The moment you tweak your spec, your project that links to it has a brand new compiler! No scripts needed at all.\n\n## Example\n\n### Codegen\nbuffered_struct.nim (...)\n```nim\nimport json, nimitai, nimitai/runtime\ninjectParser(parseJson(readFile\"buffered_struct.json\"))\n```\n\nbuffered_struct.json\n```json\n{\n  \"meta\": {\n    \"id\": \"buffered_struct\",\n    \"endian\": \"le\"\n  },\n  \"seq\": [\n    {\n      \"id\": \"len1\",\n      \"type\": \"u4\"\n    },\n    {\n      \"id\": \"block1\",\n      \"type\": \"block\",\n      \"size\": \"len1\"\n    },\n    {\n      \"id\": \"len2\",\n      \"type\": \"u4\"\n    },\n    {\n      \"id\": \"block2\",\n      \"type\": \"block\",\n      \"size\": \"len2\"\n    },\n    {\n      \"id\": \"finisher\",\n      \"type\": \"u4\"\n    }\n  ],\n  \"types\": {\n    \"block\": {\n      \"seq\": [\n        {\n          \"id\": \"number1\",\n          \"type\": \"u4\"\n        },\n        {\n          \"id\": \"number2\",\n          \"type\": \"u4\"\n        }\n      ]\n    }\n  }\n}\n```\n\ngenerated code\n```nim\ntype\n  Buffered_struct* = ref object of KaitaiStruct\n    parent*: KaitaiStruct\n    len1*: uint32\n    block1*: Buffered_structBlock\n    block1Io: KaitaiStream\n    block1Raw: seq[byte]\n    len2*: uint32\n    block2*: Buffered_structBlock\n    block2Io: KaitaiStream\n    block2Raw: seq[byte]\n    finisher*: uint32\n\n  Buffered_structBlock* = ref object of KaitaiStruct\n    parent*: Buffered_struct\n    number1*: uint32\n    number2*: uint32\n\nproc read*(_: typedesc[Buffered_structBlock]; io: KaitaiStream;\n          root: KaitaiStruct; parent: Buffered_struct): Buffered_structBlock\nproc read*(_: typedesc[Buffered_struct]; io: KaitaiStream; root: KaitaiStruct;\n          parent: KaitaiStruct): Buffered_struct\nproc read*(_: typedesc[Buffered_structBlock]; io: KaitaiStream;\n          root: KaitaiStruct; parent: Buffered_struct): Buffered_structBlock =\n  template this(): untyped =\n    result\n\n  this = Buffered_structBlock(io: io, parent: parent)\n  this.root = if root == nil: this else: root\n  this.number1 = readu4le(this.io)\n  this.number2 = readu4le(this.io)\n\nproc read*(_: typedesc[Buffered_struct]; io: KaitaiStream; root: KaitaiStruct;\n          parent: KaitaiStruct): Buffered_struct =\n  template this(): untyped =\n    result\n\n  this = Buffered_struct(io: io, parent: parent)\n  this.root = if root == nil: this else: root\n  this.len1 = readu4le(this.io)\n  this.block1Raw = readBytes(this.io, int(this.len1))\n  this.block1Io = newKaitaiStream(this.block1Raw)\n  this.block1 = Buffered_structBlock.read(this.block1io, this.root, this)\n  this.len2 = readu4le(this.io)\n  this.block2Raw = readBytes(this.io, int(this.len2))\n  this.block2Io = newKaitaiStream(this.block2Raw)\n  this.block2 = Buffered_structBlock.read(this.block2io, this.root, this)\n  this.finisher = readu4le(this.io)\n\nproc fromFile*(_: typedesc[Buffered_structBlock]; filename: string): Buffered_structBlock =\n  read(Buffered_structBlock, newKaitaiFileStream(filename), nil, nil)\n\nproc fromFile*(_: typedesc[Buffered_struct]; filename: string): Buffered_struct =\n  read(Buffered_struct, newKaitaiFileStream(filename), nil, nil)\n```\n\n### API usage\n(...) buffered_struct.nim\n```nim\nlet x = BufferedStruct.fromFile(\"buffered_struct.bin\")\necho \"Block1, number1: \" \u0026 toHex(x.block1.number1.int64, 2)\necho \"Block1, number2: \" \u0026 toHex(x.block1.number2.int64, 2)\necho \"Block2, number1: \" \u0026 toHex(x.block2.number1.int64, 2)\necho \"Block2, number2: \" \u0026 toHex(x.block2.number2.int64, 2)\n```\n\nbuffered_struct.bin (hex view)\n```bin\n10 00 00 00 42 00 00 00 43 00 00 00 ff ff ff ff\nff ff ff ff 08 00 00 00 44 00 00 00 45 00 00 00\nee 00 00 00\n```\n\noutput:\n```\nBlock1, number1: 42\nBlock1, number2: 43\nBlock2, number1: 44\nBlock2, number2: 45\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsealmove%2Fnimitai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsealmove%2Fnimitai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsealmove%2Fnimitai/lists"}