{"id":17196588,"url":"https://github.com/johnnovak/nim-riff","last_synced_at":"2025-04-09T16:17:49.874Z","repository":{"id":70773340,"uuid":"244135717","full_name":"johnnovak/nim-riff","owner":"johnnovak","description":"RIFF file handling for Nim","archived":false,"fork":false,"pushed_at":"2024-04-07T06:53:24.000Z","size":201,"stargazers_count":13,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-09T16:17:44.129Z","etag":null,"topics":["amiga","iff","interchange-format","nim","nim-lang","riff","riff-wave"],"latest_commit_sha":null,"homepage":"","language":"Nim","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"wtfpl","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/johnnovak.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG","contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-03-01T11:21:07.000Z","updated_at":"2025-01-24T23:05:38.000Z","dependencies_parsed_at":"2024-04-07T07:33:13.943Z","dependency_job_id":"583edf67-822d-4e2d-b834-44b201dd541d","html_url":"https://github.com/johnnovak/nim-riff","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/johnnovak%2Fnim-riff","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/johnnovak%2Fnim-riff/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/johnnovak%2Fnim-riff/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/johnnovak%2Fnim-riff/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/johnnovak","download_url":"https://codeload.github.com/johnnovak/nim-riff/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248065282,"owners_count":21041872,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["amiga","iff","interchange-format","nim","nim-lang","riff","riff-wave"],"created_at":"2024-10-15T01:53:40.901Z","updated_at":"2025-04-09T16:17:49.847Z","avatar_url":"https://github.com/johnnovak.png","language":"Nim","funding_links":[],"categories":[],"sub_categories":[],"readme":"# nim-riff\n\n## Overview\n\n**nim-riff** is a library for reading and writing [Resource Interchange File\nFormat (RIFF)](https://en.wikipedia.org/wiki/Resource_Interchange_File_Format)\nfiles. RIFF is heavily inspired by Electronic Arts' [Interchange File Format\n(IFF)](https://en.wikipedia.org/wiki/Interchange_File_Format), introduced in\n1985 on the best personal computer ever, the legendary [Commodore\nAmiga](https://en.wikipedia.org/wiki/Amiga).\n\nIf we're only considering existing file formats, a generic RIFF library like\nthis has little use outside of handling\n[WAV](https://en.wikipedia.org/wiki/WAV),\n[AVI](https://en.wikipedia.org/wiki/Audio_Video_Interleave) and\n[WebP](https://en.wikipedia.org/wiki/WebP) files (the most common RIFF based\nformat). What I personally think it is\nreally good for, though, is implementing your own hierarchical binary file\nformats (for example, [Gridmonger](https://github.com/johnnovak/gridmonger)\nstores its documents in RIFF format). It is an excellent alternative to\n~~bloated garbage like XML~~ other hierarchical file formats less suitable for\nstoring binary data, such as XML.\n\nTo learn more about RIFF, please refer to the [References \u0026 reading materials\nsection](#references--reading-materials).\n\n\n\n### Main features\n\n* Reading and writing of little-endian (`RIFF`) and big-endian (`RIFX`) RIFF files\n* Strict adherence to the RIFF standard\n* Convenient helper methods to navigate the chunk hierarchy (with cursor support)\n* The reader treats chunks as virtual files\n* The writer recursively auto-updates all parent chunk sizes to minimise the\n  chance of creating malformed files\n\n### Use it if\n\n* You need to store hierarchical binary data in an efficient and extensible way\n* You don't want to deal with the parsing overhead of other formats like\n  XML, JSON, etc. (such formats are ill-suited for binary data anyway)\n* You're an Amiga fan — true Amiga fans hunt for opportunities to use IFF-like\n  formats as much as possible :sunglasses:\n\n### Don't use it if\n\n* You need to store more than 4 GB in a single chunk or file\n* You want to read data from malformed RIFF files, or you want to attempt to\n  repair them. For these use cases this library is not a good fit because it\n  expects perfect RIFF files with no errors. Repairing RIFF files\n  generically is not really possible without some extra knowledge about the\n  particular file format you're dealing with (e.g. WAV, AVI, etc.), so\n  you'll probably need to something custom anyway.\n\n## Installation\n\n**nim-riff** can be installed via Nimble:\n\n    nimble install riff\n\n\n## Compiling the examples\n\nThe examples require the `simple_parseopt` module, so install that first:\n\n    nimble install simple_parseopt\n\nThen you can compile the examples in debug or release mode:\n\n    nimble examples\n    nimble examplesDebug\n\n\n## Documentation\n\nhttps://www.johnnovak.net/nim-riff/\n\n\n## Usage\n\n### Reading RIFF files\n\n#### Opening a file\n\nTo open a RIFF file for reading, you'll need to provide a filename or an\nexisting file handle to the `openRiffFile()` proc. You can also optionally\noverride the default buffer size.\n\nThis will create a `RiffReader` object on success, or raise\na `RiffReaderError` if something went wrong (just like all reader methods in\ncase of an error).\n\nWhen you're done with a reader, you can close it with the `close()` method.\n\n```nim\nimport riff\n\nvar r: RiffReader\nr = openRiffFile(\"infile\")\nr = openRiffFile(\"infile\", bufsize=8192)\n\nvar f = open(\"infile\")\nr = openRiffFile(f)\nr = openRiffFile(f, bufSize=8192)\n\nr.close()\n```\n\n\n#### Navigating the chunks\n\nThe reader has the concept of the *current chunk*, you can think of it as\na cursor. Right after successfully opening a file, the current chunk is set to\nthe root RIFF chunk, which is a group chunk itself.\n\nThe `currentChunk()` method returns information about the current chunk as\na `ChunkInfo` object, which has the following fields:\n\n* `id` *string* – 4-char chunk ID (FourCC)\n\n* `size` *uint32* – chunk data length in bytes (not including the 8-byte\n   chunk headers, nor the optional padding byte if the length is odd)\n\n* `filePos` *int64* – absolute file position of the chunk (the first byte of the chunk\n   header)\n\n* `kind` *ChunkKind* – `ckGroup` for group chunks (`RIFF` and `LIST`), `ckChunk` for normal chunks\n\n* `formatTypeId` *string* – for group chunks only: format type FourCC of the group\n\n`nextChunk()` moves the cursor to the next chunk within the current group\nchunk and returns its `ChunkInfo`, or raises an error if we're already at the\nlast subchunk. It's best to use `hasNextChunk()` before calling `nextChunk()`\nto prevent these errors.\n\nThis is a simple example that iterates through all the top-level chunks in the\nroot RIFF group chunk, and prints out their chunk infos:\n\n```nim\nimport riff\n\nvar r = openRiffFile(\"test.wav\")\nvar ci: ChunkInfo\n\n# info about the root RIFF chunk\nci = r.currentChunk\necho ci\n\n# the root RIFF chunk is a group chunk so it must be entered\nif r.hasSubChunks():\n  ci = r.enterGroup()\n  echo ci\n\n  # iterate through all top-level chunks inside the root RIFF group chunk\n  while r.hasNextChunk():\n    ci = r.nextChunk()\n    echo ci\n\nr.close()\n```\n\n##### Group chunks\n\nAs mentioned above, a chunk can be either a normal chunk (`ckChunk`) or\na group chunk (`ckGroup`) that can contain further subchunks. There are only\ntwo types of group chunks: the root `RIFF` chunk, and `LIST` chunks.\n\nWhen the cursor is at a group chunk, you can call `enterGroup()` to descend\ninto it. If the group contains subchunks (which can be checked with the\n`hasSubChunks()` method), the cursor will be set to the first child chunk, and\nthe chunk info will be returned. If the group has no subchunks, an error will\nbe raised.\n\n`exitGroup()` does the opposite; it moves the cursor up one level to the\nparent group chunk.\n\n\n##### Walking the chunk tree\n\nThe chunk hierarchy of a RIFF file is basically a tree structure, and we can\nwalk this tree with the aforementioned navigation methods:\n\n* `enterGroup()` moves the cursor to the first child node of the current group node\n* `exitGroup()` moves the cursor back to the parent node\n* `nextChunk()` iterates the cursor through sibling nodes\n\nUsing these methods, it is possible to put together a recursive algorithm that\ntraverses the whole chunk tree and prints out the chunk infos in\na hierarchical fashion:\n\n```nim\nimport strutils\nimport riff\n\nvar r = openRiffFile(\"test.grm\")\n\nproc walkChunks(depth: Natural = 0) =\n  let cc = r.currentChunk\n  echo \" \".repeat(depth * 2), cc\n\n  if cc.kind == ckGroup:\n    if r.hasSubchunks:\n      discard r.enterGroup()\n      walkChunks(depth+1)\n      r.exitGroup()\n    else:\n      echo \" \".repeat((depth+1) * 2), \"\u003cempty\u003e\"\n\n  if r.hasNextChunk:\n    discard r.nextChunk()\n    walkChunks(depth)\n\nwalkChunks()\n\nr.close()\n```\n\nThe library provides a convenient `walkChunks()` iterator that does\neffectively the same thing but without recursion. It can also traverse\nsubtrees and use any node as the starting point.\n\n```nim\nimport strutils\nimport riff\n\nvar r = openRiffFile(\"test.grm\")\n\nfor ci in r.walkChunks():\n  echo \" \".repeat((r.cursor.path.len-1) * 2), ci\n\nr.close()\n```\n\n\nExample output ([Gridmonger](/johnnovak/gridmonger) map file):\n\n```\n(id: \"RIFF\", size: 5582, filePos: 0, kind: ckGroup, formatTypeId: \"GRMM\")\n  (id: \"map \", size: 23, filePos: 12, kind: ckChunk)\n  (id: \"LIST\", size: 5508, filePos: 44, kind: ckGroup, formatTypeId: \"lvls\")\n    (id: \"LIST\", size: 5496, filePos: 56, kind: ckGroup, formatTypeId: \"lvl \")\n      (id: \"prop\", size: 20, filePos: 68, kind: ckChunk)\n      (id: \"cell\", size: 5445, filePos: 96, kind: ckChunk)\n      (id: \"note\", size: 2, filePos: 5550, kind: ckChunk)\n  (id: \"lnks\", size: 2, filePos: 5560, kind: ckChunk)\n  (id: \"disp\", size: 11, filePos: 5570, kind: ckChunk)\n```\n\n#### Reading chunk data\n\nYou can think of the current chunk as a virtual file; when you enter a chunk,\nthe \"virtual file position\", or *chunk position*, is set to the start of the\nchunk data, which is the first byte after the chunk headers.  This is chunk\nposition `0`.\n\nYou can query the current chunk position with `getChunkPos()` and set it with\n`setChunkPos()`, which works similarly to `setFilePosition()` from the\nstandard `io` library. An error will be raised if you try to set the position\nbeyond the limits of the chunk.\n\n```nim\nlet pos = r.getChunkPos()\nr.setChunkPos(20, cspSet)   # valid values are: cspSet, cspCur, cspEnd\n```\n\nYou can read the chunk data with the various `read*()` methods as shown below.\nAn error will be raised if you attempt to read past the end of the chunk.\n\n\n```nim\n# To read a specific numeric type, pass in its type as an argument\nlet i8 = r.read(uint8)\nlet f32 = r.read(float32)\nlet i64 = r.read(int64)\n\n# Reading multiple values into a buffer\nvar buf: array[100, float]\nr.read(buf, startIndex=0, numValues=buf.len)\n\nr.readChar()          # read a char\nr.readFourCC()        # read a FourCC as a string\nr.readStr(length=10)  # read the next 10 bytes as a string\n\nr.readBStr()    # read a Pascal-style string (one `length` leading\n                # byte followed by `length` bytes of character data)\n\nr.readWStr()    # read a Pascal-style string\n                # (16-bit (word) leading `length` value)\n\nr.readZStr()    # read a C-style null-terminated string\n\nr.readBZStr()   # read a Pascal-style string (byte `length`) that is also\n                # null-terminated\n\nr.readWZStr()   # read a Pascal-style string (16-bit `length`) that is also\n                # null-terminated\n```\n\n#### Cursors\n\nIt is possible to save the current chunk and chunk position as a `Cursor`\nand restore it later.\n\n```nim\nlet cur = r.cursor    # store the current cursor\nr.cursor = cur        # restore a cursor\n```\n\nThe `Cursor` object has the following fields:\n\n* `path` *seq[ChunkInfo]* – path to this chunk in the RIFF tree (the last\n    element is this chunk, the rest are the parents, right up to the root RIFF\n    chunk which is the first element)\n\n* `chunkPos` *uint32* – chunk position from the start of the chunk data\n\n* `filePos` *int64* – absolute file position from the start of the file\n\n\nA typical usage pattern is to walk through all the\nchunks in the file in the first pass, store cursors pointing to the chunks\nof interest, and then read from those chunks in the second pass using the\ncursors.\n\n\n\n### Writing RIFF files\n\n#### Creating a file\n\nYou can create a new RIFF file with the `createRiffFile()` method.  This will\ncreate a `RiffWriter` object on success, or raise a `RiffWriterError` if\nsomething went wrong (just like all writer methods in case of an error).\nYou can also optionally set the endianness of the file (default is\nlittle-endian) or override the default buffer size.\n\n\n```nim\nvar w: RiffWriter\nw = createRiffFile(filename, \"GRMM\")\nw = createRiffFile(filename, \"GRMM\", endian=littleEndian, bufSize=8192)\n```\n\nWhen you're done writing to the RIFF file, you must call the `close()` method.\n\n:warning: _Calling `close()` is very important because this ensures that the\ntotal file size in the root RIFF chunk is updated correctly! It also closes\nall currently open chunks recursively, making sure their headers are updated\nas well._\n\n#### Creating chunks\n\nYou can create chunks or list chunks with the `beginChunk()` and\n`beginListChunk()` methods, respectively. The chunk ID (or format type ID in\ncase of list chunks) needs to be passed in.\n\nCalling `endChunk()` closes the current chunk and writes the final chunk size\nto its header.\n\nThe `close()` method closes all currently open chunks recursively.\n\n```nim\nw.beginListChunk(\"lvls\")\n\nw.beginChunk(\"cell\")\n# ... write chunk data ...\nw.endChunk()\n\nw.beginChunk(\"prop\")\n# ... write chunk data ...\nw.endChunk()\n\nw.endChunk() # end of 'lvls' list chunk\n```\n\n#### Writing chunk data\n\nWriting values works analogously to the `read*()` methods:\n\n```nim\nw.write(42'u8)\nw.write(-8765'i16)\nw.write(1234.567'f64)\n\nw.writeChar('!')\nw.writeFourCC(\"ILBM\")\nw.writeStr(\"Guybrush Threepwood\")\nw.writeBStr(\"Mancomb Seepgood\")\nw.writeWStr(\"Elaine Marley\")\nw.writeZStr(\"Herman Toothrot\")\nw.writeBZStr(\"Men of Low Moral Fiber\")\nw.writeWZStr(\"Voodoo Lady\")\n```\n\n\n## References \u0026 reading materials\n\n[1] \"EA IFF 85\" Standard for Interchange Format Files\n*Electronic Arts, 1985*\nhttps://wiki.amigaos.net/wiki/EA_IFF_85_Standard_for_Interchange_Format_Files\nhttp://www.martinreddy.net/gfx/2d/IFF.txt\n\n[2] A Quick Introduction to IFF\n*AmigaOS Documentation Wiki*\nhttps://wiki.amigaos.net/wiki/A_Quick_Introduction_to_IFF\n\n[3] Resource Interchange File Format\n*Wikipedia*\nhttps://en.wikipedia.org/wiki/Resource_Interchange_File_Format\n\n[4] RIFF (Resource Interchange File Format)\n*Digital Preservation. Library of Congress*\nhttps://www.loc.gov/preservation/digital/formats/fdd/fdd000025.shtml\n\n[5] Multimedia Programming Interface and Data Specifications 1.0\n*Microsoft / IBM, August 1991*\nhttp://www-mmsp.ece.mcgill.ca/Documents/AudioFormats/WAVE/Docs/riffmci.pdf\n\n[6] Multimedia Data Standards Update\n*Microsoft, April 1994*\nhttp://www-mmsp.ece.mcgill.ca/Documents/AudioFormats/WAVE/Docs/RIFFNEW.pdf\n\n[7] Exiftool - Riff Info Tags\n*ExifTool website*\nhttps://exiftool.org/TagNames/RIFF.html#Info\n\n[8] Exchangeable image file format for digital still cameras, Exif Version 2.32\n*Camera \u0026 Imaging Products Association, May 2019*\nhttp://www.cipa.jp/std/documents/e/DC-X008-Translation-2019-E.pdf\n\n[9] Audio Interchange File Format: \"AIFF\", Version 1.3\n*Apple, January 1989*\nhttp://www-mmsp.ece.mcgill.ca/Documents/AudioFormats/AIFF/Docs/AIFF-1.3.pdf\n\n[10] AVI RIFF File Reference\n*Microsoft Dev Center, May 2018*\nhttps://docs.microsoft.com/en-us/windows/win32/directshow/avi-riff-file-reference\n\n\n## License\n\nCopyright © 2019-2024 John Novak \u003c\u003cjohn@johnnovak.net\u003e\u003e\n\nThis work is free. You can redistribute it and/or modify it under the terms of\nthe [Do What The Fuck You Want To Public License, Version 2](http://www.wtfpl.net/), as published\nby Sam Hocevar. See the [COPYING](./COPYING) file for more details.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjohnnovak%2Fnim-riff","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjohnnovak%2Fnim-riff","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjohnnovak%2Fnim-riff/lists"}