{"id":13736446,"url":"https://github.com/status-im/nim-toml-serialization","last_synced_at":"2025-04-09T16:19:16.084Z","repository":{"id":37012707,"uuid":"285478987","full_name":"status-im/nim-toml-serialization","owner":"status-im","description":"Flexible TOML serialization [not] relying on run-time type information.","archived":false,"fork":false,"pushed_at":"2024-10-15T09:51:15.000Z","size":340,"stargazers_count":43,"open_issues_count":4,"forks_count":8,"subscribers_count":19,"default_branch":"master","last_synced_at":"2025-04-09T16:19:06.331Z","etag":null,"topics":["configuration-file","nim","parser","serialization","serializer","toml"],"latest_commit_sha":null,"homepage":"","language":"Nim","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/status-im.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-APACHEv2","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-08-06T05:08:01.000Z","updated_at":"2025-04-06T14:21:21.000Z","dependencies_parsed_at":"2023-12-22T06:36:14.448Z","dependency_job_id":"623d1347-29ab-4d49-8f27-c5fc124dc12e","html_url":"https://github.com/status-im/nim-toml-serialization","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/status-im%2Fnim-toml-serialization","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/status-im%2Fnim-toml-serialization/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/status-im%2Fnim-toml-serialization/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/status-im%2Fnim-toml-serialization/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/status-im","download_url":"https://codeload.github.com/status-im/nim-toml-serialization/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248065281,"owners_count":21041872,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["configuration-file","nim","parser","serialization","serializer","toml"],"created_at":"2024-08-03T03:01:21.974Z","updated_at":"2025-04-09T16:19:16.058Z","avatar_url":"https://github.com/status-im.png","language":"Nim","funding_links":[],"categories":["Data"],"sub_categories":["Serialization"],"readme":"# nim-toml-serialization\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)\n[![License: Apache](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n[![TOML](assets/badge-TOML.svg)](https://github.com/toml-lang/toml/releases/tag/1.0.0)\n![Stability: experimental](https://img.shields.io/badge/stability-experimental-orange.svg)\n![nimble](https://img.shields.io/badge/available%20on-nimble-yellow.svg?style=flat-square)\n![Github action](https://github.com/status-im/nim-toml-serialization/workflows/CI/badge.svg)\n\nFlexible TOML serialization [not] relying on run-time type information.\n\n# Table of Contents\n- [Overview](#overview)\n  - [Spec compliance](#spec-compliance)\n  - [Nonstandard features](#nonstandard-features)\n  - [Keyed mode](#keyed-mode)\n- [Decoder](#decoder)\n  - [Parse inline table with newline](#parse-inline-table-with-newline)\n  - [Load and save](#load-and-save)\n  - [TOML we can\\['t\\] do](#toml-we-cant-do)\n  - [Option[T]](#optiont)\n  - [Bignum](#bignum)\n  - [Table](#table)\n  - [Sets and list-like](#sets-and-list-like)\n  - [Enums](#enums)\n  - [Helper functions](#helper-functions)\n- [Implementation specifics](#implementation-specifics)\n- [Installation](#installation)\n- [License](#license)\n- [Credits](#credits)\n\n## Overview\nnim-toml-serialization is a member of [nim-serialization](https://github.com/status-im/nim-serialization)\nfamily and provides several operation modes:\n\n  - Decode into Nim data types without any intermediate steps using only a **subset** of TOML.\n    - Unlike typical lexer-based parser, nim-toml-serialization is very efficient because\n      the parser converts text directly into Nim data types and uses no intermediate `token`.\n  - Decode into Nim data types mixed with `TomlValueRef` to parse any valid TOML value.\n    - Using `TomlValueRef` can offer more flexibility but also require more memory.\n      If you can avoid using a dotted key, there is no reason to use `TomlValueRef`.\n  - Decode into `TomlValueRef` from any valid TOML.\n  - Encode Nim data types into a **subset** of TOML.\n  - Encode `TomlValueRef` into full spec TOML.\n  - Both encoder and decoder support `keyed` mode.\n  - Allow skipping unknown fields using the `TomlUnknownFields` flag.\n    - Skipping unknown fields is also done efficiently, with no token produced.\n      But skipped fields should contain valid TOML values or the parser will raise an exception.\n  - Since v0.2.1 you can choose to use `OrderedTable` instead of `Table` when parsing into `TomlValueRef`\n    using `-d:tomlOrderedTable` compile time switch.\n  - Since v0.2.3, compile time decode/loadFile is allowed. It means you can initialize a const value using\n    `decode` or `loadFile`. It is also ok to use it inside a static block or other nim VM code.\n\n\u003e **Note**\u003cbr\u003e\nOn Windows, you might need to increase the stack size as nim-toml-serialization uses the stack to pass the object around.\nExample: add `--passL:\"-Wl,--stack,8388608\"` to your command line when running the compiler.\nBut you only need to do this if the object you serializing can produce deep recursion.\n\n## Spec compliance\nnim-toml-serialization implements [v1.0.0](https://github.com/toml-lang/toml/releases/tag/1.0.0)\nTOML spec and pass these test suites:\n\n  - [iarna toml test suite](https://github.com/iarna/toml-spec-tests)\n  - [burntsushi toml test suite](https://github.com/BurntSushi/toml-test)\n\n## Nonstandard features\n- TOML key comparison according to the spec is case sensitive and this is the default mode\n  for both encoder/decoder. But nim-toml-serialization also supports:\n\n  - Case insensitive key comparison.\n  - Nim ident sensitivity key comparison mode (only the first char is case sensitive).\n\n  TOML key supports Unicode chars but the comparison mentioned above only applies to ASCII chars.\n\n- TOML inline table disallows newline inside the table.\n  nim-toml-serialization provides a switch to enable newline in an inline table via `TomlInlineTableNewline`.\n\n- TOML standard does not support xHH escape sequence, only uHHHH or UHHHHHHHH.\n  Use `TomlHexEscape` to enable this feature otherwise it will raise an exception.\n\n- TOML standard requires time in HH:MM:SS format, `TomlHourMinute` flags will allow HH:MM format.\n\n- TOML standard requires array elements be separated by a comma. Whitespaces are ignored.\n  But due to a bug, the array/inline table elements can be separated by both comma and whitespace.\n  Set `TomlStrictComma` flag on to parse in strict mode, by default the strict mode is off.\n\n## Keyed mode\nWhen decoding, only objects, tuples or `TomlValueRef` are allowed at top level.\nAll other Nim basic datatypes such as floats, ints, arrays, and booleans must\nbe a value of a key.\n\nnim-toml-serialization offers `keyed` mode decoding to overcome this limitation.\nThe parser can skip any non-matching key-value pair efficiently because\nthe parser produces no token but at the same time can validate the syntax correctly.\n\n```toml\n[server]\n  name = \"TOML Server\"\n  port = 8005\n```\n\n```Nim\nvar x = Toml.decode(rawToml, string, \"server.name\")\nassert x == \"TOML Server\"\n\nor\n\nvar y = Toml.decode(rawToml, string, \"server.name\", caseSensitivity)\n```\n\nwhere `caseSensitivity` is one of:\n  - TomlCaseSensitive\n  - TomlCaseInsensitive\n  - TomlCaseNim\n\nThe key must be a valid Toml basic key, quoted key, or dotted key.\n\nGotcha:\n\n```toml\nserver = { ip = \"127.0.0.1\", port = 8005, name = \"TOML Server\" }\n```\n\nIt may be tempting to use keyed mode for the above example like this:\n```nim\nvar x = Toml.decode(rawToml, string, \"server.name\")\n```\nBut it won't work because the grammar of TOML makes it very difficult\nto `exit` from the inline table parser in a clean way.\n\n## Decoder\n```nim\n  type\n    NimServer = object\n      name: string\n      port: int\n\n    MixedServer = object\n      name: TomlValueRef\n      port: int\n\n    StringServer = object\n      name: string\n      port: string\n\n  # decode into native Nim\n  var nim_native = Toml.decode(rawtoml, NimServer)\n\n  # decode into mixed Nim + TomlValueRef\n  var nim_mixed = Toml.decode(rawtoml, MixedServer)\n\n  # decode any value into string\n  var nim_string = Toml.decode(rawtoml, StringServer)\n\n  # decode any valid TOML\n  var toml_value = Toml.decode(rawtoml, TomlValueRef)\n```\n\n## Parse inline table with newline\n```toml\n# This is a nonstandard toml\n\nserver = {\n  ip = \"127.0.0.1\",\n  port = 8005,\n  name = \"TOML Server\"\n}\n```\n\n```Nim\n  # turn on newline in inline table mode\n  var x = Toml.decode(rawtoml, Server, flags = {TomlInlineTableNewline})\n```\n\n## Load and save\n```Nim\n  var server = Toml.loadFile(\"filename.toml\", Server)\n  var ip = Toml.loadFile(\"filename.toml\", string, \"server.ip\")\n\n  Toml.saveFile(\"filename.toml\", server)\n  Toml.saveFile(\"filename.toml\", ip, \"server.ip\")\n  Toml.saveFile(\"filename.toml\", server, flags = {TomlInlineTableNewline})\n```\n\n## TOML we can['t] do\n- Date Time.\n  TOML date time format is described in [RFC 3339](https://tools.ietf.org/html/rfc3339).\n  When parsing TOML date time, use `string`, `TomlDateTime`, or `TomlValueRef`.\n\n- Date.\n  You can parse TOML date using `string`, `TomlDate`, `TomlDateTime`, or `TomlValueRef`.\n\n- Time.\n  You can parse TOML time using `string`, `TomlTime`, `TomlDateTime`, or `TomlValueRef`.\n\n- Heterogenous array.\n  When parsing a heterogenous array, use `string` or `TomlValueRef`.\n\n- Floats.\n  Floats should be implemented as IEEE 754 binary64 values.\n  The standard TOML float is float64.\n  When parsing floats, use `string` or `TomlValueRef` or `SomeFloat`.\n\n- Integers.\n  TOML integer is a 64-bit (signed long) range expected (−9,223,372,036,854,775,808 to 9,223,372,036,854,775,807).\n  When parsing integers, use `string` or `SomeInteger`, or `TomlValueRef`.\n\n- Array of tables.\n  An array of tables can be parsed via `TomlValueRef` or parsed as a field of object.\n  Parsing with keyed mode also works.\n\n- Dotted key.\n  When parsing into a nim object, the key must not be dotted.\n  The dotted key is supported via `keyed` decoding or `TomlValueRef`.\n\n## Option[T]\n  Option[T] works as usual.\n\n## Bignum\nTOML integer maxed at int64. But nim-toml-serialization can extend this to arbitrary precision bignum.\nParsing bignum is achieved via the helper function `parseNumber`.\n\n```Nim\n# This is an example of how to parse bignum with `parseNumber` and `stint`.\n\nimport stint, toml_serialization\n\nproc readValue*(r: var TomlReader, value: var Uint256) =\n  try:\n    var z: string\n    let (sign, base) = r.parseNumber(z)\n\n    if sign == Sign.Neg:\n      raiseTomlErr(r.lex, errNegateUint)\n\n    case base\n    of base10: value = parse(z, Uint256, 10)\n    of base16: value = parse(z, Uint256, 16)\n    of base8:  value = parse(z, Uint256, 8)\n    of base2:  value = parse(z, Uint256, 2)\n  except ValueError as ex:\n    raiseUnexpectedValue(r.lex, ex.msg)\n\nvar z = Toml.decode(\"bignum = 1234567890_1234567890\", Uint256, \"bignum\")\nassert $z == \"12345678901234567890\"\n```\n\n## Table\nDecoding a table can be achieved via the `parseTable` template.\nTo parse the value, you can use one of the helper functions or use `readValue`.\n\nThe table can be used to parse the top-level value, regular table, and inline table like an object.\n\nNo built-in `readValue` for the table provided, you must overload it yourself depending on your need.\n\n`Table` can be stdlib table, ordered table, table ref, or any table-like data type.\n\n```Nim\nproc readValue*(r: var TomlReader, table: var Table[string, int]) =\n  parseTable(r, key):\n    table[key] = r.parseInt(int)\n```\n\n## Sets and list-like\nSimilar to `Table`, sets and list or array-like data structure can be parsed using\n`parseList` template. It comes in two flavors, indexed and non-indexed.\n\nBuilt-in `readValue` for regular `seq` and `array` is implemented for you.\nNo built-in `readValue` for `set` or `set-like` is provided, you must overload it yourself depending on your need.\n\n```nim\ntype\n  HoldArray = object\n    data: array[3, int]\n\n  HoldSeq = object\n    data: seq[int]\n\n  WelderFlag = enum\n    TIG\n    MIG\n    MMA\n\n  Welder = object\n    flags: set[WelderFlag]\n\nproc readValue*(r: var TomlReader, value: var HoldArray) =\n  # parseList with index, `i` can be any valid identifier\n  r.parseList(i):\n    value.data[i] = r.parseInt(int)\n\nproc readValue*(r: var TomlReader, value: var HoldSeq) =\n  # parseList without index\n  r.parseList:\n    let lastPos = value.data.len\n    value.data.setLen(lastPos + 1)\n    readValue(r, value.data[lastPos])\n\nproc readValue*(r: var TomlReader, value: var Welder) =\n  # populating set also okay\n  r.parseList:\n    value.flags.incl r.parseEnum(WelderFlag)\n```\n\n## Enums\nThere are no enums in TOML specification. The reader/decoder can parse both\nthe `ordinal` or `string` representation of an enum. While on the other hand,\nthe writer/encoder only has an `ordinal` built-in writer. But that is not a limitation,\nyou can always overload the `writeValue` to produce whatever representation of\nthe enum you need.\n\nThe `ordinal` representation of an enum is TOML integer. The `string` representation\nis TOML `basic string` or `literal string`. Both multi-line basic strings(e.g. \"\"\"TOML\"\"\") and\nmulti-line literal strings(e.g. '''TOML''') are not allowed for enum value.\n\n```toml\n# fruits.toml\nfruit1 = \"Apple\"   # basic string\nfruit2 = 1         # ordinal value\nfruit3 = 'Orange'  # literal string\n```\n\n```Nim\ntype\n  Fruits = enum\n    Apple\n    Banana\n    Orange\n\n  FruitBasket = object\n    fruit1: Fruits\n    fruit2: Fruits\n    fruit3: Fruits\n\nvar x = Toml.loadFile(\"fruits.toml\", FruitBasket)\nassert x.fruit1 == Apple\nassert x.fruit2 == Banana\nassert x.fruit3 == Orange\n\n# write enum output as a string\nproc writeValue*(w: var TomlWriter, val: Fruits) =\n  w.writeValue $val\n\nlet z = FruitBasket(fruit1: Apple, fruit2: Banana, fruit3: Orange)\nlet res = Toml.encode(z)\nassert res == \"fruit1 = \\\"Apple\\\"\\nfruit2 = \\\"Banana\\\"\\nfruit3 = \\\"Orange\\\"\\n\"\n```\n\nYou can control the reader behavior when deserializing specific enum using `configureTomlDeserialization`.\n\n```Nim\nconfigureTomlDeserialization(\n    T: type[enum], allowNumericRepr: static[bool] = false,\n    stringNormalizer: static[proc(s: string): string] = strictNormalize)\n```\n\n## Helper functions\n  - `parseNumber(r: var TomlReader, value: var string): (Sign, NumberBase)`\n  - `parseDateTime(r: var TomlReader): TomlDateTime`\n  - `parseString(r: var TomlReader, value: var string): (bool, bool)`\n  - `parseAsString(r: var TomlReader): string`\n  - `parseFloat(r: var TomlReader, value: var string): Sign`\n  - `parseTime(r: var TomlReader): TomlTime`\n  - `parseDate(r: var TomlReader): TomlDate`\n  - `parseValue(r: var TomlReader): TomlValueRef`\n  - `parseEnum(r: var TomlReader, T: type enum): T`\n  - `parseInt(r: var TomlReader, T: type SomeInteger): T`\n\n`parseAsString` can parse any valid TOML value into a Nim string including a mixed array or inline table.\n\n`parseString` returns a tuple:\n  - field 0:\n    - false: is a single line string.\n    - true: is a multi-line string.\n  - field 1:\n    - false: is a basic string.\n    - true: is a literal string.\n\n`Sign` can be one of:\n  - `Sign.None`\n  - `Sign.Pos`\n  - `Sign.Neg`\n\n## Implementation specifics\nTomlTime contains a subsecond field. The spec says the precision is implementation-specific.\n\nIn nim-toml-serialization the default is 6 digits precision.\nLonger precision will be truncated by the parser.\n\nYou can override this using compiler switch `-d:tomlSubsecondPrecision=numDigits`.\n\n## Installation\n\nYou can install the development version of the library through Nimble with the following command\n```\nnimble install https://github.com/status-im/nim-toml-serialization@#master\n```\n\nor install the latest release version\n```\nnimble install toml_serialization\n```\n\n## License\n\nLicensed and distributed under either of\n\n* MIT license: [LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT\n\nor\n\n* Apache License, Version 2.0, ([LICENSE-APACHEv2](LICENSE-APACHEv2) or http://www.apache.org/licenses/LICENSE-2.0)\n\nat your option. This file may not be copied, modified, or distributed except according to those terms.\n\n## Credits\n\nA portion of the toml decoder was taken from PMunch's [`parsetoml`](https://github.com/NimParsers/parsetoml)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstatus-im%2Fnim-toml-serialization","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstatus-im%2Fnim-toml-serialization","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstatus-im%2Fnim-toml-serialization/lists"}