{"id":21102896,"url":"https://github.com/planetarium/bencodex","last_synced_at":"2025-12-29T15:43:08.854Z","repository":{"id":55109870,"uuid":"155459595","full_name":"planetarium/bencodex","owner":"planetarium","description":"Bencodex: Bencoding Extended","archived":false,"fork":false,"pushed_at":"2024-06-29T12:27:42.000Z","size":38,"stargazers_count":17,"open_issues_count":3,"forks_count":4,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-01-21T03:11:20.706Z","etag":null,"topics":["bencode","bencodex","serialization-format","specification","test-suite"],"latest_commit_sha":null,"homepage":"https://bencodex.org/","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/planetarium.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-10-30T21:38:09.000Z","updated_at":"2024-10-04T05:36:40.000Z","dependencies_parsed_at":"2024-06-29T13:51:14.073Z","dependency_job_id":null,"html_url":"https://github.com/planetarium/bencodex","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/planetarium%2Fbencodex","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/planetarium%2Fbencodex/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/planetarium%2Fbencodex/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/planetarium%2Fbencodex/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/planetarium","download_url":"https://codeload.github.com/planetarium/bencodex/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243546491,"owners_count":20308513,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bencode","bencodex","serialization-format","specification","test-suite"],"created_at":"2024-11-19T23:56:54.767Z","updated_at":"2025-12-29T15:43:03.791Z","avatar_url":"https://github.com/planetarium.png","language":null,"readme":"Bencodex: Bencoding Extended\n============================\n\n*The version of this document is **1.3**.  See also [changelog].*\n\n*There is a list of implementations.  See also [LIBRARIES.tsv](./LIBRARIES.tsv)\nfile.*\n\nBencodex is a serialization format that extends BitTorrent's [Bencoding].\nSince it is a superset of Bencoding, every valid Bencoding representation is\na valid Bencodex representation of the same meaning (i.e., represents the same\nvalue).  Bencodex adds the below data types to Bencoding:\n\n -  null\n -  Boolean values\n -  Unicode strings besides byte strings\n -  Dictionaries with both byte and Unicode string keys\n\n[Bencoding]: http://www.bittorrent.org/beps/bep_0003.html#bencoding\n[changelog]: ./CHANGES.md\n\n\nWhy not *[insert your favorite format here]*\n--------------------------------------------\n\nThe unique feature of Bencoding is forced normalization.\nAccording to Wikipedia's [Bencode] page:\n\n\u003e For each possible (complex) value, there is only a single valid bencoding;\n\u003e i.e. there is a [bijection] between values and their encodings.\n\u003e This has the advantage that applications may compare bencoded values by\n\u003e comparing their encoded forms, eliminating the need to decode the values.\n\nThis makes things really simple when an application needs to determine\nif encoded values are the same, in particular, with cryptographic hash or\ndigital signatures.\n\nThere have been countless improvements in data serialization like\nrich data types, human readability, compact binary representation,\nzero-copy serialization, and even streaming, but canonical representation\nis still not well counted.\n\nBencodex actually does not aim high in ambition; it purposes to merely\nleverage Bencoding's good things with average-level data types of modern\nserialization formats.\n\n[Bencode]: https://en.wikipedia.org/wiki/Bencode#Features_\u0026_drawbacks\n[bijection]: https://en.wikipedia.org/wiki/Bijection\n\n\nEncoding\n--------\n\nNote that notations for the semantics (i.e., the values that encodings\nrepresent) use Python's literals.\n\n -  Null is represented by `n` (`6e`).\n\n -  Boolean true is represented by `t` (`74`),\n    and false is represented by `f` (`66`).\n\n -  Byte strings are length-prefixed base 10 followed by a colon and\n    the byte string.\n\n    For example, `4:spam` (`34 3a 73 70 61 6d`) corresponds to `b\"spam\"`.\n\n -  Unicode strings are represented by `u` followed by UTF-8 byte length\n    base 10 and UTF-8 encoding of the Unicode string.\n\n    For example, `u6:단팥` (`75 36 3a eb 8b a8 ed 8c a5`) corresponds to\n    `u\"\\ub2e8\\ud325\"`.\n\n -  Integers are represented by an `i` followed by the number in base 10\n    followed by an `e`.\n\n    For example, `i3e` (`69 33 65`) corresponds to `3`,\n    and `i-3e` (`69 2d 33 65`) corresponds to `-3`.\n\n    Integers have no size limitation.\n\n    `i-0e` (`69 2d 30 65`) is invalid.  All encodings with a leading zero,\n    such as `i03e` (`69 30 33 65`), are invalid, other than `i0e` (`69 30 65`),\n    which of course corresponds to `0`.\n\n -  Lists are encoded as an `l` followed by their elements (also represented in\n    Bencodex) followed by an `e`.\n\n    For example, `l4:spamu4:eggse` (`6c 34 3a 73 70 61 6d 75 34 3a 65 67 67 73\n    65`) corresponds to `[b\"spam\", u\"eggs\"]`.\n\n -  Dictionaries are encoded as a `d` followed by a list of alternating keys\n    and their corresponding values followed by an `e`.\n\n    For example, `d3:cowu3:moou4:spam4:eggse` (`64 33 3a 63 6f 77 75 33 3a 6d\n    6f 6f 75 34 3a 73 70 61 6d 34 3a 65 67 67 73 65`) corresponds to\n    `{b\"cow\": u\"moo\", u\"spam\": b\"eggs\"}`, and `du4:spaml1:au1:bee` (`64 75 34\n    3a 73 70 61 6d 6c 31 3a 61 75 31 3a 62 65 65`) corresponds to\n    `{u\"spam\": [b\"a\", u\"b\"]}`.\n\n    Keys must be Unicode or byte strings, and appear in the certain order:\n\n     -  Unicode strings do not appear earlier than byte strings.\n\n     -  Byte strings are sorted as raw strings, not alphanumerics.\n\n     -  Unicode strings are sorted as their UTF-8 *byte* representations,\n        *not* any collation order or chart order listed by Unicode.\n\n        For example, `b` (`62`) should be followed by `á` (`C3 A1`),\n        because the byte `62` is less than the byte `C3`.\n\n    `du1:k1:v1:k1:ve` (`64 75 31 3a 6b 31 3a 76 31 3a 6b 31 3a 76 65`) is\n    invalid because `u1:k` appear earlier than `1:k`.\n\n\nTest suite\n----------\n\nThe *testsuite/* directory contains a set of Bencodex tests.  Every test case\nis a triple of *.dat* which is an arbitrary Bencodex data, a *.yaml* which\nis its corresponding value in YAML, and a *.json* which is an alternative to\nYAML and renders an AST of the Bencodex value.\n\nFor example, *list.dat* contains the below Bencodex data:\n\n~~~~ bencodex\nlu16:a Unicode string13:a byte stringi123ei-456etfndu1:au4:dictelu1:au4:listee\n~~~~\n\nwhich encodes the value corresponding to *list.yaml*, that is:\n\n~~~~ yaml\n- a Unicode string\n- !!binary \"YSBieXRlIHN0cmluZw==\"  # b\"a byte string\"\n- 123\n- -456\n- true\n- false\n- null\n- a: dict\n- [a, list]\n~~~~\n\nOr, as an alternative there's *list.json* which renders an AST of the value\nstructure:\n\n~~~~ json\n{\n  \"type\": \"list\",\n  \"values\": [\n    {\n      \"type\": \"text\",\n      \"value\": \"a Unicode string\"\n    },\n    {\n      \"base64\": \"YSBieXRlIHN0cmluZw==\",\n      \"type\": \"binary\"\n    },\n    {\n      \"decimal\": \"123\",\n      \"type\": \"integer\"\n    },\n    {\n      \"decimal\": \"-456\",\n      \"type\": \"integer\"\n    },\n    {\n      \"type\": \"boolean\",\n      \"value\": true\n    },\n    {\n      \"type\": \"boolean\",\n      \"value\": false\n    },\n    {\n      \"type\": \"null\"\n    },\n    {\n      \"pairs\": [\n        {\n          \"key\": {\n            \"type\": \"text\",\n            \"value\": \"a\"\n          },\n          \"value\": {\n            \"type\": \"text\",\n            \"value\": \"dict\"\n          }\n        }\n      ],\n      \"type\": \"dictionary\"\n    },\n    {\n      \"type\": \"list\",\n      \"values\": [\n        {\n          \"type\": \"text\",\n          \"value\": \"a\"\n        },\n        {\n          \"type\": \"text\",\n          \"value\": \"list\"\n        }\n      ]\n    }\n  ]\n}\n~~~~\n\nNote that the schema of *.json* files is formally described in [JSON Schema].\nsee also [utils/testsuite-schema.json](./utils/testsuite-schema.json).\n\nAn implementation should satisfy the below rules:\n\n -  Bytes that an encoder builds from a YAML/JSON content should be exactly\n    same to the contents of a *.dat* file that corresponds to\n    the *.yaml*/*json* file.\n\n -  A content a decoder read from a *.dat* file should be equivalent to\n    the content of a *.yaml*/*.json* file that corresponds to the *.dat* file.\n\n[JSON Schema]: https://json-schema.org/\n\n\n----\n\nThis document (*README.md*) and every content in this repository including\nthe test suite (*testsuite/*) are in the public domain.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplanetarium%2Fbencodex","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fplanetarium%2Fbencodex","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplanetarium%2Fbencodex/lists"}