{"id":13496527,"url":"https://github.com/remarshal-project/remarshal","last_synced_at":"2025-05-14T17:06:26.352Z","repository":{"id":21482282,"uuid":"24801045","full_name":"remarshal-project/remarshal","owner":"remarshal-project","description":"Convert between CBOR, JSON, MessagePack, TOML, and YAML","archived":false,"fork":false,"pushed_at":"2025-04-11T11:47:04.000Z","size":337,"stargazers_count":536,"open_issues_count":5,"forks_count":36,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-05-10T13:02:36.154Z","etag":null,"topics":["cli","command-line-tool","conversion","converter","json","messagepack","msgpack","toml","utility","yaml"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/remarshal/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/remarshal-project.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2014-10-04T20:42:34.000Z","updated_at":"2025-05-01T11:43:40.000Z","dependencies_parsed_at":"2024-01-16T09:54:22.229Z","dependency_job_id":"0e345f6f-8094-4a44-a929-f2e19fd619df","html_url":"https://github.com/remarshal-project/remarshal","commit_stats":null,"previous_names":["dbohdan/remarshal"],"tags_count":32,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/remarshal-project%2Fremarshal","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/remarshal-project%2Fremarshal/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/remarshal-project%2Fremarshal/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/remarshal-project%2Fremarshal/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/remarshal-project","download_url":"https://codeload.github.com/remarshal-project/remarshal/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254190396,"owners_count":22029632,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","command-line-tool","conversion","converter","json","messagepack","msgpack","toml","utility","yaml"],"created_at":"2024-07-31T19:01:50.335Z","updated_at":"2025-05-14T17:06:26.322Z","avatar_url":"https://github.com/remarshal-project.png","language":"Python","funding_links":[],"categories":["Python","Users"],"sub_categories":[],"readme":"# Remarshal\n\nConvert between CBOR, JSON, MessagePack, TOML, and YAML.\nWhen installed,\nRemarshal provides the command-line command `remarshal`\nas well as the short commands\n\u003ccode\u003e{cbor,json,msgpack,toml,yaml}2\u003cwbr\u003e{cbor,json,msgpack,py,toml,yaml}\u003c/code\u003e.\nYou can use these commands\nto convert between formats,\nreformat,\nand detect errors.\n\nRemarshal can also convert all supported formats to Python code.\n\n## Known limitations and quirks\n\n### YAML 1.2 only\n\nRemarshal works with YAML 1.2.\nThe last version that read and wrote YAML 1.1 was 0.17.1.\nInstall it if you need YAML 1.1.\n\n### Lossless by default; lossy must be enabled\n\nRemarshal tries to convert documents without losing information by default.\nThis means that a document converted from format A to B and then back to A should be equal to the original document.\nWhen a lossless conversion is impossible,\nRemarshal exits with an error.\n\nUse the command-line option `-k`/`--stringify` to relax this restriction.\nIt will make Remarshal do the following:\n\n- When converting to JSON,   turn boolean and null keys and date-time keys and values into strings.\n- When converting to TOML,\n  turn boolean, date-time, and null keys and null values into strings.\n\nThis is **usually what you want**.\nIt isn't the default as a safeguard against information loss.\n\n### Comments are removed\n\nRemarshal does not preserve or convert TOML and YAML comments.\n\n### Date-time conversion limitations\n\nThere are limitations\non what data can be converted\nbetween what formats.\n\n- CBOR, MessagePack, and YAML with binary fields cannot be converted\n  to JSON or TOML.\n  Binary fields can be converted between CBOR, MessagePack, and YAML.\n- The following date-time value conversions are possible:\n  - Local dates are converted between\n    [CBOR RFC 8943](https://www.rfc-editor.org/rfc/rfc8943.html)\n    dates (tag 1004),\n    [TOML Local Dates](https://toml.io/en/v1.0.0#local-date),\n    and\n    [YAML timestamps](https://yaml.org/spec/1.2.2/#tags)\n    without a time or a time zone.\n  - Local date-time is converted between\n    [TOML Local Date-Time](https://toml.io/en/v1.0.0#local-date-time)\n    and\n    [YAML timestamps](https://yaml.org/spec/1.2.2/#tags)\n    without a time zone.\n  - Date-time with a time zone\n    is converted between\n    [CBOR standard date-time strings](https://www.rfc-editor.org/rfc/rfc8949.html#stringdatetimesect)\n    (tag 0),\n    the\n    [MessagePack Timestamp extension type](https://github.com/msgpack/msgpack/blob/master/spec.md#timestamp-extension-type),\n    [TOML Offset Date-Times](https://toml.io/en/v1.0.0#offset-date-time),\n    and\n    [YAML timestamps](spec/1.2.2/#tags) with a time zone.\n- [TOML Local Time](https://toml.io/en/v1.0.0#local-time)\n  cannot be converted to a date-time in another format.\n- All date-time types can be converted to JSON\n  with the `-k`/`--stringify` option,\n  which turns them into strings.\n  Converting a document with a date-time type to JSON fails without `-k`/`--stringify`.\n- Remarshal converts YAML dates to TOML Local Dates.\n  It converts TOML Local Dates to YAML dates.\n\n### Python output\n\nConversion to Python code is one-way.\n\nThe Python output is either from\n[`repr`](https://docs.python.org/3/library/functions.html#repr)\n(the default) or formatted by\n[`pprint.pformat`](https://docs.python.org/3/library/pprint.html#pprint.pformat)\n(when you pass the option `--indent`).\nThe default `repr` format ignores `-s`/`--sort-keys`.\n\nThe style of `pprint`\nis probably not what you want your Python code to look like.\nApply your preferred Python formatter to it.\n\nThe Python output does not include the necessary `import` statements.\nYou may need to add `import datetime` before the data.\n\n## Installation\n\nYou will need Python 3.10 or later.\nEarlier versions of Python 3 will not work.\n\nThe recommended way to run Remarshal is to install the latest release\n[from PyPI](https://pypi.org/project/remarshal/)\nwith\n[pipx](https://github.com/pypa/pipx)\nor\n[uv](https://github.com/astral-sh/uv).\n\n```sh\npipx install remarshal\n# or\nuv tool install remarshal\n```\n\nRegular installation is not mandatory.\nThe command `pipx run remarshal [arg ...]` will download Remarshal and run it from a temporary location.\nIt will cache the downloaded version for up to 14 days.\nRemarshal will not be automatically upgraded during this period.\nYou can use `uvx remarshal [arg ...]` the same way.\n\nIt is also possible to install the current development version of Remarshal.\nPrefer releases unless you have a reason to run a development version.\n\n```sh\npipx install git+https://github.com/remarshal-project/remarshal\n# or\nuv tool install https://github.com/remarshal-project/remarshal\n```\n\n## Usage\n\n```none\nusage: remarshal [-h] [-v] [-f {cbor,json,msgpack,toml,yaml}] [-i \u003cinput\u003e]\n                 [--indent \u003cn\u003e] [-k] [--max-values \u003cn\u003e] [--multiline \u003cn\u003e]\n                 [-o \u003coutput\u003e] [-s] [-t {cbor,json,msgpack,python,toml,yaml}]\n                 [--unwrap \u003ckey\u003e] [--verbose] [--width \u003cn\u003e] [--wrap \u003ckey\u003e]\n                 [--yaml-style {,',\",|,\u003e}]\n                 [input] [output]\n\nConvert between CBOR, JSON, MessagePack, TOML, and YAML.\n\npositional arguments:\n  input                 input file\n  output                output file\n\noptions:\n  -h, --help            show this help message and exit\n  -v, --version         show program's version number and exit\n  -f, --from, --if, --input-format {cbor,json,msgpack,toml,yaml}\n                        input format\n  -i, --input \u003cinput\u003e   input file\n  --indent \u003cn\u003e          JSON and YAML indentation\n  -k, --stringify       turn into strings: boolean and null keys and date-time\n                        keys and values for JSON; boolean, date-time, and null\n                        keys and null values for TOML\n  --max-values \u003cn\u003e      maximum number of values in input data (default\n                        1000000, negative for unlimited)\n  --multiline \u003cn\u003e       minimum number of items to make non-nested TOML array\n                        multiline (default 6)\n  -o, --output \u003coutput\u003e\n                        output file\n  -s, --sort-keys       sort JSON, Python, and TOML keys instead of preserving\n                        key order\n  -t, --to, --of, --output-format {cbor,json,msgpack,python,toml,yaml}\n                        output format\n  --unwrap \u003ckey\u003e        only output the data stored under the given key\n  --verbose             print debug information when an error occurs\n  --width \u003cn\u003e           Python line width and YAML line width for long strings\n                        (integer or 'inf')\n  --wrap \u003ckey\u003e          wrap the data in a map type with the given key\n  --yaml-style {,',\",|,\u003e}\n                        YAML formatting style\n```\n\nInstead of `remarshal` with format arguments,\nyou can use a short command\n\u003ccode\u003e{cbor,json,msgpack,toml,yaml}2\u003cwbr\u003e{cbor,json,msgpack,py,toml,yaml}\u003c/code\u003e.\nThe `remarshal` command and the short commands\nexit with status 0 on success,\n1 on operational failure,\nand 2 on failure to parse the command line.\n\nIf no input argument `input`/`-i input` is given or its value is `-`,\nRemarshal reads input data from standard input.\nSimilarly,\nwith no `output`/`-o output` or an output argument that is `-`,\nRemarshal writes the result to standard output.\n\n### Wrappers\n\nThe options `--wrap` and `--unwrap` are available\nto solve the problem of converting CBOR, JSON, MessagePack, and YAML data to TOML\nwhen the top-level element of the data is not of a dictionary type\n(i.e., not a map in CBOR and MessagePack,\nan object in JSON,\nor an associative array in YAML).\nYou cannot represent such data as TOML directly;\nthe data must be wrapped in a dictionary first.\nPassing the option `--wrap some-key` to `remarshal` or one of its short commands\nwraps the input data in a \"wrapper\" dictionary with one key, `some-key`,\nwith the input data as its value.\nThe option `--unwrap some-key` does the opposite:\nit converts to the target format and outputs\nonly the value stored under the key `some-key`\nin the top-level dictionary element of the input data;\nthe rest of the input is discarded.\nIf the top-level element is not a dictionary or does not have the key `some-key`,\n`--unwrap some-key` causes an error.\n\nThe following shell transcript demonstrates the problem\nand how `--wrap` and `--unwrap` solve it:\n\n```\n$ echo '[{\"a\":\"b\"},{\"c\":[1,2,3]}]' | remarshal --from json --to toml\nError: cannot convert non-dictionary data to TOML; use \"--wrap\" to wrap it in a dictionary\n\n$ echo '[{\"a\":\"b\"},{\"c\":[1,2,3]}]' \\\n  | remarshal --from json --to toml --wrap main\n[[main]]\na = \"b\"\n\n[[main]]\nc = [1, 2, 3]\n\n$ echo '[{\"a\":\"b\"},{\"c\":[1,2,3]}]' \\\n  | remarshal --from json --wrap main - test.toml\n\n$ remarshal test.toml --to json\n{\"main\":[{\"a\":\"b\"},{\"c\":[1,2,3]}]}\n\n$ remarshal test.toml --of json --unwrap main\n[{\"a\":\"b\"},{\"c\":[1,2,3]}]\n```\n\n## Shell completions\n\nRemarshal provides shell-completion files for Bash and fish in the directory [`completions/`](completions/).\nYou can install fish completions automatically by running `install.fish`.\nYou will need to install Bash completions manually.\n\n## Examples\n\n### TOML to YAML\n\n```\n$ remarshal example.toml --of yaml\ntitle: TOML Example\nowner:\n  name: Tom Preston-Werner\n  organization: GitHub\n  bio: \"GitHub Cofounder \u0026 CEO\\nLikes tater tots and beer.\"\n  dob: 1979-05-27 07:32:00+00:00\ndatabase:\n  server: 192.168.1.1\n  ports:\n  - 8001\n  - 8001\n  - 8002\n  connection_max: 5000\n  enabled: true\nservers:\n  alpha:\n    ip: 10.0.0.1\n    dc: eqdc10\n  beta:\n    ip: 10.0.0.2\n    dc: eqdc10\n    country: 中国\nclients:\n  data:\n  - - gamma\n    - delta\n  - - 1\n    - 2\n  hosts:\n  - alpha\n  - omega\nproducts:\n- name: Hammer\n  sku: 738594937\n- name: Nail\n  sku: 284758393\n  color: gray\n```\n\n### JSON to TOML\n\n```\n$ curl -f 'https://archive-api.open-meteo.com/v1/era5?latitude=50.43\u0026longitude=30.52\u0026start_date=2014-10-05\u0026end_date=2014-10-05\u0026hourly=temperature_2m' \\\n  | remarshal --from json --to toml \\\n  ;\nlatitude = 50.439365\nlongitude = 30.476192\ngenerationtime_ms = 0.03254413604736328\nutc_offset_seconds = 0\ntimezone = \"GMT\"\ntimezone_abbreviation = \"GMT\"\nelevation = 147.0\n\n[hourly_units]\ntime = \"iso8601\"\ntemperature_2m = \"°C\"\n\n[hourly]\ntime = [\n    \"2014-10-05T00:00\",\n    \"2014-10-05T01:00\",\n    \"2014-10-05T02:00\",\n    \"2014-10-05T03:00\",\n    \"2014-10-05T04:00\",\n    \"2014-10-05T05:00\",\n    \"2014-10-05T06:00\",\n    \"2014-10-05T07:00\",\n    \"2014-10-05T08:00\",\n    \"2014-10-05T09:00\",\n    \"2014-10-05T10:00\",\n    \"2014-10-05T11:00\",\n    \"2014-10-05T12:00\",\n    \"2014-10-05T13:00\",\n    \"2014-10-05T14:00\",\n    \"2014-10-05T15:00\",\n    \"2014-10-05T16:00\",\n    \"2014-10-05T17:00\",\n    \"2014-10-05T18:00\",\n    \"2014-10-05T19:00\",\n    \"2014-10-05T20:00\",\n    \"2014-10-05T21:00\",\n    \"2014-10-05T22:00\",\n    \"2014-10-05T23:00\",\n]\ntemperature_2m = [\n    5.7,\n    5.3,\n    5.0,\n    4.8,\n    4.6,\n    4.6,\n    7.0,\n    8.9,\n    10.8,\n    12.2,\n    13.3,\n    13.9,\n    13.9,\n    13.7,\n    13.3,\n    12.3,\n    11.1,\n    10.2,\n    9.4,\n    8.5,\n    8.2,\n    7.9,\n    8.0,\n    7.8,\n]\n```\n\nRemarshal controls the number of items at which a TOML array becomes multiline,\nbut it does not control the line width.\nYou can use\n[`taplo fmt`](https://taplo.tamasfe.dev/cli/usage/formatting.html)\nfor finer TOML formatting.\n\n## License\n\nMIT.\nSee the file\n[`LICENSE`](LICENSE).\n\n`example.toml` from \u003chttps://github.com/toml-lang/toml\u003e.\n`example.cbor`,\n`example.json`,\n`example.msgpack`,\n`example.py`,\n`example.yml`,\n`tests/bin.msgpack`,\nand `tests/bin.yml`\nare derived from it.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fremarshal-project%2Fremarshal","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fremarshal-project%2Fremarshal","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fremarshal-project%2Fremarshal/lists"}