{"id":16117294,"url":"https://github.com/nickbabcock/jomini","last_synced_at":"2025-04-30T16:27:10.830Z","repository":{"id":26665454,"uuid":"30121926","full_name":"nickbabcock/jomini","owner":"nickbabcock","description":"Parses Paradox files into javascript objects","archived":false,"fork":false,"pushed_at":"2025-03-04T14:04:36.000Z","size":349,"stargazers_count":81,"open_issues_count":0,"forks_count":9,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-10T11:37:31.910Z","etag":null,"topics":["ck3","eu4","hoi4","imperator","paradox","parser"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nickbabcock.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-01-31T18:37:17.000Z","updated_at":"2025-03-05T20:10:27.000Z","dependencies_parsed_at":"2024-02-19T01:40:06.210Z","dependency_job_id":"d02abab1-34ba-465f-af95-8d47ec2720bd","html_url":"https://github.com/nickbabcock/jomini","commit_stats":{"total_commits":324,"total_committers":7,"mean_commits":"46.285714285714285","dds":"0.043209876543209846","last_synced_commit":"5c744b386e7eee5b18541a404f220122dd8e42c9"},"previous_names":[],"tags_count":44,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nickbabcock%2Fjomini","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nickbabcock%2Fjomini/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nickbabcock%2Fjomini/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nickbabcock%2Fjomini/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nickbabcock","download_url":"https://codeload.github.com/nickbabcock/jomini/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251741428,"owners_count":21636252,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ck3","eu4","hoi4","imperator","paradox","parser"],"created_at":"2024-10-09T20:43:52.316Z","updated_at":"2025-04-30T16:27:10.785Z","avatar_url":"https://github.com/nickbabcock.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"![CI](https://github.com/nickbabcock/jomini/workflows/CI/badge.svg)\n[![npm](https://img.shields.io/npm/v/jomini.svg)](http://npm.im/jomini)\n[![size](https://badgen.net/bundlephobia/minzip/jomini)](https://bundlephobia.com/package/jomini)\n\n# Jomini.js\n\nJomini is a javascript library that is able to read and write **plaintext** save and game files from Paradox Development Studios produced on the Clausewitz engine (Europa Universalis IV (eu4), Crusader Kings III (ck3), Hearts of Iron 4 (hoi4), Stellaris, and others)\n\n\u003e Aside: it's only by happenstance that this library and Paradox's own code share the same name (this library is older by several years).\n\n## Features:\n\n- ✔ Compatibility: Node 12+ and \u003e90% of browsers\n- ✔ Speed: Parse at over 200 MB/s\n- ✔ Correctness: The same parser underpins a [EU4 save file analyzer](https://pdx.tools), and the [Paradox Game Converters's](https://github.com/ParadoxGameConverters/EU4toVic2) ironman to plaintext converter\n- ✔ Ergonomic: Data parsed into plain javascript objects or JSON\n- ✔ Self-contained: zero runtime dependencies\n- ✔ Small: Less than 100 KB gzipped (or 70 KB when using the [slim entrypoint](#slim-module))\n\n## Quick Start\n\nQuick and easy way to add jomini to your project:\n\n```html\n\u003cbody\u003e\n  \u003cscript src=\"https://cdn.jsdelivr.net/npm/jomini@0.8.0/dist/umd/index.min.js\"\u003e\u003c/script\u003e\n  \u003cscript\u003e\n    jomini.Jomini.initialize().then((parser) =\u003e {\n      const out = parser.parseText(\"foo=bar\");\n      alert(`the value of foo is ${out.foo}`);\n    });\n  \u003c/script\u003e\n\u003c/body\u003e\n```\n\nOr if you want a more efficient way to get started:\n\n```html\n\u003cscript type=\"module\"\u003e\n  import { Jomini } from 'https://cdn.jsdelivr.net/npm/jomini@0.8.0/dist/es-slim/index_slim.min.js';\n\n  const wasmUrl = 'https://cdn.jsdelivr.net/npm/jomini@0.8.0/dist/jomini.wasm';\n  Jomini.initialize({ wasm: wasmUrl })\n    .then((parser) =\u003e {\n      const out = parser.parseText('foo=bar');\n      alert(`the value of foo is ${out.foo}`);\n    });\n\u003c/script\u003e\n```\n\nOr if Node.js is targeted or one is bundling this inside a larger application:\n\n```bash\nnpm i jomini\n```\n\n## Example\n\n```js\nimport { Jomini } from \"jomini\";\n\nconst data = `\n    date=1640.7.1\n    player=\"FRA\"\n    savegame_version={\n        first=1\n        second=9\n        third=2\n        forth=0\n    }`\n\nconst parser = await Jomini.initialize();\nconst out = parser.parseText(data);\n```\n\nWill return the following:\n\n```js\nout = {\n    date: new Date(Date.UTC(1640, 6, 1)),\n    player: \"FRA\",\n    savegame_version: {\n        first: 1,\n        second: 9,\n        third: 2,\n        forth: 0\n    }\n}\n```\n\n## Encoding\n\nIt's preferable to pass in raw bytes to `parseText` and to optionally pass in an encoding (which defaults to `utf8`) instead of passing in a string as this tends to be more efficient.\n\nIf passing in bytes to `parseText`, make sure to specify an encoding, else the strings could be deserialized incorrectly:\n\n```js\nconst jomini = await Jomini.initialize();\nconst data = new Uint8Array([0xff, 0x3d, 0x8a]);\nconst out = jomini.parseText(data, { encoding: \"windows1252\" });\n// out = { ÿ: \"Š\" }\n```\n\n## Type Narrowing\n\nBy default, jomini will attempt to narrow all values to more specific types like numbers, dates, or booleans. Type narrowing can be configured to only occur for unquoted values or disabled altogether.\n\n```js\nconst jomini = await Jomini.initialize();\nconst { root, json } = jomini.parseText(\n  'a=\"01\" b=02 c=\"yes\" d=no',\n  {\n    typeNarrowing: \"unquoted\",\n  },\n  (q) =\u003e ({ root: q.root(), json: q.json() })\n);\n\nexpect(root).toEqual({\n  a: \"01\",\n  b: 2,\n  c: \"yes\",\n  d: false,\n});\n\nexpect(json).toEqual('{\"a\":\"01\",\"b\":2,\"c\":\"yes\",\"d\":false}');\n```\n\n## Performance\n\n95-99% of the time it takes to parse a file is creating the Javascript object, so it is preferable if one can slim the object down as much as possible. This is why `parseText` accepts a callback where one can provide JSON pointer like strings to extract only the data that is necessary.\n\nBelow shows an example of extracting the player's prestige from an EU4 save file\n\n```js\nconst buffer = readFileSync(args[0]);\nconst parser = await Jomini.initialize();\nconst { player, prestige } = parser.parseText(\n  buffer,\n  { encoding: \"windows1252\" },\n  (query) =\u003e {\n    const player = query.at(\"/player\");\n    const prestige = query.at(`/countries/${player}/prestige`);\n    return { player, prestige };\n  }\n);\n```\n\nThe alternative would be:\n\n```js\nconst buffer = readFileSync(args[0]);\nconst parser = await Jomini.initialize();\nconst save = parser.parseText(buffer, { encoding: \"windows1252\" });\nconst player = save.player;\nconst prestige = save.countries[player].prestige;\n```\n\nThe faster version completes 40x faster (6.3s vs 0.16s) and uses about half the memory.\n\n## JSON\n\nThere is a middle ground in terms of performance and flexibility: using JSON as an intermediate layer:\n\n```js\nconst buffer = readFileSync(args[0]);\nconst parser = await Jomini.initialize();\nconst out = parser.parseText(buffer, { encoding: \"windows1252\" }, (q) =\u003e q.json());\nconst save = JSON.parse(out);\nconst player = save.player;\nconst prestige = save.countries[player].prestige;\n```\n\nThe keys of the stringified JSON object are in the order as they appear in the file, so this makes the JSON approach well suited for parsing files where the order of object keys matter. The other APIs are subjected to natively constructed JS objects reordering keys to suit their fancy. To process the JSON and not lose key order, you'll want to leverage a streaming JSON parser like [oboe.js](https://github.com/jimhigson/oboe.js) or [stream-json](https://github.com/uhop/stream-json).\n\nInterestingly, even though using JSON adds a layer, constructing and parsing the JSON into a JS object is still 3x faster than when a JS object is constructed directly. This must be a testament to how tuned browser JSON parsers are.\n\nThe JSON format does not change how dates are encoded, so dates are written into the JSON exactly as they appear in the original file.\n\nThe JSON generator contains options to tweak the output.\n\nTo pretty print the output:\n\n```js\nparser.parseText(buffer, { }, (q) =\u003e q.json({ pretty: true }));\n```\n\nThere is an option to decide how duplicate keys are serialized. For instance, given the following data:\n\n```\ncore=\"AAA\"\ncore=\"BBB\"\n```\n\nThe default behavior will group the two fields into one list, as shown below. This favors ergonomics, as the builtin `JSON.parse` doesn't handle duplicate keys well.\n\n```json\n{\n  \"core\": [\"AAA\", \"BBB\"]\n}\n```\n\nIf this behavior is not desirable, it can be tweaked such that the duplicate keys are preserved:\n\n```js\nparser.parseText(buffer, { }, (q) =\u003e q.json({ duplicateKeyMode: \"preserve\" }));\n```\n\nwill output:\n\n```json\n{\n  \"core\": \"AAA\",\n  \"core\": \"BBB\"\n}\n```\n\nWhether or not the above is [valid JSON is debateable](https://stackoverflow.com/q/21832701).\n\nThe remaining mode transforms key value objects to an array of key value tuples:\n\n```js\nparser.parseText(buffer, { }, (q) =\u003e q.json({ duplicateKeyMode: \"key-value-pairs\" }));\n```\n\nwill output:\n\n```json\n{\n  \"type\": \"obj\",\n  \"val\": [\n    [\"core\", \"AAA\"],\n    [\"core\", \"BBB\"]\n  ]\n}\n```\n\nThe output is ugly and verbose, but it's valid JSON and preserves the original structure. Arrays will have the type of `array`.\n\n## Data Mangling\n\nThe PDS data format is ambiguous without additional context in certain situations. A great example of this are EU4 armies. If a country only has a single army, then the parser will assume that `army` is singular object instead of an array. This can also been seen with individual units nested in an `army`. Below is an example of two armies, one army has a single unit while the other has multiple.\n\n```\narmy={\n  name=\"1st army\"\n  unit={ name=\"1st unit\" }\n}\narmy={\n  name=\"2nd army\"\n  unit={ name=\"2nd unit\" }\n  unit={ name=\"3rd unit\" }\n}\n```\n\nWithout intervention the parsed structure will be:\n\n```js\nout = {\n  army: [\n    {\n      name: \"1st army\",\n      unit: { name: \"1st unit\", },\n    },\n    {\n      name: \"2nd army\",\n      unit: [\n        { name: \"2nd unit\", },\n        { name: \"3rd unit\", },\n      ],\n    },\n  ],\n}\n\n// `army[0].unit` is an object\n// `army[1].unit` is an array! \n```\n\nThis is remedied by passing the parsed struct through `toArray` and targeting the `army.unit` property \n\n```js\ntoArray(obj, \"army.unit\");\nconst expected = {\n  army: [\n    {\n      name: \"1st army\",\n      unit: [{ name: \"1st unit\", }, ],\n    },\n    {\n      name: \"2nd army\",\n      unit: [\n        { name: \"2nd unit\", },\n        { name: \"3rd unit\", },\n      ],\n    },\n  ],\n};\n```\n\n## Write API\n\nThe write API is low level in order to clear out any ambiguities that may arise from a higher level API.\n\n```js\nconst jomini = await Jomini.initialize();\nconst out = jomini.write((writer) =\u003e {\n  writer.write_unquoted(\"data\");\n  writer.write_object_start();\n  writer.write_unquoted(\"settings\");\n  writer.write_array_start();\n  writer.write_integer(0);\n  writer.write_integer(1);\n  writer.write_end();\n  writer.write_unquoted(\"name\");\n  writer.write_quoted(\"world\");\n  writer.write_end();\n  writer.write_unquoted(\"color\");\n  writer.write_header(\"rgb\");\n  writer.write_array_start();\n  writer.write_integer(100);\n  writer.write_integer(150);\n  writer.write_integer(74);\n  writer.write_end();\n  writer.write_unquoted(\"start\");\n  writer.write_date(new Date(Date.UTC(1444, 10, 11)));\n});\n```\n\nThe return value will be a byte array that contains the following:\n\n```plain\ndata={\n  settings={\n    0 1\n  }\n  name=\"world\"\n}\ncolor=rgb {\n  100 150 74\n}\nstart=1444.11.11\n```\n\nThere is not yet an official high level API to write out arbitrary objects; however, one can adapt [this solution](https://github.com/nickbabcock/jomini/issues/5#issuecomment-1564253958) until a high level API is decided to be implemented. \n\n## Slim Module\n\nBy default, the `jomini` entrypoint includes Wasm that is base64 inlined. This is the default as most developers will probably not need to care. However some developers will care: those running the library in environments where Wasm is executable but not compilable or those who are ambitious about reducing compute and bandwidth costs for their users.\n\nTo cater to these use cases, there is a `jomini/slim` package that operates the exactly the same except now it is expected for developers to prime initialization through some other means:\n\n```js\nimport { Jomini } from \"jomini/slim\";\nimport wasm from \"jomini/jomini.wasm\";\n\nconst data = `player=\"FRA\"`;\nconst parser = await Jomini.initialize({ wasm });\nconst out = parser.parseText(data);\n```\n\n## Deno\n\nDeno is currently supported through their npm specifier. Jomini requires `--allow-read` permissions.\n\n```ts\nimport { Jomini } from \"npm:jomini@0.8.0\";\n\nconst data = await Deno.readAll(Deno.stdin);\nconst parser = await Jomini.initialize();\nconst out = parser.parseText(\n  data,\n  { encoding: \"windows1252\" },\n  (query) =\u003e query.json(),\n);\n\nconsole.log(out);\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnickbabcock%2Fjomini","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnickbabcock%2Fjomini","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnickbabcock%2Fjomini/lists"}