{"id":15168543,"url":"https://github.com/antononcube/raku-data-translators","last_synced_at":"2026-01-22T02:02:49.895Z","repository":{"id":189004283,"uuid":"679852316","full_name":"antononcube/Raku-Data-Translators","owner":"antononcube","description":"Raku package for translation of JSON specs or JSON-like data structures into other formats.","archived":false,"fork":false,"pushed_at":"2025-01-26T16:01:29.000Z","size":127,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-15T09:33:23.652Z","etag":null,"topics":["data-visualization","html","raku","rakulang"],"latest_commit_sha":null,"homepage":"https://raku.land/zef:antononcube/Data::Translators","language":"Raku","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/antononcube.png","metadata":{"files":{"readme":"README-work.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-17T19:04:23.000Z","updated_at":"2025-01-26T16:01:33.000Z","dependencies_parsed_at":"2024-11-05T18:47:20.236Z","dependency_job_id":"1fb1306f-6c33-4cef-93ee-8ff5cee91d45","html_url":"https://github.com/antononcube/Raku-Data-Translators","commit_stats":{"total_commits":78,"total_committers":1,"mean_commits":78.0,"dds":0.0,"last_synced_commit":"7edb9dd6946b455358ae57e9dd95ef8ebcc77b51"},"previous_names":["antononcube/raku-json-translators","antononcube/raku-data-translators"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antononcube%2FRaku-Data-Translators","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antononcube%2FRaku-Data-Translators/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antononcube%2FRaku-Data-Translators/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antononcube%2FRaku-Data-Translators/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/antononcube","download_url":"https://codeload.github.com/antononcube/Raku-Data-Translators/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248058996,"owners_count":21040871,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-visualization","html","raku","rakulang"],"created_at":"2024-09-27T06:22:14.785Z","updated_at":"2026-01-22T02:02:49.882Z","avatar_url":"https://github.com/antononcube.png","language":"Raku","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Data::Translators \n\nRaku package for translation of JSON specs or JSON-like data structures into other formats.\n\nIt is envisioned this package to have translators to multiple formats. For example:\n- [X] DONE HTML\n- [X] DONE JSON\n- [X] DONE R\n- [X] DONE WL\n- [ ] TODO Plain text\n- [ ] TODO Python\n- [ ] TODO Mermaid-JS\n- [ ] TODO Julia\n- [ ] TODO SQL\n\nThe main motivation for making the package is to have convenient way of making tables while doing \n[Literate programming](https://en.wikipedia.org/wiki/Literate_programming) \nwith Raku using:\n\n- Computational Markdown documents, [AAp4]\n- Jupyter notebooks, [BDp1]\n- Mathematica notebooks, [AAp4]\n\nThe use of JSON came to focus, since when working with Large Language Model (LLM) functions, [AAp3],\nvery often it is requested from LLMs to produce output in JSON format, [AA1, AA2].\n\nThe package \"Data::Reshapers\", [AAp1], would complement nicely \"Data::Translators\" and vice versa.\nThe package \"Data::TypeSystem\", [AAp2], is used for \"translation decisions\" and for conversions into more regular datasets. \n\nThe package \"Mathematica::Serializer\", [AAp5], has a very similar mission --\nit is for translating Raku data structures into Mathematica (aka Wolfram Language or WL) code.\n\nIn order to utilize \"Data::Translators\" while doing Literate programming with:\n- Computational Markdown files, then use the code chunk argument `results=asis`.\n- Jupyter notebooks, then use the code cell magic spec `%%html` or `%% \u003e html`.\n\nOne can find concrete examples for:\n- Computational Markdown files, in the [raw source code of this README](https://raw.githubusercontent.com/antononcube/Raku-Data-Translators/main/README.md)\n- Jupyter notebooks, in the [magics examples notebook](https://github.com/bduggan/raku-jupyter-kernel/blob/master/eg/magics.ipynb) at [BDp1]\n\n**Remark:** The provided converters are made for communication purposes, so they might not be\nvery performant. I have used or tested them with datasets that have less than 5000 rows.\n\n------\n\n## Installation\n\nPackage installations from both sources use [zef installer](https://github.com/ugexe/zef)\n(which should be bundled with the \"standard\" Rakudo installation file.)\n\nTo install the package from [Zef ecosystem](https://raku.land/) use the shell command:\n\n```\nzef install Data::Translators\n```\n\nTo install the package from the GitHub repository use the shell command:\n\n```\nzef install https://github.com/antononcube/Raku-JSON-Translators.git\n```\n\n\n------\n\n## Basic usage\n\n### Main use case\n\nHere is a \"main use case\" example:\n1. Get a dataset that is an array of hashes\n2. Filter or sample the records\n3. Make an HTML table with those records\n\nThe HTML table outputs can be used to present datasets nicely in:\n- Markdown documents \n- Jupyter notebooks\n\nHere we get the Titanic dataset and sample it:\n\n```perl6\nuse Data::Reshapers;\nuse Data::TypeSystem;\nuse Data::Translators;\n\nmy $tbl = get-titanic-dataset.pick(3);\n```\n\nHere is the corresponding dataset type:\n\n```perl6\ndeduce-type($tbl);\n```\n\nHere is the corresponding HTML table:\n\n```perl6, results=asis\n$tbl ==\u003e data-translation\n```\n\nWe can specify field names and HTML table attributes:\n\n```perl6, results=asis\n$tbl ==\u003e data-translation(field-names =\u003e \u003cid passengerSurvival\u003e, table-attributes =\u003e 'id=\"info-table\" class=\"table table-bordered table-hover\" text-align=\"center\"');\n```\n\nHere is how the transposed dataset is tabulated:\n\n```perl6, results=asis\n$tbl ==\u003e transpose() ==\u003e data-translation;\n```\n\n### From JSON strings\n\nHere is a JSON string translation to HTML:\n\n```perl6, results=asis\nmy $json1 = q:to/END/;\n{\n    \"sample\": [\n        {\"name\": \"json2html\", \"desc\": \"coverts json 2 html table format\", \"lang\": \"python\"},\n        {\"name\": \"testing\", \"desc\": \"clubbing same keys of array of objects\", \"lang\": \"python\"}\n    ]\n}\nEND\n\ndata-translation($json1);\n```\n\n### From HTML strings\n\nGet the data of an HTML table as a Raku dataset (array of hashmaps). Here is an HTML table string:\n\n```raku\nsink my $html = q:to/END/;\n\u003ctable\u003e\n    \u003ctr\u003e\n        \u003cth\u003eName\u003c/th\u003e\n        \u003cth\u003eAge\u003c/th\u003e\n        \u003cth\u003eCity\u003c/th\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003eJohn\u003c/td\u003e\n        \u003ctd\u003e25\u003c/td\u003e\n        \u003ctd\u003eNew York\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003eAlice\u003c/td\u003e\n        \u003ctd\u003e30\u003c/td\u003e\n        \u003ctd\u003eLondon\u003c/td\u003e\n    \u003c/tr\u003e\n\u003c/table\u003e\nEND\n```\n\nHere is the Raku dataset:\n\n```raku\ndata-translation($html, target =\u003e 'dataset')\n```\n\n### Cross-tabulated data\n\nHere is a more involved data example:\n\n```perl6, results=asis\ndata-translation(cross-tabulate(get-titanic-dataset, 'passengerSex', 'passengerSurvival'))\n```\n\nCompare the HTML table above with the following plain text table:\n\n```perl6\nto-pretty-table(cross-tabulate(get-titanic-dataset, 'passengerSex', 'passengerSurvival'))\n```\n\n### Generation of R and WL code\n\nHere is the R code version of the Titanic data sample:\n\n```perl6, output.lang=r, output.prompt=NONE\n$tbl ==\u003e data-translation(target =\u003e 'R', field-names =\u003e \u003cid passengerClass passengerSex passengerAge passengerSurvival\u003e)\n```\n\nHere is the R code version of the contingency table:\n\n```perl6, output.lang=r, output.prompt=NONE\ndata-translation(cross-tabulate(get-titanic-dataset, 'passengerSex', 'passengerSurvival'), target =\u003e 'R')\n```\n\nHere is the WL code version of the contingency table:\n\n```perl6, output.lang=r, output.prompt=NONE\ndata-translation(cross-tabulate(get-titanic-dataset, 'passengerSex', 'passengerSurvival'), target =\u003e 'WL')\n```\n\n### Nicer datasets\n\nIn order to obtain datasets or more regular datasets the function `to-dataset` can be used.\nHere a rugged dataset is made regular and converted to an HTML table:\n\n```perl6, results=asis\nmy @tbl2 = get-titanic-dataset.pick(6);\n@tbl2 = @tbl2.map({ $_.pick((1..5).pick).Hash });\n@tbl2 ==\u003e to-dataset(missing-value=\u003e'・') ==\u003e data-translation\n```\n\nHere a hash is transformed into dataset with columns `\u003cKey Value\u003e` and then converted into an HTML table:\n\n```perl6, results=asis\n{ 4 =\u003e 'a', 5 =\u003e 'b', 8 =\u003e 'c'} ==\u003e to-dataset() ==\u003e data-translation\n```\n\n------\n\n## Implementation notes\n\n- The \"need\" for this package became evident while working on the notebooks/articles [AA1, AA2]. \n- Initially, I translated plain text tables into HTML.\n  - Using LLMs or `md-interpret` provided by \"Markdown::Grammar\".\n- I considered re-using the code behind `to-pretty-table` provided by \"Data::Reshapers\", [AAp1].\n  - This is \"too much work\" and I wanted a lighter weight package.\n- Having a solution for the more general problem ***translating JSON to HTML*** seemed a much better and easier option.  \n  - For example, I hoped that someone has already solved that problem for Raku.\n- Since I did not find Raku packages for the translation I wanted, I looked for solutions into the Python ecosystem.\n  - ... And found [\"json2html\"](https://github.com/softvar/json2html).\n- Using ChatGPT-4.0 I translated the only class of that package from Python into Raku.\n- The obtained translation could be executed with relatively minor changes.\n  - I further refactored and enhanced the HTML translator to fit my most frequent Raku workflows.\n- The ingestion of JSON strings is done with the package [\"JSON::Fast\"](https://raku.land/cpan:TIMOTIMO/JSON::Fast).\n  - Hence the conversion *to* JSON \"comes for free\" using `to-json` from that package.\n- The initial versions of the package did not have the \"umbrella\" function `data-translation`.\n  - Only the \"lower level\" functions `json-to-html` and `json-to-r` were provided. \n- The \"lower level\" functions, or shortcuts, can be used: `to-html`, `to-r`, `to-wl`.\n\n------\n\n## CLI\n\nThe package provides a Command Line Interface (CLI) script. Here is its usage message:\n\n\n```shell\ndata-translation --help\n```\n\nHere is an example application (to [this file](./resources/professionals.json)):\n\n```shell, results=asis\ndata-translation ./resources/professionals.json --field-names='data;id;name;age;profession'\n```\n\n\n------\n\n## References\n\n### Articles \n\n[AA1] Anton Antonov, \n[\"Workflows with LLM functions\"](https://rakuforprediction.wordpress.com/2023/08/01/workflows-with-llm-functions/), \n(2023), \n[RakuForPrediction at WordPress](https://rakuforprediction.wordpress.com).\n\n[AA2] Anton Antonov,\n[\"TLDR LLM solutions for software manuals\"](https://rakuforprediction.wordpress.com/2023/08/15/tldr-llm-solutions-for-software-manuals/),\n(2023),\n[RakuForPrediction at WordPress](https://rakuforprediction.wordpress.com).\n\n\n### Packages\n\n[AAp1] Anton Antonov,\n[Data::Reshapers Raku package](https://github.com/antononcube/Raku-Data-Reshapers),\n(2021-2023),\n[GitHub/antononcube](https://github.com/antononcube).\n\n[AAp2] Anton Antonov,\n[Data::TypeSystem Raku package](https://github.com/antononcube/Raku-Data-TypeSystem),\n(2023),\n[GitHub/antononcube](https://github.com/antononcube).\n\n[AAp3] Anton Antonov, \n[LLM::Functions Raku package](https://github.com/antononcube/Raku-LLM-Functions), \n(2023), \n[GitHub/antononcube](https://github.com/antononcube).\n\n[AAp4] Anton Antonov,\n[Text::CodeProcessing Raku package](https://github.com/antononcube/Raku-Text-CodeProcessing),\n(2021-2023),\n[GitHub/antononcube](https://github.com/antononcube).\n\n[AAp5] Anton Antonov,\n[Mathematica::Serializer Raku package](https://github.com/antononcube/Raku-Mathematica-Serializer),\n(2021-2022),\n[GitHub/antononcube](https://github.com/antononcube).\n\n[BDp1] Brian Duggan,\n[Jupyter:Kernel Raku package](https://github.com/bduggan/raku-jupyter-kernel),\n(2017-2023),\n[GitHub/bduggan](https://github.com/bduggan).\n\n[VMp1] Varun Malhotra,\n[json2html Python package](https://github.com/softvar/json2html),\n(2013-2021),\n[GitHub/softvar](https://github.com/softvar).","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantononcube%2Fraku-data-translators","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fantononcube%2Fraku-data-translators","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantononcube%2Fraku-data-translators/lists"}