{"id":23029688,"url":"https://github.com/antononcube/raku-markdown-grammar","last_synced_at":"2025-08-14T12:34:46.700Z","repository":{"id":62971528,"uuid":"538786975","full_name":"antononcube/Raku-Markdown-Grammar","owner":"antononcube","description":"Markdown parser suitable for generation of notebooks of different kind (Mathematica, RMarkdown, Jupyter.)","archived":false,"fork":false,"pushed_at":"2024-06-07T23:57:57.000Z","size":210,"stargazers_count":7,"open_issues_count":1,"forks_count":2,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-06-08T00:45:38.626Z","etag":null,"topics":["markdown","markdown-to-html","mathematica","pod6","raku-module","rakulang"],"latest_commit_sha":null,"homepage":"https://raku.land/zef:antononcube/Markdown::Grammar","language":"Raku","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"artistic-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/antononcube.png","metadata":{"files":{"readme":"README-work.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-09-20T03:01:26.000Z","updated_at":"2024-06-07T23:58:01.000Z","dependencies_parsed_at":"2023-02-18T05:25:16.093Z","dependency_job_id":"a6469d8d-188f-4557-8eba-95820fe81f10","html_url":"https://github.com/antononcube/Raku-Markdown-Grammar","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antononcube%2FRaku-Markdown-Grammar","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antononcube%2FRaku-Markdown-Grammar/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antononcube%2FRaku-Markdown-Grammar/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antononcube%2FRaku-Markdown-Grammar/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/antononcube","download_url":"https://codeload.github.com/antononcube/Raku-Markdown-Grammar/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":229827651,"owners_count":18130395,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["markdown","markdown-to-html","mathematica","pod6","raku-module","rakulang"],"created_at":"2024-12-15T14:16:45.716Z","updated_at":"2024-12-15T14:16:46.250Z","avatar_url":"https://github.com/antononcube.png","language":"Raku","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Raku Markdown::Grammar\n\n## In brief\n\nMarkdown parser suitable for making converters of Markdown files into files of different kind of formats:\n- [X] DONE Mathematica notebook\n- [ ] TODO RMarkdown notebook\n- [X] DONE Jupyter notebook\n- [X] DONE Pod6 file\n- [X] DONE Org-mode file\n- [X] DONE HTML file\n- [X] DONE Raku data structure \n\nSee the video \n[\"Markdown to Mathematica converter (CLI and StackExchange examples)\"](https://www.youtube.com/watch?v=39ekokgnoqE), [AAv1],\nfor a (quick, 7.5 min) demo. \nThe video [AAv2] shows the conversion of a Jupyter notebook into a Mathematica notebook via Markdown representation.\nThe video [AAv3] shows conversions to POD6, Mathematica, and Jupyter and evaluations of those documents.\n(The evaluations are done with \n[\"Text::CodeProcessing\"](https://raku.land/zef:antononcube/Text::CodeProcessing), [AAp1].)\n\n### Motivation\n\n#### Mathematica notebooks\n\nI am most interested in generating Mathematica notebooks from Markdown. \n\nI have written a fair amount of Raku-related Markdown documents. Many of those Markdown documents\nwere generated from Mathematica notebooks using \n[M2MD](https://github.com/kubaPod/M2MD), [JPp1]. \nBut of course, most of the time, further changes and embellishments were made over those Markdown documents. \nHence it would be very nice to be able to go back to Mathematica.\n\n**Remark:** Raku can be used in Mathematica via the so called `RakuMode` -- see [AA1].\n\n**Remark:** Markdown documents with Raku code cells can be evaluated with Command Line Interface (CLI)\nscripts of the package \n[\"Text::CodeProcessing\"](https://raku.land/cpan:ANTONOV/Text::CodeProcessing), [AAp1]. \nFor more details see the article \n[\"Connecting Mathematica and Raku\"](https://rakuforprediction.wordpress.com/2021/12/30/connecting-mathematica-and-raku/), [AA1].\n\nHere is a flowchart that gives better idea of the workflow outlined above:\n\n```mermaid\ngraph TD\n    Rnb[Make Raku notebook in Mathematica]--\u003eLT\n    LT[Literate programming]--\u003eGE\n    GE{Good enough?}--\u003e|yes|CN\n    CN[Convert notebook to Markdown]--\u003eP\n    P[\"Publish (GitHUb/WordPress)\"]--\u003eSP\n    SP--\u003e|yes|R\n    R[Review and modify]--\u003eP\n    GE--\u003e|no|LT\n    SP{Needs \u003cbr /\u003e refinement?}--\u003e|no|SD\n    SD{Significantly \u003cbr /\u003e different?}--\u003e|yes|CM\n    SD--\u003e|no|PC\n    CM[Convert Markdown document to notebook]--\u003ePC\n    PC[\"Publish to notebook Community.wolfram.com\"]   \n```\n\n#### Pod6\n\nThe notebook formats have syntax that makes it hard to evaluate the conversions visually. \nThat is why I use Pod6 -- during development it is much easier to evaluate the Pod6 interpretations \nproduced by the package. \n(I.e. no need to use another tool to open and evaluate the obtained conversion artifact.)\n\n------\n\n## Installation\n\nFrom Zef ecosystem:\n\n```\nzef install Markdown::Grammar\n```\n\nFrom GitHub:\n\n```\nzef install https://github.com/antononcube/Raku-Markdown-Grammar.git\n```\n\n------\n\n## Round trip translation\n\nConsider the following round trip translation experiment:\n\n1. Make a Mathematica notebook\n\n2. Convert WL notebook into Markdown file with the Mathematica package [M2MD](https://github.com/kubaPod/M2MD)\n\n3. Convert the obtained Markdown file into Mathematica notebook using the Raku package \"Markdown::Grammar\"\n\n4. Compare the notebooks\n\n\nHere is the corresponding flowchart:\n\n```mermaid\ngraph TD\n    WL[Make a Mathematica notebook] --\u003e E\n    E[\"Examine notebook(s)\"] --\u003e M2MD\n    M2MD[\"Convert to Markdown with M2MD\"] --\u003e MG\n    MG[\"Convert to Mathematica with Markdown::Grammar\"] --\u003e |Compare|E\n```\n\n------\n\n## Related work\n\nHere is a table of converters from- or to Markdown:\n\n| From \\ To   | HTML                                                                                                                                                                                                                     | Jupyter                                                                                                               | Markdown                                                                    | Mathematica                                                                                                                                                                    | Pod6                                                                     | Org-mode                                                                                               | Raku                                                                     |\n|-------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------|\n| HTML        |                                                                                                                                                                                                                          | [pandoc](https://pandoc.org)                                                                                          | [pandoc](https://pandoc.org)                                                |                                                                                                                                                                                |                                                                          | [pandoc](https://pandoc.org)                                                                           |                                                                          |\n| Jupyter     | [Jupyter](https://jupyter.org)                                                                                                                                                                                           |                                                                                                                       | [Jupyter](https://jupyter.org), [jupytext](https://jupytext.readthedocs.io) |                                                                                                                                                                                |                                                                          |                                                                                                        |                                                                          |\n| Markdown    | [pandoc](https://pandoc.org), [Markit](https://raku.land/cpan:UZLUISF/Markit), [Text::Markdown](https://raku.land/zef:JJMERELO/Text::Markdown), [Markdown::Grammar](https://raku.land/zef:antononcube/Markdown::Grammar) | [jupytext](https://jupytext.readthedocs.io), [Markdown::Grammar](https://raku.land/zef:antononcube/Markdown::Grammar) |                                                                             | [Markdown2WL](https://github.com/dishmint/Markdown2WL), [md2nb](https://github.com/ConnorGray/md2nb), [Markdown::Grammar](https://raku.land/zef:antononcube/Markdown::Grammar) | [Markdown::Grammar](https://raku.land/zef:antononcube/Markdown::Grammar) | [pandoc](https://pandoc.org), [Markdown::Grammar](https://raku.land/zef:antononcube/Markdown::Grammar) | [Markdown::Grammar](https://raku.land/zef:antononcube/Markdown::Grammar) |\n| Mathematica |                                                                                                                                                                                                                          |                                                                                                                       | [M2MD](https://github.com/kubaPod/M2MD)                                     |                                                                                                                                                                                |                                                                          |                                                                                                        |                                                                          |\n| Pod6        |                                                                                                                                                                                                                          |                                                                                                                       | [Pod::To::Markdown](https://raku.land/cpan:SOFTMOTH/Pod::To::Markdown)      |                                                                                                                                                                                |                                                                          |                                                                                                        |                                                                          |\n| Org-mode    | [pandoc](https://pandoc.org), [Org-mode](https://orgmode.org)                                                                                                                                                            |                                                                                                                       | [pandoc](https://pandoc.org), [Org-mode](https://orgmode.org)               |                                                                                                                                                                                |                                                                          |                                                                                                        |                                                                          |\n\n\n**Remark:** [Pandoc](https://pandoc.org) attempts to be an universal converter, applicable for all couples of formats.\n\n**Remark:** Except \n[\"this package\"](https://github.com/antononcube/Raku-Markdown-Grammar)\nthere are no other converters from Markdown to \n[Pod6](https://docs.raku.org/language/pod).\n\n**Remark:** In general I like the idea of a Markdown-to-Mathematica converter written in Mathematica.\nThe package \n[\"Markdown2WL\"](https://github.com/dishmint/Markdown2WL) \nattempts that, but unfortunately it is fairly incomplete.\n\n**Remark:** When starting this project I did not know about Connor's package \n[\"md2nb\"](https://github.com/ConnorGray/md2nb). (Written in Rust.)\nIf I knew about it, most likely, I would have not written this package.\n\n------\n\n## Command line interface\n\nThe package provides a Command Line Interface (CLI) script, `from-markdown`. Here is its usage message:\n\n```shell\nfrom-markdown --help\n```\n\nThe CLI script `from-markdown` takes both file names and (Markdown) text. Here is an usage example for the latter:\n\n```shell\nfrom-markdown -to=pod6 'Here is data wrangling code:\n\n    obj = dfTitanic;\n    obj = GroupBy[ obj, #[\"passengerSex\"]\u0026 ];  \n    Echo[Map[ Length, obj], \"counts:\"]\n\n## References'\n```\n\n**Remark:** Using the CLI script with the option setting `--raku-cell-code-name=RakuInputExecute` produces\nRaku cells that require the Mathematica package \"RakuMode.m\", [AAp2], to be loaded and the command `RakuMode[]` \nto be executed in the result notebook. \n\n**Remark:** If CLI script's argument `--output` is a non-empty string and `--format` is \"whatever\" or \"automatic\",\nthen an attempt is made to conclude the format to convert to from the extension of the file name given to `--output`.\n\n\n------\n\n## Usage examples\n\n### Conversion of text\n\nConsider the following Markdown text:\n\n```perl6\nmy $mtext = q:to/END/;\nHere is data wrangling code:\n\n    obj = dfTitanic;\n    obj = GroupBy[ obj, #[\"passengerSex\"]\u0026 ];\n    Echo[Map[ Length, obj], \"counts:\"]\n\n## References\n\n### Articles\n\n[AA1] Anton Antonov,\n[\"Introduction to data wrangling with Raku\"](https://rakuforprediction.wordpress.com/2021/12/31/introduction-to-data-wrangling-with-raku/),\n(2021),\n[RakuForPrediction at WordPress](https://rakuforprediction.wordpress.com).\nEND\n\nsay $mtext.chars;\n```\n\nHere is the corresponding Mathematica notebook:\n\n```perl6\nuse Markdown::Grammar;\n\nfrom-markdown($mtext, to =\u003e 'mathematica')\n```\n\nHere is the corresponding Pod6 text:\n\n```perl6\nfrom-markdown($mtext, to =\u003e 'pod6')\n```\n\n### Section tree\n\nThe function `md-section-tree` extracts the Markdown blocks into a Hash-based tree that \ncorresponds to the hierarchical sections structure of the Markdown document:\n\n```perl6, results=asis\nuse Data::Translators;\nmd-section-tree($mtext) ==\u003e data-translation(field-names =\u003e \u003ctype level name content\u003e)\n```\n\nHere we just get the code blocks:\n\n```perl6\nmd-section-tree($mtext, modifier =\u003e 'code')\n```\n\n------\n\n## Parsing Markdown flavors\n\n### Obsidian \n\nThe parsing and interpretation of Obsidian flavor Markdown files is specified with the option \"flavor\".\nHere is an invocation within a Raku session:\n\n```\nfrom-markdown($mtext2, flavor =\u003e 'obsidian', to =\u003e 'jupyter');\n```\n\nHere is one of the ways to specify Obsidian file conversion via the CLI script:\n\n```\nfrom-markdown myObsidianFile.md -f obsidian -o myObsidianFile.ipynb  \n```\n\nOther examples are given in the test file \n[\"07-Jupyter-Obsidian.rakutest\"](https://github.com/antononcube/Raku-Markdown-Grammar/blob/main/t/07-Jupyter-Obsidian.rakutest).\n\nSee also the dedicated GitHub repository: [\"obsidian-2-jupyter\"](https://github.com/rcmlz/obsidian-2-jupyter), [RMr1].\n\n------\n\n## TODOs\n\nThe most important items are placed first.\n\n- [X] DONE Parsing bold, italic, code \"words.\"\n\n- [ ] TODO Parsing bold, italic, code \"phrases.\"\n\n- [X] DONE Parsing blocks with bold, italic formatting specs\n  \n- [X] DONE Parsing code blocks given with indentation.\n\n- [X] DONE Parsing and interpretation of \"deferred\" links. E.g. `[![enter image description here][1]][1]`.\n\n   - This is somewhat complicated, needing a \"second pass\" interpreter.\n   - The \"second pass\" is handled in the TOP action methods.\n   \n- [X] DONE Parsing quote lines and quote blocks\n\n- [X] DONE Parsing tables\n\n- [X] DONE Parsing LaTeX and/or [math blocks](https://github.com/fletcher/MultiMarkdown/wiki/MultiMarkdown-Syntax-Guide#math-support) \n\n- [X] DONE Parsing inlined LaTeX code.\n\n- [X] DONE Parsing LaTeX code with a docked cell button in the generated Mathematica notebook.\n\n- [X] DONE Have option(s) for default language for code cells.\n\n- [X] DONE Have option(s) for Raku code cell name.\n\n- [ ] TODO Have option(s) for skipping horizontal line specs.\n\n- [ ] TODO Parsing alternate syntax for heading 1 and 2\n\n- [ ] TODO Parsing escaping characters\n\n- [ ] TODO Parsing HTML\n\n- [ ] TODO Parsing most or all of the elements of the [extended syntax](https://www.markdownguide.org/extended-syntax/), [MC1] \n\n\n------\n\n## Acknowledgments\n\nMany thanks to Jakub (Kuba) Podkalicki for programming the package \"M2MD\", and helping me to understand\na fair amount of Mathematica's \n[low-Level notebook programming](https://reference.wolfram.com/language/guide/LowLevelNotebookProgramming.html).\n\nThanks to Rc Mlz for his request and guidance for parsing the Obsidian Markdown flavor and \ncreating the dedicated [Obsidian-to-Jupyter conversion repository](https://github.com/rcmlz/obsidian-2-jupyter), [RMr1].\n\n------\n\n## References\n\n### Articles\n\n[AA1] Anton Antonov,\n[\"Connecting Mathematica and Raku\"](https://rakuforprediction.wordpress.com/2021/12/30/connecting-mathematica-and-raku/),\n(2021),\n[RakuForPrediction at WordPress]([https://rakuforprediction.wordpress.com/).\n\n[FZ1] Faizon Zaman,\n[\"Parsing markdown files\"](https://community.wolfram.com/groups/-/m/t/2142852),\n(2021),\n[Community.wolfram.com](https://community.wolfram.com).\n\n### Guides\n\n[JG1] John Gruber, [Markdown: Syntax](https://daringfireball.net/projects/markdown/).\n\n[MC1] Matt Cone, [Markdown Guide](https://www.markdownguide.org).\n\n[RC1] Raku Community, [Raku Pod6](https://docs.raku.org/language/pod).\n\n### Packages, paclets\n\n[AAp1] Anton Antonov\n[Text::CodeProcessing Raku package](https://github.com/antononcube/Raku-Text-CodeProcessing),\n(2021-2022),\n[GitHub/antononcube](https://github.com/antononcube).\n\n[AAp2] Anton Antonov\n[RakuMode Wolfram Language paclet](https://resources.wolframcloud.com/PacletRepository/resources/AntonAntonov/RakuMode/),\n(2023),\n[Wolfram Language Paclet Repository](https://resources.wolframcloud.com/PacletRepository).\n\n[FZp1] Faizon Zaman,\n[Markdown2WL Mathematica package](https://github.com/dishmint/Markdown2WL),\n(2021),\n[GitHub/dishmint](https://github.com/dishmint/Markdown2WL).\n\n[JPp1] Jakub Podkalicki,\n[M2MD](https://github.com/kubaPod/M2MD),\n(2018-2022),\n[GitHub/kubaPod](https://github.com/kubaPod).\n\n[RMr1] Rc Mlz,\n[obsidian-2-jupyter](https://github.com/rcmlz/obsidian-2-jupyter),\n(2024),\n[GitHub/rcmlz](https://github.com/rcmlz).\n\n### Videos\n\n[AAv1] Anton Antonov,\n[\"Markdown to Mathematica converter (CLI and StackExchange examples)\"](https://www.youtube.com/watch?v=39ekokgnoqE),\n(2022),\n[Anton A. Antonov's channel at YouTube](https://www.youtube.com/channel/UC5qMPIsJeztfARXWdIw3Xzw).\n\n[AAv2] Anton Antonov,\n[\"Markdown to Mathematica converter (Jupyter notebook example)\"](https://www.youtube.com/watch?v=Htmiu3ZI05w),\n(2022),\n[Anton A. Antonov's channel at YouTube](https://www.youtube.com/channel/UC5qMPIsJeztfARXWdIw3Xzw).\n\n[AAv3] Anton Antonov,\n[\"Conversion and evaluation of Raku files\"](https://www.youtube.com/watch?v=GJO7YqjGn6o),\n(2022),\n[Anton A. Antonov's channel at YouTube](https://www.youtube.com/channel/UC5qMPIsJeztfARXWdIw3Xzw).","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantononcube%2Fraku-markdown-grammar","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fantononcube%2Fraku-markdown-grammar","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantononcube%2Fraku-markdown-grammar/lists"}