{"id":23029726,"url":"https://github.com/antononcube/raku-text-codeprocessing","last_synced_at":"2026-02-26T10:27:09.467Z","repository":{"id":47844001,"uuid":"383308299","full_name":"antononcube/Raku-Text-CodeProcessing","owner":"antononcube","description":"Raku package for processing of code chunks in documents of different types (like Markdown, Org-mode, Pod6.)","archived":false,"fork":false,"pushed_at":"2025-02-21T12:43:57.000Z","size":114,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-06-15T12:07:48.593Z","etag":null,"topics":["evaluation","literate-programming","markdown","org-mode","pod6","tangle"],"latest_commit_sha":null,"homepage":"https://raku.land/zef:antononcube/Text::CodeProcessing","language":"Raku","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"artistic-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/antononcube.png","metadata":{"files":{"readme":"README-work.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-07-06T01:45:39.000Z","updated_at":"2025-02-21T12:44:01.000Z","dependencies_parsed_at":"2023-12-16T22:22:17.920Z","dependency_job_id":"22294ff8-e595-47a5-b482-9f8a4ba1096b","html_url":"https://github.com/antononcube/Raku-Text-CodeProcessing","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/antononcube/Raku-Text-CodeProcessing","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antononcube%2FRaku-Text-CodeProcessing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antononcube%2FRaku-Text-CodeProcessing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antononcube%2FRaku-Text-CodeProcessing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antononcube%2FRaku-Text-CodeProcessing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/antononcube","download_url":"https://codeload.github.com/antononcube/Raku-Text-CodeProcessing/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antononcube%2FRaku-Text-CodeProcessing/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29856573,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-26T08:51:08.701Z","status":"ssl_error","status_checked_at":"2026-02-26T08:50:19.607Z","response_time":89,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["evaluation","literate-programming","markdown","org-mode","pod6","tangle"],"created_at":"2024-12-15T14:16:57.518Z","updated_at":"2026-02-26T10:27:09.452Z","avatar_url":"https://github.com/antononcube.png","language":"Raku","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Raku Text::CodeProcessing\n\n[![Build Status](https://app.travis-ci.com/antononcube/Raku-Text-CodeProcessing.svg?branch=main)](https://app.travis-ci.com/github/antononcube/Raku-Text-CodeProcessing)\n[![License: Artistic-2.0](https://img.shields.io/badge/License-Artistic%202.0-0298c3.svg)](https://opensource.org/licenses/Artistic-2.0)\n\n## In brief\n\nThe main goal of this package is to facilitate \n[Literate Programming](https://en.wikipedia.org/wiki/Literate_programming)\nwith Raku.\n\nThe package has functions and a script for the evaluations of\ncode chunks in documents of different types (like \n[Markdown](https://daringfireball.net/projects/markdown/), \n[Org Mode](https://orgmode.org), \n[Pod6](https://docs.raku.org/language/pod).)\n\nThere is also a script for extracting code chunks.\n\n------\n\n## Installation\n\nPackage installations from both sources use [zef installer](https://github.com/ugexe/zef)\n(which should be bundled with the \"standard\" [Rakudo](https://rakudo.org) installation file.)\n\nTo install the package from [Zef ecosystem](https://raku.land)\nuse the shell command:\n\n```\nzef install Text::CodeProcessing\n```\n\nTo install the package from the GitHub repository use the shell command:\n\n```\nzef install https://github.com/antononcube/Raku-Text-CodeProcessing.git\n```\n\n------\n\n## Usage\n\n### Main function\n\nThe package provides the function `FileCodeChunksEvaluation` for the \nevaluation of code chunks in files. The first argument is a file name string:\n\n```\nFileCodeChunksEvaluation( $fileName, ... )\n```\nHere are the (optional) parameters:\n\n- `Str :$outputFileName` : output file name\n  \n- `Str :$evalOutputPrompt = 'AUTO'` : code chunk output prompt\n\n- `Str :$evalErrorPrompt = 'AUTO'` : code chunk error prompt\n\n- `Bool :$noteOutputFileName = False` : whether to print out the name of the new file\n\n- `Bool :$promptPerLine = True` : whether to put prompt to each output or error line or just the first one\n\nWhen the prompt arguments are given the value `'AUTO'` then the actual prompt values are selected according to the file type:\n\n- Markdown : `evalOutputPrompt = '# '`, `evalErrorPrompt = '#ERROR: '`\n\n- Org-mode : `evalOutputPrompt = ': '`, `evalErrorPrompt = ':ERROR: '`\n\n- Pod6 : `evalOutputPrompt = '# '`, `evalErrorPrompt = '#ERROR: '`\n\n-------\n\n## Document parameters\n\nA document can have an YAML header. If that header contains parameter specifications\nthen the corresponding parameters values are replaced in the document before evaluation.\n\nHere is a Markdown document string with an YAML header and Raku code that uses the parameters:\n\n````\n----\ntitle: \"Replacement example\"\ndate: 2025-06-19\nparams:\n  partSize: 0.25\n  dataDirName: \"~/fake-data\"\n  exportQ: FALSE\n----\n\nGetting data from %params\u003cdataDirName\u003e:\n\n```raku\nsay %params\u003cpartSize\u003e;\nsay (%params{\"exportQ\"} ?? '' !! 'do not ') ~ 'export it';\n```\n````\n\nSee the test file [\"08-header-parameters.rakutest\"](./t/08-header-parameters.rakutest) for \nmore detailed examples.\n\n**Remark:** Both forms of hashmap retrieval, `%params\u003cpartSize\u003e` and `%params{'partSize'}`, are replaced with the corresponding parameter value.\n\nThis Command Line Interface (CLI) invocation over [\"Template.md\"](./resources/Template.md) \ngenerates 10 random integers and their Russian numeric word forms:\n\n```\nfile-code-chunks-eval ./resources/Template.md\n```\n\nThis CLI invocation over the same template generates 7 random integers and their Bulgarian numeric word forms:\n\n```\nfile-code-chunks-eval ./resources/Template.md --params=\"{to-lang=\u003e'Bulgarian', :7sample-size}\"\n```\n\n-------\n\n## Command Line Interface \n\nThe package provides Command Line Interface (CLI) scripts, \n[`file-code-chunks-eval`](bin/file-code-chunks-eval) and\n[`file-code-chunks-extract`](bin/file-code-chunks-extract).\n\nHere are script invocation examples for the code chunks evaluation in a file named \"doc.md\":\n\n```\nfile-code-chunks-eval doc.md\n```\n\n```\nfile-code-chunks-eval file-code-chunks-eval.raku --evalOutputPrompt=\"## OUTPUT :: \" --evalErrorPrompt=\"## ERROR :: \" -o=doc_newly_weaved.md doc.md\n```\n\nHere is a script invocation example for code extraction from code chunks in a file named \"doc.md\":\n\n```\nfile-code-chunks-extract -o=doc_new_extract.md doc.md\n```\n\nIf no output file name is specified then the script\n[`file-code-chunks-eval`](bin/file-code-chunks-eval)\n([`file-code-chunks-extract`](bin/file-code-chunks-extract))\nmakes a new file in the same directory with the string\n\"_woven\" (\"_tangled\") inserted into the input file name.\n\n\n### `file-code-chunks-eval`\n\n```shell\nfile-code-chunks-eval --help\n```\n\n### `file-code-chunks-extract`\n\n```shell\nfile-code-chunks-extract --help\n```\n\n### `cronify`\n\nThe script `cronify` facilitates periodic execution of a shell command (with parameters.)\nIt heavily borrows ideas and code from the chapter \"Silent Cron, a Cron Wrapper\" of the book,\n\"Raku Fundamentals\" by Moritz Lenz, [ML1].\n\n```shell\ncronify --help\n```\n\n------\n\n## Implementation notes\n\nThe implementation uses a greatly reduced version of the class\n[`Jupyter::Kernel::Sandbox`](https://github.com/bduggan/p6-jupyter-kernel/blob/master/lib/Jupyter/Kernel/Sandbox.rakumod)\nof Raku Jupyter kernel package/repository [BD1].\n(See the class [REPLSandbox](./lib/Text/CodeProcessing/REPLSandbox.rakumod).)\n\nJust using \n[`EVAL`](https://docs.raku.org/routine/EVAL), \n(as in [SO1]) did not provide state persistence between code chunks evaluations.\nFor example, creating and assigning variables or loading packages in the first code chunk\ndid not make those variables and packages \"available\" in the subsequent code chunks.\n\nThat problem is resolved by setting up a separate Raku REPL (sandbox) object. \n\n-----\n\n## TODO\n\nThe following TODO items are ordered by priority, the most important are on top. \n \n- [X] Provide a function that works on strings.\n  (By refactoring the main function `FileCodeChunksEvaluation`.)\n    \n- [ ] TODO Add unit tests for:\n\n  - [X] DONE Code chunks evaluation\n      \n  - [X] DONE Persistence of REPL state(s)\n  \n  - [ ] TODO REPL availability\n    \n  - [X] DONE File code chunks evaluation \n\n  - [X] DONE File code extraction from chunks \n\n- [X] DONE Implement handling of code chunk parameters.\n\n- [X] DONE Shell code chunks execution.  \n\n- [X] DONE Implement output code cell generation that is marked as being of specified language.\n  - Done via the code chunk parameter `outputLang`.\n  \n- [X] DONE Comprehensive help for the CLI functions. \n\n- [X] DONE Implement document-wide, template parameters.\n  - Similar to YAML .Rmd files parameters specs.\n  - [X] DONE YAML header parsing and interpretation.\n  - [X] DONE Parameter substitution.\n    1. [X] Globally over the whole document \n    2. [ ] Per code chunk\n       - This means considering inline evaluations, like \\`\\`\\`1_000.sqrt\\`\\`\\`.\n  - [X] DONE Unit tests.\n  \n- [ ] TODO Implement data arguments for code chunks.\n  (As in [Babel org-mode](https://orgmode.org/manual/Environment-of-a-Code-Block.html).)\n\n- [ ] TODO Implement evaluation of Raku code chunks in Mathematica notebooks.\n\n- [ ] TODO Make the functionalities to work with languages other than Raku.\n  - This is both difficult and low priority. (Except for shell.)\n\n- [ ] TODO Refactor the sub arguments to use kebab-case (not just camelCase.)\n  - This is of _very low_ priority. \n\n-----\n\n## References\n\n### Articles\n\n[AA1] Anton Antonov,\n[\"Conversion and evaluation of Raku files\"](https://rakuforprediction.wordpress.com/2022/11/05/conversion-and-evaluation-of-raku-files),\n(2022),\n[RakuForPrediction at WordPress](https://rakuforprediction.wordpress.com).\n\n[DS1] Daniel Sockwell,\n[\"Weaving Raku: semiliterate programming in a beautiful language\"](https://www.codesections.com/blog/weaving-raku/),\n(2020),\n[codesections.com](https://www.codesections.com).\n\n[SO1] Suman Khanal et al.,\n[\"Capture and execute multiline code and incorporate result in raku\"](https://stackoverflow.com/q/57127263),\n(2017),\n[Stack Overflow](https://stackoverflow.com).\n\n### Books\n\n[ML1] Moritz Lenz,\n[\"Raku Fundamentals: A Primer with Examples, Projects, and Case Studies\"](https://www.google.com/books/edition/Raku_Fundamentals/MvyRzQEACAAJ?hl=en),\n2nd ed. (2020), Apress.\n\n### Repositories\n\n[BD1] Brian Duggan et al.,\n[p6-jupyter-kernel](https://github.com/bduggan/p6-jupyter-kernel),\n(2017-2020),\n[GitHub/bduggan](https://github.com/bduggan).\n\n### Videos\n\n[AAv1] Anton Antonov,\n[\"Conversion and evaluation of Raku files\"](https://www.youtube.com/watch?v=GJO7YqjGn6o),\n(2022)\n[Anton Antonov's YouTube channel](https://www.youtube.com/@AAA4prediction).","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantononcube%2Fraku-text-codeprocessing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fantononcube%2Fraku-text-codeprocessing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantononcube%2Fraku-text-codeprocessing/lists"}