{"id":13686707,"url":"https://github.com/Zestylogic/FMark","last_synced_at":"2025-05-01T09:32:31.467Z","repository":{"id":47412609,"uuid":"119062941","full_name":"Zestylogic/FMark","owner":"Zestylogic","description":"Markdown parser to html with many useful extensions and a purely functional implementation.","archived":false,"fork":false,"pushed_at":"2024-05-30T20:46:26.000Z","size":2404,"stargazers_count":8,"open_issues_count":27,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-26T06:42:38.952Z","etag":null,"topics":["fsharp","fsharp-markdown","lexer","markdown","paragraph","parser"],"latest_commit_sha":null,"homepage":"https://zestylogic.github.io/FMark","language":"F#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Zestylogic.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-01-26T14:45:32.000Z","updated_at":"2024-05-30T10:53:40.000Z","dependencies_parsed_at":"2024-05-30T12:23:58.702Z","dependency_job_id":"b69badef-2251-4426-aec3-b93e4741e403","html_url":"https://github.com/Zestylogic/FMark","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Zestylogic%2FFMark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Zestylogic%2FFMark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Zestylogic%2FFMark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Zestylogic%2FFMark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Zestylogic","download_url":"https://codeload.github.com/Zestylogic/FMark/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251852897,"owners_count":21654482,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fsharp","fsharp-markdown","lexer","markdown","paragraph","parser"],"created_at":"2024-08-02T15:00:38.282Z","updated_at":"2025-05-01T09:32:31.159Z","avatar_url":"https://github.com/Zestylogic.png","language":"F#","funding_links":[],"categories":["F# #"],"sub_categories":[],"readme":"# FMark by ZestyLogic 🍋♟\nFSharp markdown implementation with additional functionality.\n\nNamely:\n\n- Macros\n- Includes\n- Spreadsheet functionality\n- Table of Contents generation\n- Citations and footnotes\n\nSee [example.fmark](examples/example.fmark) for usage examples.\n\n# How to use\n\n## FMark Plugin Visual Studio Code\n\nFunctions:\n\n- Open Preview (ctrl+shift+M)\n\nOpen a live preview of the HTML that will be generated by FMark.\n\n- Create HTML File\n\nCreate an HTML file in the same directory as the .fmark file, with .html instead of .fmark.\n\n## Command Line Interface\n```\nUSAGE: FMark [--help] [--output \u003cpath\u003e] [--loglevel \u003cdebug|info|warning|error|fatal\u003e] [--format \u003chtml|markdown\u003e] [--test [\u003csequential\u003e]] [\u003cpath\u003e]\n\nINPUT:\n    \u003cpath\u003e                specify input file path.\n\nOPTIONS:\n    --input, -i \u003cpath\u003e CLI flag for input (not necessary, default argument is input.)\n    --output, -o \u003cpath\u003e specify output file path.\n    --loglevel, -l \u003cdebug|info|warning|error|fatal\u003e set the log level  ('0:DEBUG', '1:INFO', '2:WARNING', '3:ERROR' ,'4:FATAL').\n    --format, -f \u003chtml|markdown\u003e specify format, by default: html.\n    --test, -t [\u003csequential\u003e] run CI tests.\n    --help display this list of options.\n```\nNote: Markdown generation is incomplete and should only be used for property based testing.\n\n\n# Modules\n\n```\nOverall Flowchart:\n\n            ┌─────────────────────┐                      ┌───────────────┐\nSource ───\u003e │ Lex and Preprocessor│ ───\u003e Token list ───\u003e │ TOCite Parse  │ ──\u003e Token list with identifiers ┐\n            └─────────────────────┘           │          └───────────────┘                                 │\n                                    │         │                                                            │\n                                    │         └──────────────────────────────\u003e Header+Footer list ────\u003e────┤\n                                    │                                                                      │\n                                    │          ┌──────────────┐                                            │\n                                    └────────\u003e │ Markalc Parse│────────── Table ────────────\u003e──────────────│\n                                               └──────────────┘                                            │\n                                                                                                           │\n                                   ┌─────────┐                            ┌─────────────┐                  │\n             Final Document  \u003c──── │ HTMLGen │ \u003c──── ParsedObj list \u003c──── │ Main Parser │ \u003c────────────────┘\n                                   └─────────┘                            └─────────────┘\n```\n\n1. [Lexer and Preprocessor](FMark/src/Common/Lexer/README.md)\n2. [TOCite: Preparsing headers and footers](FMark/src/Common/TOCite/README.md)\n3. [Markalc: Spreadsheet functionality](FMark/src/Common/Markalc/README.md)\n4. [Main Parser](FMark/src/Common/Parser/README.md)\n\n# Vanilla Markdown \n\n## Implemented\n- HTML passthrough\n- Lists\n- Links (relative links within file don't work in VSCode, use `Create HTML File` and preview that.)\n- Images (images don't work in VSCode, use `Create HTML File` and preview that.)\n- Headers\n- Styling (bold, italic, bold and italic, strikethrough)\n- Paragraphs\n- Quotes\n- Code blocks\n- Tables\n\n## Design decisions\n\n### Lists\n\n* A list ends with two endlines, i.e., `\\r\\n\\r\\n`\n* List items are separated by one endline\n* List type is determined by the first item.\n  * `NUMBER ; DOT` will be ordered list\n  * ` - ` or ` * ` will be unordered list\n* If the first item does not start with `NUMDER;DOT;Space| \"- \" | \"* \"`, the list type will be default to unordered list\n* Ordered list start number is determined by the first item\n  * The subsequent counting is `+1` for each list item\n* Two spaces as one level of indentation.\n* A tab, `\\t` character will be interpreted as four spaces, therefore two levels of indentation.\n\n### Header\n\n* All headers must be preceded by two endlines, with an exception made for the first line in a file.\n\n`\\n\\n# header\\n` is a header, otherwise unless the header is the first line in a file it won't be processed as a header.\n\n### Styling\n\n* When using underscores for styling, there must be a space (or endline on the right) on either side in order for it to be recognised.\n* Asterisks for styling work as usual.\n\ne.g.\n\n`[ ]_em_[ ]`, `[ ]__a strong__[ ]`, or `[ ]_em_$`, `[ ]__a strong__$`\n\n### Paragraphs\n\nA paragraph is some characters that does not match:\n* CodeBlock\n* Header\n* ContentTable (Table of Contents object)\n* List\n* Quote\n* Table\n* Reference\n\nIt terminates with two endlines. Any elements mentioned above cannot exist in a paragraph. Otherwise, they will become normal text, and will not be rendered as expected.\n\n# Building\n\nTo build fmark, dotnet and fable have to be installed first. To build the javascript as well, yarn or npm are needed.\n\n## Using the Build script\n\n``` shell\n# build only javascript\nbuild -b js\n\n# build and test every single module\nbuild -b testall\n\n# build js and cli\nbuild -b all\n```\n\n# Markdown extensions\n\n## Macros\n\n### Supported Constructs\n\nThese are the supported constructs in the preprocessor.\n\n|Supported|Syntax|Description|Tested|\n|---|---|---|---|\n|Simple Macro|`{% macro name value %}`| Sets the Macro `name` equal to the string `value`|Unit Test|\n|Function Macro|`{% macro name(arg1; arg2) value %}`|Sets the Macro `name` equal to the string `value` with two parameters.|Unit Test|\n|Simple Evaluation|`{{ macro_name }}`|Evaluates the macro `macro_name` and replaces the evaluation with the evaluated body of the macro.|Unit Test|\n|Function Evaluation|`{{ macro_name(arg 1; arg 2) }}`|Evaluates the macro `macro_name` with the arguments `arg 1` and `arg 2` and replaces the evaluation with the evaluated body of the macro.|Unit Test|\n|File Include|`{{ include relative/path/to/file }}`|Includes and preprocesses a file using a relative or absolute path. The macros declared in that file will then be available in the current file|Unit Test|\n|Complex Macro Evaluation|`{{ x( {{ y( {{z}} ; Hello ) }} ; {{z}} ) }}`|Nested macro evaulations are supported. This way, default arguments can be created for other macros.|Unit Test|\n\n### Supported Features\n\nThese are the features that are currently supported by the preprocessor.\n\n|Feature|Example|Description|Tested|\n|---|---|---|---|\n| Simple whitespace control|`{% macro x y %}` evaluates to `y` and not ` ` `y` ` `.|Removes whitespace and newlines in macros where one wouldn't expect them to be added to the macro body.|Unit Test|\n|Shadowing of macros through|`{% macro x x %} {% macro y(x) {{ x }} %}` with `{{ y(z) }}` will evaluate to `z` but `{{ x }}` outside of the macro will always evaluate to `x`.|Macros can be shadowed by arguments of other macros.|Unit Test|\n|Nested macros|`{% macro x {% macro y %} %}`|Macro y is only defined inside macro x and cannot be seen outside of the scope of x.|Unit Test|\n|Shadowing of macros through|`{% macro x x %} {% macro y {% macro x z %} {{x}} %} y: {{ y }}, x: {{ x }}` will evaluate to `y: z, x: x`|Macros can be shadowed by other macros which will be used instead for evaluation.|Unit Test|\n|Evaluation of large strings|`{{ x(This is the first argument; This is the second argument) }}`|One can pass large strings as arguments to the macros.|Unit Test|\n|Escaping of characters inside argument|`{{ x(arg 1 with a \\); arg 2 with a \\;) }}`|One can esape all the special characters inside macros and substitutions|Unit Test|\n|Escaping macros|`\\{% macro x y %}`|This will escape the whole macro and not evaluate it|Unit Test|\n|Escaping Subsitutions|`\\{{ x }}`| will not evaluate the substitution but instead output it literally|Unit Test|\n|Outputting unmatched subsituttion|`{{ x }}` -\u003e `{{ x }}` if not in scope|If the subsitution is not matched, it will output it as it got it|Unit Test|\n|Nested Evaluations|`{{ x( {{y}} ) }}`|Arguments can now be evaluated inside them.|Unit Test|\n\n### Usage\n\nTo use the preprocessor and the lexer, a string or a list of strings can be used, depending on if there are multiple\nlines or not. For a single string, the following can be used.\n\nFor string, the `preprocess` and `lex` functions.\n\n``` f#\n[\u003cEntryPoint\u003e]\nlet main =\n    let inputString = (* Read the string *)\n\n    inputString\n    |\u003e preprocess\n    |\u003e lex\n    ...\n```\n\nFor a list of strings, one can use the `preprocessList` and `lexList` functions.\n\n``` f#\n[\u003cEntryPoint\u003e]\nlet main =\n    let inputStringList = (* Read the string list *)\n    \n    inputStringList\n    |\u003e preprocessList\n    |\u003e lexList\n    ...\n```\n\n### Example\n\nIn markdown using the preprocessor, one can then write the following:\n\n```\nText before macro\n{% macro Hello(arg1; arg2)\nThis is text inside the macro, with semicolons;\n{% macro local(arg1; arg2)\nThis is the second macro\n%}\nNow back in the first macro.\n{{ local(arg1; arg2) }}\n%}\nOutside both macros\nShould be printed as not in scope: {{ local(arg1; arg2) }}\n\n{{ Hello(arg1; arg2) }}\n```\n\nwhich then evaluates to\n\n```\nText before macro\nOutside both macros\nShould be printed as not in scope: {{ local(arg1; arg2) }}\n\n\nThis is text inside the macro, with semicolons;\nNow back in the first macro.\n\nThis is the second macro \n\n\n```\n\nMore complicated macros can also be created by writing html in the macros. Due to the \nhtml passthrough in the lexer, the html will be copied over literally to the output html.\n\n## Lexer\n\nThe lexer supports HTML pass through, which can be used to display raw html in markdown. Using this with\nmacros can give very useful and interesting functions.\n\n## Interface to the Parser\n\nThe interface to the parser was done using the following `Token` type, which the parser takes in \nand can parse.\n\n``` f#\ntype Token =\n    | CODEBLOCK of string * Language\n    | LITERAL of string\n    | WHITESPACE of size: int\n    | NUMBER of string\n    | HASH | PIPE | EQUAL | MINUS | PLUS | ASTERISK | DOT\n    | DASTERISK | TASTERISK | UNDERSCORE | DUNDERSCORE | TUNDERSCORE | TILDE | DTILDE\n    | TTILDE | LSBRA | RSBRA | LBRA | RBRA | BSLASH | SLASH | LABRA | RABRA | LCBRA\n    | RCBRA | BACKTICK | TBACKTICK | EXCLAMATION | ENDLINE | COLON | CARET | PERCENT\n```\n\n## Features\n\nSupports escaping of all the special characters defined in [Types](/FMark/FMark/src/Common/Types.fs). This is done by adding\na `\\` in front of the character that should be escaped.\n\nTokens that match multiple characters can also be escaped by just putting a `\\` before it. For example, \n`***` can be escaped by writing `\\***`.\n\n## Extensibility\n\nIt can easily be extended by adding the type of the token to `Token` above. Then the string\nhas to be linked to the token by adding it as a tuple of type `string * Token` to a list called\n`charList` in the [Lexer](/FMark/FMark/src/Common/Lexer/Lexer.fs).\n\n## Spreadsheet functionality\nSpreadsheet functions will evaluate in-place, if they are incorrectly formatted then Markalc will leave the cell unchanged as if it were normal text inside. \n\nTo delineate an expression, start the cell with the `=` operator, e.g.\n\nCalcs|39|42|\n|---|---|---|\n||`=6*5+SUM{4,5}`|`=[1,1]+3`|\n\n### Supported\n\n#### Arithmetic\n\n|Function|Syntax|Effect|Tested|\n|---|---|---|---|\n|Modulo|`a%b`|a modulo b|Property based, all integer inputs.\n|Power|`a^b`|a to the power of b|Property based, all integer inputs.\n|Multiply|`a*b`|a times b|Property based, all integer inputs.\n|Division|`a/b`|a divided by b|Property based, all integer inputs.\n|Subtraction|`a-b`|a minus b|Property based, all integer inputs.\n|Addition|`a+b`|a plus b|Property based, all integer inputs.\n|Brackets|`a+(b-c)`|Specify order of operation.|Unit tested.\n|Cell References|`[0,2]`|Evaluates to contents of cell specified (row 0, col 2)|Unit tested|\n|Range Cell Ref|`[0,0]:[2,0]`|Evaluates to list of cell references.|Unit tested.|\n|Keyword Cell Ref|`[col=2,row=0]`|Alternative syntax, order doens't matter.|Unit tested|\n|Specify Decimal Places|`=5.333,2`|Comma then a number specifies number of decimal places.|Unit tested|\n\n##### Cell reference behaviour\n\n|Referenced cell|Return|\n|---|---|\n|Contains valid expression `=...`|Contents of cell|\n|Does not contain valid expression|nan|\n|Is not in table|nan|\n|Contains circular reference|nan|\n\n#### Functions\n\nSupported functions:\n\n|Function name| Effect| Tested|\n|---|---|---|\n|`SUM`|Adds all arguments.|Unit tested.|\n|`AVG`|Calculates mean of all arguments.|Unit tested.|\n|`MIN`|Returns minimum of all arguments.|Unit tested.|\n|`MAX`|Returns maximum of all arguments.|Unit tested.|\n\nFunctions support Range Cell References: `[0,0]:[2,0]` will evaluate to `[0,0],[1,0],[2,0]`. This can be used with other arguments, e.g. `SUM{[0,0]:[2,0],1,3+2,[0,0]}`\n\n### Unsupported\n\n|Feature|Example|Workaround|\n|---|---|---|\n|Negative numbers  |`=-a+b`|Subtract number from zero in brackets e.g. `=(0-a)+b`|\n|Integer operations|`=5+2`|All calculations done in floating point.|\n|Allow formatting around equation|`*=5+2*`|None|\n|String operations|e.g. Excel CONCAT|None|\n|Assume empty cells are zero|Summing over column with empty cells|Put `=0` in each cell|\n\n## Table of Contents\n\nTable of Contents can be build anywhere with `%%TOC`. Relative linking is supported,\nclicking an item from the contents table will jump to the respective header when viewed\nin html.\n\n|Feature|Example|Exaplanation\n|---|---|---|\n|depth|`%%TOC depth=3`|Build a TOC with headers level less than 3\n|exclude|`%%TOC excludes=[Appendix;Acknowledgement]`|Exclude headers with name `Appendix` and `Acknowledgement`|\n\nMultiple features can be stacked with `,` like so:\n\n`%%TOC depth=3, excludes=[Appendix;Acknowledgement]`\n\n## Citations\n\nFMark supports simple footers and styled references.\n\n* If multiple citations with the same reference ID is found, the first citation will be used inline.\n  * One `[^fmark]` inline, and two `[^fmark], some author`. The first will be used.\n* Reference to non-exist citation will result in `\"[Reference: %refID not found!]\"` in the HTML generated.\n\n* The list is sorted, with footnotes in order of their numerical IDs first,\nReferences in order of apperance after. Thus the order each explanatory text\ncomes in the fmark file does not matter.\n\n### Simple Footers\n\nSimple footers have numerical IDs.\n```\nthis is inline[^1], and so on.\n\n[^1], this is the text to explain such line.\n      this line as well.\nBut not this line.\n```\nThe body will show up with a superscripted link:\n\u003e this is inline\u003csup\u003e1\u003c/sup\u003e , and so on.\n\u003e\n\u003e But not this line.\n\nThe footer texts will be collected at the end of the document.\n\u003e 1: this is the text to explain such line. this line as well.\n\n### Styled References\n\nStyled references have alphabetic IDs.\n\nSupported rendering styles:\n\n||Chicago|Harvard|IEEE|\n|---|---|---|---|\n|Book|Yes|Yes|-|\n|Website|Yes|Yes|-|\n\nSupported data fields\n\n|Field|Explanation|\n|---|---|\n|type|`'Book'` or `'Website'`|\n|author|Author with surname at the end|\n|title|Title|\n|year|The year it is written|\n|url|Address for website|\n|access|Date of access for websites, in `yyyy-mm-dd` format|\n\nPick a style with `%%RefStyle`, or it will default to Harvard.\n```\n%%RefStyle = Harvard\n```\n\nThen follow `field1=data1, field2=data2, ...` to use references.\n\nSpaces around equal sign are allowed, so `f = d`, `f= d`, `f =d` or `f=d`\nall have the same effect.\n\n```\nThis is a citation[^Mark]. This is another[^FMark] one.\n\n[^Mark], type= Book, author= Mark Smith, title= Not a real book, year= 2018\n[^FMark], type= Website, author= FMark Smith, title= Not a real website, year= 2017 url= www.example.com/website access= 2018-3-4\n```\n\nWith Harvard, it will look like this:\n\u003e This is a citation(Smith, 2018). This is another(Smith, 2017) one.\n\nAt the end of the document:\n\u003e Smith, M. (2018) *Not a real book*.\n\u003e \n\u003e Smith, F. (2017) *Not a real website*. Available from: www.example.com/website [Accessed 4th March 2018].\n\nif Chicago style is chosen:\n\n\u003e This is a citation(Smith 2018). This is another(Smith 2017) one.\n\nAt the end of the document:\n\u003e Mark Smith. 2018. *Not a real book*.\n\u003e \n\u003e FMark Smith. 2017. \"Not a real website.\" Accessed March 3, 2018. https://www.example.com/website\n\n## Latex maths rendering\n\nFmark includes support for Latex maths rendering.\n\nSurround an equation written in Latex with `$$` in order to use this feature.\n\nExample: `$$2+2=4-1=3$$`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FZestylogic%2FFMark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FZestylogic%2FFMark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FZestylogic%2FFMark/lists"}