{"id":16866500,"url":"https://github.com/saulecabrera/markdownif","last_synced_at":"2025-04-11T10:15:06.011Z","repository":{"id":57520990,"uuid":"214052091","full_name":"saulecabrera/markdownif","owner":"saulecabrera","description":"Safe + Fast Markdown NIFs","archived":false,"fork":false,"pushed_at":"2020-04-19T19:02:58.000Z","size":82,"stargazers_count":10,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-25T07:02:08.020Z","etag":null,"topics":["elixir","markdown","nif","rust"],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/saulecabrera.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-10-10T00:40:52.000Z","updated_at":"2023-11-13T17:07:43.000Z","dependencies_parsed_at":"2022-09-13T22:52:15.030Z","dependency_job_id":null,"html_url":"https://github.com/saulecabrera/markdownif","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/saulecabrera%2Fmarkdownif","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/saulecabrera%2Fmarkdownif/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/saulecabrera%2Fmarkdownif/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/saulecabrera%2Fmarkdownif/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/saulecabrera","download_url":"https://codeload.github.com/saulecabrera/markdownif/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247959798,"owners_count":21024842,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elixir","markdown","nif","rust"],"created_at":"2024-10-13T14:50:47.364Z","updated_at":"2025-04-11T10:15:05.979Z","avatar_url":"https://github.com/saulecabrera.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Markdownif\n\n[![Build Status](https://github.com/saulecabrera/markdownif/workflows/CI/badge.svg)](https://github.com/saulecabrera/markdownif/actions)\n\n\nMarkdown NIFs built on top of [pulldown-cmark](https://github.com/raphlinus/pulldown-cmark) using [Rustler](https://github.com/rusterlium/rustler).\n\n\n## Installation\n\nIn your `deps` add:\n\n```elixir\n{:markdownif, \"~\u003e 0.9.0\"}\n```\n\n## Requirements\n\n- Markdownif requires Elixir 1.7+ and OTP 21+\n- pulldown-cmark requires Rust 1.34+\n\n## Features\n\n- Fast\n- Safe: the bindings are written in Rust using [Rustler](https://github.com/rusterlium/rustler)\n\n## Usage\n\nThis library implements bindings for [pulldown-cmark](https://github.com/raphlinus/pulldown-cmark) which is a pull parser for [CommonMark](https://commonmark.org/).\n\nIt supports the following optional features:\n\n- Foonote parsing\n- GitHub flavored tasklists\n- Github flavored tables\n- Strikethrough\n\n\nTo convert a markdown string into HTML use `Markdownif.to_html/2`:\n\n```elixir\nMarkdownif.to_html(\"__Foo__\")\n\n\u003cp\u003e\u003cstrong\u003eFoo\u003c/strong\u003e\u003c/p\u003e\n```\n\nAll optional features all disabled by default, to enable any of them pass a `%Markdownif.Features{}` struct as a second parameter to `Markdownif.to_html/2`:\n\n\n- `Markdownif.to_html(input, %Markdownif.Features{footnotes: true})` to enable footnote parsing\n- `Markdownif.to_html(input, %Markdownif.Features{tables: true})` to enable table support\n- `Markdownif.to_html(input, %Markdownif.Features{tasklists: true})` to enable tasklist support \n- `Markdownif.to_html(input, %Markdownif.Features{strikethrough: true})` to enable strikethrough support\n\n### Input size and dirty scheduling\n\nMarkdownif makes use of dirty schedulers to handle long running NIFs.\n\nTo avoid any unexpected behaviour in the VM, the Erlang team recommends that NIFs should be as fast as possible, ideally they should run within 1ms, if this is not possible,\nwork should be chunked or the function should be flagged as a dirty NIF.\n\nIn the case of this library, since chunking is not an option, dirty NIFs are used instead. Two main factors are taken into account to decide when to flag the execution as dirty:\n\n- Input size\n- Dirty scheduling overhead\n\nWhen loading a dirty NIF the execution doesn't take place right away, instead it's enqueued into the run queue of a dirty scheduler, this is not a free operation.\n\nRoughly, the following calls are made:\n\n1. `static_schedule_dirty_cpu_nif` performs the NIF scheduling\n2. `erts_call_dirty_nif` calls the NIF\n3. `dirty_nif_finalizer` returns the result of the NIF\n\n[This post](https://medium.com/@jlouis666/erlang-dirty-scheduler-overhead-6e1219dcc7) suggests that instead of relying on the 1ms rule for dirty scheduling, it's viable to use dirty schedulers when the dirty scheduling overhead is low enough that it's not significant to the overall NIF execution time. That is when the scheduling overhead is 10% or less of the overall execution time. \n\nIn this case, running the [tracing script](https://github.com/saulecabrera/markdownif/blob/master/bench/trace_dirty_nif.d) shows that with an input of 2049 bytes the scheduling overhead is 3250ns and the total execution time is 26500ns; the overhead represents ~12% of the total execution time. \n\nTaking all this into account, dirty schedulers will be used with all inputs greater than 2048 bytes.\n\n## Benchmarks\n\nThe following benchmark results compares Markdownif to [markdown](https://github.com/devinus/markdown) and [earmark](https://github.com/pragdave/earmark)\n\n```sh\nOperating System: macOS\nCPU Information: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz\nNumber of Available Cores: 8\nAvailable memory: 16 GB\nElixir 1.9.1\nErlang 21.0\n\nBenchmark suite executing with the following configuration:\nwarmup: 2 s\ntime: 10 s\nmemory time: 0 ns\nparallel: 1\ninputs: none specified\nEstimated total run time: 1.20 min\n\nBenchmarking Earmark.as_html!/1 120027 bytes bytes...\nBenchmarking Earmark.as_html!/1 2049 bytes...\nBenchmarking Markdown.to_html/1 120027 bytes bytes...\nBenchmarking Markdown.to_html/1 2049 bytes...\nBenchmarking Markdownif.to_html/1: 120027 bytes bytes...\nBenchmarking Markdownif.to_html/1: 2049 bytes...\n\nName                                               ips        average  deviation         median         99th %\nMarkdown.to_html/1 2049 bytes                 38275.79      0.0261 ms    ±40.58%      0.0240 ms      0.0640 ms\nMarkdownif.to_html/1: 2049 bytes              33590.81      0.0298 ms    ±48.69%      0.0270 ms      0.0800 ms\nMarkdown.to_html/1 120027 bytes bytes           939.05        1.06 ms    ±10.10%        1.02 ms        1.47 ms\nMarkdownif.to_html/1: 120027 bytes bytes        868.94        1.15 ms    ±12.93%        1.12 ms        1.70 ms\nEarmark.as_html!/1 2049 bytes                   163.35        6.12 ms    ±11.52%        5.97 ms        9.22 ms\nEarmark.as_html!/1 120027 bytes bytes             5.24      190.74 ms     ±3.63%      189.62 ms      229.73 ms\n\nComparison:\nMarkdown.to_html/1 2049 bytes                 38275.79\nMarkdownif.to_html/1: 2049 bytes              33590.81 - 1.14x slower +0.00364 ms\nMarkdown.to_html/1 120027 bytes bytes           939.05 - 40.76x slower +1.04 ms\nMarkdownif.to_html/1: 120027 bytes bytes        868.94 - 44.05x slower +1.12 ms\nEarmark.as_html!/1 2049 bytes                   163.35 - 234.32x slower +6.10 ms\nEarmark.as_html!/1 120027 bytes bytes             5.24 - 7300.72x slower +190.71 ms\n\n```\n\n\n## LICENSE\n\nMIT License\n\nCopyright (c) 2019 Saúl Cabrera\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsaulecabrera%2Fmarkdownif","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsaulecabrera%2Fmarkdownif","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsaulecabrera%2Fmarkdownif/lists"}