{"id":19716660,"url":"https://github.com/rwillians/rinha-de-compilers--elixir-transcompiler","last_synced_at":"2025-04-29T20:30:46.123Z","repository":{"id":193374940,"uuid":"688675772","full_name":"rwillians/rinha-de-compilers--elixir-transcompiler","owner":"rwillians","description":"Source-to-Source Transcompiler from `.rinha` to Elixir. Runs on ERTS (Erlang Runtime System).","archived":false,"fork":false,"pushed_at":"2023-09-25T20:51:30.000Z","size":96,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-05T19:05:23.163Z","etag":null,"topics":["elixir","rinha-de-compilers"],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rwillians.png","metadata":{"files":{"readme":"README.bkp.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-09-07T21:30:35.000Z","updated_at":"2023-09-20T18:02:00.000Z","dependencies_parsed_at":"2023-09-13T13:46:42.836Z","dependency_job_id":"5c0db6a5-debd-49da-878b-5280da35b08d","html_url":"https://github.com/rwillians/rinha-de-compilers--elixir-transcompiler","commit_stats":null,"previous_names":["rwillians/rinha-de-compiladores--gambi-elixir","rwillians/rinha-de-compilers--elixir-transpiler","rwillians/rinha-de-compilers--elixir-transcompiler"],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rwillians%2Frinha-de-compilers--elixir-transcompiler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rwillians%2Frinha-de-compilers--elixir-transcompiler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rwillians%2Frinha-de-compilers--elixir-transcompiler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rwillians%2Frinha-de-compilers--elixir-transcompiler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rwillians","download_url":"https://codeload.github.com/rwillians/rinha-de-compilers--elixir-transcompiler/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251578284,"owners_count":21612009,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elixir","rinha-de-compilers"],"created_at":"2024-11-11T22:43:00.129Z","updated_at":"2025-04-29T20:30:45.708Z","avatar_url":"https://github.com/rwillians.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003ca href=\"https://github.com/aripiprazole/rinha-de-compiler\" alt=\"Link para o repositório da Rinha de Compiladores\" target=\"_blank\"\u003e\n    \u003cimg src=\"https://raw.githubusercontent.com/aripiprazole/rinha-de-compiler/main/img/banner.png\" alt=\"Logo da Rinha de Compilers\"\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\n---\n\n# A Source-to-Source Transcompiler\n\nThe core idea here is to use Elixir (at compile time) to parse a `.rinha` program, transpile it to Elixir AST and then compile it as an Elixir program.\n\n\n## How to use it?\n\nYou just need to create a module where your transpiled `rinha` program will live. To transpcompile the code, all you gotta do is use the `Transcompile` module:\n\n```elixir\n# lib/rinha/fib.ex\n\ndefmodule Rinha.Fib do\n  use Transcompiler,\n    source: {:file, path: \".rinha/files/fib.rinha\"},\n    parser: Rinha.Parser\nend\n\n```\n\nAll functions defined in your `.rinha` program file will be extracted from the syntax tree then transpiled as Elixir's `def` functions (public module functions). That's necessary to allow for recursive functions. As for the rest of the tree, all script-like procedural code will be transpiled into a `main/0` public function in the same module.\n\n\n## Running it\n\n\u003e **Note**\n\u003e I assume you have `asdf-vm` installed (because you should 👀 -- it's like nvm, but for anything basically).\n\n1.  Clone the repo (yes, that `--recursive` flag is important):\n\n    ```sh\n    git clone --recursive git@github.com:rwillians/rinha-de-compiladores.git\n    ```\n\n2.  Install Elixir and Erlang with the versions specified in the file `.tool-versions`:\n\n    ```sh\n    asdf install\n    ```\n\n3.  Install dependencies:\n\n    ```sh\n    mix deps.get\n    ```\n\n4.  Compile dependencies (shouldn't be timmed):\n\n    ```sh\n    mix deps.compile\n    ```\n\n5.  Compile the main source code (that's the one you want to time):\n\n    ```sh\n    mix compile\n    ```\n\n### Run any `.rinha` program:\n\n```sh\nmix run play.exs \"./relative/path/to/program.rinha\"\n```\n\nTo compile once then execute `n` times:\n\n```sh\nmix run play.exs \"./relative/path/to/program.rinha\" 1000\n```\n\nAnd, if you're running that many times you'll probably want to redirect `stdout` to `/dev/null`:\n\n```sh\nmix run play.exs \"./relative/path/to/program.rinha\" 1000000 \u0026\u003e/dev/null\n```\n\n### Run using the REPL:\n\n```sh\niex -S mix\n```\n\nCall whatever function you'd like to see working:\n\n```elixir\nRinha.Fib.main()\n```\n\nNote that functions specified in the program are public functions, meaning you could call `fib/1` from the REPL as well:\n\n```elixir\nRinha.Fib.fib(15)\n```\n\nYou can also play with the other test programs:\n\n```elixir\nRinha.Combination.main()\n```\n\n```elixir\nRinha.Sum.main()\n```\n\n\n## How does it work?\n\nLet's take `Rinha.Fib` as an example:\n\n```elixir\n# lib/rinha/fib.ex\n\ndefmodule Rinha.Fib do\n  use Transcompiler,\n    source: {:file, path: \".rinha/files/fib.rinha\"},\n    parser: Rinha.Parser\nend\n```\n\nWhen you use `use Transcompiler`, we first take that `path` to the `fib.rinha` program and make sure it's associated with your módule (e.g.: `Rinha.Fib`) so that, whenever `fib.rinha` is changed, then the módulo is recompiled:\n\n```elixir\n# lib/transcompiler.ex\n\ndefmodule Transcompiler do\n  # ...\n\n  defmacro __using__(opts) do\n    # ...\n\n    quote do\n      @external_resource unquote(path)\n\n      # ...\n    end\n  end\n\n  # ...\nend\n```\n\nThen the contents of `fib.rinha` is read and parsed into a generic AST:\n\n```elixir\n# lib/transcompiler.ex\n\ndefmodule Transcompiler do\n  #...\n\n  defmacro __using__(opts) do\n    # ...\n\n    quote do\n      # ...\n\n      ast = File.read!(unquote(path))\n            #   ^  read the contents of the file\n\n            |\u003e unquote(parser).parse(unquote(path))\n            #  ^ calls function `parse/2` from the `parser`\n            #    given when using `use Transcompiler`\n\n            # |\u003e Ex.Tuple.unwrap!()\n            # |\u003e unquote(__MODULE__).transpile(__MODULE__)\n\n      # ...\n    end\n  end\n\n  # ...\nend\n```\n\nThe parser is implemented using [NimbleParsed](https://github.com/dashbitco/nimble_parsec) for parser combinators that are compiled to functions where rules are choosen via pattern matching (meaning, it's fast!):\n\n```elixir\n# lib/rinha/parser.ex\n\ndefmodule Rinha.Parser do\n  # ...\n\n  defparsec :bool,\n            choice([\n              string(\"true\") |\u003e replace(true),\n              string(\"false\") |\u003e replace(false)\n            ])\n            |\u003e unwrap_and_tag(:boolean)\n\n  defparsecp :let,\n             ignore(string(\"let\"))\n             |\u003e ignore(times(space, min: 1))\n             |\u003e parsec(:var)\n             |\u003e ignore(times(space, min: 1))\n             |\u003e ignore(string(\"=\"))\n             |\u003e ignore(times(space, min: 1))\n             |\u003e unwrap_and_tag(parsec(:expr), :value)\n             |\u003e ignore(optional(string(\";\")))\n             |\u003e tag(:let)\n\n  defparsecp :binary_op,\n             empty()\n             |\u003e unwrap_and_tag(parsec(:term), :lhs)\n             |\u003e ignore(times(space, min: 1))\n             |\u003e unwrap_and_tag(parsec(:operator), :op)\n             |\u003e ignore(times(space, min: 1))\n             |\u003e unwrap_and_tag(parsec(:term), :rhs)\n             |\u003e tag(:binary_op)\n\n  # ...\n\n  def parse(program, filename) do\n    with {:ok, [{:file, exprs}], \"\", _, _, _} \u003c- file(program),\n         do: {:ok, to_common_ast({:file, exprs}, %{filename: filename})}\n  end\n\n  # ...\n\n  defp to_common_ast({:string, value}, ctx) do\n    %Transcompiler.String{\n      value: value,\n      location: %Transcompiler.Location{filename: ctx.filename}\n    }\n  end\nend\n```\n\nParsing a program results in a generic AST like this:\n\n```elixir\n# \"let a = k == 0;\"\n\n%Transcompiler.File{\n  name: \"foo.rinha\",\n  block: [\n    %Transcompiler.Let{\n      var: %Transcompiler.Variable{name: :a, location: %Transcompiler.Location{}},\n      value: %Transcompiler.BinaryOp.Eq{\n        lhs: %Transcompiler.Variable{name: :k, location: %Transcompiler.Location{}},\n        rhs: %Transcompiler.Integer{value: 0, location: %Transcompiler.Location{}},\n        location: %Transcompiler.Location{}\n      },\n      location: %Transcompiler.Location{}\n    }\n  ],\n  location: %Transcompiler.Location{}\n}\n```\n\nThere's a total of 27 types of tokens that can be composed into that generic AST. Each token implements a `Transpiler` protocol, which introduces the function `to_elixir_ast` that is capable of taking a specific type of AST node and recursively transpile it to Elixir AST.\n\n```elixir\n# lib/transcompiler/transpiler.ex\n\ndefprotocol Transcompiler.Transpiler do\n  @spec to_elixir_ast(ast :: struct, env :: module) :: Macro.t()\n  def to_elixir_ast(ast, env)\nend\n```\n\n```elixir\n# lib/transcompiler/binary_op.add.ex\n\ndefimpl Transcompiler.Transpiler, for: Transcompiler.BinaryOp.Add do\n  def to_elixir_ast(ast, env) do\n    {:+, [context: env, imports: [{1, Kernel}, {2, Kernel}]], [\n      Transcompiler.Transpiler.to_elixir_ast(ast.lhs, env),\n      Transcompiler.Transpiler.to_elixir_ast(ast.rhs, env),\n    ]}\n  end\nend\n```\n\nNow that we have a generic AST and that we're capable of transpiling it to Elixir AST, let's get back to `Transcompiler` module. It takes the generic AST and transpiles it to Elixir AST then apply such AST to the module which invoked `use Transcompile`:\n\n```elixir\n# lib/transcompiler.ex\n\ndefmodule Transcompiler do\n  # ...\n\n  defmacro __using__(opts) do\n    # ...\n\n    quote do\n      # ...\n\n      ast = File.read!(unquote(path))\n            #    ^ read the contents for `fib.rinha`\n\n            |\u003e unquote(parser).parse(unquote(path))\n            #                  ^ parses into generic AST\n\n            |\u003e Ex.Tuple.unwrap!()\n            #          ^ raises and error if something goes wrong\n            #            with parsing\n\n            |\u003e unquote(__MODULE__).transpile(__MODULE__)\n            #                      ^ recursively transpiles generic\n            #                        AST into Elixir AST\n\n      Module.eval_quoted(__MODULE__, ast)\n      #     ^ applies Elixir AST to the module which invoked\n      #       `use Transcompiler`\n    end\n  end\n\n  # ...\nend\n```\n\nAnd that's it. Now the `.rinha` code is Elixir code; get's compiled as Elixir code; and runs as any Elixir code.\n\n\n## Where to find me\n\n|      Name | Link                                                 |\n|----------:|:-----------------------------------------------------|\n| 𝕏 Twitter | [@rwillians_](https://twitter.com/rwillians_)        |\n|  LinkedIn | [@rwillians](https://www.linkedin.com/in/rwillians/) |\n|    GitHub | [@rwillians](https://github.com/rwillians)           |\n|    Resume | [rwillians.com](https://rwillians.com/resume)        |\n\n\n## What's next?\n\n- [ ] `#debug` add line number and character offset to `Transcompiler.Location` on all tokens;\n- [ ] `#improvement` support functions to be declared anywhere in the file, not only at the root scope;\n- [ ] `#dx` `#debug` provide nicer error messages when parsing fails.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frwillians%2Frinha-de-compilers--elixir-transcompiler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frwillians%2Frinha-de-compilers--elixir-transcompiler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frwillians%2Frinha-de-compilers--elixir-transcompiler/lists"}