{"id":16919329,"url":"https://github.com/antonmedv/ultra-tiny-compiler","last_synced_at":"2025-07-16T06:45:11.668Z","repository":{"id":57385577,"uuid":"55230149","full_name":"antonmedv/ultra-tiny-compiler","owner":"antonmedv","description":"Ultra Tiny Compiler","archived":false,"fork":false,"pushed_at":"2019-03-28T10:50:05.000Z","size":13,"stargazers_count":180,"open_issues_count":0,"forks_count":12,"subscribers_count":10,"default_branch":"master","last_synced_at":"2024-05-02T01:09:22.447Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"CoffeeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/antonmedv.png","metadata":{"files":{"readme":"readme.litcoffee","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-04-01T12:16:01.000Z","updated_at":"2023-09-21T22:11:22.000Z","dependencies_parsed_at":"2022-09-26T16:50:50.040Z","dependency_job_id":null,"html_url":"https://github.com/antonmedv/ultra-tiny-compiler","commit_stats":null,"previous_names":["elfet/ultra-tiny-compiler"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antonmedv%2Fultra-tiny-compiler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antonmedv%2Fultra-tiny-compiler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antonmedv%2Fultra-tiny-compiler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antonmedv%2Fultra-tiny-compiler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/antonmedv","download_url":"https://codeload.github.com/antonmedv/ultra-tiny-compiler/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221673963,"owners_count":16861746,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-13T19:44:02.987Z","updated_at":"2024-10-27T12:14:13.301Z","avatar_url":"https://github.com/antonmedv.png","language":"CoffeeScript","funding_links":[],"categories":["Compilers and Interpreters"],"sub_categories":["Educational and Toy Projects"],"readme":"![Ultra Tiny Compiler](http://medv.io/assets/ultra-tiny-compiler.png)\n\nThis is Ultra Tiny Compiler for any C-like expression into lisp.  \nIf remove all comments, source code will be less then **\u003c90** lines of actual code. \nBy the way, you are viewing source code itself (yes, this readme file also is source code).\nIt's written in [literate coffescript](http://coffeescript.org/#literate) and fully [tested](tests.litcoffee) [![Build Status](https://travis-ci.org/antonmedv/ultra-tiny-compiler.svg?branch=master)](https://travis-ci.org/antonmedv/ultra-tiny-compiler) and published on [npm](https://www.npmjs.com/package/ultra-tiny-compiler). \nYou can install it via `npm install ultra-tiny-compiler`.\n\nWhat this compiler can do?\n\n##### Examples\n\n| Input               | Output                 |\n|---------------------|------------------------|\n| `1 + 2 * 3`         | `(+ 1 (* 2 3))`        |\n| `1 + exp(i * pi)`   | `(+ 1 (exp (* i pi)))` |\n| `pow(1 + 1 / n, n)` | `(pow (+ 1 (/ 1 n)) n)`|\n\n\n#### Theory\n\nWe begin by describing grammar of C-like expressions, more precisely context-free grammar.\nA context-free grammar has four components:  \n 1. A set of _terminal_ symbols (tokens).\n 2. A set of _nonterminals_ (syntactic variables).\n 3. A set of _productions_, where each production consists of a nonterminal, called the head or left side of the production, an arrow, and a sequence of terminals and/or nonterminals, called the body or right side of the production.\n 4. A designation of one of the nonterminals as the start symbol.\n \nFor notational convenience, productions with the same nonterminal as the head can have their bodies grouped by symbol |.\n\nHere is our grammar for C-like expressions:\n\n_expr_ → _expr_ + _term_ | _expr_ - _term_ | _term_  \n_term_ → _term_ * _factor_ | _term_ / _factor_ | _factor_  \n_factor_ → ( _expr_ ) | _atom_ | _call_  \n_call_ → _atom_ ( _list_ )  \n_list_ → _list_ , _expr_ | _expr_  \n_atom_ → [a-z0-9]+  \n\nHere _expr_, _term_, _factor_, ... a nonterminals and +, -, *, /, (, ), ... a terminals, _expr_ is a start symbol.\n\nUsually compiler consist of several parts: lexer, parser, code generator.\nLexer reads input stream, split it into tokens and pass them into parser.\nParser uses grammar to build parse tree and generate [abstract syntax tree](https://en.wikipedia.org/wiki/Abstract_syntax_tree) (AST for short). \nAST resemble parse tree to an extent. However, in AST interior nodes represent programming constructs while in the parse tree, the interior nodes represent nonterminals.\nCode generator traverses AST and generates code.\n\nBut we a going to simplify our compiler and combine parse phase with code generation phase. \nTo accomplish this we will use syntax-directed definitions (semantic actions) for translating expressions into postfix notation.\n\n_expr_ → _expr_ + _term_ {puts(\"+\")}  \n_expr_ → _expr_ - _term_ {puts(\"-\")}  \n_atom_ → [a-z0-9]+ {puts(_atom_)}  \n...  \n\nExpression `9-5+2` will be translating into `95-2+` with this semantic actions.\n\nFor parsing we will use simple form of [recursive-descent parsing](https://en.wikipedia.org/wiki/Recursive_descent_parser), called predictive parsing, in which the lookahead symbol unambiguously determines the flow of control through the procedure body for each nonterminal.\nIt is possible for a recursive-descent parser to loop forever. A problem arises with left-recursive productions like _expr_ → _expr_ + _term_.\n\nA left-recursive production can be eliminated by rewriting the offending production:\n\n_A_ → _A_ ⍺ | β\n\nInto right-recursive production:\n\n_A_ → β _R_  \n_R_ → ⍺ _R_ | ∅\n\nUsing this technique we can transform our grammer into new grammar:\n\n_expr_ → _term_ _exprR_  \n_exprR_ → + _term_ _exprR_ | - _term_ _exprR_ | ∅  \n_term_ → _factor_ _termR_  \n_termR_ → * _factor_ _termR_ | / _factor_ _termR_ | ∅  \n_factor_ → ( _expr_ ) | _atom_ | _call_  \n_call_ → _atom_ ( _list_ )  \n_list_ → _expr_ _listR_  \n_listR_ → , _expr_ _listR_ | ∅  \n_atom_ → [a-z0-9]+  \n\nSemantic actions embedded in the production are simply carried along in the transformation, as if they were terminals.\n\nOkay, let's write some code. We need only one function `compile(input: string)` which will do all the work.\n\n    compile = (input) -\u003e \n\nFirst start with lexer what will be splitting input sequence of characters into tokens.  \nFor example next input string `pow(1, 2 + 3)` will be transformed into array `['pow', '(', '1', ',', '2', '+', '3', ')']`.\n\n      tokens = []\n      \nEvery of our tokens will be one character, only atoms can be any length of letters or numbers.\n      \n      is_atom = /[a-z0-9]/i\n\nIterate throw characters of input. Note what inside this loop may be another loop \nwhich also increments `i` value.\n\n      i = 0\n      while i \u003c input.length \n        switch char = input[i]\n\nWhen meet one of next character, put it as token and continue to next character.\n\n          when \"+\", \"-\", \"*\", \"/\", \"(\", \")\", \",\"\n            tokens.push char\n            i++\n\nSkip whitespaces.\n\n          when \" \"\n            i++\n\nIf character is unknown,\n\n          else\n\nLoop through each character in sequence until we encounter a character that is not an atom.\n\n            if is_atom.test char\n              tokens.push do -\u003e\n                value = ''\n                while char and is_atom.test char\n                  value += char\n                  char = input[++i]\n                value\n\nThrow error on an unknown character.\n\n            else\n              throw \"Unknown input char: #{char}\"\n\nWe need a function which will be giving us a token from a tokens stream. Let's call it _next_.\n\n      next = -\u003e tokens.shift()\n\nLookahead is very important part of our compiler. Allows to determine what kind of production we are hitting next.\n\n      lookahead = next()\n\nAnother important function which can match given terminal with `lookahead` terminal and move `lookahead` to next token.\n\n      match = (terminal) -\u003e\n        if lookahead == terminal then lookahead = next()\n        else throw \"Syntax error: Expected token #{terminal}, got #{lookahead}\"\n\nSometimes we a going to hit into wrong production, and we need a function which allows us to return to previous state.\n\n      recover = (token) -\u003e\n        tokens.unshift lookahead\n        lookahead = token\n\nNext, we a going to write our production rules. Each nonterminal represents corresponding function call,\neach terminal represents `match` function call. Also, we omitted ∅ production.  \n`expr` function represents next production rule:  \n_expr_ → _term_ _exprR_ \n      \n      expr = -\u003e\n        term(); exprR()\n\nWill be using lookahead for determine which production to use. \nHere also our first semantic actions which puts + or - onto stack. \nEnsure preserve ordering of semantic actions.  \n_exprR_ → + _term_ {puts(\"+\")} _exprR_ | - _term_ {puts(\"-\")} _exprR_ | ∅\n\n      exprR = -\u003e\n        if lookahead == \"+\"\n          match(\"+\"); term(); puts(\"+\"); exprR()\n        else if lookahead == \"-\"\n          match(\"-\"); term(); puts(\"-\"); exprR()\n\n_term_ → _factor_ _termR_\n\n      term = -\u003e\n        factor(); termR()\n\n_termR_ → * _factor_ {puts(\"*\")} _termR_ | / _factor_ {puts(\"/\")} _termR_ | ∅\n\n      termR = -\u003e \n        if lookahead == \"*\"\n          match(\"*\"); factor(); puts(\"*\"); termR()\n        else if lookahead == \"/\"\n          match(\"/\"); factor(); puts(\"/\"); termR()\n\nNext goes tricky production rule. First, we lookahead if there `(`, \nwhich will mean what current we at expression in brackets.\nSecond, try use production rule for function call.\nThird, if function call production fails, consider current token as an atom.  \n_factor_ → ( _expr_ ) | _atom_ | _call_\n\n      factor = -\u003e\n        if lookahead == \"(\"\n          match(\"(\"); expr(); match(\")\")\n        else \n          atom() unless call() \n\nCall production should allow recovering if we chose it incorrectly.  \nRemember current token, and move to next token with `match`. If after goes an `(`,\nwhen we choose call production correctly, otherwise recover to previous state.\nThen we hitting list of arguments, puts special _mark_ onto stack, this allows us to\nknow how many arguments contains in this list.  \n_call_ → _atom_ ( _list_ )\n\n      call = -\u003e\n        token = lookahead\n        match(lookahead)\n        if lookahead == \"(\"\n          puts(false, \"mark\"); match(\"(\"); list(); match(\")\"); puts(token, \"call\")\n          true\n        else\n          recover(token)\n          false\n\n_list_ → _expr_ _listR_ \n\n      list = -\u003e\n        expr(); listR();\n\n_listR_ → , _expr_ _listR_ | ∅\n\n      listR = -\u003e\n        if lookahead == \",\"\n          match(\",\"); expr(); listR();\n\nAt the bottom lies atom production. If we went down to this production, we expecting to get an atom.  \nOtherwise, it's some syntax error.  \n_atom_ → [a-z0-9]+\n\n      atom = -\u003e\n        if is_atom.test lookahead\n          match(puts(lookahead, \"atom\"))\n        else\n          throw \"Syntax error: Unexpected token #{lookahead}\"\n          \n\nIn semantic rules, we use `puts` function, which records operators and atoms into `stack` in [reverse polish notation](https://en.wikipedia.org/wiki/Reverse_Polish_notation) (RPN).  \nBut instead recording entire program in RPN, we are going to do code generation on the fly. For that, we must understand\nhow to [postfix algorithm](https://en.wikipedia.org/wiki/Reverse_Polish_notation#Postfix_algorithm) works.\n\nFor example, we have a stream of tokens: _1_, _2_, _3_, +, -  \n\n1. Put _1_ onto stack.\n2. Put _2_ onto stack.\n3. Put _3_ onto stack.\n4. When + pop two values (3,2) from stack and put `(+ 2 3)` onto stack. \n5. When - pop two values (`(+ 2 3)`,1) from stack and put `(- 1 (+ 2 3))` onto stack back.\n\nGenerated code will be on top of stack and stack size will be one, if stream was complete.\n\n      stack = []\n      puts = (token, type=\"op\") -\u003e\n        switch type\n\nThen operators comes in, pop two values from stack,\ngenerate code for that operator and push generated code back into stack.\n\n          when \"op\"\n            op = token\n            y = stack.pop()\n            x = stack.pop()\n            stack.push \"(#{op} #{x} #{y})\"\n\nDo same thing for _call_, but instead of gathering two values from stack,\ntake all values, until `false` shows up from stack. `false` represents special\nmark to know their arguments ends.\n\n          when \"call\"\n            func = token\n            args = []\n            while arg = stack.pop()\n              break unless arg\n              args.unshift arg\n            stack.push \"(#{func} #{args.join(' ')})\"\n\nAny other atoms or marks push to stack as it appears.\n\n          else \n            stack.push token\n\nReturn token from puts function for reuse.\n\n        token\n\nAt this point, we described all function what we need, and now we can start parsing \nand compilation at same time.\n\n      expr()\n\nIf parsing pass well, we end up with `stack` with only one item in it.\nIt's our compiled code. Let's return it from the function.\n\n      stack[0]\n\nExpose the world.\n\n    module.exports = compile\n\nWhat's it. We just wrote out compiler.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantonmedv%2Fultra-tiny-compiler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fantonmedv%2Fultra-tiny-compiler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantonmedv%2Fultra-tiny-compiler/lists"}