{"id":13831534,"url":"https://github.com/c3d/xl","last_synced_at":"2025-04-07T10:22:34.853Z","repository":{"id":4890005,"uuid":"6045500","full_name":"c3d/xl","owner":"c3d","description":"A minimalist, general-purpose programming language based on meta-programming and parse tree rewrites","archived":false,"fork":false,"pushed_at":"2023-02-05T17:43:56.000Z","size":22597,"stargazers_count":277,"open_issues_count":46,"forks_count":15,"subscribers_count":16,"default_branch":"master","last_synced_at":"2025-03-31T08:08:58.004Z","etag":null,"topics":["compiler","compiler-design","compilers-design","dialects","domain-specific-language","extensible-language","extension-language","functional-languages","homoiconic","metaprogramming","programming-language","programming-languages","rewrites","xl-language","xl-parse-tree"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"google/certificate-transparency","license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/c3d.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2012-10-02T13:48:32.000Z","updated_at":"2025-03-31T05:54:55.000Z","dependencies_parsed_at":"2023-02-19T01:01:42.304Z","dependency_job_id":null,"html_url":"https://github.com/c3d/xl","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/c3d%2Fxl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/c3d%2Fxl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/c3d%2Fxl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/c3d%2Fxl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/c3d","download_url":"https://codeload.github.com/c3d/xl/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247632028,"owners_count":20970102,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compiler","compiler-design","compilers-design","dialects","domain-specific-language","extensible-language","extension-language","functional-languages","homoiconic","metaprogramming","programming-language","programming-languages","rewrites","xl-language","xl-parse-tree"],"created_at":"2024-08-04T10:01:30.493Z","updated_at":"2025-04-07T10:22:34.831Z","avatar_url":"https://github.com/c3d.png","language":"C++","readme":"# XL - An extensible language\n\n\u003e **WARNING**: XL is a work in progress. Even if there are some bits\n\u003e and pieces that happen to already work, XL is presently not suitable\n\u003e for any serious programming. Examples given below may sometimes simply not\n\u003e work. Take it as a painful reminder that the work is far from finished,\n\u003e and, who knows, as an idea for a contribution.\n\u003e See [HISTORY](docs/HANDBOOK.adoc#history) for how we came to the present mess, and\n\u003e [Compiler status](#compiler-status) for information about what is\n\u003e expected to work, and [Compiler overview](#compiler-overview) for a\n\u003e quick overview of the compiler internals.\n\nXL is an extensible programming language designed to accomodate a\nvariety of programming needs with ease. Being _extensible_ means that\nthe language is designed to make it very easy for programmers to adapt\nthe language to suit their needs, for example by adding new\nprogramming constructs. In XL, extending the language is a routine\noperation, much like adding a function or creating a class in more\ntraditional programming languages.\n\nAs a validation of this bold claim, XL has a single fundamental operator,\nthe [definition operator](#semantics-one-operator-to-rule-them-all),\nwhich you write `[Pattern] is [Implementation`, where] `[Pattern]` is\na program pattern, like `X+Y`, and `[Implementation]` explains how to\ntranslate that pattern, for example `Add X, Y`.\n\nEverything that is built-in in most other programming languages, from\nbasic data types to arithmetic to conditionals to loops is provided by\nthe standard library in XL. You can replace these constructs if you\nwant, or add your own. Adding a new kind of loop is not more difficult\nin XL than adding a function, and it uses the same syntax.\n\n* [Simple examples](#a-few-simple-examples)\n* [Dialects and use cases](#dialects-and-use-cases)\n* [If you come from another language](#if-you-come-from-another-language)\n* [One operator to rule them all](#semantics-one-operator-to-rule-them-all)\n* [Syntax: Look, Ma, no keywords!](#syntax-look-ma-no-keywords)\n* [XL as a functional language](#XL-as-a-functional-language)\n* [Subtelty #1: Expression vs statement](#subtlety-1-expression-vs-statement)\n* [Subtelty #2: Infix vs. Prefix](#subtlety-2-infix-vs-prefix)\n* [Subtelty #3: Delayed evaluation](#subtlety-3-delayed-evaluation)\n* [Subtelty #4: Closures](#subtlety-4-closures)\n\nFor more information, please consult the\n[XL handbook](https://c3d.github.io/xl/HANDBOOK.html),\nalso available in [asciidoc format](docs/HANDBOOK.adoc) and\n[PDF format](docs/HANDBOOK.pdf)\n\n**WARNING** This documentation, like the compiler, is work in progress\nand presently extremely messy, incomplete and inaccurate.\n\n\n## A few simple examples\n\nA program computing the factorial of numbers between 1 and 5 would be\nwritten as follows:\n```xl\n0! is 1\nN! is N * (N-1)!\n\nfor I in 1..5 loop\n    print \"The factorial of \", I, \" is \", I!\n```\n\n\nAs a testament to its extensible nature, fundamental operations in XL\nare defined in the standard library, including operations that would\nbe implemented using keywords in more traditional languages. For\nexample, the `if` statement in XL is defined by the following code:\n\n```xl\nif [[true]]  then TrueClause else FalseClause   is TrueClause\nif [[false]] then TrueClause else FalseClause   is FalseClause\nif [[true]]  then TrueClause                    is TrueClause\nif [[false]] then TrueClause                    is false\n```\n\nSimilarly, the `while` loop is defined as follows:\n\n```xl\nwhile Condition loop Body is\n    if Condition then\n        Body\n        while Condition loop Body\n```\n\nThe standard library also provides implementations for usual\noperations. For example, if you evaluate `1+3`, this is done through a\ndefinition for `+` on `integer` values that looks like the following\n(where `...` denotes some implementation-dependent code):\n\n```xl\nX:integer + Y:integer is ...\n```\n\n\n## Dialects and use cases\n\nTwo dialects of XL further demonstrate the extensibility of the language\n\n* [Tao3D](http://tao3d.sf.net) focuses on real-time 3D animations\n  and can be used as a scriptable presentation software, or as someone\n  once described it, a sort of real-time 3D LaTeX Lisp. In Tao3D, you\n  describe a slide with a program that looks like the following code:\n\n  ```xl\n  import Slides\n\n  slide \"A simple slide example\",\n      * \"This looks like some kind of markdown language\"\n      * \"But code makes it powerful: your mouse is on the \" \u0026 position\n      position is\n          if mouse_x \u003c 0 then \"left\" else \"right\"\n  ```\n\n   \u003eThe examples above use the new syntax in XL, with `is` as its definition\n   \u003eoperator. Older variants of the language used `-\u003e` instead. If\n   \u003eyou downloaded a pre-built binary of Tao3D, chances are that you need\n   \u003eto replace `is` with `-\u003e` for the code above to work as intended.\n\n* [ELFE](http://github.com/c3d/elfe), formerly ELIOT (Extensible\n  Language for the Internet of things) was an experiment on how to\n  write distributed software that looks like a single program, for\n  example to control swarms of devices in the context of the Internet\n  of Things. An example of a simple ELFE program would be:\n\n  ```xl\n  WORKER is \"worker.mycorp.com\"\n  MIN_TEMP is 25\n  MAX_TEMP is 55\n\n  invoke WORKER,\n      every 2s,\n          reply\n              display temperature\n\n  display Temp:real is\n      print \"The temperature of \", WORKER, \" is \", Temp\n  ```\n\n\u003e The present branch, `bigmerge`, is an ongoing effort to _reconverge_\n\u003e the various dialects of XL. At the moment, it should pass most of\n\u003e the ELFE-level tests, although this is not actively tested. Getting\n\u003e it to support Tao3D is somewhat more difficult and may take some time.\n\n\n## If you come from another language\n\nIf you are familiar with other programming languages, here are a few\nthings that may surprise you about XL.\n\n* There are no keywords. In C, `if` is a keyword. In XL, it's just a name.\n* The language is designed primarily to be readable and writable by humans.\n  For example, there are [special parsing rules](#subtlety-1-expression-vs-statement) to match how we read the code.\n* The language is _homoiconic_, i.e. programs are data, like in Lisp.\n  This forms the basis of XL extensibility.\n* Evaluation is defined entirely in terms of rewrites of a very simple abstract.\n  syntax tree that represents the program being evaluated.\n* The precedence of all operators is dynamic, in the sense that it's\n  loaded from a [configuration file](src/xl.syntax)\n* The language is primarily defined by its own\n  [standard library](src/builtins.xl), rather than by special rules in\n  the compiler.\n\n\n### Semantics: One operator to rule them all\n\nXL has one fundamental operator, `is`, the _definition operator_.\nThis operator can be read as *transforms into*, i.e. it transforms the\ncode that is on the left into the code that is on the right.\n\n\u003cdetails\u003e\n\u003csummary\u003eIt can define simple variables\u003c/summary\u003e\n\n```xl\npi              is      3.1415926\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eIt can define lists\u003c/summary\u003e\n\n```xl\nwords           is      \"xylophage\", \"zygomatic\", \"barfitude\"\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eIt can define functions\u003c/summary\u003e\n\n```xl\nabs X           is      if X \u003c 0 then -X else X\n```\n\n\u003c/details\u003e\n\u003cdetails\u003e\n\u003csummary\u003eIt can define operators\u003c/summary\u003e\n\n```xl\nX ≠ Y           is      not X = Y\n```\n\n\n\u003c/details\u003e\n\u003cdetails\u003e\n\u003csummary\u003eIt can define specializations for particular inputs\u003c/summary\u003e\n\n```xl\n0!              is      1\nN!  when N \u003e 0  is      N * (N-1)!\n```\n\n\u003c/details\u003e\n\u003cdetails\u003e\n\u003csummary\u003eIt can define notations using arbitrary combinations of operators\u003c/summary\u003e\n\n```xl\nA in B..C       is      A \u003e= B and A \u003c= C\n```\n\n\u003c/details\u003e\n\u003cdetails\u003e\n\u003csummary\u003eIt can define optimizations using specializations\u003c/summary\u003e\n\n```xl\nX * 1           is      X\nX + 0           is      X\n```\n\n\u003c/details\u003e\n\u003cdetails\u003e\n\u003csummary\u003eIt can define program structures\u003c/summary\u003e\n\n```xl\nloop Body       is      Body; loop Body\n```\n\n\n\u003c/details\u003e\n\u003cdetails\u003e\n\u003csummary\u003eIt can define types\u003c/summary\u003e\n\n```xl\ntype complex    is      polar or cartesian\ntype cartesian  is      cartesian(re:number, im:number)\ntype polar      is      polar(mod:number, arg:number)\n```\n\nNote that types in XL indicate the shape of parse trees. In other\nwords, the `cartesian` type above will match any parse tree that takes\nthe shape of the word `cartesian` followed by two numbers, like for\nexample `cartesian(1,5)`.\n\u003c/details\u003e\n\u003cdetails\u003e\n\u003csummary\u003eIt can define higher-order functions, i.e. functions that return functions\u003c/summary\u003e\n\n```xl\nadder N         is      (X is N + X)\nadd3            is      adder 3\n\n // This will compute 8\n add3 5\n```\nThis makes XL a truly functional language.\n\n\u003c/details\u003e\n\u003cdetails\u003e\n\u003csummary\u003eIt can define maps associating a key to a value\u003c/summary\u003e\n\n```xl\nmy_map is\n    0 is 4\n    1 is 0\n    8 is \"World\"\n    27 is 32\n    N when N \u003c 45 is N + 1\n\n// The following is \"World\"\nmy_map 8\n\n// The following is 32\nmy_map[27]\n\n// The following is 45\nmy_map (44)\n```\n\nThis provides a functionality roughly equivalent to `std::map` in\nC++. However, it's really nothing more than a regular function with a\nnumber of special cases. The compiler can optimize some special kinds\nof mapping to provide an efficient implementation.\n\n\u003c/details\u003e\n\u003cdetails\u003e\n\u003csummary\u003eIt can define templates (C++ terminology) or generics (Ada terminology)\u003c/summary\u003e_\n\n```xl\n// An (inefficient) implementation of a generic 1-based array type\ntype array [1] of T is\n    Value : T\n    1 is Value\ntype array [N] of T when N \u003e 1 is\n    Head  : array[N-1] of T\n    Tail  : T\n    I when I\u003cN is Head[I]\n    I when I=N is Tail\n\nA : array[5] of integer\nfor I in 1..5 loop\n    A[I] := I * I\n```\n\n\u003c/details\u003e\n\u003cdetails\u003e\n\u003csummary\u003eIt can define variadic functions\u003c/summary\u003e\n\n```xl\nmin X       is X\nmin X, Y    is { Z is min Y; if X \u003c Z then X else Z }\n\n// Computes 4\nmin 7, 42, 20, 8, 4, 5, 30\n```\n\u003c/details\u003e\n\nIn short, the single `is` operator covers all kinds of declarations\nthat are found in other languages, using a single, easy to read\nsyntax.\n\n\n### Syntax: Look, Ma, no keywords!\n\nXL has no keywords. Instead, the syntax relies on a rather simple\n[recursive descent](https://en.wikipedia.org/wiki/Recursive_descent_parser)\n[parser](src/parser.cpp).\n\nTHe parser generates a parse tree made of 8 node types. The first four\nnode types are leaf nodes:\n\n * `Integer` is for integer numbers such as `2` or `16#FFFF_FFFF`.\n * `Real` is for real numbers such as `2.5` or `2#1.001_001_001#e-3`\n * `Text` is for text values such as `\"Hello\"` or `'World'`. Text can\n   be encoded using UTF-8\n * `Name` is for names and symbols such as `ABC` or `**`\n\nThe last four node types are inner nodes:\n\n * `Infix` are nodes where a named operator separates the operands,\n   e.g. `A+B` or `A and B`.\n * `Prefix` are nodes where the operator precedes the operand, e.g.\n   `+X` or `sin X`. By default, functions are prefix.\n * `Postfix` are nodes where the operator follows the operand, e.g.\n   `3%` or `5km`.\n * `Block` are nodes with only one child surrounded by delimiters,\n   such as `(A)`, `[A]` or `{A}`.\n\nOf note, the line separator is an infix that separates statements,\nmuch like the semi-colon `;`. The comma `,` infix is traditionally\nused to build lists or to separate the argument of\nfunctions. Indentation forms a special kind of block.\n\nFor example, the following code:\n\n```xl\n    tell \"foo\",\n        if A \u003c B+C then\n            hello\n        world\n```\n\nparses as a prefix `tell`, with an infix `,` as its right argument. On\nthe left of the `,` there is the text `\"foo\"`. On the right, there is\nan indentation block with a child that is an infix line separator. On\nthe left of the line separator is the `if` statement. On the right is\nthe name `world`.\n\nThis parser is dynamically configurable, with the default priorities\nbeing defined by the [xl.syntax](src/xl.syntax) file.\n\nParse trees are the fundamendal data structure in XL. Any data or\nprogram can be represented as a parse tree. Program evaluation is\ndefined as transformation of parse trees.\n\n\n### XL as a functional language\n\nXL can be seen as a functional language, where functions are\nfirst-class entities, i.e. you can manipulate them, pass them around,\netc:\n\n```xl\nadder X:integer is (Y is Y + X)\n\nadd3 is adder 3\nadd5 is adder 5\n\nprint \"3+2=\", add3 2\nprint \"5+17=\", add5 17\nprint \"8+2=\", (adder 8) 2\n```\n\nHowever, it is a bit different in the sense that the core data\nstructure is the parse tree. Some specific parse trees, for example\n`A+B`, are not naturally reduced to a function call, although they are\nsubject to the same evaluation rules based on tree rewrites.\n\n\n### Subtlety #1: expression vs. statement\n\nThe XL parse tree is designed to represent programs in a way that\nis relatively natural for human beings. In that sense, it departs from\nlanguages such as Lisp or SmallTalk.\n\nHowever, being readable for humans requires a few special rules to\nmatch the way we read expressions. Consider for example the following:\n\n```xl\nwrite sin X, cos Y\n```\n\nMost human beings parse this as meaning `write (sin(X),cos(Y))`,\ni.e. we call `write` with two values resulting from evaluating `sin X`\nand `cos Y`. This is not entirely logical. If `write` takes\ncomma-separated arguments, why wouldn't `sin` also take\ncomma-separated arguments? In other words, why doesn't this parse as\n`write(sin(X, cos(Y))`?\n\nThis shows that humans have a notion of *expressions*\nvs. *statements*. Expressions such as `sin X` have higher priority\nthan commas and require parentheses if you want multiple arguments. By\ncontrast, statements such as `write` have lower priority, and will\ntake comma-separated argument lists. An indent or `{ }` block begins a\nstatement, whereas parentheses `()` or square brackets `[]` begin an\nexpression.\n\nThere are rare cases where the default rule will not achieve the\ndesired objective, and you will need additional parentheses.\n\n### Subtlety #2: infix vs. prefix\n\nAnother special rule is that XL will use the presence of a space on\nonly one side of an operator to disambiguate between an infix or a\nprefix. For example:\n\n```xl\nwrite -A   // write (-A)\nB - A      // (B - A)\n```\n\n### Subtlety #3: Delayed evaluation\n\nWhen you pass an argument to a function, evaluation happens only when\nnecessary. Deferred evaluation may happen multiple times, which is\nnecessary in many cases, but awful for performance if you do it by\nmistake.\n\nConsider the following definition of `every`:\n\n```xl\nevery Duration, Body is\n    loop\n        Body\n        sleep Duration\n```\n\nIn that case, we want the `Body` to be evaluated every iteration,\nsince this is typically an operation that we want to execute at each\nloop. Is the same true for `Duration`? Most likely, no.\n\nOne way to force evaluation is to give a type to the argument. If you\nwant to force early evaluation of the argument, and to check that it\nis a real value, you can do it as follows:\n\n```xl\nevery Duration:real, Body is\n    loop\n        Body\n        sleep Duration\n```\n\n### Subtlelty #4: Closures\n\nLike many functional languages, XL ensures that the value of\nvariables is preserved for the evaluation of a given body. Consider\nfor example:\n\n```xl\nadder X:integer is (Y is Y + X)\nadd3 := adder 3\n```\n\nIn that case, `adder 3` will bind `X` to value `3`, but then the\nreturned value outlives the scope where `X` was declared. However, `X`\nis referred to in the code. So the returned value is a *closure* which\nintegrates the binding `X is 3`.\n\n\n## Compiler status\nWork items for the XL compiler (will be turned into GitHub issues)\n\n### Language definition\n- [X] Language definition documentation [published](https://c3d.github.com/xl)\n- [X] Quickly-scanned To-Do list (this section)\n- [ ] Finish [chapter on compilation](https://c3d.github.io/xl/#compiling-xl)\n- [ ] Finish [chapter on programming paradigms](https://c3d.github.io/xl/#programming-paradigms)\n- [ ] Description of the [module system](https://c3d.github.io/xl/#modules)\n- [ ] Description of the [standard library](https://c3d.github.io/xl/#standard-library)\n- [ ] Decide whether it's `import`, `use` or both (`import` doing the\n      importing  and `use` bringing the referenced expression in scope).\n\n### Recent language changes\n- [X] Switch from `-\u003e` to `is` as the definition operator\n- [ ] Switch type definition from `type Pattern` to `matching Pattern`\n      ([issue #5](https://github.com/c3d/xl/issues/5))\n- [ ] Implement syntactic sugar (can it be lib only?), e.g. `type X is Y`\n      and `module X with Y` or `to Copy(...) is blah`\n      ([issue #6](https://github.com/c3d/xl/issues/6))\n- [ ] Support for nested functions and proper nested scopes.\n      See [issue #8](https://github.com/c3d/xl/issues/8).\n- [ ] Scope injection and scoping, i.e. meaning of `scope.Foo` and\n      `scope Foo` in the language. Deal with `(scope) foo`, etc.\n      See [scoping](https://c3d.github.io/xl/#scoping) and\n      [issue #9](https://github.com/c3d/xl/issues/9).\n- [ ] Safe implementation of `for` loop using scope injection\n      (see [name parameters](https://c3d.github.io/xl/#name-parameters)).\n      This is [issue #7](https://github.com/c3d/xl/issues/7).\n- [ ] Implement metabox (`[[true]]`). This is [issue #10](https://github.com/c3d/xl/issues/10).\n- [ ] Write the interface and implementation of the `type` type\n      ([issue #11](https://github.com/c3d/xl/issues/11)).\n- [ ] Implement union types (`T1 or T2`), as well as `and` and `not`.\n      See [issue #12](https://github.com/c3d/xl/issues/12)\n- [ ] Revisit dynamic dispatch based on types.\n      See [issue #13](https://github.com/c3d/xl/issues/13)\n- [ ] Implement type inheritance checks (`Derived like Base`).\n      See [issue #14](https://github.com/c3d/xl/issues/14)\n- [ ] Generate `lifetime` values.\n      See [issue #15](https://github.com/c3d/xl/issues/15)\n- [ ] Implement `own` and `ref` types.\n      See [issue #16](https://github.com/c3d/xl/issues/16)\n- [ ] Implement `in`, `out`, `inout` types.\n      See [issue #17](https://github.com/c3d/xl/issues/17)\n- [ ] Implement creation and destruction.\n      See [issue #18](https://github.com/c3d/xl/issues/18)\n\n### Symbol table\n- [x] Symbol table as an XL parse tree\n- [ ] Symbol table storing actual definitions, using `is` operator.\n      Currently, the symbol table uses a \"nonsensical\" prefix.\n      See [issue #19](https://github.com/c3d/xl/issues/19)\n\n### Scanner\n- [x] Scanner\n- [ ] Scan version numbers correctly, e.g. `1.20.5` not parsed as `(1.2).5`\n- [ ] Better Unicode classification of letters versus\n      punctuation. Currently, only ASCII punctuation is recognized as such.\n- [ ] Option to transparently convert `-\u003e` to `is` (for Tao3D importing)\n- [ ] Scanner-time processing of `syntax` statements from imported files\n\n### Parser\n- [x] Parser\n- [ ] \"Binary\" terminal node holding arbitrary binary data\n- [ ] \"Literal\" terminal node holding arbitrary text parsed from regexp\n- [ ] Add more error checks for failure cases, better error reporting\n\n### Renderer\n- [x] Renderer (`-show` option)\n- [x] Style renderer with `-style` option and `.stylesheet` files\n- [x] Preserving comments in renderer output\n- [ ] Rendering binary or regexp nodes\n\n### Runtime\n- [x] Foreign function interface (FFI) with easy macro (see `native.h`\n- [ ] Tag all useful runtime functions with FFI\n- [ ] Find a strategy for migrating Tao3D 1500 runtime calls\n- [ ] Cleanup the runtime from obsolete / useless functions\n\n### Interpreter (-O0)\n- [x] Simple interpreter (`-O0` or `-i`)\n- [ ] Explicit FFI for interpreter (`extern` C syntax)\n- [ ] Add \"sandboxed\" mode (on by default for interpreter) that\n      disables the above\n- [ ] Connect `native.h` FFI to interpreter\n- [ ] Implement modules in interpreter\n- [ ] Implement `error` and `compile_error` evaluation rules in interpreter\n- [ ] Update type system to recognize `matching` prefix instead of `type`\n\n### Bytecode interpreter (-O1) BROKEN, LIKELY NOT TO BE REPAIRED\n- [X] Bytecode interpreter (broken, do not use)\n- [ ] Fix bytecode interpreter or get rid of its remnants (`-O2`)\n- [ ] Redesign bytecode interpreter as emitting LLVM byte code?\n\n### LLVM \"fast compiler\" (-O2)\nSimplistic compiler that does only run-time type analysis\n- [X] Fast compiler functionally similar to what was used in Tao3D\n- [ ] Find strategy to re-connect it to Tao3D\n\n### LLVM optimized compiler (-O3)\n- [x] LLVM-based compiler\n- [x] LLVM-CRAP (adapting to multiple versions of LLVM)\n- [x] `native.h` for building FFI\n- [x] Option to emit LLVM bitcode (`-B` or `-emit_ir`)\n- [ ] Option to pass bitcode to LLVM bitcode compiler\n      (Automatically do something like `xl -B ... | llc -filetype=asm`)\n- [ ] Option to directly emit disassembly\n- [ ] Pass LLVM options to LLVM (`-llvm-foo`)\n- [ ] Replace algo-W type analysis with forward-only type analysis.\n- [ ] Implement\n      [auto-boxing](https://c3d.github.io/xl/#machine-representation)\n      for\n      [parameter scopes](https://c3d.github.io/xl/#pattern-matching-scope)\n- [ ] Self-compiling compiler\n- [ ] Rebuild Tao3D on top of \"new\" XL\n\n### Build system\n- [X] Portable auto-configuring makefile (Using `make-it-quick`)\n- [X] Targets for `opt`, `debug`, `release` and `check`\n- [X] Target for `install`\n- [ ] Fix issue with too much stuff being rebuilt for `install`\n\n### CI / Testing\n- [X] Basic QE framework / testing script (`alltest` script, `make check`)\n- [X] Fix breakage in `make check` with macOS Catalina DYLD_LIBRARY_PATH\n- [X] Basic CI in GitLab\n- [X] Repair breakage in GitLab CI\n- [X] Add tests for variants of LLVM in GitLab CI\n- [X] Basic CI in GitHub\n\n### Modules\n- [ ] Reimplement module system\n\n### Standard library\n- [x] Standard library\n- [x] Overall structure (WIP)\n- [ ] Types\n- [ ] Arrays\n- [ ] STL-style containers\n- [ ] Algorithms\n- [ ] I/Os\n- [ ] Threading / synchronization\n- [ ] Math and numerics\n- [ ] Standard math functions\n- [X] Complex module\n- [ ] Vectors\n- [ ] Matrices\n- [ ] Text processing\n- [ ] Time\n- [ ] Graphics (Tao3D based?)\n\n## Compiler overview\n\nThe interpreter / compiler recently went through a rather heavy\nmerge of several incompatible branches. As a result, it inherited\nthe best of the various branches, but is overall quite broken.\nThere are actually several, somewhat incompatible versions of\nthe language, some of which reside in different binaries, some\nin the primary binary.\n\nThe primary binary resides in the `src` directory. It is written\nin C++, and there is currently no real plan to self-compile it,\nalthough there are plans to use it as a basis for a self-compiling\ncompiler bootstrap someday.\n\nThat primary binary contains a single scanner and parser for the\nXL language, but three different ways to evaluate it, which are\ninstances of the C++ `Evaluator` class. These three ways to evaluate\nXL are selected using the `-O` option.\n\n* `-O0` or `-i` selects an interpreter. This interpreter is\nessentially similar to what used to be the ELFE implementation\nof the language, i.e. a very small implementation that performs\nevaluation using parse tree rewrites. It sort of works, passes\nmost tests with `make check`, and is overall sane, if a bit slow\n(similar to `bash` in my testing). It can be used for example\nas an extension language for your application, and does not draw\nmuch in terms of dependencies. You would add your own vocabulary\nusing simple-to-write \"modules\". See the `Makefile` for examples.\nThat part is the only one I can advertise as possibly useful.\nIn particular, it correctly runs the examples in the `demo` directory,\nwhich are the older ELFE demos, i.e. distributed programming\nfrom a single source code.\n\n* `-O1` selects the `FastCompiler`, which is basically an\nadaptation of the compiler used in the Tao3D program, with\nthe intent to allow the `master` branch of XL to be able to\nsupport Tao3D again without major incompatibilities. It generates\nmachine code using LLVM, but the generated code is relatively\ninefficient because it manipulates the parse tree. For example,\nan integer value is always represented by an `Integer` pointer,\nso there is always an indirection to access it. Also, while\nforward-porting that compiler to a version of the compiler that\nhad dropped it, I broke a number of things. So under repair,\nand not currently able to support Tao3D yet.\n\n* `-O2` and above select the `Compiler` class, which is an\nongoing attempt at implementing XL the way I always wanted to,\nusing type inference to generate efficient code. Presently, the\ntype inference is so badly broken that it's likely to reject\na number of very valid programs, including the basic factorial\nexample. I have hope, though. At some point, that implementation\nwas able to compete with C on relatively simple programs, but\nonly with a lot of type annotations. I'm trying to achieve the\nsame result without the type annotations. We're getting there.\nLike `-O1`, `-O2` output uses LLVM to generate machine code, but\nthat time, it's good machine code.\n\nIf you think 3 implementations is bad, wait. There is more.\nThere is a `Bytecode` class that is yet another evaluator\nthat attempted to generate a bytecode so as to accelerate\ninterpreted evaluation, without having to bring in LLVM and\nall the risks associated with dynamic code generation (e.g. if\nyou use XL as an extension language). Unfortunately, that\nbytecode experiment went nowhere. It's slow, ridden with bugs,\nand has severely damaged other parts of the compiler. I can't\nwait to expurge it from the compiler.\n\nSo now that's it, right? Well... No.\n\nYou see, the current XL started life as a \"runtime\" language\nfor the \"real\" XL. The original XL looked more like Ada,\nand had very different type semantics.\nSee [HISTORY](docs/HANDBOOK.adoc#history)\nfor all the gory details. Suffice it to say here that this\ncompiler resides in the `xl2` directory (because, yes, it was\nalready version 2 of the language). One reason for me to keep\nit is that it's the only version of the compiler that ever\ncame close to self-compiling it. So I keep it around to remind\nmyself of various neat tricks that XL made possible, like the\n`translate` instruction.\n\nNow, you are really done, right? Well... There's one more.\n\nSee, I really want the compiler to self-compile. So in order\nto prepare for that, there is a `native` directory where I\nstore tidbits of what the future compiler and library would\nlook like. Except that this is really an exploratory scratchpad,\nso the various modules are not even consistent with one another...\nBut ultimately, if everything goes according to plan, the C++\ncompiler should be able to compile `native` in order to generate\na compiler that would compile itself.\n","funding_links":[],"categories":["C++","Other"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fc3d%2Fxl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fc3d%2Fxl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fc3d%2Fxl/lists"}