{"id":17129563,"url":"https://github.com/attila-lendvai/maru","last_synced_at":"2025-04-10T03:13:24.020Z","repository":{"id":13389082,"uuid":"16077246","full_name":"attila-lendvai/maru","owner":"attila-lendvai","description":"Maru - a tiny self-hosting lisp dialect","archived":false,"fork":false,"pushed_at":"2025-03-24T11:39:19.000Z","size":4192,"stargazers_count":195,"open_issues_count":5,"forks_count":10,"subscribers_count":15,"default_branch":"maru.10","last_synced_at":"2025-04-10T03:13:17.285Z","etag":null,"topics":["bootstrap-process","bootstrapping","lisp","maru","self-hosted","self-hosting","sexpression-language"],"latest_commit_sha":null,"homepage":"http://piumarta.com/software/maru/","language":"Common Lisp","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/attila-lendvai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-01-20T17:10:05.000Z","updated_at":"2025-03-24T11:23:27.000Z","dependencies_parsed_at":"2024-12-01T10:45:43.669Z","dependency_job_id":null,"html_url":"https://github.com/attila-lendvai/maru","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/attila-lendvai%2Fmaru","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/attila-lendvai%2Fmaru/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/attila-lendvai%2Fmaru/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/attila-lendvai%2Fmaru/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/attila-lendvai","download_url":"https://codeload.github.com/attila-lendvai/maru/tar.gz/refs/heads/maru.10","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248148240,"owners_count":21055548,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bootstrap-process","bootstrapping","lisp","maru","self-hosted","self-hosting","sexpression-language"],"created_at":"2024-10-14T19:09:58.344Z","updated_at":"2025-04-10T03:13:24.009Z","avatar_url":"https://github.com/attila-lendvai.png","language":"Common Lisp","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Maru\n\n## What\n\nMaru is a programming language. It's a\n[self-hosting](https://en.wikipedia.org/wiki/Self-hosting_(compilers)),\nyet tiny [lisp](https://en.wikipedia.org/wiki/Lisp_(programming_language))\ndialect: a symbolic expression evaluator that can compile its own implementation to\n[machine code](https://en.wikipedia.org/wiki/Machine_code),\nin about 2000 LoC altogether.\n\n\u003e Maru is in particular trying to be malleable at the very lowest levels,\n\u003e so any special interest that cannot be accommodated easily within the common\n\u003e platform would be a strong indicator of a deficiency within the platform\n\u003e that should be addressed rather than disinherited. (Ian Piumarta)\n\nThis repo is also a place for exploration in the land of\n[bootstrapping](doc/bootstrap.md) and computing system\ndevelopment. My primary drive with Maru is to clearly\nand formally express that which is mostly treated as black magic:\nthe bootstrapping of a language on top of other languages (which\nincludes the previous developmental stage of the same language).\n\n## Meta\n\nThis document aims to present an overview of Maru. There are various\ndocuments in the [`doc/`](doc/) directory that discuss some topics in\nmore detail.\n\n## How\n\nMaru's architecture is described in [doc/how.md](doc/how.md).\n\n#### Build instructions\n\nTo test a bootstrap cycle using one or all of the backends:\n\n```\nmake test-bootstrap-x86    # defaults to the libc platform\nmake PLATFORM=[libc,linux] test-bootstrap[-llvm,-x86]\n```\n\n\u003cdetails\u003e\n\u003csummary\u003ePlatform specific instructions\u003c/summary\u003e\n\n#### Nix and NixOS\n\nMy primary platform. There's a `default.nix` file in the repo, so you\ncan run `nix-shell` to enter into the same environment that I use when\nI work on Maru.\n\n#### Debian, and derivatives\n\n```\nsudo apt install make time rlwrap\n```\n\nYou will need LLVM, and/or a C compiler (any version beyond LLVM 8 should work):\n\n```\nsudo apt install llvm clang\n```\n\nFor now the x86 backend only supports 32 bit mode. To use it you will\nneed to have support for compiling and running 32 bit C code. On\nDebian based x86_64 systems this will install all the necessary\nlibraries:\n\n```\nsudo apt install gcc-multilib\n```\n\n#### MacOS\n\nPlease note that recent MacOS versions don't support 32 bit\nexecutables anymore, but Maru's LLVM backend should work fine. I don't\ntest it regularly, so things may not always work out of the box.\n\n1. Make sure XCode is installed. In a Terminal:\n\n```\nxcode-select --install\n```\n\n2. [Install Homebrew](https://brew.sh/)\n\n```\n/bin/bash -c \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)\"\n```\n\n3. [Install LLVM using Homebrew](https://embeddedartistry.com/blog/2017/02/24/installing-llvm-clang-on-osx/)\n```\nbrew install llvm\necho export PATH=\"$(brew --prefix llvm)/bin:$PATH\" \u003e\u003e ~/.bash_profile\nsource ~/.bash_profile\n```\n\n#### Other platforms\n\nCurrently Maru should work everywhere where there's a `libc`, and either the\nGNU toolchain, or LLVM is available.\n\nPatches are welcome for other platforms.\n\u003c/details\u003e\n\n## Who\n\nOriginally written by [Ian Piumarta](https://www.piumarta.com/software/maru/),\nat around 2011. Full commit history is available in the\n[`piumarta`](https://github.com/attila-lendvai/maru/tree/piumarta)\nbranch.\n\nThe current gardener is [attila@lendvai.name](mailto:attila@lendvai.name).\n\n## Where\n\nBugs and patches: [maru github page](https://github.com/attila-lendvai/maru).\n\nDiscussion: [maru-dev google group](https://groups.google.com/forum/#!forum/maru-dev).\n\n## Why\n\n* Programming badly needs\n[better foundations](https://github.com/nagydani/seedling/blob/master/RATIONALE.md),\nand Maru is part of this exploration. The foundations should get **smaller, simpler,\nmore self-contained, and more approachable** by people who set out to learn programming.\n\n* I'm fascinated by bootstrapping issues.\nWe lose a lot of value by not capturing the history of the growth of a language, including\nthe formal encoding of its build instructions. They are useful both for educational purposes,\nand also for practical reasons: to have a minimal seed that is very simple to\nport to a new architecture, and then have a self-contained, formal bootstrap process that\ncan automatically \"grow\" an entire computing system on top of that freshly laid, tiny foundation.\n\n* Ian seems to have abandoned Maru, and his published archive couldn't\nbe run as-is. But it's an interesting piece of code that deserves a\nrepo and a maintainer to keep bitrot at bay.\n\n* This work is full of puzzles that are a whole lot of fun to solve!\n\n## Contribution\n\nYou are very welcome to contribute, but beware that until further notice\n**this repo will receive forced pushes** (i.e.\n**`git push -f`** rewriting git history (except the `piumarta` branch)). This will stop\neventually when I settle with\na build setup that nicely facilitates bootstrapping multiple, parallel paths of\nlanguage development. Please make sure that you open a branch for your work,\nand/or that you are ready for some `git fetch` and `git rebase`.\n\n## Status\n\n### Maru's status\n\nBackporting and bootstrapping the latest semantics from the `piumarta`\nbranch is done: the `eval.l` in the latest branch of this repo should\nbe semantically equivalent with the `eval.l` that resides in the\n`piumarta` branch, although, we have arrived to this state on two\ndifferent paths:\n\n  - Ian, while evolving Maru, kept his `eval.c` and `eval.l`\n    semantically in sync\n\n  - in contrast, I have bootstrapped the new features: I started out\n    from a minimal version of the `eval.l` + `eval.c` couple (the\n    [original version](https://www.piumarta.com/software/maru/)\n    published on Ian's website). Then I bootstrapped the features of\n    the later stages of `eval.l` using an earlier stage of itself. I\n    only use the 2300 LoC of throwaway C code as the initial stepping\n    stone in the bootstrap process, but once the first step is made\n    the C code is left behind.\n\nThere is one major bug left that I failed to fix while I was actively\nhacking on Maru. It's discussed in\nhttps://github.com/attila-lendvai/maru/issues/8.\n\n### Notable new features\n\nThere are several Maru stages/branches now, introducing non-trivial\nnew features. Some that are worth mentioning:\n\n  - Introduction of [*platforms*](platforms.md), and notably the\n    `linux` platform that compiles to a statically linked executable\n    that only uses Linux kernel\n    [`syscalls`](https://en.wikibooks.org/wiki/X86_Assembly/Interfacing_with_Linux);\n    From a practical perspective this is almost equivalent with\n    running directly on the bare metal (i.e. all dynamically allocated\n    memory needs to be managed by our own GC, all IO behind our own\n    abstractions, etc).\n\n  - The host and the slave are isolated while bootstrapping which makes it possible to\n    do things like reordering types (changing their type id in the target),\n    or changing their object layout.\n\n  - Relying on this isolation, the code in `eval.l` now looks pretty much the same\n    as something that is meant to be loaded into the evaluator (i.e. the function\n    implementing `car` in `eval.l` is now called `car`). This paves the way for\n    metacircularity: to be able to \"bring alive\" the evaluator by loading it\n    verbatim into another instance of itself (as opposed to compiling it to\n    machine code and giving it to a CPU to bring it alive).\n\n  - The addition of an LLVM backend.\n\n### Future plans\n\n\u003cdetails\u003e\n\u003csummary\u003eAssorted TODO list\u003c/summary\u003e\n\n  - Make Maru Scheme compatible, either by forking it, or by some sort\n    of a compatibility layer that is loadable into vanilla Maru. Then\n    consider how that relates to [GNU Mes](https://www.gnu.org/software/mes/).\n\n  - Finish the proof of concept in `tests/test-elf.l` to compile the\n    Linux plaform directly into an ELF binary. This would reduce the\n    list of external dependencies to a single one (GNU Make).\n\n  - Rewrite the build process in Maru; eliminate dependency on GNU Make.\n\n  - Replace the hand-written parser in `eval.l` with something\n    generated by a parser generator, maybe the\n    [PEG](https://en.wikipedia.org/wiki/Parsing_expression_grammar)\n    compiler. More generally, make the parser extendable.\n\n  - Implement modules and phase separation along with what is outlined in\n    [Submodules in Racket - You Want it When, Again?](https://www.cs.utah.edu/plt/publications/gpce13-f-color.pdf).\n    Part of this is already done and is used in the bootstrap process.\n\n  - Compile to, and bootstrap on the bare metal of some interesting\n    targets. It's already demonstrated by the Linux platform. Another\n    one could be\n    [pc-bios](https://github.com/cirosantilli/x86-bare-metal-examples),\n    or [EFI](https://kuroko-lang.github.io/),\n    because it's easily testable using QEMU. Or port it to an ARM\n    board (like Raspberry Pi)? Or maybe even attempt a C64 port?\n\n  - Revive all the goodies in the `piumarta` branch, but in a structured way.\n\n  - Investigate [Cranelift](https://github.com/bytecodealliance/wasmtime/tree/main/cranelift)\n    and consider adding it as a backend.\n\n  - Simplify the types-are-objects part and its bootstrap, and maybe even\n    make it optional?\n\n  - Weed out some of the added bloat/complexity (e.g. compile closures\n    instead of `\u003cselector\u003e`s, and use them to implement streams; write a\n    tree shaker; etc).\n\n  - Merge the language and API that the compiler and the evaluator understands;\n    i.e. make the level-shifted code (`eval.l` \u0026 co.) less different than code\n    understood by the evaluator. This would mean that we can e.g. load/compile\n    `source/buffer.l` both into the level-shifted code and into the evaluator.\n    This is slowly happening, but it's nowhere near done, and I'm not even sure\n    what being done means here.\n\n  - Use LLVM's [tablegen](https://llvm.org/docs/TableGen/index.html)\n    definitions to generate machine code assemblers for various CPU\n    architectures. It requires either the reimplementation of the\n    tablegen parser/logic in Maru (doesn't seem to be trivial), or\n    writing C++ code (uhh!) to compile the data to something that is\n    easy to work with in Maru.\n\n  - Maybe add [PEG-based tree rewriter](https://www.piumarta.com/S3-2010/)\n    to the repo as a branch, and use it as a bootstrap stage. It seems to\n    be an earlier, or different iteration of the same idea.\n\n  - Introduce a simplified language that drops some langauge features,\n    e.g. remove *forms* and the *expand* protocol. Make sure that this\n    language can bootstrap itself off of C99. Then reintroduce *forms*\n    and *expand* by using this simplified Maru as the bootstrap host.\n\n  - Understand and incorporate François René Rideau's model of\n    [First Class Implementations: Climbing up the Semantic Tower](https://www.youtube.com/watch?v=fH51qhI3hq0),\n    (see this [couple of page summary](https://github.com/fare/climbing), or\n    see his [page on reflection](http://fare.tunes.org/reflection.html))\n\n\u003c/details\u003e\n\n### History and perspective\n\n#### Around 2010-2013\nMaru was developed as part of Alan Kay's *Fundamentals of New Computing* project,\nby the *[Viewpoints Research Institute](https://web.archive.org/web/20111002111856/http://www.viewpointsresearch.org/fonc_wiki/index.php/Idst)*.\nThe goal of the project was to implement an entirely new, self-hosting computing system, with GUI, in 20.000 lines of code.\n\nAt some point VPRI went quiet and closed down in 2018. Much of their online content disappeared, and the team (probably) also dissolved.\n\nTheir annual reports:\n[2007](http://www.vpri.org/pdf/tr2007008_steps.pdf),\n[2008](http://www.vpri.org/pdf/tr2008004_steps08.pdf),\n[2009](http://www.vpri.org/pdf/tr2009016_steps09.pdf),\n[2010](http://www.vpri.org/pdf/tr2010004_steps10.pdf),\n[2011](http://www.vpri.org/pdf/tr2011004_steps11.pdf),\n[2012](http://www.vpri.org/pdf/tr2012001_steps.pdf).\n\n#### This git repo\n\nThe `piumarta` branch of this git repo is a conversion of Ian Piumarta's Mercurial\nrepo that was once available at\n[http://piumarta.com/hg/maru/](http://piumarta.com/hg/maru/).\nTo the best of my knowledge\nthis is the latest publicly available state of Ian's work. This repo was full of\nassorted code, probably driving the VPRI demos.\n\nThe `piumarta` branch will be left stale (modulo small fixes and cleanups).\nMy plan is to eventually revive most of the goodies from this branch, but in a\nmore organized and approachable manner, and also paying attention to the\nbootstrapping issues.\n\nIan published another Mercurial repo somewhere halfway in the commit history\nwith only a couple of commits from around 2011. I assume that it was meant to hold\nthe [minimal/historical version](https://www.piumarta.com/software/maru/)\nof Maru that can already self-host. I started out\nmy work from this minimal repo (hence the divergence between the `piumarta` and\nthe `maru.x` branches in this repo).\n\n#### Other instances\n\nThere are some other copies/versions of Maru. Here are the ones\nthat I know about and contain interesting code:\n\n- [github.com/melvinzhang/maru](https://github.com/melvinzhang/maru)\n\n- [below-the-top](https://github.com/jbulow/below-the-top) is some kind of\n  generic sexp tokenizer and evaluator written in Common Lisp that can be\n  configured so that it can bootstrap Maru. I haven't tried it myself.\n\n#### Related projects\n\nA list of projects that are relevant in this context:\n\n  - Robert van Engelen's\n    [tinylisp](https://github.com/Robert-van-Engelen/tinylisp) (99\n    lines of C with with GC and a REPL), his\n    [paper](https://github.com/Robert-van-Engelen/tinylisp/blob/main/tinylisp.pdf),\n    and its [big brother](https://github.com/Robert-van-Engelen/lisp),\n    which is 1k LoC of heavily commented C. No compiler, they are not\n    self-hosting.\n\n  - [sectorlisp](https://justine.lol/sectorlisp2/)\n    ([github](https://github.com/jart/sectorlisp)): LISP with GC in\n    436 bytes. It doesn't have a compiler, i.e. it cannot\n    self-host. It only has a C implementation, an x86 assembly\n    impementation (in the form of a boot sector), and John McCarthy's\n    Lisp in Lisp evaluator. It would be an interesting project to add\n    a compiler to it and see how the end result compares to Maru. Or\n    to start growing a language as demonstrated in this repo, but\n    starting out from sectorlisp. Note that sectorlisp is not\n    equivalent to the first stage of Maru (the `maru.1` git branches),\n    because that can already self-host, i.e. it can bootstrap itself\n    off of the C implementation.\n\n  - [Seedling](https://github.com/nagydani/seedling/): a ladder of\n    languages, with a minimalistic core language at the bottom called\n    *Seed* (it's a Forth like). Seed can self-host in less than 1k\n    LoC. The higher level languages above Seed are (going to be)\n    extensions of it, and are implemented on top of Seed. Porting to a\n    new architecture will be trivial. And an interesting tidbit: the\n    initial bootstrap was done not by using another programming\n    language/compiler, but by pen and paper!\n\n  - [bootstrappable.org](http://bootstrappable.org/): a community\n    around bootstrapping, and making/keeping projects\n    bootstrapable. It brings together many interesting projects:\n    [stage0](https://github.com/oriansj/stage0) (~500 byte self-hosting hex assembler),\n    [live-bootstrap](https://github.com/fosslinux/live-bootstrap/),\n    [GNU Mes](https://www.gnu.org/software/mes/) (Scheme + C, mutually self-hosting each other),\n    [m2-planet](https://github.com/oriansj/m2-planet) (a tiny C compiler).\n\n  - [Kalyn](https://intuitiveexplanations.com/tech/kalyn): a subset of\n    Haskell semantics (mostly; not lazy), but with Lisp\n    syntax. Entirely (!)  self-hosting over x86-64 in 4-5 kLoC. The\n    project feels of high standard, including its documentation.\n\n  - [nanohs](https://github.com/bkomuves/nanohs): a tiny self-hosting\n    subset of Haskell.\n\n  - [PEG-based tree rewriter](https://www.piumarta.com/S3-2010/):\n    runnable code to accompany Ian Piumarta's paper called *PEG-based\n    tree rewriter provides front-, middle- and back-end stages in a\n    simple compiler*. Ian wrote this before Maru, and there are\n    several similarities between the two. See the [mailing list\n    thread](https://groups.google.com/g/maru-dev/c/0Cnq1RB-Ahk).\n\n  - [blynn's Haskell compiler](https://github.com/blynn/compiler):\n    bootstrap a Haskell compiler incrementally from C, with extensive\n    documentation..\n\n  - [RefPerSys](http://refpersys.org/): a mostly symbolic artificial\n    intelligence long-term project, with ambitious Artificial General\n    Intelligence goals. It contains interesting and relevant ideas, e.g. in\n    [refpersys-design.pdf](http://starynkevitch.net/Basile/refpersys-design.pdf).\n\n  - [Project Oberon](http://www.projectoberon.com/): a project which encompasses CPU,\n    language, operating system and user interface, and which can be run on a relatively\n    inexpensive FPGA board, and simple enough for one person to understand it all.\n\n  - [tort](https://github.com/kstephens/tort): Inspired by Ian\n    Piumarta's idst, maru and other small runtimes. Core is\n    approx. 5000 lines of C.\n\n  - [kernel](http://web.cs.wpi.edu/~jshutt/kernel.html): \"Kernel is a\n    conservative, Scheme-like dialect of Lisp in which **everything**\n    is a first-class object.\" (including special forms) You may want\n    to also see\n    [this blog](https://axisofeval.blogspot.com/search/label/kernel).\n\n  - [Compiling a Lisp: Overture](https://bernsteinbear.com/blog/compiling-a-lisp-0/):\n    Educational article series about constructing a simple Lisp compiler,\n    implemented in C.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fattila-lendvai%2Fmaru","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fattila-lendvai%2Fmaru","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fattila-lendvai%2Fmaru/lists"}