{"id":13993846,"url":"https://github.com/boothby/repiet","last_synced_at":"2025-10-25T18:18:48.148Z","repository":{"id":50582671,"uuid":"174581365","full_name":"boothby/repiet","owner":"boothby","description":"A compiler for the esoteric language Piet, targeting multiple backends.","archived":false,"fork":false,"pushed_at":"2019-11-25T05:23:06.000Z","size":360,"stargazers_count":79,"open_issues_count":0,"forks_count":5,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-04-24T14:09:17.918Z","etag":null,"topics":["piet","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/boothby.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-03-08T17:35:05.000Z","updated_at":"2024-03-20T10:05:32.000Z","dependencies_parsed_at":"2022-08-31T20:31:51.804Z","dependency_job_id":null,"html_url":"https://github.com/boothby/repiet","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/boothby%2Frepiet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/boothby%2Frepiet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/boothby%2Frepiet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/boothby%2Frepiet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/boothby","download_url":"https://codeload.github.com/boothby/repiet/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":227156399,"owners_count":17739293,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["piet","python"],"created_at":"2024-08-09T14:02:35.416Z","updated_at":"2025-10-25T18:18:48.052Z","avatar_url":"https://github.com/boothby.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Repiet: The Brutalizing Piet Recompiler\n\n\u003cimg src=\"assets/welcome.png\" align=\"right\" width=\"100%\" title=\"A Piet program welcoming you to repiet.\" alt=\"A Piet program welcoming you to repiet; a mostly-white field with thin, multicolored lines on the left side and a diagonal arrangement of red tee-shapes on the right side.  Execute it to find out what it does!\"\u003e\n\nRepiet is a compiler for the [Piet](http://www.dangermouse.net/esoteric/piet.html)\ngraphical language, written in Python.  The name \"repiet\" is meant to convey that\nit can recompile Piet programs into Piet.  Additionally, there are Python and C++\nbackends (more to come, and please contribute!).\n\n\u003cimg src=\"assets/program.png\" align=\"right\" width=\"20%\" title=\"An example program which will be used throughout this document to illustrate various compiler passes.\" alt=\"A small, colorful, square Piet program which is used to illustrate various compiler passes.\"\u003e\n\nTo compile a Piet program, we lex it, parse it, and optionally perform some\noptimizations.  The result of parsing and subsequent passes are parse graphs; our\nintermediate representation.  The compiler (`repiet.compiler.compiler`)\ncollaborates with backends to grok this representation, and emit a program in a\ntarget language.\n\nWe describe `repiet` as a \"brutalizing recompiler\" because it will take your\nbeautiful, hand-crafted Piet program and reconstruct it in a\n[brutalist](https://en.wikipedia.org/wiki/Brutalist_architecture) style.  Who\nknows why you'd want that.  The images referenced in this README are a somewhat\n[randomly-generated Piet program](https://codegolf.stackexchange.com/a/3226) --\ntweaked a little bit so it does *something* and also terminates.  Spoiler: it\ntakes a number from the user, emits the corresponding ascii character, and \nterminates, but spends most of its time NOPping.\n\n## First Pass: Lexing\n\n\u003cimg align=\"left\" src=\"assets/lexed.png\" width=\"20%\" title=\"A visualization of a lexed Piet program\"\nalt=\"A visualization of a lexed Piet program, where pixel regions have been merged into 'lexemes', which contain numbers indicating the number of codels they contain; their corners decorated with arrows indicating dp/cc  directions, and sliding regions are decorated with arrows indicating how sliding through those regions proceeds.\"\u003e\n\nThe lexer (`repiet.lexer.Lexer`) computes lexemes, or cardinally-connected sets\nof same-colored pixels, and identifies whitespace and blocking features \n(by default, these are all pixels which are neither white nor coding colors; this\nmay be controlled through `opinions` described in `repiet.util`).  As the lexemes\nare computed, we locate eight corners of each; corresponding to the  eight (`dp`,\n`cc`) states.  As whitespace is located, we identify the 4 pixels reached by\nsliding in each of four `dp` directions.\n\n\nThe role of the lexer is to eliminate pixel-based computations from subsequent\npasses.\n\n\n## Second Pass: Parsing\n\n\u003cimg align=\"left\" src=\"assets/parsed.png\" width=\"20%\" title=\"A visualization of a parsed Piet program.\" \nalt = \"A visualiation of a parsed Piet program, where lexemes have been split into one or more parse nodes, and unreachable nodes have been discarded.  Parse nodes retain the shapes of their original lexemes, contain text indicating the operation emitted by that node, and have arrows to their destinations -- branching operations have multiple outgoing arrows.  The initial node is indicated with a large triangle, and a termination point is indicated with a heavy box containing an X.\"\u003e\n\nThe parser begins in the upper-left corner of the image, and computes a parse\ngraph whose nodes correspond to (lexeme, `dp`, `cc`) states.  For each node, we\ncompute the operation (if any) resulting from the (`dp`, `cc`)-ward transition\nfrom the lexeme, as well if the next visited lexeme (if any).  If the operation\n is `switch` or `pointer`, there are two or four possible next-visited lexemes\nrespectively.\n\n\u003cimg align=\"right\" src=\"assets/parsed_c.png\" width=\"20%\" title=\"The recompiled version of the example program with optimization level zero (parsing only).\"\nalt=\"The recompiled version of the example program with optimization level zero (parsing only).\"\u003e\n\nThe role of the parser is to eliminate the `dp` and `cc` from subsequent passes.\nThus, interpreters and compilers of the intermediate representations need only\ngrok parse graph, handle input, output, and other stack-based operations.\n\n### Intermediate Representation\n\nA parse graph is a [rooted digraph](https://en.wikipedia.org/wiki/Rooted_digraph)\ncontaining Nodes (`repiet.util.Node`) which consist of a name, zero or more\noperations, and a list of outgoing neighbor's names. To compile this\nrepresentation into another language, one need only implement the\n[trampoline](https://en.wikipedia.org/wiki/Trampoline_(computing)) pattern.\n\nA node may have zero, one, two, or four children.  These respectively correspond\nto halting, jumping, and the `switch` or `pointer` operations.  Hence, a node's\noperation list is constrained such that only the final operation may be `switch`\nor `pointer`.  To implement these two operations, a compiled program pops a value\nfrom the stack, takes that value modulo 2 or 4, respectively, and uses the result\nas the index to the list of children.  If the stack is empty, the value is taken\nto be zero.\n\nThe backends (`repiet.backends`) are quite rudimentary, and only grok this very\nsimple IR.  Thus, the only optimizations available to us are those which perform\nsurgery on parse graphs.\n\n## Optimizing Passes\n\n\u003cimg src=\"assets/traced.png\" align=\"left\" width=\"20%\" title=\"A visualization of a traced program.\"\nalt=\"A visualiation of a traced Piet program, where parse nodes have been merged into trace nodes.  Parse nodes retain the shapes of their original lexemes, and have arrows to their destinations -- branching operations have multiple outgoing arrows.  The initial node is indicated with a large triangle, and a termination point is indicated with a heavy box containing an X.\"\u003e\n\nWe implement a Tracer (`repiet.tracer.Tracer`) which collects a parse graph into\nsequences of non-branching operations.  The result is a new parse graph, typically\nwith fewer nodes. Depending on the backend chosen, this may be a slight \noptimization.\n\n\u003cimg src=\"assets/traced_c.png\" align=\"right\" width=\"10%\" title=\"The recompiled version of the example program with optimization level one (parsing and tracing).\"\nalt=\"The recompiled version of the example program with optimization level one (parsing and tracing).\"\u003e\n\nAdditionally, we implement a rudimentary Static Evaluator \n(`repiet.optimizer.StaticEvaluator`), which maintains compile-time stack while\ntracing through instructions.  Presently, the static evaluator stops whenever the\nprogram (a) takes input from the user, (b) tries to pop from an empty stack, or \n(c) attempts to roll beyond the depth of the stack. Further optimizations may be\npossible: the values coming off of an empty stack could be treated as symbols,\nfor example.  Currently the backends are incapable of representing symbolic\nvariables, so such operations would require quite a bit of work.\n\nThe static evaluator is a work in progress, and it looks reasonable to interpose\na stack-depth analyzer between the tracer and static evaluator: the language spec\nrecommends skipping instructions that pop from an empty stack, which is especially\nvisible in this page's working example.  The existing static evaluator does not\ndrop those instructions, but they're apparently rare in hand-crafted or assembled\nPiet programs.\n\n# Installing and Using `repiet`\n\nRepiet is a `python` package, with a standard `setup.py`.  To get the very latest,\nfetch the git repo and install from there.\n\n    git clone https://github.com/boothby/repiet.git\n    cd repiet\n    python setup.py install\n \nOtherwise, just run `pip install repiet` and you're off to the races.  The primary\ninterface to `repiet` is the module's `__main__`, but you can also `import` it\n`repiet` from python and poke around the module structure.  The `__main__` can be\nused either as an executable python module,\n\n    python -m repiet ...\n\nor depending on how it's been installed, directly from the command line\n\n    repiet ...\n\n## But is it Faster Than C?\n\n\u003cimg src=\"assets/wc.png\" align=\"right\" width=\"20%\" title=\"A wc utility, which is waaaaay faster than C.\" title=\"A wc utility, which is waaaaay faster than C.\"\u003e\n\nYes!  Well, let's back up a minute for the folks missing context.  Recently there\nhave been a spate of blog posts of folks claiming that their pet language is\n\"faster than C\" by implementing a feature-incomplete version of the \n[wc](http://man7.org/linux/man-pages/man1/wc.1.html) utility and running a single\nbenchmark to demonstrate superiority.\n\nAfter cloning the `repiet` repo, point your command line at that directory.  We've\nplaced a `wc` utility into the `assets` directory, and we're going to time it\nversus the `wc` supplied by my operating system (you don't need to know any\ndetails about my system, of course).\n\n    $ cd assets\n    $ repiet wc.png -o wc.c --backend c --codel_size 10 -O 2\n    $ gcc wc.c -o wc -O3\n  \nOkay, now our wc utility is built; let's grab some data to test.\n\n    $ wget https://github.com/dwyl/english-words/raw/master/words_dictionary.json\n\nNow, we're off to the races!  Don't worry, I didn't cherrypick these benchmarks\nat all!\n\n    $ time cat words_dictionary.json | wc -w\n    740204\n\n    real\t0m0.145s\n    user\t0m0.136s\n    sys\t0m0.036s\n\nWow that's fast.  Now let's our hand-optimized Piet hotrod!\n\n    $ time cat words_dictionary.json | ./wc\n    740204\n    real\t0m0.131s\n    user\t0m0.140s\n    sys\t0m0.028s\n\nHoly wowzers, that's extremely peppy!  Good thing I only ran those tests once,\nand didn't show a distribution of runtimes!\n\nSo there you go, Piet is faster than C.  Tell your grandma.\n\n# Notes on Style and Quality\n\nThis project was written purely for the amusement of the author.  Hence, the code\nin this project is idiomatic and [golfed](https://en.wikipedia.org/wiki/Code_golf)\nto a minor extent.  Where this could cause difficulty for the reader, extensive\ncomments have been written to explain what's going on.  Other folks have written\ntools for Piet which use short-codes for various operations.  The 3-letter opcodes\nused by `repiet` are probably not used elsewhere.\n\nAlso it might be worth pointing out that the author has never taken a course on\ncompilers, read a book about compilers, or generally done much more than read\nbasic definitions about compilers.  If somebody's got some well-reasoned opinions\nabout how this project sucks, please be nice, but do share.  Several parts are\nsignificantly works in progress, and there are \"tests\" but they are not integrated\nand do not provide a reasonable degree of coverage.\n\n# Credits and License\n\nThe current version of this code is almost entirely the work of Kelly Boothby, and\ndistributed under the MIT license. If you look far enough back in the git history,\nyou'll find an interpreter written by Ross Tucker which is distributed on DMM's\nwebsite without any mention of licesnse.  That interpreter was hijacked, modified\ninto a rudimentary compiler, and deleted once obsolete.  Two lines of Ross's code\nsurvive untouched, where the virtual machine performs a `roll` operation.  It is\nthe author's opinion that these lines fall under fair use.\n\nRedistribution of Ross's code (which appears in early commits) is a bit of a grey\narea, and we only claim copyright for the diffs to that code and for the other \nfiles in the repo.  If anybody knows Ross, and Ross takes issue with that code\nliving in the history, please submit an issue and we can revise history to make\ngood.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fboothby%2Frepiet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fboothby%2Frepiet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fboothby%2Frepiet/lists"}