{"id":19297175,"url":"https://github.com/larmel/lacc","last_synced_at":"2025-04-22T08:31:11.861Z","repository":{"id":18324303,"uuid":"21503354","full_name":"larmel/lacc","owner":"larmel","description":"A simple, self-hosting C compiler","archived":false,"fork":false,"pushed_at":"2022-05-21T19:35:42.000Z","size":2792,"stargazers_count":893,"open_issues_count":7,"forks_count":64,"subscribers_count":39,"default_branch":"master","last_synced_at":"2024-11-09T23:02:22.398Z","etag":null,"topics":["c","compiler"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/larmel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-07-04T17:01:29.000Z","updated_at":"2024-11-08T16:16:49.000Z","dependencies_parsed_at":"2022-07-11T10:52:46.728Z","dependency_job_id":null,"html_url":"https://github.com/larmel/lacc","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/larmel%2Flacc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/larmel%2Flacc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/larmel%2Flacc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/larmel%2Flacc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/larmel","download_url":"https://codeload.github.com/larmel/lacc/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250206137,"owners_count":21392193,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["c","compiler"],"created_at":"2024-11-09T23:01:46.374Z","updated_at":"2025-04-22T08:31:10.400Z","avatar_url":"https://github.com/larmel.png","language":"C","readme":"lacc: A simple, self-hosting C compiler\n=======================================\nThis is a toy project of mine, with the goal of making a compiler for C, written in C, which is able to compile itself.\n\nFeatures\n--------\n * Complete support for C89, in addition to some features from later standards.\n * Target x86_64 assembly GNU syntax (-S), binary ELF object files (-c), or pure  preprocessing (-E).\n * Rich intermediate representation, building a control flow graph (CFG) with basic blocks of three-address code for each function definition. This is the target for basic dataflow analysis and optimization.\n\nInstall\n-------\nClone and build from source, and the binary will be placed in `bin/lacc`.\n\n    git clone https://github.com/larmel/lacc.git\n    cd lacc\n    ./configure\n    make\n    make install\n\nDefault configuration includes some basic probing of the environment to detect which machine and libc we are compiling for.\nRecognized platforms currently include Linux with glibc or musl, and OpenBSD.\n\nCertain standard library headers, such as `stddef.h` and `stdarg.h`, contain definitions that are inherently compiler specific, and are provided specifically for lacc under [lib/](lib/).\nIf you want to use `bin/lacc` directly without installing the headers, you can override the location by setting `--libdir` to point to this folder directly.\n\nThe `install` target will copy the binary and headers to the usual locations, or as configured with `--prefix` or `--bindir`.\nThere is also an option to set `DESTDIR`.\nExecute `make uninstall` to remove all the files that were copied.\n\nSee `./configure --help` for options to customize build and install parameters.\n\nUsage\n-----\nCommand line interface is kept similar to GCC and other compilers, using mostly a subset of the same flags and options.\n\n    -E         Output preprocessed.\n    -S         Output GNU style textual x86_64 assembly.\n    -c         Output x86_64 ELF object file.\n    -dot       Output intermediate representation in dot format.\n    -o         Specify output file name. If not speficied, default to input file\n               name with suffix changed to '.s', '.o' or '.dot' when compiling\n               with -S, -c or -dot, respectively. Otherwise use stdout.\n    -std=      Specify C standard, valid options are c89, c99, and c11.\n    -I         Add directory to search for included files.\n    -w         Disable warnings.\n    -g         Generate debug information (DWARF).\n    -O{0..3}   Set optimization level.\n    -D X[=]    Define macro, optionally with a value. For example -DNDEBUG, or\n               -D 'FOO(a)=a*2+1'.\n    -f[no-]PIC Generate position-independent code.\n    -v         Print verbose diagnostic information. This will dump a lot of\n               internal state during compilation, and can be useful for debugging.\n    --help     Print help text.\n\nArguments that do not match any option are taken to be input files.\nIf no compilation mode is specified, lacc will act as a wrapper for the system linker `/bin/ld`.\nSome common linker flags are supported.\n\n    -Wl,       Specify linker options, separated by commas.\n    -L         Add linker include directory.\n    -l         Add linker library.\n    -shared    Passed to linker as is.\n    -rdynamic  Pass -export-dynamic to linker.\n\nAs an example invocation, here is compiling [test/c89/fact.c](test/c89/fact.c) to object code, and then using the system linker to produce the final executable.\n\n    bin/lacc -c test/c89/fact.c -o fact.o\n    bin/lacc fact.o -o fact\n\nThe program is part of the test suite, calculating 5! using recursion, and exiting with the answer.\nRunning `./fact` followed by `echo $?` should print `120`.\n\nImplementation\n--------------\nThe compiler is written in C89, counting around 19k lines of code in total.\nThere are no external dependencies except for the C standard libary, and some system calls required to invoke the linker.\n\nThe implementation is organized into four main parts; preprocessor, parser, optimizer, and backend, each in their own directory under [src/](src/).\nIn general, each module (a `.c` file typically paired with an `.h` file defining the public interface) depend mostly on headers in their own subtree.\nDeclarations that are shared on a global level reside in [include/lacc/](include/lacc).\nThis is where you will find the core data structures, and interfaces between preprocessing, parsing, and code generation.\n\n### Preprocessor\nPreprocessing includes reading files, tokenization, macro expansion, and directive handling.\nThe interface to the preprocessor is `peek(0)`, `peekn(1)`, `consume(1)`, `try_consume(1)`, and `next(0)`, which looks at a stream of preprocessed `struct token` objects.\nThese are defined in [include/lacc/token.h](include/lacc/token.h).\n\nInput processing is done completely lazily, driven by the parser calling these functions to consume more input.\nA buffer of preprocessed tokens is kept for lookahead, and filled on demand when peeking ahead.\n\n### Intermediate Representation\nCode is modeled as control flow graph of basic blocks, each holding a sequence of three-address code statements.\nEach external variable or function definition is represented by a `struct definition` object, defining a single `struct symbol` and a CFG holding the code.\nThe data structures backing the intermediate representation can be found in [include/lacc/ir.h](include/lacc/ir.h).\n\nVisualizing the intermediate representation is a separate output target.\nIf the -dot option is specified, a dot formatted text file is produced as output.\n\n    bin/lacc -dot -I include src/backend/compile.c -o compile.dot\n    dot -Tps compile.dot -o compile.ps\n\nBelow is an example from a function found in [src/backend/compile.c](src/backend/compile.c), showing a slice of the complete graph.\nThe full output can be generated as a PostScript file by running the commands shown.\n\n![Example intermediate representation](doc/cfg.png)\n\nEach basic block in the graph has a list of statements, most commonly `IR_ASSIGN`, which assigns an expression (`struct expression`) to a variable (`struct var`).\nExpressions also contain variable operands, which can encode memory locations, addresses and dereferenced pointers at a high level.\n\n * `DIRECT` operands refer to memory at `*(\u0026symbol + offset)`, where symbol is a variable or temporary at a specific location in memory (for example stack).\n * `ADDRESS` operands represent exactly the address of a `DIRECT` operand, namely `(\u0026symbol + offset)`.\n * `DEREF` operands refer to memory pointed to by a symbol (which must be of pointer type).\n The expression is `*(symbol + offset)`, which requires two load operations to map to assembly. Only `DEREF` and `DIRECT` variables can be target for assignment, or l-value.\n * `IMMEDIATE` operands hold a constant number, or string. Evaluation of immediate operands do constant folding automatically.\n\n### Parser\nThe parser is hand coded recursive descent, with main parts split into [src/parser/declaration.c](src/parser/declaration.c), [src/parser/initializer.c](src/parser/initializer.c), [src/parser/expression.c](src/parser/expression.c), and [src/parser/statement.c](src/parser/statement.c).\nThe current function control flow graph, and the current active basic block in that graph, are passed as arguments to each production.\nThe graph is gradually constructed as new three-address code instructions are added to the current block.\n\nThe following example shows the parsing rule for bitwise or expressions, which\nadds a new `IR_OP_OR` operation to the current block.\nLogic in `eval_expr` will ensure that the operands `value` and `block-\u003eexpr` are valid, terminating in case of an error.\n\n    static struct block *inclusive_or_expression(\n        struct definition *def,\n        struct block *block)\n    {\n        struct var value;\n\n        block = exclusive_or_expression(def, block);\n        while (try_consume('|')) {\n            value = eval(def, block, block-\u003eexpr);\n            block = exclusive_or_expression(def, block);\n            block-\u003eexpr = eval_or(def, block, value, eval(def, block, block-\u003eexpr));\n        }\n\n        return block;\n    }\n\nThe latest evaluation result is always stored in `block-\u003eexpr`.\nBranching is done by instantiating new basic blocks  and maintaining pointers.\nEach basic block has a true and false branch pointer to other blocks, which is how branches and gotos are modeled.\nNote that at no point is there any syntax tree structure being built.\nIt exists only implicitly in the recursion.\n\n### Optimizer\nThe main motivation for building a control flow graph is to be able to do dataflow analysis and optimization.\nThe current capabilities here are still limited, but it can easily be extended with additional and more advanced analysis and optimization passes.\n\nLiveness analysis is used to figure out, at every statement, which symbols may later be read.\nThe dataflow algorithm is implemented using bit masks for representing symbols, numbering them 1-63.\nAs a consequence, optimization only works on functions with less than 64 variables.\nThe algorithm also has to be very conservative, as there is no pointer alias analysis (yet).\n\nUsing the liveness information, a transformation pass doing dead store elimination can remove `IR_ASSIGN` nodes which provably do nothing, reducing the size of the generated code.\n\n### Backend\nThere are three backend targets: textual assembly code, ELF object files, and dot for the intermediate representation.\nEach `struct definition` object yielded from the parser is passed to the [src/backend/compile.c](src/backend/compile.c) module.\nHere we do a mapping from intermediate control flow graph representation down to a lower level IR, reducing the code to something that directly represents x86_64 instructions.\nThe definition for this can be found in [src/backend/x86_64/encoding.h](src/backend/x86_64/encoding.h).\n\nDepending on function pointers set up on program start, the instructions are sent to either the ELF backend, or text assembly.\nThe code to output text assembly is therefore very simple, more or less just a mapping between the low level IR instructions and their GNU syntax assembly code.\nSee [src/backend/x86_64/assemble.c](src/backend/x86_64/assemble.c).\n\nDot output is a separate pipeline that does not need low level IR to be generated.\nThe compile module will simply forward the CFG to [src/backend/graphviz/dot.c](src/backend/graphviz/dot.c).\n\nCorrectness\n-----------\nTesting is done by comparing the runtime output of programs compiled with lacc and the system compiler (cc).\nA collection of small standalone programs used for validation can be found under the [test/](test/) directory.\nTests are executed using [check.sh](test/check.sh), which will validate preprocessing, assembly, and ELF outputs.\n\n    $ test/check.sh bin/lacc test/c89/fact.c\n    [-E: Ok!] [-S: Ok!] [-c: Ok!] [-c -O1: Ok!] :: test/c89/fact.c\n\nA complete test of the compiler is done by going through all test cases on a self-hosted version of lacc.\n\n    make -C test\n\nThis will first use the already built `bin/lacc` to produce `bin/bootstrap/lacc`, which in turn is used to build `bin/selfhost/lacc`.\nBetween the bootstrap and selfhost stages, the intermediate object files are compared for equality.\nIf everything works correctly, these stages should produce identical binaries.\nThe compiler is ''good'' when all tests pass on the selfhost binary.\nThis should always be green, on every commit.\n\n### Csmith\nIt is hard to come up with a good test suite covering all possible cases.\nIn order to weed out bugs, we can use [csmith](https://embed.cs.utah.edu/csmith/) to generate random programs that are suitable for validation.\n\n    ./csmith.sh\n\nThe [csmith.sh](test/csmith.sh) script runs csmith to generate an infinite sequence of random programs until something fails the test harness.\nIt will typically run thousands of tests without failure.\n\n![Still successful after running 4314 tests](doc/csmith.png)\n\nThe programs generated by Csmith contain a set of global variables, and functions making mutations on these.\nAt the end, a checksum of the complete state of all variables is output.\nThis checksum can then be compared against different compilers to find discreptancies, or bugs.\nSee [doc/random.c](doc/random.c) for an example program generated by Csmith, which is also compiled correctly by lacc.\n\n### Creduce\nWhen a bug is found, we can use [creduce](https://embed.cs.utah.edu/creduce/) to make a minimal repro.\nThis then will end up as a new test case in the normal test suite.\n\nPerformance\n-----------\nSome effort has been put into making the compiler itself fast (although the generated code is still very much unoptimized).\nServing as both a performance benchmark and correctness test, we use the [sqlite](https://sqlite.org/download.html) database engine.\nThe source code is distributed as a single ~7 MB (7184634 bytes) large C file spanning more than 200 K lines (including comments and whitespace), which is perfect for stress testing the compiler. \n\nThe following experiments were run on a laptop with an i5-7300U CPU, compiling version 3.20.1 of sqlite3.\nMeasurements are made from compiling to object code (-c).\n\n### Compilation speed\n\nIt takes less than 200 ms to compile the file with lacc, but rather than time we look at a more accurate sampling of CPU cycles and instructions executed.\nHardware performance counter data is collected with `perf stat`, and memory allocations with `valgrind --trace-children=yes`.\nIn valgrind, we are only counting contributions from the compiler itself (`cc1` executable) while running GCC.\n\nNumbers for lacc is from an optimized build produced by `make CC='clang -O3 -DNDEBUG' bin/lacc`.\nEach compiler is invoked with arguments `-c sqlite/sqlite3.c -o foo.o`.\n\n| Compiler       |        Cycles |   Instructions | Allocations | Bytes allocated | Result size |\n|:---------------|--------------:|---------------:|------------:|----------------:|------------:|\n| lacc           |   412,334,334 |    619,466,003 |      49,020 |      24,244,107 |   1,590,482 |\n| tcc (0.9.27)   |   245,142,166 |    397,514,762 |       2,909 |      23,020,917 |   1,409,716 |\n| gcc (9.3.0)    | 9,958,514,599 | 14,524,274,665 |   1,546,790 |   1,111,331,606 |   1,098,408 |\n| clang (10.0.0) | 4,351,456,211 |  5,466,808,426 |   1,434,072 |     476,529,342 |   1,116,992 |\n\nThere is yet work to be done to get closer to [TCC](http://bellard.org/tcc/), which is probably one of the fastest C compilers available.\nStill, we are within reasonable distance from TCC performance, and an order of magnitude better than GCC and clang.\n\n### Codegen quality\n\nFrom the above table, we can see that the size of the sqlite object file generated by lacc is larger than those generated by other compilers, suggesting that we output less optimal code.\n\nTo compare the relative quality of code generated from lacc and GCC, we can look at the number of dynamic instructions executed by the selfhost binary versus a binary built by GCC.\nWe run the same test as above, compiling sqlite to object code.\nTargets for the test are the default compiler build (`bin/lacc`) produced by GCC, and the selfhost binary (`bin/selfhost/lacc`) produced by lacc.\nBoth of these targets are produced without any optimizations enabled, and without defining `NDEBUG`.\n\n| Compiler        | Cycles        | Instructions   |\n|:----------------|--------------:|---------------:|\n| lacc            |   946,315,653 |  1,481,608,726 |\n| lacc (selfhost) | 1,417,112,690 |  2,046,996,115 |\n\nThe selfhosted binary is slower to compile sqlite than the compiler built by GCC, showing that lacc indeed generates rather inefficient code.\nImproving the backend with better instruction selection is a priority, so these numbers should hopefully get closer in the future.\n\nReferences\n----------\nThese are some useful resources for building a C compiler targeting x86_64.\n\n * The C Programming Language, Second Edition. Brian W. Kernighan, Dennis M. Ritchie.\n * [System V Application Binary Interface](http://www.x86-64.org/documentation/abi.pdf).\n * [Intel® 64 and IA-32 Architectures Software Developer’s Manuals](http://www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html), specifically the instruction set reference.\n","funding_links":[],"categories":["C"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flarmel%2Flacc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flarmel%2Flacc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flarmel%2Flacc/lists"}