{"id":13438537,"url":"https://github.com/jnz/q3vm","last_synced_at":"2025-03-20T06:30:38.688Z","repository":{"id":47765033,"uuid":"145262628","full_name":"jnz/q3vm","owner":"jnz","description":"Q3VM - Single file (vm.c) bytecode virtual machine/interpreter for C-language input","archived":false,"fork":false,"pushed_at":"2024-07-24T00:13:34.000Z","size":1925,"stargazers_count":837,"open_issues_count":10,"forks_count":59,"subscribers_count":25,"default_branch":"master","last_synced_at":"2024-10-28T00:23:09.741Z","etag":null,"topics":["bytecode","embedded","interpreter","quake3","vm"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jnz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"COPYING.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-08-19T00:51:02.000Z","updated_at":"2024-10-25T07:49:06.000Z","dependencies_parsed_at":"2024-01-16T12:46:40.632Z","dependency_job_id":"39e84f84-ad92-4500-a53d-5204dadddd5e","html_url":"https://github.com/jnz/q3vm","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jnz%2Fq3vm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jnz%2Fq3vm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jnz%2Fq3vm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jnz%2Fq3vm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jnz","download_url":"https://codeload.github.com/jnz/q3vm/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244564898,"owners_count":20473155,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bytecode","embedded","interpreter","quake3","vm"],"created_at":"2024-07-31T03:01:06.297Z","updated_at":"2025-03-20T06:30:33.676Z","avatar_url":"https://github.com/jnz.png","language":"C","readme":"Q3VM Readme\n===========\n\nA lightweight (single file: `vm.c`) embeddable interpreter/Virtual Machine (VM) for compiled bytecode files (`.qvm`) based on good old C-language input (`.c`). A complete C compiler to generate `.qvm` files is included (LCC). The interpreter is based on the Quake III Arena virtual machine (hence the name q3vm) but the interpreter is not tied to Quake III Arena and can be used for any kind of project. For example code that needs to run in a sandbox.\n\n\u003c!-- [![Build Status](https://travis-ci.org/jnz/q3vm.svg?branch=master)](https://travis-ci.org/jnz/q3vm) --\u003e\n[![codecov](https://codecov.io/gh/jnz/q3vm/branch/master/graph/badge.svg)](https://codecov.io/gh/jnz/q3vm)\n[![Coverity](https://scan.coverity.com/projects/16570/badge.svg)](https://scan.coverity.com/projects/jnz-q3vm)\n\n      ___   _______     ____  __\n     / _ \\ |___ /\\ \\   / /  \\/  |\n    | | | |  |_ \\ \\ \\ / /| |\\/| |\n    | |_| |____) | \\ V / | |  | |\n     \\__\\_______/   \\_/  |_|  |_|\n     \n     vm.c / vm.h\n\nJan Zwiener, 2018-2024. Mail: jan@zwiener.org\n\nRead the excellent introduction to the Q3VM by Fabien Sanglard:\n\n * http://fabiensanglard.net/quake3/qvm.php\n\nGif: compiling a simple *hello world* example (`main.c`) and run it with the virtual machine interpreter `q3vm`.\n\n![gif](demo_vm.gif?raw=1)\n\nInstallation\n------------\n\nThe [vm.c](src/vm/vm.c?raw=1) and [vm.h](src/vm/vm.h?raw=1) files can be\ndropped into an existing C project and compiled along with it. Implement\nthe 4 callback functions in your project: `Com_malloc`,  `Com_free`, `Com_Error`\nand `systemCalls`.\n\nFeatures\n--------\n\n * Small and lightweight (one .c file to include without dependencies)\n * Battle-tested (20 years of use in Quake III Arena)\n * VM and LCC forked from the well maintained ioquake3 code base\n * Tool tested (static code analysis, test coverage, Valgrind)\n * No need to learn a new scripting language (e.g. Lua)\n * Static type checking in the language (C)\n * Static memory allocation in C, no unpredictable garbage collector\n * Plan B: you can always go back to native code, as .c files are the input\n * Great tool landscape for C. Use the tools that are available for C\n * Computed gotos are used to speed up the interpreter if you compile with GCC (see benchmark section) \n * Much faster than the Triseism Q3VM interpreter (see benchmark section)\n\nUse Cases\n---------\n\n * Sandbox for code you don't fully trust (e.g. download the bytecode from a web server)\n * Mods for hobby game engines\n * There are many virtual machines, but not many are so small, with static typing and no garbage collector\n * Learn about virtual machines in general, but directly have a C compiler available for the virtual machine\n * Sandbox for embedded applications, e.g. plug-ins for IoT applications on microcontrollers (bounded CPU time, bounded memory area, restrict access to peripheral devices)\n * There is also a historical value: learn about the Quake III engine\n\n\nQuick Intro\n-----------\n\nTwo things are required:\n\n * The interpreter\n * A bytecode binary .qvm file\n\nRun:\n\n    \u003e q3vm.exe bytecode.qvm\n\nThe q3vm.exe standalone interpreter is not required, it is more of a demo\napplication.  You can easily add the interpreter as a single .c file to your\nproject (`vm.c` and the header `vm.h`).  Call `VM_Create` and `VM_Call` to run\nthe bytecode in your application:\n\n```c\n    #include \"vm.h\"\n\n    vm_t vm;\n    int result;\n    \n    VM_Create(\u0026vm, \"my test\", pointerToByteCodeBuffer, sysCall);\n    result = VM_Call(\u0026vm, 12345);\n    VM_Free(\u0026vm);\n```\n\nThe `pointerToByteCodeBuffer` is some memory location where the bytecode is\nlocated. You can e.g. load it from a file and store it in a byte array. See\n`main.c` for an example implementation.\n\nData can be exchanged with the bytecode by the return value (result) and\narguments to `VM_Call`. Here just a 12345 is passed to the bytecode. It is up\nto the `vmMain` function in the bytecode what to do with that value.  You can pass\nmore (up to 12) optional arguments to the bytecode:\ne.g. `VM_Call(\u0026vm, 0, 1, 2, 3, 4)`.\n\nThe `sysCall` is a callback function that you define so that the interpreter\ncan call native functions from your code. E.g. a logging function or some time\ncritical function that you don't want to implement in the bytecode. Again,\ncheck `main.c` for an example. Also check the section *How to add a custom\nnative function* for more information.\n\nA few callback functions are required, read the section *Callback functions\nrequired in host application* for more information.\n\nAnd normally you should also check if `VM_Create` returns 0 (i.e. everything is\nOK).\n\nFolder structure\n----------------\n\n    ├─ bin/             LCC compiler and q3asm linker output binaries\n    │  ├─ linux/        Linux target folder for LCC compiler and q3asm linker\n    │  └─ win32/        Precompiled lcc.exe and q3asm.exe for Windows\n    ├─ build/           Temp. directory for object files\n    ├─ doxygen/         Doxygen config and API documentation output\n    ├─ example/         Example \"hello world\" firmware project (bytecode.qvm)\n    ├─ lcc/             The LCC compiler (compile .c files to .asm files)\n    ├─ msvc/            Microsoft Visual Studio 2015 project file for q3vm\n    ├─ q3asm/           Linker: link the LCC .asm files to a .qvm bytecode file\n    ├─ src/             q3vm standalone console application source code\n    │  └─ vm/           The core VM source, copy that folder into your project\n    └─ test/            Test environment\n\nAPI Documentation\n-----------------\n\nCall `make doxygen` to autogenerate the API documentation in the `doxygen/html`\ndirectory. Doxygen is required as well as the dot command (part of graphviz).\nInstall it with `sudo apt-get install doxygen graphviz` on Debian or Ubuntu.\n\n    \u003e make doxygen\n\nBut you can also read `vm.h` directly for the API documentation.\n\n\nBuild VM/interpreter\n--------------------\n\nOn **Linux**:\n\n    \u003e make\n\nOn **Windows**:\n\nUse the **Visual Studio 2015** project `q3vm.sln` in the `msvc` subfolder.\n\nOr install MinGW64 and add the MinGW64 bin\\ directory to your path.\nSo that you have gcc.exe and mingw32-make.exe available at the command prompt.\n\n * [https://www.mingw-w64.org/downloads/](https://www.mingw-w64.org/downloads/)\n\nCompile with:\n\n    \u003e mingw32-make\n\nBuild example bytecode firmware\n-------------------------------\n\n**Windows**:\n\nThe LCC compiler (lcc.exe) is included in the ./bin/win32 directory.\nYou need make (mingw32-make) from the MinGW64 installation in\nyour path. The Makefile calls LCC and q3asm to generate `bytecode.qvm`:\n\n    cd example\n    mingw32-make\n\nIf you don't want to use make, you can do the steps from the make file\nmanually at the command line. Compile every `.c` source code with `LCC`:\n\n    \u003e lcc -S -Wf-target=bytecode -Wf-g YOUR_C_CODE.c\n\nThis will create .asm output files. Then link the .asm files with `q3asm` (based on a bytecode.q3asm\nlinker script):\n\n    \u003e q3asm -f bytecode\n\nThe output of q3asm is a `.qvm` file that you can run with q3vm.\n\n\n**Linux**:\n\nBuild LCC:\n\n    \u003e make lcc\n\nBuild q3asm\n\n    \u003e make q3asm\n\nBuild the example bytecode:\n\n    \u003e make example/bytecode.qvm\n\nCallback functions required in host application\n-----------------------------------------------\n\n**malloc and free**:\n\nThe following functions are required in the host application for\nmemory allocation:\n\n```c\n    void* Com_malloc(size_t size, vm_t* vm, vmMallocType_t type);\n    {\n        (void)vm;\n        (void)type;\n        return malloc(size);\n    }\n    \n    void Com_free(void* p, vm_t* vm, vmMallocType_t type)\n    {\n        (void)vm;\n        (void)type;\n        free(p);\n    }\n```\n\nThe host can simply call `malloc` and `free` or use a custom memory allocation\nfunction or use static memory (e.g. in an embedded application). Each VM only\ncalls `Com_malloc` once per malloc type. This can be used as a help for the static memory\nallocation in an embedded environment without `malloc()` and `free()`.\n\n**Error handling**:\n\nThe following function needs to be implemented in the host application:\n\n```c\n    void Com_Error(vmErrorCode_t level, const char* error)\n    {\n        fprintf(stderr, \"Err (%i): %s\\n\", level, error);\n        exit(level);\n    }\n```\n\nThe error id is given by the `vmErrorCode_t` parameter. The `error` string describes\nwhat went wrong.  It is up to the host application how to deal with the error.\nIn this simple example we just print the error string and exit the application.\nThe error code is stored in the `vm_t::lastError` variable.\n\nHow to add a custom native function\n-----------------------------------\n\nLet's say we want to add a native function to convert a string to an integer:\n`stringToInt`.  We want to add the function to our virtual machine (step 1) and\ncall it from our example code (step 2).  (Note: there is already the `atoi` function in\nthe bytecode, but this is just an example on how to call `atoi` as a native\nfunction and deal with address translation).\n\n\n**Step 1)** Add the native function to the host application\n\nOpen `src/main.c` and modify the `systemCalls` function. Add `case -5:` for the\nnew native function. We just use the next free id (here -5) as an identifier.\nThe identifier will be important in step 2.  The first argument\nfor `stringToInt` is the address of a string. The address is in the virtual\nmachine address space, so we can't directly use that argument (`args[1]`) for\nthe native call to `atoi`. There is a helper macro that will translate the\naddress for use: `VMA`. We need to give `VMA` the pointer argument from the\nbytecode and the virtual machine context (`vm`) to translate it.\nThe function `VM_MemoryRangeValid` makes sure that the memory range is valid. This is e.g.\nimportant for the `memcpy` call, so that the VM cannot write outside of\nthe sandbox memory.\nIt is also possible to call the VM recursively again with `VM_Call`.\n\n```c\n    /* Call native functions from the bytecode: */\n    int systemCalls(vm_t* vm, int* args)\n    {\n        const int id = -1 - args[0];\n    \n        switch (id)\n        {\n        case -1: /* PRINTF */\n            return printf(\"%s\", (const char*)VMA(1, vm));\n\n        case -2: /* ERROR */\n            return fprintf(stderr, \"%s\", (const char*)VMA(1, vm));\n    \n        case -3: /* MEMSET */\n            if (VM_MemoryRangeValid(args[1]/*addr*/, args[3]/*len*/, vm) == 0)\n            {\n                memset(VMA(1, vm), args[2], args[3]);\n            }\n            return args[1];\n    \n        case -4: /* MEMCPY */\n            if (VM_MemoryRangeValid(args[1]/*addr*/, args[3]/*len*/, vm) == 0 \u0026\u0026\n                VM_MemoryRangeValid(args[2]/*addr*/, args[3]/*len*/, vm) == 0)\n            {\n                memcpy(VMA(1, vm), VMA(2, vm), args[3]);\n            }\n            return args[1];\n    \n        case -5: /* stringToInt */                             // \u003c NEW !!!\n            return atoi(VMA(1, vm));                           // \u003c NEW !!!\n    \n        default:\n            fprintf(stderr, \"Bad system call: %i\\n\", id);\n        }\n        return 0;\n    }\n```\n\n**Step 2)** Tell the bytecode about this function\n\nNow we need to tell our example project about this new function `strintToInt`.\nOpen `example/g_syscalls.asm` and add the last line. The identifier -5 is\nimportant for the mapping.\n\n    code\n    \n    equ trap_Printf             -1\n    equ trap_Error              -2\n    equ memset                  -3\n    equ memcpy                  -4\n    equ stringToInt             -5\n\n**Step 3)** Perform an example call to `strintToInt`\n\nEdit `example/main.c` and add the function declaration:\n\n```c\n    int stringToInt(const char* a);\n```\n\nAnd call it somewhere from the `vmMain` function:\n\n```c\n    char* myStr = \"1234\";\n    printf(\"\\\"%s\\\" -\u003e %i\\n\", myStr, stringToInt(myStr));\n```\n\nCompile everything:\n\n    \u003e make \u0026\u0026 make example/bytecode.qvm\n\nAnd run it:\n\n    \u003e ./q3vm example/bytecode.qvm\n\nOriginal comments by John Carmack\n---------------------------------\n\nJohn Carmack's .plan for Nov 03, 1998:\n\n*I had been working under the assumption that Java was the right way to go, but recently I reached a better conclusion.*\n*The programming language for QuakeArena mods is interpreted ANSI C. (well, I am dropping the double data type, but otherwise it should be pretty conformant)*\n*The game will have an interpreter for a virtual RISC-like CPU. This should have a minor speed benefit over a byte-coded, stack based java interpreter. Loads and stores are confined to a preset block of memory, and access to all external system facilities is done with system traps to the main game code, so it is completely secure.*\n*The tools necessary for building mods will all be freely available: a modified version of LCC and a new program called q3asm. LCC is a wonderful project - a cross platform, cross compiling ANSI C compiler done in under 20K lines of code. Anyone interested in compilers should pick up a copy of \"A retargetable C compiler: design and implementation\" by Fraser and Hanson.*\n*You can't link against any libraries, so every function must be resolved. Things like strcmp, memcpy, rand, etc. must all be implemented directly. I have code for all the ones I use, but some people may have to modify their coding styles or provide implementations for other functions.*\n*It is a fair amount of work to restructure all the interfaces to not share pointers between the system and the games, but it is a whole lot easier than porting everything to a new language. The client game code is about 10k lines, and the server game code is about 20k lines.*\n*The drawback is performance. It will probably perform somewhat like QC. Most of the heavy lifting is still done in the builtin functions for path tracing and world sampling, but you could still hurt yourself by looping over tons of objects every frame. Yes, this does mean more load on servers, but I am making some improvements in other parts that I hope will balance things to about the way Q2 was on previous generation hardware.*\n*There is also the amusing avenue of writing hand tuned virtual assembly assembly language for critical functions.*\n*I think this is The Right Thing.*\n\nStatic code analysis\n--------------------\n\nCall `make analysis` and `make valgrind` to check the VM with:\n\n * clang static code analysis (scan-build)\n * cppcheck\n * Valgrind\n\nclang-format\n------------\n\nRun the following command to reformat a file according to the coding style:\n\n    \u003e clang-format -i -style=file input.c\n\n\nDebugging\n---------\n\nBuild `vm.c` with `#define DEBUG_VM` in `vm.h` to enable more checks and debug\nfunctions. Call `VM_Debug()` to enable debug printfs.  This\nrequires the symbol file of the `.qvm`: the `.map` file in the same directory\nas the `.qvm`. The `.map` file is automatically generated for each `.qvm`.\n\nCall at the end of a session `VM_VmProfile_f(vm)` to see a VM usage summary.\n\nBenchmarks\n----------\n\nTime to run `test/test.qvm`.\n\nSmaller numbers are better (multiple runs, smallest number used).\n\n\n| Interpreter          | Time     |\n|----------------------|----------|\n| Default Interpreter  |  3.063 s |\n| w. computed gotos    |  1.771 s |\n| Native executable    |  0.307 s |\n\n\nEnvironment:\n\n * Ubuntu 17.10\n * GCC: 7.2.0-8ubuntu3.2\n * CPU: Intel(R) Core(TM) i5-3320M CPU @ 2.60GHz\n * Version Git hash: 8e46048f475a53f99f9e6656e030835b6011f2ca\n * Date: 2018.08.31\n\nCommand line:\n\n    time ./q3vm test/test.qvm\n    time ./test/test_native\n\nBenchmark vs. Triseism Q3 interpreter (seismiq executable).\n\nTestfirmware: `test/example_test.qvm`\n\nSmaller numbers are better (multiple runs, smallest number used).\n\n\n| Interpreter          | Time     |\n|----------------------|----------|\n| Q3VM                 |  1.222 s |\n| Triseism project     | 10.903 s |\n\n * Ubuntu 17.10\n * GCC: 7.2.0-8ubuntu3.2\n * CPU: Intel(R) Core(TM) i5-3320M CPU @ 2.60GHz\n * Version Git hash: bb4848c25b2b95c08b9aa03bb6ac46ef4948d900\n\nVersion History\n---------------\n\n * v1.3 - q3asm from ioquake3 added\n\n * v1.2 - Debug features enabled (compile with `-DDEBUG_VM`)\n\n * v1.1 - LCC from ioquake3 added\n\nTODO\n----\n\nKnown limitations, bugs, missing features:\n\n * The Quake III Arena JIT compiler (e.g. for x86) is not added.\n\nLICENSE\n-------\n\nSee COPYING.txt for details on the license. Basically the Quake III Arena\nGPL 2 source code license has been inherited.\n\nBe aware that LCC has its own non-commercial license which is described in\nlcc/COPYRIGHT.\n\nFurther information\n-------------------\n\n * http://fabiensanglard.net/quake3/qvm.php\n * http://users.suse.com/~lnussel/talks/fosdem_talk_2013_q3.pdf\n\nCredits\n=======\n\nThis project is based on the Quake 3 and ioquake3 source:\n\n * https://github.com/id-Software/Quake-III-Arena (id Software)\n * https://github.com/ioquake/ioq3\n * https://icculus.org/projects/triseism/triseism.html\n\nComputed gotos are used:\n\n * https://eli.thegreenplace.net/2012/07/12/computed-goto-for-efficient-dispatch-tables\n \n","funding_links":[],"categories":["C","Scripting","Virtualization"],"sub_categories":["Books"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjnz%2Fq3vm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjnz%2Fq3vm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjnz%2Fq3vm/lists"}