https://github.com/cslarsen/minijit

A basic x86-64 JIT compiler written from scratch in stock Python
https://github.com/cslarsen/minijit

assembly compiler jit jit-compiler machine-code python x86-64

Last synced: about 1 month ago
JSON representation

A basic x86-64 JIT compiler written from scratch in stock Python

• Host: GitHub
• URL: https://github.com/cslarsen/minijit
• Owner: cslarsen
• Created: 2017-10-15T23:17:54.000Z (over 7 years ago)
• Default Branch: master
• Last Pushed: 2018-04-21T18:02:58.000Z (almost 7 years ago)
• Last Synced: 2024-12-08T08:31:29.810Z (about 2 months ago)
• Topics: assembly, compiler, jit, jit-compiler, machine-code, python, x86-64
• Language: Python
• Homepage: https://csl.name/post/python-jit/
• Size: 31.3 KB
• Stars: 227
• Watchers: 10
• Forks: 13
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
MiniJIT

=======

Contains code for the posts

  * Writing a basic x86-64 JIT compiler from scratch in stock Python

    https://csl.name/post/python-jit/

  * JIT compiling a tiny subset of Python to x86-64 from scratch — in Python

    https://csl.name/post/python-compiler/

You need a UNIX/POSIX system and an x86-64 compatible CPU. I've tested this on

Linux and macOS, using Python 2.7 and 3+

The ~500 lines of code relies only on standard Python libraries and contains a

Python bytecode converter, peephole optimizer and x86-64 machine code

assembler. The code is meant to be simple to understand and pedagogical.

Finally, there is a decorator that automatically swaps out Python functions

with native code:

    >>> from jitcompiler import jit, disassemble

    >>> @jit

    ... def foo(a, b): return a + b

    ... 

    --- Installing JIT for 

    >>> foo(10, -2)

    --- JIT-compiling 

    8

    >>> print(disassemble(foo))

    0x10bb3c000 48 89 fb       mov rbx, rdi

    0x10bb3c003 48 89 f0       mov rax, rsi

    0x10bb3c006 48 01 d8       add rax, rbx

    0x10bb3c009 c3             ret 

How to run tests

----------------

The first one patches up some machine code and runs it at runtime

    $ python mj.py

The second one JIT compiles Python bytecode to machine code at runtime

    $ python tests.py

If you have the `capstone` module installed, it will display an in-memory

disassembly as well.

You can also run the decorator test. It defines a function like this

    import jitcompiler

    #...

    @jitcompiler.jit

    def foo(a, b):

        return a*a - b*b

On the first *call* to `foo`, it will be compiled to native code and swap out

the original Python function. It treats all arguments as signed 64-bit

integers. If you have the Capstone module installed, it will also print a

disassembly. To run:

    $ python test-decorator.py

    Definition point of foo

    Installing JIT for 

    Calling foo

    JIT-compiling 

    Installed native code for 

    Calling function 

    foo(1, 2) => -3

    Calling function 

    foo(2, 3) => -5

    Disassembly of foo

    0x7f86765f9000 48 89 fb       mov rbx, rdi

    0x7f86765f9003 48 89 f8       mov rax, rdi

    0x7f86765f9006 48 0f af c3    imul rax, rbx

    0x7f86765f900a 50             push rax

    0x7f86765f900b 48 89 f3       mov rbx, rsi

    0x7f86765f900e 48 89 f0       mov rax, rsi

    0x7f86765f9011 48 0f af c3    imul rax, rbx

    0x7f86765f9015 48 89 c3       mov rbx, rax

    0x7f86765f9018 58             pop rax

    0x7f86765f9019 48 29 d8       sub rax, rbx

    0x7f86765f901c c3             ret

If you want to get serious about this

-------------------------------------

  * Check out a full-blown assembler library for Python:

    https://github.com/Maratyszcza/PeachPy

References

----------

  * Intel assembly manuals:

    https://software.intel.com/en-us/articles/intel-sdm

  * x86 Reference: http://ref.x86asm.net/

  * Capstone disassembler:

    http://www.capstone-engine.org/lang_python.html

License

-------

Put in the public domain in 2017 by the author Christian Stigen Larsen