Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/mschuldt/maml

The Mega Awesome Microcontroller Language!
https://github.com/mschuldt/maml

Last synced: 1 day ago
JSON representation

The Mega Awesome Microcontroller Language!

Awesome Lists containing this project

README 

        
* Overview

MAML compiles select blocks of a Python program to bytecode and sends it to

an Arduino for execution. The virtual machine running on the Arduino may be

paused/resumed and its environment can be fully inspected and modified from

Python as the program is running. This enables writing programs for

Arduino that benefit from interactive development, friendly syntax, and

close interoperability with the desktop.



The Python decorators @block(once) and @block(chain) are used to designate

functions whose body will be compiled and sent to the Arduino for execution.

The rest of the Python program runs as normal and may exit after sending

the bytecode or keep running and provide an application that interacts

with the Arduino.

The bodies of these decorated functions contain a subset of the Python

language with modified semantics.



* Usage

to compile, send, and run a file on the desktop:

#+Begin_SRC text

./run.sh filename.maml

#+END_SRC

to start the VM:

#+Begin_SRC text

./avm

#+END_SRC

to compile and send code to a running VM on the desktop

#+Begin_SRC text

./maml.py -d filename.maml

#+END_SRC



to compile and send code to an Arduino

#+Begin_SRC text

./maml.py -a filename.maml

#+END_SRC

if you want to interact with it:

#+Begin_SRC text

python3 -i maml.py -a filename.maml

#+END_SRC

* Examples

For working examples see the [[file:examples/][examples]] directory.

* Arduino API

Instantiate the arduino object (required).

Set desktop=True if the VM is running on the Desktop, defaults to False

#+Begin_SRC python

arduino = Arduino(desktop=True)

#+END_SRC

Define a block that is executed once on the VM.

The body of all @block functions are executed in global scope on the Arduino,

Therefor variables defined in one block are available to all others.

#+Begin_SRC python

@block(once)

def blockname():

   ...

#+END_SRC

Define a block that is inserted into the VM mainloop and repeatedly executed

#+Begin_SRC python

@block(chain)

def blockname():

   ...

#+END_SRC

Define a function

#+Begin_SRC python

@function

def funcname(a,b,c):

   ...

#+END_SRC

Send a block or function to the VM

#+Begin_SRC python

arduino.send(blockname)

#+END_SRC

Set a variable in the VM environment

#+Begin_SRC python

arduino.set("variableName", value)

#+END_SRC

Get the value of a variable in the VM (currently only supports integers)

#+Begin_SRC python

arduino.get("variableName")

#+END_SRC

Pause or Resume The VM

#+Begin_SRC python

arduino.pause()

arduino.resume()

#+END_SRC

Dump all variable values in the VM as well as the values on the stack

#+Begin_SRC python

arduino.dump()

#+END_SRC



* Building the VM

build the virtual machine for use on the desktop

#+Begin_SRC text

make

#+END_SRC

generates the file 'avm.ino', the Arduino Virtual Machine,

that can be copied into the Arduino IDE

#+Begin_SRC text

make ino

#+END_SRC

* Dependencies

 - g++ (The VM does not compile with LLVM)

 - python3

 - pyserial  (pip3 install pyserial)

 - Arduino IDE 1.5.8 (http://arduino.cc/en/Main/Software)

 - Emacs



Maml is currently only known to run on the Arduino Mega 2560.

The Uno does not work for unknown reasons (never able to connect to the VM on it),

The Due does does not work because of changes to the Serial Library.



* Testing

to run all tests in maml/tests/ and maml/false_tests:

#+Begin_SRC text

  make test

#+END_SRC

** Using gdb to debug the VM

start the avm process with gdb

then inject code in separate terminal by running python file:

#+Begin_SRC text

 ./maml.py -d filename.maml

#+END_SRC

using ./run.sh will not work because it creates its own avm subprocess

** Testing bytecode compilation

to print the compiled code from a file use:

#+Begin_SRC text

 ./maml_compile filename.py

#+END_SRC

this will print something like:

#+Begin_SRC text

  block: 'test'

      [22, 2, 8, 0]

#+END_SRC

to print with human readable opcodes instead,

at the top of maml_opcodes.py set "debug = True"

Now maml_compile will print something like

#+Begin_SRC text

  block: 'test'

      ['SOP_INT', 2, 'OP_GLOBAL_STORE', 0]

#+END_SRC



* Desktop version limitations

- Arduino.get an Arduino.dump do not work in the desktop version

  (this is just because the output of the VM is not piped back to maml.py)

* Features

some features in the ~order they where implemented



- arithmetic

- communication to vm on desktop using files and signals

- defining/calling c primitive functions

- global variables

- if/elif/else conditions

- conditionals

- while loops

- strings

- (linked) lists

- array and list literals

- type declarations checking (currently buggy and disabled)

- getting/setting values from the VM

- dumping all variable and stack values from the VM

- pausing and resuming the VM

- defining/calling functions



* Defining new c primitives

primitives are separated into files depending on their compile target.

- [[file:primitives.c][primitives.c]]                for both Desktop and Arduino

- [[file:arduino_only_primitives.c][arduino_only_primitives.c]]   not compiled for desktop

- [[file:desktop_only_primitives.c][desktop_only_primitives.c]]   not compiled for desktop



primitive functions are defined in normal C/C++ but with the

_DEFUN_ tag above them.

declare primitives with the _DECL_ tag. See the files for examples.

* Internals

Internal structures and how to add compiler or VM features.

TODO: This is very incomplete.

** serial protocol

** bytecode and opcode format

Operands are mixed in with opcodes. All opcode operands come before the

opcodes in the bytecode when it is transmitted to the arduino but

comp after the operands in the threaded code that is generated within the VM.



TODO: opcode formats

** lsdjlskjd

if an opcode has integer operands it must insert SOP_INT before the number.

This allows the number to be serialized. In serial_in, SOP_INT must

be explicitly skipped with SKIP before calling READ_INT.



** adding a new opcode / VM case

using 'pop' as an example.



in maml_opcodes.py add the opcode definition:



#+Begin_SRC python

 OP_POP = OP("OP_POP")

#+END_SRC



in avm.c add the case in loop():



#+Begin_SRC c

  pop:

   --top;

   NEXT(code);

#+END_SRC



in avm.c at the top of loop() defined a label variable:



#+Begin_SRC c

 void* l_pop;

#+END_SRC



and below that, in loop(), add the label address assignment:



#+Begin_SRC c

 l_pop = &pop;

#+END_SRC



at the bottom of avm.c in serial_in(), add a case to the switch statement

that reads in the bytecode and adds the address to the code array,

if this opcode has operands, they are read in now, see case SOP_INT or

SOP_PRIM_CALL for and example of that.



#+Begin_SRC c

    case OP_POP:

      NL;

      code_array[i++] = l_pop;

      break;

#+END_SRC



** adding new feature

using 'if' as an example.



general steps (lots of them may not be used)

- ast translation

- ast checking function

- code generation function

- new opcode

- serialization

- de-serialization, convert to threaded form

- new vm case

*** AST translation

first attempt to get the ast of the example:

#+Begin_SRC text

  ./maml_ast.py filename.py

#+END_SRC

This will likely result in an error such as:

#+Begin_SRC text

  ...

    return eval(ast.dump(ast.parse(code),include_attributes=True))

  File "", line 1, in 

NameError: name 'If' is not defined

#+END_SRC

(If there is no error, the ast will be dumped. skip this section)

This means we need to define the AST node translation function for 'If'

in maml_ast.py. Before we do that we need to know what parameters the

translation function will take. Get a dump of the raw Python ast using:



#+Begin_SRC text

./py_ast.py filename.py

#+END_SRC

(include in filename.py only the new feature, py_ast.py will not extract

code from maml blocks)



The output is:

#+Begin_SRC python

Module(body=[If(test=Num(n=1, lineno=1, col_offset=3), body=[Expr(value=Call(func=Name(id='print_i', ctx=Load(), lineno=2, col_offset=4), args=[Num(n=11, lineno=2, col_offset=12)], keywords=[], starargs=None, kwargs=None, lineno=2, col_offset=4), lineno=2, col_offset=4)], orelse=[Expr(value=Call(func=Name(id='print_i', ctx=Load(), lineno=4, col_offset=4), args=[Num(n=22, lineno=4, col_offset=12)], keywords=[], starargs=None, kwargs=None, lineno=4, col_offset=4), lineno=4, col_offset=4)], lineno=1, col_offset=0)])

#+END_SRC



from this we can see that the If function takes parameters

'test', 'body', 'orelse', 'lineno', and 'col_offset'.



now define the translation function that goes in maml_ast.py:



#+Begin_SRC python

def If(test, body, orelse, lineno=None, col_offset=None):

    return {'type': 'if',

            'test': test,

            'body': body,

            'else': orelse,

            'lineno': lineno,

            'col_offset': col_offset}

#+END_SRC

At this point various changes can be made to the ast if it makes the compilation

step easier.

lineno and col_offset are optional and should be given None default values.



Multiple translation functions may have to be defined for each new feature.



run ./maml_ast.py filename.py again to verify correct ast creation.



*** define ast checking function

We are compiling a subset of Python so we need to check that the programmer

is not trying to use features that are not supported.

Do not check for syntactic correctness, Python does that for us.

The checking function should raise an error if a problem is found.

It's return result is ignored.



the ast checking function takes the format:

#+Begin_SRC python

@check('if')

def _(ast):

   #checking code here

#+END_SRC



These functions are collected in the middle of maml_compile.py

In this case of 'if' there is nothing to check for.



The checking function is automatically called before compilation function.

*** define bytecode compilation function



all compilation functions take the form:

#+Begin_SRC python

@node('if')

def _(ast, btc, env, top):

  #compilation code

#+END_SRC



AST is the ast node of the corresponding type.

Generated code is appended to BTC.

In recursive calls to 'gen_bytecode', the TOP parameter should be False.



*** new opcodes



TODO

*** (de)serialization, threaded code

TODO

*** vm case

TODO

* .lock files

When running the VM on the desktop, it creates a while PID.lock

where PID is the process id of the VM. This prevents the compiler

from interrupting the VM at a bad time to inject code.

These should be cleaned up by the VM but often are not - you may delete them safely

after the VM terminates