Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/jazzycamel/micropython-uprotobuf

A lightweight implementation of Google's Protocol Buffers (protobuf) for micropython
https://github.com/jazzycamel/micropython-uprotobuf

Last synced: 3 months ago
JSON representation

A lightweight implementation of Google's Protocol Buffers (protobuf) for micropython

Awesome Lists containing this project

README 

        
# micropython-uprotobuf



## Project Status

This project is very much a work in progress and, as such, is incomplete. Currently protocol specifications can be compiled

via the plugin and messages can be parsed. 



### Things To Do

* Handle repeats and groups



## Linux Usage

It is assumed that Google Protobuf has been installed and the compiler (`protoc`) is on the `$PATH`. It is also assumed that

a version of Python (preferably 3) is also available and the `protobuf` module has been installed (`pip install protobuf`).



Assuming you have a protocol specification file available (in this case called `test1.proto`) containing something like the following:



```proto

syntax = "proto2";



package test1;



message test1 {

  enum test1enum {

    ValueA=1;

    ValueB=2;

    ValueC=3;

  }



  required int32 a = 1;

  required string b = 2;

  required int64 c = 4;

  required float d = 5;

  required double e = 6;

  required test1enum f = 7 [default=ValueA];

  optional bool g = 8;

  repeated fixed32 h = 9;

  required sint32 i = 10;

  required sfixed64 j = 11;

}

```



then a micropython compatible module can be created using the uprotobuf plugin as follows: 



```sh

$ git clone https://github.com/jazzycamel/micropython-uprotobuf.git

$ cd micropython-uprotobuf

$ chmod +x uprotobuf_plugin.py

$ protoc test1.proto --plugin=protoc-gen-custom=uprotobuf_plugin.py --custom_out=. test1.proto

```

Note: on Windows use

```

> protoc test1.proto --plugin=protoc-gen-custom=uprotobuf_plugin.bat --custom_out=. --proto_path=your\path\to\test1.proto test1.proto

```



This will generate a python module named `test1_upb2.py`, which can be imported and used by micropython, containing a

class named `Test1Message`. This class can currently be used to parse binary messages as follows:



```python

from test1_upb2 import Test1Message



message=Test1Message()

ok=message.parse("\x08\x96\x01")

```



## Windows Usage

It is again assumed, as in "Linux Usage", that you will have installed protoc and placed it on the PATH, installed the protobuf Python module, and created the 'test1.proto' file.



A micropython compatible module can be created using the uprotobuf plugin as follows: 



1. Download the code and change into the correct directory:



    ```cmd

    git clone https://github.com/jazzycamel/micropython-uprotobuf.git

    cd micropython-uprotobuf

    ```



2. Create a file in this directory called 'uprotobuf_plugin.bat' with the following contents: 



    ```bat 

    @py  

    ```



    For example:



    ```bat

    @py C:\Users\me\Desktop\buf\uprotobuf_plugin.py

    ```

    Replace @py with the @ sign and then however you run the python command in CMD. This might be python or a filepath to the python executable.



3. Run the command

    ```bat

    protoc test1.proto --custom_out=. --plugin=protoc-gen-custom= test1.proto

    ```



    For example:



    ```bat

    protoc test1.proto --custom_out=. --plugin=protoc-gen-custom=C:\Users\me\Desktop\buf\micropython-uprotobuf\uprotobuf_plugin.bat test1.proto

    ```



Why do we need to do this? Modern versions of Protoc on Windows won't accept .py files for the plugin files but they will accept a batch script which triggers the Python file.



## Command Explanation

```--plugin=protoc-gen-custom=``` means the protobuf generated code will be generated by a plugin with name "custom", which can be run by running ``````.

The output of the plugin to generate the code is saved at the file indicated with ```--_out``` where `````` is the name of the plugin, in this case custom. 



## Note

This plugin was written for protobuf version 2. Version 3 has not been tested and may or may not work.



## Client/Server Example

In the `test` directory there are client and server scripts that demonstrate micropython encoding a message via

micropython-uprotobuf, transmitting that message via TCP and then decoding the message via the standard Google

protobuf python implementation.



First, run the server as follows:

```sh

$ cd tests/server

$ python3 server.py 

```



Then run the client as follows:

```sh

$ cd tests/client

$ micropython client.py

```



Both server and client use a protocol specification that can be found in `tests/proto/tests.proto`. There is also a 

script named `generate.sh` that is used to generate the python and micropython modules used by the example scripts.