Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.
Awesome Lists | Featured Topics | Projects
https://github.com/grumpyhome/grumpy

Grumpy is a Python to Go source code transcompiler and runtime.
https://github.com/grumpyhome/grumpy
golang hacktoberfest python transpiler
Last synced: 3 days ago
JSON representation
Grumpy is a Python to Go source code transcompiler and runtime.
Host: GitHub
URL: https://github.com/grumpyhome/grumpy
Owner: grumpyhome
License: apache-2.0
Fork: true (alanjds/grumpy)
Created: 2018-07-28T14:17:35.000Z (over 6 years ago)
Default Branch: master
Last Pushed: 2021-06-03T04:01:00.000Z (over 3 years ago)
Last Synced: 2024-08-02T05:06:54.584Z (3 months ago)
Topics: golang, hacktoberfest, python, transpiler
Language: Go
Homepage:
Size: 1.95 MB
Stars: 419
Watchers: 10
Forks: 18
Open Issues: 83
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project

README

        # Grumpy: Go running Python

[![Build Status](https://travis-ci.org/grumpyhome/grumpy.svg?branch=master)](https://travis-ci.org/grumpyhome/grumpy)

[![Join the chat at https://gitter.im/grumpy-devel/Lobby](https://badges.gitter.im/grumpy-devel/Lobby.svg)](https://gitter.im/grumpy-devel/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)

[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)

## Overview

Grumpy is a Python to Go source code transcompiler and runtime that is intended

to be a near drop-in replacement for CPython 2.7. Grumpy has no VM, because it

compiles Python source code to Go source code which is then compiled to native

code (rather than to bytecode). The compiled Go source code uses Go library

called Grumpy runtime, which replaces Python C API for system calls (although

the API is incompatible with CPython's).

## Limitations

### Things that will probably never be supported by Grumpy

1. `exec`, `eval` and `compile`: These dynamic features of CPython are not

   supported by Grumpy because Grumpy modules consist of statically-compiled Go

   code. Supporting dynamic execution would require bundling Grumpy programs

   with the compilation toolchain, which would be unwieldy and impractically

   slow.

2. C extension modules: Grumpy has a different API and object layout than

   CPython and so supporting C extensions would be difficult. In principle it's

   possible to support them via an API bridge layer like the one that

   [JyNI](http://jyni.org) provides for Jython, but it would be hard to maintain and

   would add significant overhead when calling into and out of extension

   modules.

### Things that Grumpy will support but doesn't yet

There are three basic categories of incomplete functionality:

1. [Language features](https://github.com/google/grumpy/wiki/Missing-features#language-features):

   Most language features are implemented with the notable exception of

   [old-style classes](http://stackoverflow.com/questions/54867/what-is-the-difference-between-old-style-and-new-style-classes-in-python).

   There are also a handful of operators that aren't yet supported.

2. [Builtin functions and types](https://github.com/google/grumpy/wiki/Missing-features#builtins):

   There are a number of missing functions and types in `__builtins__` that have

   not yet been implemented. There are also a lot of methods on builtin types

   that are missing.

3. [Standard library](https://github.com/google/grumpy/wiki/Missing-features#standard-libraries):

   The Python standard library is very large and much of it is pure Python, so

   as the language features and builtins get filled out, many modules will

   just work. But there are also a number of libraries in CPython that are C

   extension modules which will need to be rewritten.

4. C locale support: Go doesn't support locales in the same way that C does. As such,

   some functionality that is locale-dependent may not currently work the same as in

   CPython.

## Running Grumpy

### Pre-requisites

Python 2.7

pip 

Go 1.10+ 

##### Mac Python 2.7

OSX users can go to https://www.python.org/downloads/release/python-2718/

Once you've succesfull installed Python2.7 please ensure that pip is also 

installed on your system by running the command "python -m pip --version". 

If pip is not installed please follow the instructions found here 

https://pip.pypa.io/en/stable/installing/#

Once pip has succesfully been installed please run the following command

    python -m pip install --upgrade pip setuptools

##### Linux Python 2.7

For Debian based distos you can install Python 2 with.

    sudo apt install python2

Next ensure that pip is installed by running "python -m pip --version" 

or, "python2 -m pip version". If it returns pip [version number] then 

pip is already installed. If it doesn't run the command "sudo apt install python-pip".

Note: Again, replace apt with the appropriate package tool.

##### Go Compiler

Go can be downloaded here: https://golang.org/dl/ 

Please see the official go installation documentation to install Go: https://golang.org/doc/install

If you wish to build Go from source see the documentation here: https://golang.org/doc/install/source

## Installing Grumpy

The commands ahead assumes that you have Golang installed and a recent

version of Python 2 and `pip`.

### Method 0: binary package

For convenience, a Python package is provided from the PyPI. During install,

many Grumpy will be compiled and stored inside your Python installation.

You need Golang preinstalled anyway for the installation to be successful.

```sh

pip2 install -U grumpy-runtime -I --no-cache

(wait about 5 minutes)

echo "print 'hello, world'" | grumpy run

```

### Method 1: make run:

The simplest way to execute a Grumpy program is to use `make run`, which wraps a

shell script called grumprun that takes Python code on stdin and builds and runs

the code under Grumpy:

```sh

cd grumpy-tools-src

python2 setup.py develop

cd ../grumpy-runtime-src

echo "print 'hello, world'" | make run

```

### Method 2: grumpc and grumprun:

For more complicated programs, you'll want to compile your Python source code to

Go using grumpc (the Grumpy compiler) and then build the Go code using `go

build`. Since Grumpy programs are statically linked, all the modules in a

program must be findable by the Grumpy toolchain on the GOPATH. Grumpy looks for

Go packages corresponding to Python modules in the \_\_python\_\_ subdirectory

of the GOPATH. By convention, this subdirectory is also used for staging Python

source code, making it similar to the PYTHONPATH.

The first step is to set up the shell so that the Grumpy toolchain and libraries

can be found. From the root directory of the Grumpy source distribution run:

```sh

cd grumpy-tools-src

python2 setup.py develop

cd ../grumpy-runtime-src

make

export PATH=$PWD/build/bin:$PATH

export GOPATH=$PWD/build

export PYTHONPATH=$PWD/build/lib/python2.7/site-packages

```

You will know things are working if you see the expected output from this

command:

```sh

cd grumpy-runtime-src

echo 'import sys; print sys.version' | grumprun

```

Next, we will write our simple Python module into the \_\_python\_\_ directory:

```sh

cd grumpy-runtime-src

echo 'def hello(): print "hello, world"' > $GOPATH/src/__python__/hello.py

```

To build a Go package from our Python script, run the following:

```sh

cd grumpy-runtime-src

mkdir -p $GOPATH/src/__python__/hello

grumpc -modname=hello $GOPATH/src/__python__/hello.py > \

    $GOPATH/src/__python__/hello/module.go

```

You should now be able to build a Go program that imports the package

"\_\_python\_\_/hello". We can also import this module into Python programs

that are built using grumprun:

```sh

cd grumpy-runtime-src

echo 'from hello import hello; hello()' | grumprun

```

grumprun is doing a few things under the hood here:

1. Compiles the given Python code to a dummy Go package, the same way we

   produced \_\_python\_\_/hello/module.go above

2. Produces a main Go package that imports the Go package from step 1. and

   executes it as our \_\_main\_\_ Python package

3. Executes `go run` on the main package generated in step 2.

## Developing Grumpy

There are three main components and depending on what kind of feature you're

writing, you may need to change one or more of these.

### Grumpy Tools

Grumpy converts Python programs into Go programs and

`grumpy transpile` is the CLI tool responsible for parsing Python code

and generating Go code from it. `grumpy transpile` is written in Python

and uses the [`pythonparser`](https://github.com/m-labs/pythonparser)

module to accomplish parsing.

The CLI main entrypoint lives at `grumpy-tools-src/grumpy_tools/cli.py`.

It is supported by a number of Python modules in the

`grumpy-tools-src/grumpy_tools/compiler` subdir.

### Grumpy Runtime

The Go code generated by `grumpy transpile` performs operations

on data structures that represent Python objects in running Grumpy programs.

These data structures and operations are defined in the `grumpy` Go library

(source is in the `grumpy-runtime-src/runtime` subdir of the source

distribution). This runtime is analogous to the Python C API and many of the

structures and operations defined by `grumpy` have counterparts in CPython.

### Grumpy Standard Library

Much of the Python standard library is written in Python and thus "just works"

in Grumpy. These parts of the standard library are copied from CPython 2.7

(possibly with light modifications). For licensing reasons, these files are kept

in the `grumpy-runtime-src/third_party` subdir.

The parts of the standard library that cannot be written in pure Python, e.g.

file and directory operations, are kept in the `grumpy-runtime-src/lib` subdir.

In CPython these kinds of modules are written as C extensions. In Grumpy they

are written in Python but they use native Go extensions to access facilities not

otherwise available in Python.

### Source Code Overview

- `grumpy-tools-src/grumpy_tools/compiler`: Python package implementating Python -> Go transcompilation logic.

- `grumpy-runtime-src/lib`: Grumpy-specific Python standard library implementation.

- `grumpy-runtime-src/runtime`: Go source code for the Grumpy runtime library.

- `grumpy-runtime-src/third_party/ouroboros`: Pure Python standard libraries copied from the

   [Ouroboros project](https://github.com/pybee/ouroboros).

- `grumpy-runtime-src/third_party/pypy`: Pure Python standard libraries copied from PyPy.

- `grumpy-runtime-src/third_party/stdlib`: Pure Python standard libraries copied from CPython.

- `grumpy-tools-src/grumpy_tools/`: Transcompilation and utility CLI.

## Contact

Questions? Comments? Drop us a line at [[email protected]](https://groups.google.com/forum/#!forum/grumpy-users)

or join our [Gitter channel](https://gitter.im/grumpy-devel/Lobby)