Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/stisa/jupyternim

A Jupyter kernel for nim
https://github.com/stisa/jupyternim

hacktoberfest ipython jupyter jupyter-kernel jupyter-kernels nim nteract

Last synced: 3 months ago
JSON representation

A Jupyter kernel for nim

Awesome Lists containing this project

README 

        
![logo.svg](logo.svg) Jupyter Nim 

====



This is a beta  [jupyter](http://jupyter.org/) kernel for nim written in nim.  

Works with `notebook`, `lab`, `nteract`, should even work in `vscode-python`.  

If you use `nteract` or `vscode-python`, there are still some problems, please report them.  

For `jupyter lab`, you can also install the companion extension by `jupyter labextension install jupyternim-labextension`  

that provides syntax highlighting.

  

Look at [example-notebook](examples/example-notebook.ipynb) for some examples,  

and at [example-display](examples/example-display.ipynb) for examples of displaying latex, md, etc.  



NOTE: running a notebook with this creates a directory `~/.jupyternim` in which it stores blocks of code, pngs, compiled outputs, etc.

Compilation output should be automatically cleaned up starting from version 0.6.0.



NOTE2: `nteract` support is very wip, also `nteract` doesn't add a cellId to notebook cells so changing types is buggy, I'll work on it and maybe also provide a patch for nteract, but it's low priority, help appreciated.



Installation 

------------

TL,DR:

```

nimble install jupyternim -y

```

Done!



`jupyternim -v` has some details about how it was compiled.



### Prereqs



- a working `nim` installation ( [download](http://nim-lang.org/download.html) )

- a `zeromq` installation. Currently tested only with [ZeroMQ](http://zeromq.org/intro:get-the-software) 4.2. **It must be in PATH or the kernel won't run**.

  - you may already have this installed, it will be checked when you install `jupyternim` to see if you need to install it yourself

- some kind of jupyter environment, some examples:

  - `jupyter`  ( I recomend [miniconda3](http://conda.pydata.org/miniconda.html) and adding jupyter with `conda install jupyter` )

  - `nteract` (get it [here](https://nteract.io/))

  - `vscode`+`vscode-python` extension ( poor naming, but `vscode-python` also provides `jupyter` support )



### Long version:



The kernel should be automatically compiled and registered by doing `nimble install jupyternim` ( or `nimble install https://github.com/stisa/jupyternim` if it's not in nimble yet).



Alternatively, try the following:



- clone this repo: `git clone https://github.com/stisa/jupyternim`

- then go into the cloned dir `cd jupyternim`

- register to nimble with `nimble install`

- compile with `nimble dev`, this will give you a debug version

- run `jupyternim`to register the kernel

- run `jupyter notebook`



Note that [ZeroMQ](http://zeromq.org/intro:get-the-software) is dinamically linked, so it needs to be installed **and added to path**  



### HotCodeReloading

To enable the **very** experimental hotcodereloading support, you'll need to recompile `jupyternim` with `-d:useHcr` and then overwrite the one in `~/.nimble/pkgs/jupyternim-` with it.  

The hotcodereloading mostly works, but there are various bugs that prevent its use. For examples, printing a float crashes it.



Editing

-------



`TAB` : completion request, for example `p a pop up with possible completions  

![completion](examples/jlabcompletion.gif)



TODO:

- [ ] `shift+TAB` : inspection request, for example `echo` -> a pop with a description of echo 

- [ ] support the debugger protocol, support variable value inspection



Magics:

-------



**passing flags**



`#>flags < --some > < --d:flag >`

Pass flags to nim compiler, default is `--verbosity:0 -d:release`.  

Passing new flags overwrites all other previous flags, even default ones.

Example: 

```nim

#>flags -d:test



echo "hi"

when defined test:

  echo "test defined"

else:

  echo "test not defined"

```

Outputs:

```

hi

test defined

```

TODO: provide a way to override default compilation output file



### Delete old temp files

`#>clear all`



### Displaying data

To send back data to display, you can use the module [jupyternimpkg/display](src/jupyternimpkg/display.nim), example:  



```nim

import nimPNG, jupyternimpkg/display



let file = r"..\\src\\jupyternimspec\\logo-64x64.png"

show dkPngFile, [64, 64]: # [width, height]

    file

```



If your file type is not supported by the `display` module yet, you need to implement the proc yourself.

Just write to stdout a string containing a json object defined as

```json

{   // is base64 encoded for binary formats, eg png

    "data": {"": }, 

    "metadata": {"": {}},

    "transient": {}

}

```

enclosed in `##` and `##` markers.

For example, to display a PNG image, simply:

```nim

import json, base64

var 

  img = readFile(path).encode # encode the png file with base64

  w = 320 # displayed width

  h = 240 # displayed height

var content: JsonNode = %*{

    "data": {"image/png": img }, 

    "metadata": %*{"image/png": {"width": w, "height":h}},

    "transient": %*{}

  }



echo "##" & $content $ "##"

```

Consider sending a pr for the display module if you end up having to do this.



TODO

----

- [ ] Finish implementing messaging ( completion, introspection, history, update_display_data... )

- [ ] Connect to nimsuggest via socket, parse its output for introspection requests

- [ ] Documentation lookup magic? 

  - eg. put docs in a subfolder, then `#>doc sequtils` opens a browser to the correct `.html` page ( possibly local )  

- [ ] improve hotcodereloading (probably needs work on the compiler side)

- [ ] convince jupyter notebook maintainers to provide cellId, so I can stop patching the javascript

- [ ] find a better way to fake a repl than re running prior code and discarding output (we have HCR! Buggy though)

- [ ] use `JNsession` as name for temp files (allows multiple open kernels)

- [ ] a better way to handle `display_data` than string delimiters



General structure

-----------------



### [jupyternim](src/jupyternim.nim)

Handles init, start, stop of the various loops, as well as initial installation of the kernelspec. 



### [jupyternimpkg/messages](src/jupyternimpkg/messages.nim)

Handles message specifications exposing low level procs to decode, encode messages



### [jupyternimpkg/sockets](src/jupyternimpkg/sockets.nim)

Defines sockets types, how they are created, how their loops work, how they send and receive messages



### [jupyternimpkg/display](src/jupyternimpkg/display)

Handle preparing and sending back data to display



### [jupyternimspec](src/jupyternimspec/)

Logos for jupyter, a `kernel.js` file to load syntax highlight and patch jupyter notebook to send

a cellId.



Internal Notes

--------------

Messages must be multipart  

signature-must-be-lowercase  



http://nim-lang.org/docs/tinyc.html  



[Jupyter Kernel Docs](https://jupyter-client.readthedocs.io/en/latest/kernels.html#kernels)  

[IHaskell](http://andrew.gibiansky.com/blog/ipython/ipython-kernels)  

[Messaging Docs](https://jupyter-client.readthedocs.io/en/latest/messaging.html)  

[Async logger in nim](https://hookrace.net/blog/writing-an-async-logger-in-nim/)