Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/gregsexton/ob-ipython
org-babel integration with Jupyter for evaluation of (Python by default) code blocks
https://github.com/gregsexton/ob-ipython
Last synced: 2 months ago
JSON representation
org-babel integration with Jupyter for evaluation of (Python by default) code blocks
- Host: GitHub
- URL: https://github.com/gregsexton/ob-ipython
- Owner: gregsexton
- Created: 2015-06-06T17:51:02.000Z (over 9 years ago)
- Default Branch: master
- Last Pushed: 2022-09-07T10:12:05.000Z (over 2 years ago)
- Last Synced: 2024-08-02T01:26:19.762Z (6 months ago)
- Language: Emacs Lisp
- Homepage:
- Size: 641 KB
- Stars: 737
- Watchers: 57
- Forks: 111
- Open Issues: 101
-
Metadata Files:
- Readme: README.org
Awesome Lists containing this project
README
#+TITLE: Readme
** What is this?
An Emacs library that allows Org mode to evaluate code blocks using
a Jupyter kernel (Python by default).
** Screenshot
[[./screenshot.jpg]]
** How do I install this?
*** First, you need IPython
Before installing, you'll need Jupyter (>= 1.0) and IPython (>=
5.0) installed and working. You will also need the [[http://jupyter.readthedocs.org/en/latest/install.html][Jupyter]] console
and client (~jupyter_console~, ~jupyter_client~) libraries. All of
this should be trivially installable using pip.
*** Install the Emacs plugin
This package is in MELPA. I recommend installing from there.
Otherwise, for manual installation, you'll need the following
elisp dependencies first:
* https://github.com/magnars/dash.el
* Including dash-functional
* https://github.com/magnars/s.el
* https://github.com/rejeep/f.el
Then just drop this somewhere in your load path and ~(require
'ob-ipython)~.
Lastly, activate ~ipython~ in Org-Babel:
#+BEGIN_SRC emacs-lisp
(org-babel-do-load-languages
'org-babel-load-languages
'((ipython . t)
;; other languages..
))
#+END_SRC
** How do I use it?
Open an org file, add a SRC block and evaluate as you would any Org
SRC block (usually =C-c C-c=). Here I will run through some example
blocks.
This is the most basic ipython block. You *must* provide a session
argument. You can name the session if you wish to separate state.
You can also pass a connection json of an existing ipython session
as a session name in order to connect to it.
The result returned by ob-ipython should be renderable by org so
it's recommended to always use ~:results raw drawer~.
#+BEGIN_SRC org
,#+BEGIN_SRC ipython :session :results raw drawer
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
,#+END_SRC
#+END_SRC
Here we evaluate some code with a function definition using a named
session.
#+BEGIN_SRC org
,#+BEGIN_SRC ipython :session mysession :exports both :results raw drawer
def foo(x):
return x + 9
[foo(x) + 7 for x in range(7)]
,#+END_SRC
,#+RESULTS:
: [16, 17, 18, 19, 20, 21, 22]
#+END_SRC
To get a graphic out, you must ensure that you have evaluated
~%matplotlib inline~ first. A file will be generated for you (see
the ~ob-ipython-resources-dir~ custom var if you want to change the
path).
#+BEGIN_SRC org
,#+BEGIN_SRC ipython :session :exports both :results raw drawer
plt.hist(np.random.randn(20000), bins=200)
,#+END_SRC
#+END_SRC
If you provide an ipyfile argument, this filename will be used
instead of generating one.
#+BEGIN_SRC org
,#+BEGIN_SRC ipython :session :ipyfile /tmp/image.png :exports both :results raw drawer
plt.hist(np.random.randn(20000), bins=200)
,#+END_SRC
#+END_SRC
In order to make an svg graphic rather than a png, you may specify
the output format globally to IPython.
#+BEGIN_EXAMPLE
%config InlineBackend.figure_format = 'svg'
#+END_EXAMPLE
If you wish to use a specific Jupyter kernel, you may pass the
kernel option. This enables you to use ob-ipython with languages
other than Python. You need to have the Jupyter kernel installed
and working before you can use this.
When mixing code from different languages you will need to make use
of the session argument.
#+BEGIN_SRC org
,#+BEGIN_SRC ipython :session :kernel clojure
(+ 1 2)
,#+END_SRC
,#+RESULTS:
: 3
#+END_SRC
ob-ipython supports providing variables and even tables to code.
#+BEGIN_SRC org
,#+TBLNAME: data_table
| a | 1 | 2 |
| b | 2 | 3 |
| c | 3 | 4 |
,#+BEGIN_SRC ipython :session :exports both :var x=2 :var data=data_table
(x, data)
,#+END_SRC
,#+RESULTS:
: (2, [['a', 1, 2], ['b', 2, 3], ['c', 3, 4]])
#+END_SRC
Asynchronous execution is supported. Use the ~:async t~ option.
#+BEGIN_SRC org
,#+BEGIN_SRC ipython :session :ipyfile /tmp/image.png :exports both :async t :results raw drawer
import time
time.sleep(3)
plt.hist(np.random.randn(20000), bins=200)
,#+END_SRC
#+END_SRC
** Experimental: Jupyter kernel support
This package is starting to transition from the original
ipython-only support to full jupyter support.
If you have other kernels installed, you should be able to evaluate
blocks by providing jupyter-X as the language, where X is the
language name recognised by jupyter. For example, you can do
something like this:
#+BEGIN_SRC org
,#+BEGIN_SRC jupyter-R :results raw drawer
x <- 3
x
,#+END_SRC
#+END_SRC
Notice, when providing languages like this, you do not need to
(although you may) provide a session argument. A default session is
created per language. This should also try to provide support for
per-language modes when editing.
** Working with a remote session
First, follow the instructions [[https://github.com/ipython/ipython/wiki/Cookbook:-Connecting-to-a-remote-kernel-via-ssh][here]] to get access to a remote kernel.
You can then pass the name of the local json file as a session arg to
use this tunnel.
Essentially the instructions boil down to
- scp the remote json file over (will be in =/run/user/*/jupyter/=)
- use =jupyter console --existing ./kernel-remote.json --ssh
remote-host= to connect locally. This will give you a prompt that
should allow you to evaluate expressions locally but have them run
on the remote machine.
- use the newly created local config file printed by the previous step
as the argument to =:session=. Be sure to include the =.json= suffix
so that ob-ipython knows to use this as the config rather than
spawning a new session.
** What features are there outside of Org SRC block evaluation?
* You can ask the running IPython kernel for documentation. Open a
SRC block, place the point on the thing you're interested in and
run =M-x ob-ipython-inspect=. I recommend you bind this to a key.
* ~ob-ipython-completions~ queries the kernel for completions at a
position. You may use this to hook up any completion mechanism.
We already provide a company backend. With company installed, add
~(add-to-list 'company-backends 'company-ob-ipython)~ somewhere
in your config. This should then work while editing a src block.
* It's often easier to play with code using a REPL. With the point
in an ipython SRC block, you can open a REPL connected to the
current kernel by running =C-c C-v C-z=.
* If evaluated code produces an error, this will be displayed
nicely in a buffer using IPython's traceback support.
* Stdout/err from code evaluation is displayed in a popup buffer.
This is great for debugging or getting verbose output that is
best left out of documents (e.g progress updates). If you wish to
capture output in your document use the =:results output= SRC
block header.
* You can interrupt or kill a running kernel. This is helpful if
things get stuck or really broken. See =M-x
ob-ipython-interrupt-kernel= and =M-x ob-ipython-kill-kernel=,
respectively.
** Tips and tricks
Here are a few things I have setup to make life better. These
aren't provided with ob-ipython, but are recommended.
* Be sure to use ~%matplotlib inline~, otherwise graphics won't work.
* I use yasnippet to create src blocks. Here is the snippet I use.
It takes care of generating unique file names (when I want one)
so I don't have to think about this.
#+BEGIN_SRC snippet
# -*- mode: snippet -*-
# name: ipython block
# key: py
# --
,#+BEGIN_SRC ipython :session ${1::ipyfile ${2:$$(let ((temporary-file-directory "./")) (make-temp-file "py" nil ".png"))} }:exports ${3:both}
$0
,#+END_SRC
#+END_SRC
These days I use this more often:
#+BEGIN_SRC snippet
,#+BEGIN_SRC ipython :session :exports both :results raw drawer
$0
,#+END_SRC
#+END_SRC
* I use the following Org settings:
#+BEGIN_SRC emacs-lisp
(setq org-confirm-babel-evaluate nil) ;don't prompt me to confirm everytime I want to evaluate a block
;;; display/update images in the buffer after I evaluate
(add-hook 'org-babel-after-execute-hook 'org-display-inline-images 'append)
#+END_SRC
* Export with the =LaTeX= backend using the =minted= package for
source block highlighting fails for =ipython= blocks by default
with the error
: Error: no lexer for alias 'ipython' found
To use the =python= lexer for =ipython= blocks, add this setting:
#+BEGIN_SRC emacs-lisp
(add-to-list 'org-latex-minted-langs '(ipython "python"))
#+END_SRC
# * Install pandoc and anything ipython renders as html will be
# converted to org. This is mostly useful for getting nice tables
# automatically.
* ob-ipython will display anything back from ipython with the mime
type 'text/org' verbatim. This allows you and others to create
[[https://www.safaribooksonline.com/blog/2014/02/11/altering-display-existing-classes-ipython/][formatters]] that output raw org. For example, drop this in your
ipython startup file to have arrays and dataframes rendered as
org tables:
#+BEGIN_SRC python
import IPython
from tabulate import tabulate
class OrgFormatter(IPython.core.formatters.BaseFormatter):
def __call__(self, obj):
try:
return tabulate(obj, headers='keys',
tablefmt='orgtbl', showindex='always')
except:
return None
ip = get_ipython()
ip.display_formatter.formatters['text/org'] = OrgFormatter()
#+END_SRC
** Help, it doesn't work
First thing to do is check that you have all of the required
dependencies. Several common problems have been resolved in the
project's issues, so take a look there to see if your problem has a
quick fix. Otherwise feel free to cut an issue - I'll do my best to
help.
** Alternatives
*** Why not use IPython notebook?
I tried using the IPython notebook but quickly became frustrated
with trying to write code in a web browser. This provides another
option for creating documents containing executable Python code,
but in Emacs - with everything that entails.
*** Why not use [[https://millejoh.github.io/emacs-ipython-notebook/][EIN]]?
EIN is really great. It kept me happy for quite a while but I
started to feel constrained by the cell format of IPython
notebooks. What I really wanted was to embed code in Org
documents. It's hard to compete with Org mode! A few key points in
favour of Org:
* In my opinion, Org's markup is better than Markdown.
* Org's organisational, editing and navigation facilities are much
better than EIN.
* Org's tables...
* Org can export to multiple formats.
* I like how Org opens a new buffer when editing code so that you
can use a Python major mode rather than trying to handle
multiple major modes in one.
I also found myself hitting bugs in EIN where evaluation and doc
lookup would just stop working. I regularly had to kill and reopen
buffers or restart the IPython kernel and this was getting
frustrating.
*** How does this compare to regular Org Python integration (ob-python)?
I think this is more robust. The executed code is sent to a
running IPython kernel which has an architecture designed for this
purpose. The way ob-python works feels like a bit of a hack. I ran
in to race conditions using ob-python where the Org buffer would
update its results before the Python REPL had finished evaluating
the code block. This is what eventually drove me to write this.
It's easier to get plots and images out of this. I also provide
several features I missed when using plain ob-python, such as
looking up documentation and getting IPython-style tracebacks when
things go wrong.
You can also use IPython-specific features such as ~%timeit~.