Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/Calysto/metakernel
Jupyter/IPython Kernel Tools
https://github.com/Calysto/metakernel
Last synced: about 1 month ago
JSON representation
Jupyter/IPython Kernel Tools
- Host: GitHub
- URL: https://github.com/Calysto/metakernel
- Owner: Calysto
- License: bsd-3-clause
- Created: 2014-08-24T20:53:45.000Z (over 10 years ago)
- Default Branch: main
- Last Pushed: 2024-08-21T02:11:02.000Z (4 months ago)
- Last Synced: 2024-11-07T02:42:48.701Z (about 1 month ago)
- Language: Python
- Homepage:
- Size: 3.54 MB
- Stars: 346
- Watchers: 17
- Forks: 84
- Open Issues: 36
-
Metadata Files:
- Readme: README.rst
- Changelog: CHANGELOG.md
- License: LICENSE.txt
Awesome Lists containing this project
- awesome-jupyter-resources - GitHub - 19% open · ⏱️ 09.08.2022): (Jupyter内核)
- best-of-jupyter - GitHub - 22% open · ⏱️ 04.04.2024): (Jupyter Kernels)
README
A Jupyter kernel base class in Python which includes core magic functions (including help, command and file path completion, parallel and distributed processing, downloads, and much more).
.. image:: https://badge.fury.io/py/metakernel.png/
:target: http://badge.fury.io/py/metakernel.. image:: https://coveralls.io/repos/Calysto/metakernel/badge.png?branch=main
:target: https://coveralls.io/r/Calysto/metakernel.. image:: https://github.com/Calysto/metakernel/actions/workflows/tests.yml/badge.svg?query=branch%3Amain++
:target: https://github.com/Calysto/metakernel/actions/workflows/tests.yml.. image:: https://anaconda.org/conda-forge/metakernel/badges/version.svg
:target: https://anaconda.org/conda-forge/metakernel.. image:: https://anaconda.org/conda-forge/metakernel/badges/downloads.svg
:target: https://anaconda.org/conda-forge/metakernelSee Jupyter's docs on `wrapper kernels
`_.Additional magics can be installed within the new kernel package under a `magics` subpackage.
Features
-------------
- Basic set of line and cell magics for all kernels.
- Python magic for accessing python interpreter.
- Run kernels in parallel.
- Shell magics.
- Classroom management magics.
- Tab completion for magics and file paths.
- Help for magics using ? or Shift+Tab.
- Plot magic for setting default plot behavior.Kernels based on Metakernel
---------------------------- matlab_kernel, https://github.com/Calysto/matlab_kernel
- octave_kernel, https://github.com/Calysto/octave_kernel
- calysto_scheme, https://github.com/Calysto/calysto_scheme
- calysto_processing, https://github.com/Calysto/calysto_processing
- java9_kernel, https://github.com/Bachmann1234/java9_kernel
- xonsh_kernel, https://github.com/Calysto/xonsh_kernel
- calysto_hy, https://github.com/Calysto/calysto_hy
- gnuplot_kernel, https://github.com/has2k1/gnuplot_kernel
- spylon_kernel, https://github.com/mariusvniekerk/spylon-kernel
- wolfram_kernel, https://github.com/mmatera/iwolfram
- sas_kernel, https://github.com/sassoftware/sas_kernel
- pysysh_kernel, https://github.com/Jaesin/psysh_kernel
- calysto_bash, https://github.com/Calysto/calysto_bash... and many others.
Installation
----------------
You can install Metakernel through ``pip``:.. code:: bash
pip install metakernel --upgrade
Installing `metakernel` from the `conda-forge` channel can be achieved by adding `conda-forge` to your channels with:
.. code:: bash
conda config --add channels conda-forge
Once the `conda-forge` channel has been enabled, `metakernel` can be installed with:
.. code:: bash
conda install metakernel
It is possible to list all of the versions of `metakernel` available on your platform with:
.. code:: bash
conda search metakernel --channel conda-forge
Use MetaKernel Magics in IPython
--------------------------------Although MetaKernel is a system for building new kernels, you can use a subset of the magics in the IPython kernel.
.. code:: python
from metakernel import register_ipython_magics
register_ipython_magics()Put the following in your (or a system-wide) ``ipython_config.py`` file:
.. code:: python
# /etc/ipython/ipython_config.py
c = get_config()
startup = [
'from metakernel import register_ipython_magics',
'register_ipython_magics()',
]
c.InteractiveShellApp.exec_lines = startupUse MetaKernel Languages in Parallel
To use a MetaKernel language in parallel, do the following:
1. Make sure that the Python module `ipyparallel` is installed. In the shell, type:
.. code:: bash
pip install ipyparallel
2. To enable the extension in the notebook, in the shell, type:
.. code:: bash
ipcluster nbextension enable
3. To start up a cluster, with 10 nodes, on a local IP address, in the shell, type:
.. code:: bash
ipcluster start --n=10 --ip=192.168.1.108
4. Initialize the code to use the 10 nodes, inside the notebook from a host kernel ``MODULE`` and ``CLASSNAME`` (can be any metakernel kernel):
.. code:: bash
%parallel MODULE CLASSNAME
For example:
.. code:: bash
%parallel calysto_scheme CalystoScheme
5. Run code in parallel, inside the notebook, type:
Execute a single line, in parallel:
.. code:: bash
%px (+ 1 1)
Or execute the entire cell, in parallel:
.. code:: bash
%%px
(* cluster_rank cluster_rank)Results come back in a Python list (Scheme vector), in ``cluster_rank`` order. (This will be a JSON representation in the future).
Therefore, the above would produce the result:
.. code:: bash
#10(0 1 4 9 16 25 36 49 64 81)
You can get the results back in any of the parallel magics (``%px``, ``%%px``, or ``%pmap``) in the host kernel by accessing the variable ``_`` (single underscore), or by using the ``--set_variable VARIABLE`` flag, like so:
.. code:: bash
%%px --set_variable results
(* cluster_rank cluster_rank)Then, in the next cell, you can access ``results``.
Notice that you can use the variable ``cluster_rank`` to partition parts of a problem so that each node is working on something different.
In the examples above, use ``-e`` to evaluate the code in the host kernel as well. Note that ``cluster_rank`` is not defined on the host machine, and that this assumes the host kernel is the same as the parallel machines.
Configuration
-------------
``Metakernel`` subclasses can be configured by the user. The
configuration file name is determined by the ``app_name`` property of the subclass.
For example, in the ``Octave`` kernel, it is ``octave_kernel``. The user of the kernel can add an ``octave_kernel_config.py`` file to their
``jupyter`` config path. The base ``MetaKernel`` class offers ``plot_settings`` as a configurable trait. Subclasses can define other traits that they wish to make
configurable.As an example:
.. code:: bash
cat ~/.jupyter/octave_kernel_config.py
# use Qt as the default backend for plots
c.OctaveKernel.plot_settings = dict(backend='qt')Documentation
-----------------------Example notebooks can be viewed here_.
Documentation is available online_. Magics have interactive help_ (and online).
For version information, see the Changelog_.
.. _here: http://nbviewer.jupyter.org/github/Calysto/metakernel/tree/main/examples/
.. _help: https://github.com/Calysto/metakernel/blob/main/metakernel/magics/README.md
.. _online: http://Calysto.github.io/metakernel/
.. _Changelog: https://github.com/Calysto/metakernel/blob/main/CHANGELOG.md