https://github.com/adtech-labs/spylon-kernel

Jupyter kernel for scala and spark
https://github.com/adtech-labs/spylon-kernel

jupyter-kernels kernel metakernel scala spark team-platform

Last synced: about 4 hours ago
JSON representation

Jupyter kernel for scala and spark

• Host: GitHub
• URL: https://github.com/adtech-labs/spylon-kernel
• Owner: adtech-labs
• License: other
• Created: 2016-11-21T02:23:14.000Z (about 8 years ago)
• Default Branch: master
• Last Pushed: 2024-01-11T09:22:04.000Z (about 1 year ago)
• Last Synced: 2025-01-16T12:55:36.071Z (about 9 hours ago)
• Topics: jupyter-kernels, kernel, metakernel, scala, spark, team-platform
• Language: Python
• Homepage:
• Size: 199 KB
• Stars: 187
• Watchers: 11
• Forks: 38
• Open Issues: 26
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# spylon-kernel

[![Project Status: Inactive – The project has reached a stable, usable state but is no longer being actively developed; support/maintenance will be provided as time allows.](https://www.repostatus.org/badges/latest/inactive.svg)](https://www.repostatus.org/#inactive)

[![Build Status](https://travis-ci.org/Valassis-Digital-Media/spylon-kernel.svg?branch=master)](https://travis-ci.org/maxpoint/spylon-kernel)

[![codecov](https://codecov.io/gh/Valassis-Digital-Media/spylon-kernel/branch/master/graph/badge.svg)](https://codecov.io/gh/maxpoint/spylon-kernel)

A Scala [Jupyter kernel](http://jupyter.readthedocs.io/en/latest/projects/kernels.html) that uses [metakernel](https://github.com/Calysto/metakernel) in combination with [py4j](https://www.py4j.org/).

## Prerequisites

* Apache Spark 2.1.1 compiled for Scala 2.11

* Jupyter Notebook

* Python 3.5+

## Install

You can install the spylon-kernel package using `pip` or `conda`.

```bash

pip install spylon-kernel

# or

conda install -c conda-forge spylon-kernel

```

## Using it as a Scala Kernel

You can use spylon-kernel as Scala kernel for Jupyter Notebook. Do this when you want

to work with Spark in Scala with a bit of Python code mixed in.

Create a kernel spec for Jupyter notebook by running the following command:

```bash

python -m spylon_kernel install

```

Launch `jupyter notebook` and you should see a `spylon-kernel` as an option

in the *New* dropdown menu.

See [the basic example notebook](./examples/basic_example.ipynb) for information

about how to intiialize a Spark session and use it both in Scala and Python.

## Using it as an IPython Magic

You can also use spylon-kernel as a magic in an IPython notebook. Do this when

you want to mix a little bit of Scala into your primarily Python notebook.

```python

from spylon_kernel import register_ipython_magics

register_ipython_magics()

```

```scala

%%scala

val x = 8

x

```

## Using it as a Library

Finally, you can use spylon-kernel as a Python library. Do this when you

want to evaluate a string of Scala code in a Python script or shell.

```python

from spylon_kernel import get_scala_interpreter

interp = get_scala_interpreter()

# Evaluate the result of a scala code block.

interp.interpret("""

    val x = 8

    x

""")

interp.last_result()

```

# Release Process

Push a tag and submit a source dist to PyPI.

```

git commit -m 'REL: 0.2.1' --allow-empty

git tag -a 0.2.1 # and enter the same message as the commit

git push origin master # or send a PR

# if everything builds / tests cleanly, release to pypi

make release

```

Then update https://github.com/conda-forge/spylon-kernel-feedstock.