https://github.com/manuzhang/jupyterlab_spark

Spark Application UI extension for JupyterLab
https://github.com/manuzhang/jupyterlab_spark

jupyterlab jupyterlab-extension spark typescript

Last synced: 5 months ago
JSON representation

Spark Application UI extension for JupyterLab

• Host: GitHub
• URL: https://github.com/manuzhang/jupyterlab_spark
• Owner: manuzhang
• License: apache-2.0
• Created: 2019-04-20T06:31:59.000Z (about 6 years ago)
• Default Branch: master
• Last Pushed: 2021-09-05T13:13:14.000Z (over 3 years ago)
• Last Synced: 2024-11-07T14:42:37.347Z (5 months ago)
• Topics: jupyterlab, jupyterlab-extension, spark, typescript
• Language: TypeScript
• Homepage:
• Size: 1.2 MB
• Stars: 10
• Watchers: 5
• Forks: 2
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-jupyter-resources - GitHub - 40% open · ⏱️ 05.09.2021): (JupyterLab扩展)

README

        
# jupyterlab_spark

[![npm version](https://img.shields.io/npm/v/jupyterlab_spark?style=flat-square)](https://www.npmjs.com/package/jupyterlab_spark)

[![Build status](https://img.shields.io/github/workflow/status/manuzhang/jupyterlab_spark/Build?style=flat-square)](https://github.com/manuzhang/jupyterlab_spark/actions)

A JupyterLab extension to show Spark Application UI in a JupyterLab panel

## Prerequisites

* JupyterLab

## Installation

```bash

jupyter labextension install jupyterlab_spark

```

## Usage

> Note: the extension only works with Spark application running in client mode.

0. Add the following configuration to `spark-defaults.conf` to allow framing from localhost

```

spark.ui.allowFramingFrom    http://localhost

```

1. Click on the "Application UI" item from the new "Spark" menu 

![spark_menu](image/spark_menu.png)

2. Input the Spark application id in the popped up dialog and click on "CREATE"

![input_app_id](image/input_app_id.png)

The id can be found in driver log or output of executing `sparkContext.applicationId`

3. Here is your Spark application UI. Enjoy!

![spark_app_ui](image/spark_app_ui.png)

## Development

For a development install (requires npm version 4 or later), do the following in the repository directory:

```bash

npm install

npm run build

jupyter labextension link .

```

To rebuild the package and the JupyterLab app:

```bash

npm run build

jupyter lab build

```

## Acknowledgement

Thanks to the [xkcd exntension tutorial](https://jupyterlab.readthedocs.io/en/stable/developer/xkcd_extension_tutorial.html) and [cookiecutter template](https://github.com/jupyterlab/extension-cookiecutter-ts). I'm especially inspired by the [jupyterlab_tensorboard](https://github.com/chaoleili/jupyterlab_tensorboard) project where I also 

shamelessly borrowed some codes.