Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/cameronraysmith/notebooks
Container image for jupyter with multiple kernels on Arch Linux
https://github.com/cameronraysmith/notebooks
archlinux binder docker dotfiles julia jupyter jupyter-notebook jupyterlab maxima python r zsh
Last synced: about 3 hours ago
JSON representation
Container image for jupyter with multiple kernels on Arch Linux
- Host: GitHub
- URL: https://github.com/cameronraysmith/notebooks
- Owner: cameronraysmith
- License: mit
- Created: 2020-06-25T17:14:40.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2023-02-18T02:37:30.000Z (almost 2 years ago)
- Last Synced: 2024-04-13T15:06:50.286Z (10 months ago)
- Topics: archlinux, binder, docker, dotfiles, julia, jupyter, jupyter-notebook, jupyterlab, maxima, python, r, zsh
- Language: Shell
- Homepage: https://hub.docker.com/r/cameronraysmith/notebooks
- Size: 177 KB
- Stars: 3
- Watchers: 3
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# notebooks
[![Binder][mybinder-badge]][mybinder]
[](https://microbadger.com/images/cameronraysmith/notebooks)
## about
This is a [Docker][] configuration for running [jupyter][] || [lab][] with kernels for [Haskell][], [julia][], [maxima][] ([robert-dodier/maxima-jupyter][]), [python][], and [R][] with support for multi-kernel notebooks via [sos-notebook](https://github.com/vatlab/sos-notebook) on [Arch Linux][]. The name and framework are based on [AustinRochford/notebooks][].
See the [Makefile](Makefile) for relevant commands.
## tl;dr
If you have the [Google Cloud SDK](https://cloud.google.com/sdk/docs/install) setup on your machine and are logged into an account with [GCP GPU quota of 1 or greater](https://cloud.google.com/compute/quotas#requesting_additional_quota), you should be able to run
``` bash
make initialize_gcp_insecure
```
and after about 15 minutes, access jupyter lab via HTTP at the external IP of the VM. However, you may want to investigate how to setup SSL/HTTPS access with `initialize_gcp_secure`.
You can stop with `make stop_gcp` and restart the insecure machine with `make startup_gcp_insecure`. Any data you save at the path `$(DATA_DISK)/$(USER_NAME)` should be (re)attached via a persistent disk named `$(DATA_DISK)` in your GCP account. Data you place anywhere else will be lost on stopping the container or VM. When you open jupyter lab, the file browser will be pointed at `$(DATA_DISK)/$(USER_NAME)/$(NOTEBOOK_DIR)`.
You can destroy everything except your `$(DATA_DISK)` by running `make delete_previous_gcp`. This is controlled by `GCP_VM_PREVIOUS=$(GCP_VM)`, which may need to be edited if changing other variables causes this value to fall out of sync with the state of your GCP account.
## container images
### images
Images are available from [Docker hub](https://hub.docker.com/r/cameronraysmith/notebooks) and [GitHub Container Registry](https://ghcr.io/cameronraysmith/notebooks).
### contents
In addition to the jupyter kernels listed in the [about](#about) section, the following software is installed by the [Dockerfile](Dockerfile)
* [arch linux packages](https://archlinux.org/packages/) listed in [etc/pkglist-01.txt](etc/pkglist-01.txt) and [etc/pkglist-02.txt](etc/pkglist-02.txt)
* [yay](https://aur.archlinux.org/packages/yay/) AUR helper
* [jupyter lab](https://jupyterlab.readthedocs.io/en/stable/) dependencies along with a number of jupyter lab extensions and their dependencies (not listed here as this has recently changed with the upgrade to jupyter lab `>=3.0`). A default configuration is in [etc/jupyter_notebook_config.py](etc/jupyter_notebook_config.py), which may be overridden by supplying container arguments in the [cloud](#jupyter-notebook-security) setup.
* [python](https://github.com/python) packages listed in [etc/python-libraries.txt](etc/python-libraries.txt)
* [julia](https://github.com/JuliaLang/IJulia.jl) packages listed in [etc/Project.toml](etc/Project.toml)
* [thoughtbot/rcm](https://github.com/thoughtbot/rcm) and [thoughtbot/dotfiles](https://github.com/thoughtbot/dotfiles)
* [zsh](https://www.zsh.org/) and [ohmyzsh](https://github.com/ohmyzsh/ohmyzsh)
* [romkatv/powerlevel10k](https://github.com/romkatv/powerlevel10k) with default configuration in [etc/p10k.zsh](etc/p10k.zsh)
* [hlissner/doom-emacs](https://github.com/hlissner/doom-emacs)
## setup
### local
This section is deprecated. I use the containers generated by this project exclusively in a cloud computing environment. I currently only support GCP.
### cloud
Interest in supporting cloud services other than GCP is welcome.
#### Google Cloud Platform (GCP)
##### setup
You will need to install the [Google Cloud SDK](https://cloud.google.com/sdk/docs/install). Ensure that you are logged in to an account associated with GCP.
``` bash
$ gcloud auth list
Credentialed Accounts
ACTIVE ACCOUNT
*
To set the active account, run:
$ gcloud config set account `ACCOUNT`
```
You will of course require an active configuration and project.
``` bash
$ gcloud config configurations describe
is_active: true
name:
properties:
compute:
region: us-central1
zone: us-central1-f
core:
account:
project:
$ gcloud projects describe
createTime: '2020-07-20T11:36:22.364Z'
lifecycleState: ACTIVE
name: notebooks
projectId:
projectNumber: ''
```
Once the Google Cloud SDK is configured, follow the list of Make targets that proceed from `initialize_gcp_insecure` in the [Makefile](Makefile). Insecure only refers to the lack of SSL (i.e. the notebook will be served over HTTP as opposed to HTTPS). To setup SSL, see `initialize_gcp_secure` and the section on [Cloudflare](#cloudflare). After running one of the initialize targets, machines can be stopped with the `stop_gcp` target and returned to a working state with the appropriate `startup` target such as `startup_gcp_insecure`.
##### jupyter notebook security
If you would only like to implement password-based authentication, you will need to follow the instructions in the jupyter notebook documentation on [preparing a hashed password](https://jupyter-server.readthedocs.io/en/latest/operators/public-server.html#preparing-a-hashed-password). You can then edit the relevant line in the [Makefile](Makefile). The password in the git history (`%HfuQRa@X%9&8MxM`) should obviously be treated as compromised. You can generate the associated salted, hashed password in python
```python
from jupyter_server.auth import passwd
passwd()
Enter password: %HfuQRa@X%9&8MxM
Verify password: %HfuQRa@X%9&8MxM
'sha1:0af606f6f6ce:11fe6ae47992d2d7a9015d322cf75e5a77c57149'
```
The following is no longer needed. However, for reference, prior to jupyter server, this was done as follows with the notebook package:
```python
from notebook.auth import passwd
passwd()
Enter password: %HfuQRa@X%9&8MxM
Verify password: %HfuQRa@X%9&8MxM
'argon2:$argon2id$v=19$m=10240,t=10,p=8$hQQSNsDLkgTth1v7IjN4Ig$G+O1EfHDdKq/hOZUODBnQA'
```
If you are using an older version of jupyter you may need to do this. Note that the `$` in this salted, hashed password had to be escaped as `\$$`.
``` bash
\$$argon2id\$$v=19\$$m=10240,t=10,p=8\$$hQQSNsDLkgTth1v7IjN4Ig\$$G+O1EfHDdKq/hOZUODBnQA
```
##### data
It is assumed that data will be managed via a persistent disk named `$DATA_DISK` that will be attached in read-write mode to one running instance at a time. If you would like to run multiple instances of this container at the same time, you will need to account for the need to create multiple persistent disks.
#### Cloudflare
This section is only relevant if you would like to access the jupyter notebook server at a custom domain via SSL. Cloudflare is used for [managing Origin CA certificates](https://support.cloudflare.com/hc/en-us/articles/115000479507). See the variables required by [scripts/cloudflare-update.sh](scripts/cloudflare-update.sh) and checked by the Make target `check_cf_env_set` to setup the environment as necessary.
Once you have created the `cf-cert.pem` and `cf-key.pem` files, place them at `./etc/certs`. The Make target `ssl_cert_copy_to_gcp` can then be used to copy them to the [data persistent disk](#data). So long as this disk is maintained and the certificates remain valid, this process should not need to be repeated. Of course, anyone who gains access to this disk will be able to read the contents of the certificates unless additional security measures are taken.
## file listing
```bash
▶ tree -I 'maxima-jupyter'
.
├── Dockerfile
├── Dockerfile.dev
├── LICENSE
├── Makefile
├── README.md
├── VERSION
├── etc
│ ├── Project.toml
│ ├── certs
│ │ ├── cf-cert.pem
│ │ └── cf-key.pem
│ ├── gitconfig.local
│ ├── jupyter_notebook_config.py
│ ├── p10k.zsh
│ ├── pkglist-01.txt
│ ├── pkglist-02.txt
│ ├── pkglist-startup.txt
│ ├── pkglist-yay.txt
│ ├── plugin.jupyterlab-settings
│ ├── python-libraries.txt
│ ├── themes.jupyterlab-settings
│ ├── tracker.jupyterlab-settings
│ └── zshrc.local
├── notebooks
│ ├── ...
└── scripts
├── cloudflare-update.sh
├── install-cos-gpu.sh
└── startup.sh
```
## LICENSE
This code is distributed under the [MIT License](http://opensource.org/licenses/MIT).
[mybinder-badge]: https://mybinder.org/badge_logo.svg
[mybinder]: https://mybinder.org/v2/gh/cameronraysmith/notebooks/master?urlpath=lab
[Docker]: https://www.docker.com/
[jupyter]: https://jupyter.org/
[lab]: https://jupyterlab.readthedocs.io/
[Haskell]: https://github.com/gibiansky/IHaskell
[python]: https://ipython.readthedocs.io/
[julia]: https://github.com/JuliaLang/IJulia.jl
[maxima]: http://maxima.sourceforge.net/
[robert-dodier/maxima-jupyter]: https://github.com/robert-dodier/maxima-jupyter
[R]: https://irkernel.github.io/
[Arch Linux]: https://www.archlinux.org/
[AustinRochford/notebooks]: https://github.com/AustinRochford/notebooks
[microscaling/microbadger]: https://github.com/microscaling/microbadger