Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/gosecure/jupyterhub-workshop-environment

Opinionated JupyterHub deployment for workshops relying on GitHub for Authentication
https://github.com/gosecure/jupyterhub-workshop-environment

jupyter-notebook jupyterhub jupyterlab workshop

Last synced: over 1 year ago
JSON representation

Opinionated JupyterHub deployment for workshops relying on GitHub for Authentication

Awesome Lists containing this project

README 

          
= README



Opionated JupyterHub deployment for workshops relying on GitHub for Authentication and Digital Ocean as the infrastructure provider.



This is a drastic departure of our previous environment available here: https://github.com/GoSecure/jupyterhub-workshop-environment-legacy



Features:



* Supports JupyterLab and Jupyter Notebook (Python and Terminals)

* Read-only shared files to all users (in users' homes)

* Writeable and shared scratch directory available to all users (in users' homes)

* Users have their own copies of notebook, created on first login from a template directory

* Server deployed and managed by Terraform

* Out of the box LetsEncrypt HTTPS

* Very close to upstream https://tljh.jupyter.org/[The Littlest JupyterHub (TLJH)] should make upgrades seamless



Originally forked from https://github.com/jupyterhub/jupyterhub-deploy-docker[jupyterhub-deploy-docker] in 2017 but little left if anything today.



== Setup Instructions



=== Prepare for Deployment



These steps should be done locally before provisioning the JupterHub

server to facilitate deployment.



==== Setup GitHub Authentication



Create a GitHub application reflecting the purpose of your JupyterHub

workshop. It will be used to allow users to login and gain access to

the workshop servers. You should know the following information ahead

of time:



* Workshop Name (Application Name)

* Workshop Server URL (Doesn't need to resolve yet)

* Workshop Description

* Workshop Icon if desired.



*NOTE*: Applications for workshops should be created under your organization at

https://github.com/organizations//settings/applications/new



Ask organisation admin to create the application with the required information.



Populate `terraform.tfvars` with the following content:



  github_client_id  = ""

  github_client_secret = ""

  # GitHub Oauth Callback will be set to https:///hub/oauth_callback by default

  #github_oauth_callback = "https:///hub/oauth_callback"



By default our setup is open to anyone to create an account like for an open

workshop with no pre-registrations (different options available below).



==== Setup Administrator Accounts



The first administrator account is configured in the `terraform.tfvars` file with `jupyterhub_admin`.

It must be a GitHub username. Once authenticated this user will be an administrator.

Other admin accounts are configured in the `/opt/tljh/config/config.yaml` file.

The administration GUI (`/hub/admin`) also allows you to elevate accounts via "Edit".

These accounts can use `sudo` without a password.



==== Workshop Files



There are three ways to expose files to the workshop attendees.



Files exposed to attendees all rely to using the Linux `/etc/skel/` home directory template. Remember: once the user is created, things in `skel` will not be copied again. You must destroy a user for it to be recreated (see instructions below under "Operational Procedures").



===== Read-only shared with all participants



* Locally: `workshop-data/`

* Deployed to: `/srv/workshop/`

* Available to participants as: `workshop-readonly/` in their home directory



All users will have access to a directory in their own homes (`/home/jupyter-/`) called `workshop-readonly/` that will be mapped to the servers' `/srv/workshop/`.



Changes made in `/srv/workshop/*` will be reflected instantly to all participants.



This is accomplished via a symlink in `/etc/skel/`.



===== Files for participants (writable and executable)



* Locally: `workshop-data/assignments/`

* Deployed to: `/srv/workshop/assignments` and `/etc/skel/your-personal-lab/`

* Available to participants as: `your-personal-lab/` in their home directory



On container creation, a directory called `your-personal-lab/` will be created and

populated from the servers' `/etc/skel/your-personal-lab/` content.



===== Scratch for participants (shared and writable)



* Locally: None

* On the server: `/srv/scratch/`

* Available to participants as: `scratch/` in their home directory



A `scratch/` directory is available in users' home which is world-writable.

Creator ownership is retained, you can't alter someone else's files but you can read them.



Enabled by the https://github.com/kafonek/tljh-shared-directory[tljh-shared-directory] plugin.



=== Deploying the Server



Ensure that you have a recent version of Terraform installed and get it ready:



    terraform init



Configure your Digital Ocean credentials and other settings in `terraform.tfvars`:



----

# the following values are used as tags in digital ocean so only lowercase, dash and underscore allowed

workshop_name = ""

tag_owner = ""

tag_event = ""

do_token = ""

# default is 1 CPU 2 GB, you can find droplet sizes here: https://slugs.do-api.dev/

#instance_size = "c-4"

# default region is Toronto (tor1)

instance_region = "nyc3"

jupyterhub_admin = ""

# A URL to a Python requirements.txt file for dependencies to be installed on the system

workshop_requirements_url = ""



# GitHub Authentication Parameters

github_client_id  = ""

github_client_secret = ""

----



Spawn and provision the droplet. It might take a while to setup and provision, so be patient.



    terraform validate

    terraform plan

    terraform apply



NOTE: During the apply, you should hurry up and update the DNS of `workshop_domain` to point to the newly deployed droplet public IP.

      Failure to do so will end-up in repeated Let's Encrypt certificate creation failures and will result in a block of one hour before you can attempt to create the certificate again.

      If it happens. Shutdown jupyterhub, make sure DNS has propagated and start jupyterhub again after one hour.



=== Server-Side Configuration



To connect to the server via SSH, run:



    ./bin/ssh-connect.sh



=== More Customization



At this point you have a 100% standard TLJH installation.

You can rely on their documentation to further customize the installation:



* https://tljh.jupyter.org/en/latest/howto/index.html[How-To Guides]

* https://tljh.jupyter.org/en/latest/topic/customizing-installer.html#installing-python-packages-in-the-user-environment[Installing more packages in the environment]

* https://tljh.jupyter.org/en/latest/topic/index.html[Topic Guides]

* https://tljh.jupyter.org/en/latest/troubleshooting/index.html[Troubleshooting]



== Operational Procedures



=== Upgrading JupyterHub



Re-running the TLJH installer should upgrade JupyterHub



=== Deleting a user



This is useful when testing the initial setup of a user's home directory.



* In the `/hub/admin` GUI: Edit User -> Delete

* In a root shell: `userdel -r jupyter-`



=== Reveal Solutions



From an admin notebook:



    !sudo chmod go+rx /srv/workshop/solutions/

    !sudo chmod go+r /srv/workshop/solutions/*



From an admin shell:



    sudo chmod go+rx /srv/workshop/solutions/

    sudo chmod go+r /srv/workshop/solutions/*



== Debugging



=== Logging



Using `systemctl status` on the jupyterhub or traefik services:



    systemctl status jupyterhub



Checking the logs with `journalctl`:



    journalctl -f -u traefik