{"id":19350668,"url":"https://github.com/slevin48/jupyter_docker","last_synced_at":"2025-02-24T10:23:25.745Z","repository":{"id":112496419,"uuid":"325822874","full_name":"slevin48/jupyter_docker","owner":"slevin48","description":"Deploy Jupyter Server with Docker (on Heroku)","archived":false,"fork":false,"pushed_at":"2021-01-30T13:53:40.000Z","size":44,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-06T15:33:34.318Z","etag":null,"topics":["docker","heroku","jupyter"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/slevin48.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-12-31T15:07:43.000Z","updated_at":"2021-02-07T10:23:56.000Z","dependencies_parsed_at":"2023-05-15T09:15:46.375Z","dependency_job_id":null,"html_url":"https://github.com/slevin48/jupyter_docker","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slevin48%2Fjupyter_docker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slevin48%2Fjupyter_docker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slevin48%2Fjupyter_docker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slevin48%2Fjupyter_docker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/slevin48","download_url":"https://codeload.github.com/slevin48/jupyter_docker/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240459002,"owners_count":19804667,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","heroku","jupyter"],"created_at":"2024-11-10T04:33:28.117Z","updated_at":"2025-02-24T10:23:25.716Z","avatar_url":"https://github.com/slevin48.png","language":"Python","readme":"\u003e [Original Post](https://www.codingforentrepreneurs.com/blog/jupyter-production-server-on-docker-heroku)\n\n\u003e [Original repo](https://github.com/codingforentrepreneurs/Jupyter-x-Docker-on-Heroku/)\n\n[![Jupyter x Docker on Heroku Post](https://static.codingforentrepreneurs.com/media/cfe-blog/jupyter-production-server-on-docker-heroku/Jupyter_x_Docker_to_Heroku.jpg)](https://www.codingforentrepreneurs.com/blog/jupyter-production-server-on-docker-heroku)\n\nJupyter is a tool for running interactive notebooks; basically add Python with Markdown and you've got Jupyter. if you haven't used it before, I recommend you do. \n\nIn this post, I'm going to show you how to deploy a Jupyter Notebook server on [Heroku using Docker](https://devcenter.heroku.com/articles/container-registry-and-runtime). \n\nVersions of the images created:\n![dockers.png](dockers.png)\n*(largest version contains git + sci-py stack)*\n\n## The big caveat\nJupyter has the ability to create new notebooks and they will 100% save on your deployed docker-based Jupyter server... but they will **disappear** as soon as you deploy a new version. That's because containers, by their very nature, are ephemeral by default. \n\nThis caveat doesn't mean we shouldn't do this... it just means it is a HUGE consideration when using this guide over something like http://colab.research.google.com.\n\nNear the bottom, I'll show you how to package all your Jupyter contents, download it, and unpackage it again when you deploy.\n\n\n### Final Project Structure\n\n```\ncfe_jupyter\n|   Dockerfile\n│   Pipfile  \n│   Pipfile.lock\n│\n└───conf\n│   │   jupyter.py\n|\n└───nbs\n│   │   notebook.tar.gz\n│   \n└───scripts\n    │   Dockerfile\n    │   d_build.sh\n    |   d_run.sh\n    |   deploy.sh\n    |   entrypoint.sh\n```\n\n## How it's done.\n\n#### 1. Use `pipenv` and install `jupyter`\n\n```\npip install pipenv\ncd path/to/your/project/\npipenv install jupyter --python 3.8\n```\n\n#### 2. Create Jupyter Configuration\n\n**Generate Default Config**\n```\njupyter notebook --generate-config\n```\nThis command creates the default `jupyter_notebook_config.py` file on your local machine. Mine was stored on `~/.jupyter/jupyter_notebook_config.py`\n\n**Create `conf/jupyter.py`**\n```\nmkdir conf\necho \"\" \u003e conf/jupyter.py\n```\nIn `conf/jupyter.py` add:\n\n```python\nimport os\nc = get_config()\n# Kernel config\nc.IPKernelApp.pylab = 'inline'  # if you want plotting support always in your notebook\n# Notebook config\nc.NotebookApp.notebook_dir = 'nbs'\nc.NotebookApp.allow_origin = u'cfe-jupyter.herokuapp.com' # put your public IP Address here\nc.NotebookApp.ip = '*'\nc.NotebookApp.allow_remote_access = True\nc.NotebookApp.open_browser = False\n# ipython -c \"from notebook.auth import passwd; passwd()\"\nc.NotebookApp.password = u'sha1:8da45965a489:86884d5b174e2f64e900edd129b5ef0d2f784a65'\nc.NotebookApp.port = int(os.environ.get(\"PORT\", 8888))\nc.NotebookApp.allow_root = True\nc.NotebookApp.allow_password_change = True\nc.ConfigurableHTTPProxy.command = ['configurable-http-proxy', '--redirect-port', '80']\n```\nA few noteable setup items here:\n\n- `c.NotebookApp.notebook_dir` I set as `nbs` which means you should create a directory as `nbs` for your default notebooks directory. In my case, jupyter will open right to this directory ignoring all others.\n- `c.NotebookApp.password` - this has to be a hashed password. To create a new one, just run `ipython -c \"from notebook.auth import passwd; passwd()\"` on your command line.\n- `c.NotebookApp.port` - Heroku sets this value in our environment variables thus `int(os.environ.get(\"PORT\", 8888))` as our default.\n\n\nTest your new configuration locally with: `jupyter notebook --config=./conf/jupyter.py`\n\n\n#### 3.Create a notebook under -\u003e `nbs/Load_Unload.ipynb`\nThis will be how you can handle the ephemeral nature of Docker containers with Jupyter notebooks. Just create a new notebook called `Load_Unload.ipynb`, and add the following:\n\n```python\nmode = \"unload\"\n\nif mode == 'unload':\n    # Zip all files in the current directory\n    !tar chvfz notebook.tar.gz *\n\nelif mode == 'load:\n    # Unzip all files in the current directory\n    !!tar -xv -f notebook.tar.gz\n```\n\n\n#### 4. Add your `Dockerfile`\nThis is the absolute minimum setup here. You might want to add additional items as needed. Certain packages, especially the ones for data science, require additional installs for our docker-based linux server.\n\n```dockerfile\nFROM python:3.8.2-slim\n\nENV APP_HOME /app\nWORKDIR ${APP_HOME}\n\nCOPY . ./\n\nRUN pip install pip pipenv --upgrade\nRUN pipenv install --skip-lock --system --dev\n\nCMD [\"./scripts/entrypoint.sh\"]\n```\n\u003e The most noteable part of this all is that (1) I'm using `pipenv` locally and in docker and (2) I both install `pipenv` and run `pipenv install --system` to install all pipenv dependancies to the entire docker container (instead of in a virtual environment within the container as well).\n\n\n\n#### 5. Create `scripts/entrypoint.sh`\n\nI perfer using a `entrypoint.sh` script for the `CMD` in Dockerfiles. \n\n```bash\n#!/bin/bash\n\n/usr/local/bin/jupyter notebook --config=./conf/jupyter.py\n```\n\n\n\n#### 6. Build \u0026 Run Docker Locally\n\n```\ndocker build -t cfe-jupyter -f Dockerfile .\n\ndocker run --env PORT=8888 -it -p 8888:8888 cfe-jupyter\n```\n\n#### 7. Heroku Setup\n\n##### 1. Create heroku app\n```\nheroku create cfe-jupyter\n```\n- Change `cfe-jupyter` to your app name\n\n##### 2. Login to Heroku Container Registry\n```\nheroku container:login\n\n```\n\n#### 7. Push \u0026 Release To Heroku\n\n```bash\nheroku container:push web\nheroku container:release web \n```\n\n- `web` is the default for our `Dockerfile`. \n- On the commands above, you might have to append `-a \u003cyour-app-name\u003e` like `heroku container:push web -a cfe-jupyter \n\n\n#### 8. That's it\n```\nheroku open\n```\nThis should allow you to open up your project.\n\n\n## Full Reference\n\n\n\n### `Dockerfile`\n\n```dockerfile\nFROM python:3.8.2-slim\n\nENV APP_HOME /app\nWORKDIR ${APP_HOME}\n\nCOPY . ./\n\nRUN pip install pip pipenv --upgrade\nRUN pipenv install --skip-lock --system --dev\n\nCMD [\"./scripts/entrypoint.sh\"]\n```\n\n### `Pipfile`\n```\n[[source]]\nname = \"pypi\"\nurl = \"https://pypi.org/simple\"\nverify_ssl = true\n\n[dev-packages]\n\n[packages]\njupyter = \"*\"\n\n[requires]\npython_version = \"3.8\"\n```\n\n### `scripts/d_build.sh`\n```bash\ndocker build -t cfe-jupyter -f Dockerfile .\n```\n\n\n### `scripts/d_run.sh`\n```bash\ndocker run --env PORT=8888 -it -p 8888:8888 cfe-jupyter\n```\n\n\n### `scripts/deploy.sh`\n```bash\nheroku container:push web\nheroku container:release web\n```\n\n\n### `scripts/entrypoint.sh`\n\n```bash\n#!/bin/bash\n\n/usr/local/bin/jupyter notebook --config=./conf/jupyter.py\n```\n\n\n### `conf/jupyter.py`\n```python\nimport os\nc = get_config()\n# Kernel config\nc.IPKernelApp.pylab = 'inline'  # if you want plotting support always in your notebook\n# Notebook config\nc.NotebookApp.notebook_dir = 'nbs'\nc.NotebookApp.allow_origin = u'cfe-jupyter.herokuapp.com' # put your public IP Address here\nc.NotebookApp.ip = '*'\nc.NotebookApp.allow_remote_access = True\nc.NotebookApp.open_browser = False\n# ipython -c \"from notebook.auth import passwd; passwd()\"\nc.NotebookApp.password = u'sha1:8da45965a489:86884d5b174e2f64e900edd129b5ef0d2f784a65'\nc.NotebookApp.port = int(os.environ.get(\"PORT\", 8888))\nc.NotebookApp.allow_root = True\nc.NotebookApp.allow_password_change = True\nc.ConfigurableHTTPProxy.command = ['configurable-http-proxy', '--redirect-port', '80']\n```\n\n\n### Create a notebook under -\u003e `nbs/Load_Unload.ipynb`\n\n```python\nmode = \"unload\"\n\nif mode == 'unload':\n    # Zip all files in the current directory\n    !tar chvfz notebook.tar.gz *\n\nelif mode == 'load:\n    # Unzip all files in the current directory\n    !!tar -xv -f notebook.tar.gz\n```\n\n\n### Bonus Installs\nYou might need additional packages (like `numpy` or `pandas` or `opencv`) in your project. Here's what you need to do in your `Dockerfile`, (on [our repo](https://github.com/codingforentrepreneurs/Jupyter-x-Docker-on-Heroku) the final docker file is listed as `Dockerfile.Bonus`) just update it to the following:\n\n```dockerfile\nFROM python:3.8.2-slim\n\nENV APP_HOME /app\nWORKDIR ${APP_HOME}\n\nCOPY . ./\n\n# Install Ubuntu dependencies\n# libopencv-dev = opencv dependencies\nRUN apt-get update \u0026\u0026 apt-get install -y --no-install-recommends \\\n        tzdata \\\n        libopencv-dev \\ \n        build-essential \\\n        libssl-dev \\\n        libpq-dev \\\n        libcurl4-gnutls-dev \\\n        libexpat1-dev \\\n        gettext \\\n        unzip \\\n        supervisor \\\n        python3-setuptools \\\n        python3-pip \\\n        python3-dev \\\n        python3-venv \\\n        python3-urllib3 \\\n        git \\\n        \u0026\u0026 \\\n    apt-get clean \u0026\u0026 \\\n    rm -rf /var/lib/apt/lists/*\n\n# Upgrade PIP\nRUN pip install pip pipenv --upgrade\n\n# sklearn opencv, numpy, and pandas\nRUN pip install scikit-learn opencv-contrib-python numpy pandas\n\n# tensorflow (including Keras)\nRUN pip install tensorflow keras\n\n# pytorch (cpu)\nRUN apt-get update \u0026\u0026 apt-get -y install gcc mono-mcs \u0026\u0026 rm -rf /var/lib/apt/lists/*\nRUN pip install torch==1.5.0+cpu torchvision==0.6.0+cpu -f https://download.pytorch.org/whl/torch_stable.html\n\n# fastai\nRUN pip install fastai\n\n# Project installs\nRUN pipenv install --skip-lock --system --dev\n\n\nCMD [\"./scripts/entrypoint.sh\"]\n```","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fslevin48%2Fjupyter_docker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fslevin48%2Fjupyter_docker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fslevin48%2Fjupyter_docker/lists"}