{"id":13740638,"url":"https://github.com/gnzsnz/jupyter-quant","last_synced_at":"2025-05-08T20:32:09.062Z","repository":{"id":190823081,"uuid":"683414504","full_name":"quantbelt/jupyter-quant","owner":"quantbelt","description":"A dockerized Jupyter quant research environment.","archived":false,"fork":false,"pushed_at":"2024-06-17T17:10:23.000Z","size":320,"stargazers_count":69,"open_issues_count":10,"forks_count":8,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-06-17T19:21:17.623Z","etag":null,"topics":["ai","machine-learning","quant-trading","quantitative-analysis","quantitative-trading","trading-strategies"],"latest_commit_sha":null,"homepage":"","language":"Dockerfile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/quantbelt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-26T13:56:09.000Z","updated_at":"2024-06-17T19:21:37.320Z","dependencies_parsed_at":"2024-06-17T19:21:35.380Z","dependency_job_id":"82fd8916-9404-49d5-8ec6-4122f9382760","html_url":"https://github.com/quantbelt/jupyter-quant","commit_stats":null,"previous_names":["gnzsnz/jupyter-quant","quantbelt/jupyter-quant"],"tags_count":25,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quantbelt%2Fjupyter-quant","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quantbelt%2Fjupyter-quant/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quantbelt%2Fjupyter-quant/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quantbelt%2Fjupyter-quant/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/quantbelt","download_url":"https://codeload.github.com/quantbelt/jupyter-quant/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224765531,"owners_count":17366131,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","machine-learning","quant-trading","quantitative-analysis","quantitative-trading","trading-strategies"],"created_at":"2024-08-03T04:00:50.808Z","updated_at":"2024-11-15T10:30:55.836Z","avatar_url":"https://github.com/quantbelt.png","language":"Dockerfile","funding_links":[],"categories":["Python"],"sub_categories":["Quant Research Environment","定量研究环境"],"readme":"# Jupyter Quant\n\nA dockerized Jupyter quant research environment.\n\n## Highlights\n\n- It can be used as a docker image or pypi package.\n- Includes tools for quant analysis, statsmodels, pymc, arch, py_vollib,\n  zipline-reloaded, PyPortfolioOpt, etc.\n- The usual suspects are included, numpy, pandas, sci-py, scikit-learn,\n  yellowbricks, shap, optuna.\n- [ib_async](https://github.com/ib-api-reloaded/ib_async) for Interactive Broker\n  connectivity. Works well with\n  [IB Gateway](https://github.com/gnzsnz/ib-gateway-docker) docker image.\n  [ib_insync](https://github.com/erdewit/ib_insync/) has been invaluable all\n  this time, RIP Ewald.\n- Includes all major Python packages for statistical and time series analysis,\n  see [requirements](https://github.com/gnzsnz/jupyter-quant/blob/master/requirements.txt).\n  For an extensive list check\n  [list installed packages](#list-installed-packages) section.\n- [Zipline-reloaded](https://github.com/stefan-jansen/zipline-reloaded/),\n  [pyfolio-reloaded](https://github.com/stefan-jansen/pyfolio-reloaded)\n  and [alphalens-reloaded](https://github.com/stefan-jansen/alphalens-reloaded).\n- [ib_fundamental](https://github.com/quantbelt/ib_fundamental) for IBKR\n  fundamental data.\n- You can install it as a python package, just `pip install -U jupyter-quant`\n- Designed for [ephemeral](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#create-ephemeral-containers)\n  containers. Relevant data for your environment will survive your container.\n- Optimized for size, it's a 2GB image vs 4GB for jupyter/scipy-notebook\n- Includes jedi language server, jupyterlab-lsp, black and isort.\n- It does NOT include conda/mamba. All packages are installed with pip under\n  `~/.local/lib/python`. Which should be mounted in a dedicated volume to\n  preserve your environment.\n- Includes Cython, Numba, bottleneck and numexpr to speed up things\n- sudo, so you can install new packages if needed.\n- bash and stow, so you can [BYODF](#install-your-dotfiles) (bring your\n  dotfiles). Plus common command line utilities like git, less, nano (tiny), jq,\n  [ssh](#install-your-ssh-keys), curl, bash completion and others.\n- Support for [apt cache](https://github.com/gnzsnz/apt-cacher-ng). If you have\n  other Linux boxes using it can leverage your package cache.\n- It does not include a built environment. If you need to install a package\n  that does not provide wheels you can build your wheels, as explained\n  in [common tasks](#build-wheels-outside-the-container)\n\n## Quick Start\n\nTo use `jupyter-quant` as a [pypi package](https://pypi.org/project/jupyter-quant/)\nsee [install quant package](#install-jupyter-quant-package).\n\nCreate a `docker-compose.yml` file with this content\n\n```yml\nservices:\n  jupyter-quant:\n    image: gnzsnz/jupyter-quant:${IMAGE_VERSION}\n    environment:\n      APT_PROXY: ${APT_PROXY:-}\n      BYODF: ${BYODF:-}\n      SSH_KEYDIR: ${SSH_KEYDIR:-}\n      START_SCRIPTS: ${START_SCRIPTS:-}\n      TZ: ${QUANT_TZ:-}\n    restart: unless-stopped\n    ports:\n      - ${LISTEN_PORT}:8888\n    volumes:\n      - quant_conf:/home/gordon/.config\n      - quant_data:/home/gordon/.local\n      - ${PWD}/Notebooks:/home/gordon/Notebooks\n\nvolumes:\n  quant_conf:\n  quant_data:\n```\n\nYou can use `.env-dist` as your starting point.\n\n```bash\ncp .env-dist .env\n# verify everything looks good\ndocker compose config\ndocker compose up\n```\n\n## Volumes\n\nThe image is designed to work with 3 volumes:\n\n1. `quant_data` - volume for ~/.local folder. It contains caches and all Python\n   packages. This enables to install additional packages through pip.\n2. `quant_conf` - volume for ~/.config, all config goes here. This includes\n   jupyter, ipython, matplotlib, etc\n3. Bind mount (but you could use a named volume) - volume for all notebooks,\n   under `~/Notebooks`.\n\nThis allows to have ephemeral containers and to keep your notebooks (3), your\nconfig (2) and your additional packages (1). Eventually, you would need to\nupdate the image, in this case, your notebooks (3) can move without issues,\nyour config (2) should still work but no warranty and your packages in\n`quant_data` could still be used but you should refresh it with a new image.\nEventually, you would need to refresh (1) and less frequently (2)\n\n## Common tasks\n\n### Get running server URL\n\n```bash\ndocker exec -it jupyterquant jupyter-server list\nCurrently running servers:\nhttp://40798f7a604a:8888/?token=\nebf9e870d2aa0ed877590eb83b4d3bbbdfbd55467422a167 :: /home/gordon/Notebooks\n```\n\nor\n\n```bash\ndocker logs -t jupyter-quant 2\u003e\u00261 | grep '127.0.0.1:8888/lab?token='\n```\n\nYou will need to change hostname (40798f7a604a in this case) or 127.0.0.1 by\nyour docker host ip.\n\n### Show jupyter config\n\n```bash\ndocker exec -it jupyter-quant jupyter-server --show-config\n```\n\n### Set password\n\n```bash\ndocker exec -it jupyter-quant jupyter-server password\n```\n\n### Get command line help\n\n```bash\ndocker exec -it jupyter-quant jupyter-server --help\ndocker exec -it jupyter-quant jupyter-lab --help\n```\n\n### List installed packages\n\n```bash\ndocker exec -it jupyter-quant pip list\n# outdated packages\ndocker exec -it jupyter-quant pip list -o\n```\n\n### Pass parameters to jupyter-lab\n\n```bash\ndocker run -it --rm gnzsnz/jupyter-quant --core-mode\ndocker run -it --rm gnzsnz/jupyter-quant --show-config-json\n```\n\n### Run a command in the container\n\n```bash\ndocker run -it --rm gnzsnz/jupyter-quant bash\n```\n\n### Build wheels outside the container\n\nBuild wheels outside the container and import wheels into the container\n\n```bash\n# make sure python version match .env-dist\ndocker run -it --rm -v $PWD/wheels:/wheels python:3.11 bash\npip wheel --no-cache-dir --wheel-dir /wheels numpy\n```\n\nThis will build wheels for numpy (or any other package that you need) and save\nthe file in `$PWD/wheels`. Then you can copy the wheels in your notebook mount\n(3 above) and install it within the container. You can even drag and drop into\nJupyter.\n\n### Install your dotfiles\n\n`git clone` your dotfiles to `Notebook/etc/dotfiles`, set environment variable\n`BYODF=/home/gordon/Notebook/etc/dotfiles` in your `docker-compose.yml` When\nthe container starts up stow will create links like `/home/gordon/.bashrc`\n\n### Install your SSH keys\n\nYou need to define environment variable `SSH_KEY_DIR` which should point to a\nlocation with your keys. The suggested place is\n`SSH_KEYDIR=/home/gordon/Notebooks/etc/ssh`, make sure the director has the\nright permissions. Something like `chmod 700 Notebooks/etc/ssh` should work.\n\nThe `entrypoint.sh` script will create a symbolic link pointing to\n`$SSH_KEYDIR` on `/home/gordon/.ssh`.\n\nWithin Jupyter's terminal, you can then:\n\n```shell\n# start agent\neval $(ssh-agent)\n# add keys to agent\nssh-add\n# open a tunnel\nssh -fNL 4001:localhost:4001 gordon@bastion-ssh\n```\n\n### Run scripts at start up\n\nIf you define `START_SCRIPTS` env variable with a path, all scripts on that\ndirectory will be executed at start up. The sample `.env-dist` file contains\na commented line with `START_SCRIPTS=/home/gordon/Notebooks/etc/start_scripts`\nas an example and recommended location.\n\nFiles should have a `.sh` suffix and should run under `bash`. in directory\n[start_scripts](https://github.com/quantbelt/jupyter-quant/tree/master/start_scripts)\nyou will find example scripts to load ssh keys and install python packages.\n\n### Install jupyter-quant package\n\nJupyter-quant is available as a package in [pypi](https://pypi.org/project/jupyter-quant/).\nIt's a meta-package that pulls all dependencies in it's highest possible version.\n\nDependencies:\n\n- hdf5 (see below)\n- TA-lib see [instructions](https://pypi.org/project/TA-Lib/)\n\n```bash\n# ubuntu/debian, see install instructions above for TA-lib\nsudo apt-get install libhdf5-dev\n\n# osx\nbrew install hdf5 ta-lib\n```\n\nInstall [pypi package](https://pypi.org/project/jupyter-quant/).\n\n```bash\npip install -U jupyter-quant\n```\n\nAdditional options supported are\n\n```bash\npip install -U jupyter-quant[bayes] # to install pymc \u0026 arviz/graphviz\n\npip install -U jupyter-quant[sk-util] # to install skfolio \u0026 sktime\n```\n\n`jupyter-quant` it's a meta-package that pins all it's dependencies versions.\nIf you need/want to upgrade a dependency you can uninstall `jupyter-quant`,\nalthough this can break interdependencies. Or install from git, where it's\nupdated regularly.\n\n```bash\n# git install\npip install -U git+https://github.com/quantbelt/jupyter-quant.git\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgnzsnz%2Fjupyter-quant","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgnzsnz%2Fjupyter-quant","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgnzsnz%2Fjupyter-quant/lists"}