{"id":19858945,"url":"https://github.com/mmattamala/docker_setup","last_synced_at":"2025-05-02T02:31:18.410Z","repository":{"id":63316936,"uuid":"561741357","full_name":"mmattamala/docker_setup","owner":"mmattamala","description":"Personal docker setup with all the tools and packages I use","archived":false,"fork":false,"pushed_at":"2024-12-09T15:14:01.000Z","size":95,"stargazers_count":4,"open_issues_count":0,"forks_count":4,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-06T21:25:10.344Z","etag":null,"topics":["deep-learning","deployment","docker","gpu","jetson-orin","jetson-xavier","machine-learning","nvidia","robot-operating-system","robotics","ros"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mmattamala.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-04T11:41:36.000Z","updated_at":"2024-12-09T15:14:41.000Z","dependencies_parsed_at":"2024-11-12T14:37:11.695Z","dependency_job_id":null,"html_url":"https://github.com/mmattamala/docker_setup","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mmattamala%2Fdocker_setup","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mmattamala%2Fdocker_setup/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mmattamala%2Fdocker_setup/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mmattamala%2Fdocker_setup/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mmattamala","download_url":"https://codeload.github.com/mmattamala/docker_setup/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251972492,"owners_count":21673613,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","deployment","docker","gpu","jetson-orin","jetson-xavier","machine-learning","nvidia","robot-operating-system","robotics","ros"],"created_at":"2024-11-12T14:25:03.363Z","updated_at":"2025-05-02T02:31:18.404Z","avatar_url":"https://github.com/mmattamala.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# docker_setup\n\n\u003e ⚠️ I do not recomment this package for deployment. It was a testbench to learn docker concepts, specially when integrated with NVidia devices.\n\u003e In its current state, this repo is only a good reference on tricks and configurations to make stuff work.\n\u003e If you need to create a docker image for your application, I instead recommend the [`docker-for-robotics`](https://github.com/2b-t/docker-for-robotics/) repository\n\n\nPersonal package to work with Docker with all the tools and packages I use: ROS, OpenCV with CUDA support, PyTorch.\nIt allows to:\n\n1. **Build new images** that rely on some of the packages mentioned above.\n2. **Run them** in an easy way.\n3. **Set up containers as background process** to allow easy deployment on our robots.\n\nThe available images can be found in [DockerHub](https://hub.docker.com/u/mmattamala)\n\n## Acknowledgments\n\nThe setup here is inspired by advice and snippets by Simone Arreghini and Jonas Frey (ETH Zurich).\n\n## General overview\n\n* `bin`: stores all the scripts to build the images and run the containers, as well as the default settings.\n* `stages`: files that define the different stages to install all the dependencies, creating intermediate images.\n* `targets`: scripts to load all the environment variables to setup different target options (cpu, gpu).\n* `Dockerfile`: the main Dockerfile that builds the images.\n* `entrypoint.sh`: the entrypoint script that is executed when the container is executed. It sources the `.bashrc` file and it can also run other stuff.\n\n## Dependencies\n\nThis package assumes you have installed:\n\n* Docker \u003e20.10.10: [link](https://docs.docker.com/engine/install/ubuntu/)\n* Docker buildx: [link](https://github.com/docker/buildx#linux-packages)\n* NVidia docker: [link](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html)\n\nAdditionally, if you are working on a Jetson board, you need:\n\n* Jetson Stats: [link](https://github.com/rbonghi/jetson_stats)\n\nPlease also check the [Troubleshooting](#troubleshooting) section below for more common errors I had while using Docker (like enabling support for NVidia GPUs).\n\n## Building approach\n\nThis repo follows the approach of manually creating different images to incrementally add more dependencies. This can be also addressed by [multi-stage builds](https://docs.docker.com/build/building/multi-stage/#use-a-previous-stage-as-a-new-stage) but I decided to do it manually to have more control on the workflow.\n\n## Using the repo\n\n### Setting up the image targets\n\nThe targets define different configurations we can build images for, i.e, CPU or GPU-based platforms. For example, the script [`gpu.sh`](targets/gpu.sh) configures all the variables required for my laptop.\n\nIf you want to create images for other GPUs or platforms, you can use [these files](targets) as a reference.\n\n### Building the images\n\nWhile staying in the `docker_setup` folder, we can build the images using the `build.sh` script. For instance, to build all the stages of the `cpu` target:\n\n```sh\n./bin/build.sh --target=cpu\n```\n\nAll the options are:\n\n```sh\nUsage: build.sh --target=TARGET [OPTIONS]\n\nOptions:\n  --target=\u003ctarget\u003e        Target to be built: [cpu gpu jetson_xavier]\n  --stage=\u003cstage\u003e          Last stage to be built: [01-base 02-cuda 03-opencv 04-ros 05-ml 06-extra]\n  --no-push                DO NOT push images to DockerHub\n```\n\n### Running a container\n\nTo run a container from an image we built, we instead use the `run.sh` script. For example:\n\n```sh\n./bin/run.sh --target=cpu --git=$HOME/git\n```\n\nAll the options are:\n\n```sh\nUsage: run.sh --target=TARGET|--image=IMAGE [OPTIONS]\n\nOptions:\n  --target=\u003ctarget\u003e        Selected target (available ones): [cpu gpu jetson_xavier]\n  --image=\u003cimage\u003e          Tag or image id (if not using a specific target)\n  --git=\u003cgit_folder\u003e       Git folder to be mounted (Default: /home/matias/git)\n```\n\n## Background installation\n\nThe scripts in this package also allow to have a container running as a background process, so it can be easily used to test GPU applications or launch processes via [Procman](https://github.com/ori-drs/procman_ros).\nThe following instructions explain how to to this in a clean computer (e.g a Jetson board).\n\n### Pre-installation\n\n1. Make the `git` folder if it doesn't exist `mkdir $HOME/git \u0026\u0026 cd $HOME/dir`\n2. Clone this repo: `git clone git@github.com:mmattamala/docker_setup.git`\n3. Create a new entrypoint for the container under `entrypoints/your_platform_name\u003e.sh`. You can find some examples in the [`entrypoints`](entrypoints/) folder\n\nThe entrypoint is important because it allows us to launch processes at boot. Check the section below for a detailed explanation of the [`jetson_xavier_cerberus.sh`](entrypoints/jetson_xavier_cerberus.sh) example:\n\n\u003cdetails\u003e\n  \u003csummary\u003e Click for full explanation \u003c/summary\u003e\n\n```sh\n#!/bin/bash\n\n# Fix for skimage\nexport LD_PRELOAD=/usr/local/lib/python3.8/dist-packages/skimage/_shared/../../scikit_image.libs/libgomp-d22c30c5.so.1.0.0\n\n# User specific enviornment configuration\nexport ENV_WORKSTATION_NAME=jetson\n\n# Assess if catkin_ws can be sourced\ncatkin_ws=$(echo ~/catkin_ws/devel/setup.bash)\nif [ -f \"$catkin_ws\" ]\nthen\n    # Assess if procman is build\n    out=$(source ~/.bashrc \u0026\u0026 source ~/catkin_ws/devel/setup.bash \u0026\u0026 echo roscd procman_ros)\n    ref=$(echo roscd: No such package/stack \\'procman_ros\\')\n    if [ $out == $ref ]; then\n        source ~/.bashrc \u0026\u0026 source ~/catkin_ws/devel/setup.bash  \n        echo \"Warning: procman_ros is not build within the catkin_ws. Therefore the deputy cannot be started!\"\n    else\n        source ~/.bashrc \u0026\u0026 source ~/catkin_ws/devel/setup.bash  \u0026\u0026 rosrun procman_ros deputy -i anymal_cerberus_xavier\n    fi\nelse\n    echo \"Warning: Catkin_ws does not exist!\"\nfi\n```\n\nThis entrypoint sources the `.bashrc`, `catkin_ws` and automatically starts a `procman deputy` with the correct name set to `anymal_cerberus_xavier` if `procman_ros` is correctly build within the catkin_ws. Make sure that the entrypoint can be executed otherwise the service can not run and the container will not start.\n\n\u003e 💡 Running the idealing procman deputy inside the container does not take up any resources, therefore it can always run in the background, even when not used.\n\u003c/details\u003e\n\n### Installation of the container\n\nWhile being in `~/git/docker_setup`, execute:\n\n```sh\n./bin/install.sh --name=jetson_xavier_cerberus --target=jetson_xavier --git=$HOME/git --entrypoint=jetson_xavier_cerberus.sh\n```\n\nThe previous script will do as follows:\n\n1. It will pull the image specified by the target. In this case the target is `jetson_xavier`, which is defined in the [targets](targets) folder: [jetson_xavier](targets/jetson_xavier.sh)\n2. A new container will be created under the name `jetson_xavier_cerberus`. This is the name that will be used by the Docker daemon, and it will mount the `$HOME/git` folder under `/root/git` in the container. This container is initialized using the [`scripts/run.sh`](bin/run.sh) script described before.\n3. A background service is added under: `/etc/systemd/system/docker-setup-jetson_xavier_cerberus.service`. This will ensure that the container is launched on startup, and it will execute what is defined within the entrypoint. \n4. Finally, the installation script also defines some variables and extra files that are sourced in the `bashrc` of the host.\n\n```sh\n# == Docker setup ini ==\nexport DOCKER_SETUP_ROOT=/home/tutuna/git/docker_setup\nexport DOCKER_SETUP_CONTAINER_NAME=jetson_xavier_cerberus\nsource $DOCKER_SETUP_ROOT/bin/commands.sh\nexport DOCKER_SETUP_SERVICE=docker-setup-jetson_xavier_cerberus\n# == Docker setup end ==\"\n```\n\n5. Make sure to source your `.bashrc`\n\n### Post-installation\n\n#### Sanity checks\n\nAfter sourcing your `.bashrc`, you can execute some basic tests to confirm that everything works as intended:\n\n1. Run `docker ps` to see a running container with the name `jetson_xavier_cerberus`\n\n```sh\nuser@hostname:~/git/docker_setup$ docker ps\nCONTAINER ID   IMAGE                                                           COMMAND                 CREATED      STATUS         PORTS     NAMES\naece32b86eb5   mmattamala/devel-jetson:ubuntu20.04-noetic-cuda10.2.0-r32.5.0   \"/entrypoint.sh bash\"   1 min ago   Up 15 months             jetson_xavier_cerberus\n```\n\n2. Run `dsstatus` to check the status of the background service. It should look like:\n\n```sh\n● docker-setup-gpu.service - Docker Setup service\n     Loaded: loaded (/etc/systemd/system/docker-setup-gpu.service; enabled; vendor preset: enabled)\n     Active: activating (auto-restart) since Mon 2023-04-10 16:07:42 BST; 1s ago\n    Process: 73193 ExecStart=/usr/bin/docker start gpu (code=exited, status=0/SUCCESS)\n   Main PID: 73193 (code=exited, status=0/SUCCESS)\n```\n\n3. Run `dsbash` to start a bash terminal on the background container.\n\n```sh\nbash: /root/catkin_ws/devel/setup.bash: No such file or directory\n(docker) root@hostname:~# \n```\n\n\u003e 💡 Both `dstatus` and `dsbash` are defined in [`bin/commands.sh`](`bin/commands.sh`), which is sourced in your `bashrc`.\n\n\u003e ⚠️ The error `bash: /root/catkin_ws/devel/setup.bash: No such file or directory` is expected as the `catkin_ws` has not been set up yet.\n\n#### Setting up your catkin_ws\n\n1. Outside the container: clone in the `$HOME/git` folder all the repos/packages you need. They will be automatically mounted to the container.\n2. Open a new terminal on the container with `dsbash`\n3. Set up the catkin workspace as usual. Here we show an example with `procman_ros`:\n\n```sh\nmkdir -p ~/catkin_ws/src/\nsource /opt/ros/noetic/setup.bash\ncd ~/catkin_ws/\ncatkin init\ncatkin config --extend /opt/ros/noetic\ncatkin config --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo\n\nln -s /root/git/procman_ros /root/catkin_ws/src/\n\ncd $HOME/catkin_ws\ncatkin build procman_ros\n```\n\n\u003e 💡 By defining `catkin_ws` inside the container we avoid conflicts between the host user and container's `root` user.\n\n4. Now log out of the container using `exit` or _Ctrl+D_ and restart the container `docker restart jetson_xavier_cerberus`. This will ensure that the procman deputy is loaded with the correct packages available within the `catkin_ws`.\n\n#### Running code\n\n\u003cdetails\u003e\n  \u003csummary\u003e Launching code within the container \u003c/summary\u003e\n\n\u003c/br\u003e\n\n  The simplest way is just running:\n\n  ```sh\n  dsbash \u003cyour_command\u003e\n  ```\n\n  `dsbash` should pass your command to the container to execut it.\n\n  Otherwise, you can init a terminal on the container:\n  \n  ```sh\n  dsbash\n  ```\n\n  And then run your command as usual:\n\n  ```sh\n  \u003cyour_command\u003e\n  ```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003e Launching code on the robot \u003c/summary\u003e\n\n\u003c/br\u003e\n\n1. Ssh into the jetson\n```sh\nssh anymal-cerberus-jetson\n```\n2. Start a tmux session\n```sh\ntmux\n```\n3. Execute dsbash\n```sh\ndsbash\n```\n4. Execute the anymal_rsl code inside the container\n```sh\nrosrun anymal_rsl jetson.py\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003e Launching code with procman (work in progress) \u003c/summary\u003e\n\n\u003c/br\u003e\n\n\u003e ⚠️ This is not fully documented yet\n\n1. Define on OPC a procman configuration `~/procman.pmd` containing the following:\n\n```sh\ngroup \"0.basic\" {\n    cmd \"0.5.jetson MAN\" {\n        exec = \"rosrun anymal_c_rsl jetson.py\";\n        host = \"anymal_cerberus_xavier\";\n    }\n}\n```\n\nWe recommend to add an alias to the OPC `.bashrc` to easily access procman:\n\n```sh\nalias procman=\"rosrun procman_ros sheriff -l $HOME/procman.pmd\"\n```\n\nStart procman (don not forget to source your `.bashrc` before):\n\n```sh\nprocman\n```\n\n\u003e ⚠️ If you change things in `catkin_ws` and you want to start them via the procman, remember to restart the container such that `catkin_ws` is sourced when procman restarts.\n\n\u003c/details\u003e\n\n### Uninstallation\n\nWhile being in `~/git/docker_setup`, execute:\n\n```sh\n./bin/install.sh --name=jetson_xavier_cerberus --uninstall\n```\n\nThis will stop and remove the container, as well as the background services.\n\n## Troubleshooting\n\nJust writing down some we have found working with Docker in different platforms and how to fix them.\n\n### To avoid using sudo\n\nAdd your user to the Docker group\n\nCreate the docker group if it does not exist\n\n```sh\nsudo groupadd docker\n ```\n\nAdd your user to the docker group.\n\n```sh\nsudo usermod -aG docker $USER\n```\n\n### Enable the nvidia runtime\n\nStop the Docker daemon:\n\n```sh\nsudo service docker stop\n```\n\nOpen the docker daemon file\n\n```sh\nsudo nano /etc/docker/daemon.json\n```\n\nAdd the `default-runtime` entry to the `daemon.json` file and save:\n\n```json\n{\n    \"runtimes\": {\n        \"nvidia\": {\n            \"path\": \"nvidia-container-runtime\",\n            \"runtimeArgs\": []\n        }\n    },\n    \"default-runtime\": \"nvidia\"\n}\n```\n\nStart the daemon again\n\n```sh\nsudo service docker restart\n```\n\n### Change default directory for docker images\n\nThe images are stored in `/var/lib/docker` by default. To change it, modify the same `daemon.json` file to add the `data-root` entry:\n\n```json\n{\n    \"data-root\": \"/media/drs-orin/orin_ssd/docker\",\n    \"runtimes\": {\n        \"nvidia\": {\n            \"path\": \"nvidia-container-runtime\",\n            \"runtimeArgs\": []\n        }\n    },\n    \"default-runtime\": \"nvidia\"\n}\n```\n\nIf you need more information when doing this on a Jetson, check this [link](https://forums.developer.nvidia.com/t/change-docker-image-storage-location-to-nvme-ssd/156882/2) from the NVidia forums.\n\n### If you get `unknown flag: --build-arg` when building images\n\nYou are missing [Docker buildx](https://github.com/docker/buildx#linux-packages). Follow the instructions so set it up in your system.\n\n### Mount extra devices when running the container (e.g cameras)\n\nYou can use the `--flags` parameter. For example, to load a camera with descriptor `/dev/video2` as `video0` in the container:\n\n```sh\n./bin/run.sh --target=cpu --flags=\"-v=/dev/video2:/dev/video0\"\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmmattamala%2Fdocker_setup","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmmattamala%2Fdocker_setup","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmmattamala%2Fdocker_setup/lists"}