{"id":27185965,"url":"https://github.com/bdusell/singularity-tutorial","last_synced_at":"2025-04-09T17:55:51.459Z","repository":{"id":49718700,"uuid":"180221339","full_name":"bdusell/singularity-tutorial","owner":"bdusell","description":"Tutorial for using Singularity containers","archived":false,"fork":false,"pushed_at":"2020-10-01T02:44:12.000Z","size":6637,"stargazers_count":95,"open_issues_count":1,"forks_count":11,"subscribers_count":6,"default_branch":"master","last_synced_at":"2023-08-04T07:40:23.009Z","etag":null,"topics":["container","cuda","cudnn","pytorch","singularity"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bdusell.png","metadata":{"files":{"readme":"README-ND.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-04-08T19:44:47.000Z","updated_at":"2023-08-03T08:04:02.000Z","dependencies_parsed_at":"2022-09-24T19:41:31.498Z","dependency_job_id":null,"html_url":"https://github.com/bdusell/singularity-tutorial","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bdusell%2Fsingularity-tutorial","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bdusell%2Fsingularity-tutorial/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bdusell%2Fsingularity-tutorial/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bdusell%2Fsingularity-tutorial/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bdusell","download_url":"https://codeload.github.com/bdusell/singularity-tutorial/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248083593,"owners_count":21045122,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["container","cuda","cudnn","pytorch","singularity"],"created_at":"2025-04-09T17:55:47.172Z","updated_at":"2025-04-09T17:55:51.446Z","avatar_url":"https://github.com/bdusell.png","language":null,"readme":"# How to Install Literally Anything Using Containers\n\nBrian DuSell\u003cbr\u003e\nApr 9, 2019\n\nGrad Tutorial Talk\u003cbr\u003e\nDept. of Computer Science and Engineering\u003cbr\u003e\nUniversity of Notre Dame\n\n## Abstract\n\nHave you ever spent an inordinate amount of time trying to install\nsomething on the CRC without root privileges? Have you ever wrecked your\ncomputer trying to update CUDA? Have you ever wished you could install two\nversions of the same package at once? If so, containers may be what's missing\nin your life. In this talk, I will show you how to install software using\nSingularity, a container system that allows you to install software in a fully\nportable, self-contained Linux environment where you have full administrative\nrights. Singularity can be installed on any Linux machine (with techniques\navailable for running it on Windows and Mac) and is available on the CRC, thus\nensuring that your code runs in a consistent environment no matter which\nmachine you run it on. Singularity is compatible with Docker images and lets\nyou effortlessly install any CUDA version of your choosing provided that your\nNvidia drivers have been set up properly. My tutorial will consist of walking\nyou through using Singularity to run a GPU-accelerated PyTorch program for deep\nlearning on the CRC. Note: If you want to follow along, please ensure that you\nhave a directory under the `/scratch365` directory on the CRC's filesystem.\n\n## Introduction\n\nThis tutorial will introduce you to [Singularity](https://www.sylabs.io/singularity/),\na containerization system for scientific computing environments that is\navailable on Notre Dame's CRC computing cluster. Containers allow you to\npackage the environment that your code depends on inside of a portable unit.\nThis is extremely useful for ensuring that your code can be run portably\non other machines. It is also useful for installing software, packages,\nlibraries, etc. in environments where you do not have root privileges, like the\nCRC. I will show you how to install PyTorch with GPU support inside of a\ncontainer and run a simple PyTorch program to train a neural net.\n\n## The Portability Problem\n\nThe programs we write depend on external environments, whether that environment\nis explicitly documented or not. A Python program assumes that a Python\ninterpreter is available on the system it is run on. A Python program that uses\nset comprehension syntax, e.g.\n\n```python\n{ x * 2 for x in range(10) }\n```\n\nassumes that you're using Python 3. A Python program that uses the function\n`subprocess.run()` assumes that you're using at least version 3.5. A Python\nprogram that calls `subprocess.run(['grep', '-r', 'foo', './my/directory'])`\nassumes that you're running on a \\*nix system where the program `grep` is\navailable.\n\nWhen these dependencies are undocumented, it can become painful to run a\nprogram in an environment that is different from the one it was developed in.\nIt would be nice to have a way to package a program together with its\nenvironment, and then run that program on any machine.\n\n## The Installation Problem\n\nThe CRC is a shared scientific computing environment with a shared file system.\nThis means that users do not have root privileges and cannot use a package\nmanager like `yum` or `apt-get` to install new libraries. If you want to\ninstall something on the CRC that is not already there, you have a few options:\n\n* If it is a major library, ask the staff to install/update it for you\n* Install it in your home directory (e.g. `pip install --user` for Python\n  modules) or other non-standard directory\n* Compile it yourself in your home directory\n\nWhile it is almost always possible to re-compile a library yourself without\nroot privileges, it can be very time-consuming. This is especially true when\nthe library depends on other libraries that also need to be re-compiled,\nleading to a tedious search for just the right configuration to stitch them all\ntogether. CUDA also complicates the situation, as certain deep learning\nlibraries need to be built on a node that has a GPU (even though the GPU is\nnever used during compilation!).\n\nFinally, sometimes you deliberately want to install an older version of a\npackage. But unless you set up two isolated installations, this could conflict\nwith projects that still require the newer versions.\n\nTo take an extreme (but completely real!) example, older versions of the deep\nlearning library [DyNet](https://dynet.readthedocs.io/en/latest/) could only be\nbuilt with an old version of GCC, and moreover needed to be compiled on a GPU\nnode with the CRC's CUDA module loaded in order to work properly. In May 2018,\nthe CRC removed the required version of GCC. This meant that if you wanted to\ninstall or update DyNet, you needed to re-compile that version of GCC yourself\n*and* figure out how to configure DyNet to build itself with a compiler in a\nnon-standard location.\n\n## The Solution: Containers\n\nContainers are a software isolation technique that has exploded in popularity\nin recent years, particularly thanks to [Docker](https://www.docker.com/).\nA container, like a virtual machine, is an operating system within an operating\nsystem. Unlike a virtual machine, however, it shares the kernel with the host\noperating system, so it incurs no performance penalty for translating machine\ninstructions. Instead, containers rely on special system calls that allow the\nhost to spoof the filesystem and network that the container has access to,\nmaking it appear from inside the container that it exists in a separate\nenvironment.\n\nToday we will be talking about an alternative to Docker called Singularity,\nwhich is more suitable for scientific computing environments (Docker is better\nsuited for things like cloud applications, and there are reasons why it would\nnot be ideal for a shared environment like the CRC). The CRC currently offers\n[Singularity 3.0](https://www.sylabs.io/guides/3.0/user-guide/), which is\navailable via the `singularity` command.\n\nSingularity containers are instantiated from **images**, which are files that\ndefine the container's environment. The container's \"root\" file system is\ndistinct from that of the host operating system, so you can install whatever\nsoftware you like as if you were the root user. Installing software via the\nbuilt-in package manager is now an option again. Not only this, but you can\nalso choose a pre-made image to base your container on. Singularity is\ncompatible with Docker images (a very deliberate design decision), so it can\ntake advantage of the extremely rich selection of production-grade Docker\nimages that are available. For example, there are pre-made images for fresh\ninstallations of Ubuntu, Python, TensorFlow, PyTorch, and even CUDA. For\nvirtually all major libraries, getting a pre-made image for X is as simple as\nGoogling \"X docker\" and taking note of the name of the image.\n\nAlso, because your program's environment is self-contained, it is not affected\nby changes to the CRC's software and is no longer susceptible to \"software\nrot.\" There is also no longer a need to rely on the CRC's modules via `module\nload`. Because the container is portable, it will also run just as well on your\nlocal machine as on the CRC. In the age of containers, \"it runs on my machine\"\nis no longer an excuse.\n\n## Basic Workflow\n\nSingularity instantiates containers from images that define their environment.\nSingularity images are stored in `.sif` files. You build a `.sif` file by\ndefining your environment in a text file and providing that definition to the\ncommand `singularity build`.\n\nBuilding an image file does require root privileges, so it is most convenient\nto build the image on your local machine or workstation and then copy it to\nyour `/scratch365` directory in the CRC. The reason it requires root is because\nthe kernel is shared, and user permissions are implemented in the kernel. So if\nyou want to do something in the container as root, you actually need to *be*\nroot on the host when you do it.\n\nThere is also an option to build it without root privileges. This works by\nsending your definition to a remote server and building the image there, but I\nhave had difficulty getting this to work.\n\nOnce you've uploaded your image to the CRC, you can submit a batch job that\nruns `singularity exec` with the image file you created and the command you\nwant to run. That's it!\n\n## A Simple PyTorch Program\n\nI have included a PyTorch program,\n[`train_xor.py`](examples/xor/train_xor.py),\nthat trains a neural network to compute the XOR function and then plots the\nloss as a function of training time. It can also save the model to a file. It\ndepends on the Python modules `torch`, `numpy`, and `matplotlib`.\n\n## Installing Singularity\n\n[Singularity 3.0](https://www.sylabs.io/guides/3.0/user-guide/index.html)\nis already available on the CRC via the `singularity` command.\n\nAs for installing Singularity locally, the Singularity docs include detailed\ninstructions for installing Singularity on major operating systems\n[here](https://www.sylabs.io/guides/3.0/user-guide/installation.html).\nInstalling Singularity is not necessary for following the tutorial in real\ntime, as I will provide you with pre-built images.\n\n## Defining an Image\n\nThe first step in defining an image is picking which base image to use. This\ncan be a Linux distribution, such as Ubuntu, or an image with a library\npre-installed, like one of PyTorch's\n[official Docker images](https://hub.docker.com/r/pytorch/pytorch/tags). Since\nour program depends on more than just PyTorch, let's start with a plain Ubuntu\nimage and build up from there.\n\nLet's start with the basic syntax for definition files, which is documented\n[here](https://www.sylabs.io/guides/3.0/user-guide/definition_files.html).\nThe first part of the file is the header, where we define the base image and\nother meta-information. The only required keyword in the header is `Bootstrap`,\nwhich defines the type of image being imported. Using `Bootstrap: library`\nmeans that we are importing a library from the official\n[Singularity Library](https://cloud.sylabs.io/library).\nUsing `Bootstrap: docker` means that we are importing a Docker image from a\nDocker registry such as\n[Docker Hub](https://hub.docker.com/).\nLet's import the official\n[Ubuntu 18.04](https://cloud.sylabs.io/library/_container/5baba99394feb900016ea433)\nimage.\n\n```\nBootstrap: library\nFrom: ubuntu:18.04\n```\n\nThe rest of the definition file is split up into several **sections** which\nserve special roles. The `%post` section defines a series of commands to be run\nwhile the image is being built, inside of a container as the root user. This\nis typically where you install packages. The `%environment` section defines\nenvironment variables that are set when the image is instantiated as a\ncontainer. The `%files` section lets you copy files into the image. There are\n[many other types of section](https://www.sylabs.io/guides/3.0/user-guide/definition_files.html#sections).\n\nLet's use the `%post` section to install all of our requirements using\n`apt-get` and `pip3`.\n\n```\n%post\n    # Downloads the latest package lists (important).\n    apt-get update -y\n    # Runs apt-get while ensuring that there are no user prompts that would\n    # cause the build process to hang.\n    # python3-tk is required by matplotlib.\n    DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \\\n        python3 \\\n        python3-tk \\\n        python3-pip\n    # Reduce the size of the image by deleting the package lists we downloaded,\n    # which are no longer needed.\n    rm -rf /var/lib/apt/lists/*\n    # Install Python modules.\n    pip3 install torch numpy matplotlib\n```\n\nEach line defines a separate command (lines can be continued with a `\\`).\nUnlike normal shell scripts, the build will be aborted as soon as one of the\ncommands fails. You do not need to connect the commands with `\u0026\u0026`.\n\nThe final build definition is in the file\n[version-1.def](examples/xor/version-1.def).\n\n## Building an Image\n\nSupposing we are on our own Ubuntu machine, we can build this definition into\na `.sif` image file using the following command:\n\n```bash\ncd examples/xor\nsudo singularity build version-1.sif version-1.def\n```\n\n[View the screencast](https://bdusell.github.io/singularity-tutorial/casts/version-1.html)\n\nThis ran the commands we defined in the `%post` section inside a container and\nafterwards saved the state of the container in the image `version-1.sif`.\n\n## Running an Image\n\nLet's run our PyTorch program in a container based on the image we just built.\n\n```bash\nsingularity exec version-1.sif python3 train_xor.py --output model.pt\n```\n\nThis program does not take long to run. Once it finishes, it should open a\nwindow with a plot of the model's loss and accuracy over time.\n\n[![asciicast](https://asciinema.org/a/Lqq0AsJSwVgFoo1Hr8S7euMe5.svg)](https://asciinema.org/a/Lqq0AsJSwVgFoo1Hr8S7euMe5)\n\n![Plot](images/plot.png)\n\nThe trained model should also be saved in the file `model.pt`. Note that even\nthough the program ran in a container, it was able to write a file to the host\nfile system that remained after the program exited and the container was shut\ndown. If you are familiar with Docker, you probably know that you cannot write\nfiles to the host in this way unless you explicitly **bind mount** two\ndirectories in the host and container file system. Bind mounting makes a file\nor directory on the host system synonymous with one in the container.\n\nFor convenience, Singularity\n[binds a few important directories by\ndefault](https://www.sylabs.io/guides/3.0/user-guide/bind_paths_and_mounts.html):\n\n* Your home directory\n* The current working directory\n* `/tmp`\n* `/proc`\n* `/sys`\n* `/dev`\n\nYou can add to or override these settings if you wish using the\n[`--bind` flag](https://www.sylabs.io/guides/3.0/user-guide/bind_paths_and_mounts.html#specifying-bind-paths)\nto `singularity exec`. This is important to remember if you want to access a\nfile that is outside of your home directory on the CRC -- otherwise you may end\nup with cryptic persmission errors.\n\nIt is also important to know that, unlike Docker, environment variables are\ninherited inside the container for convenience.\n\n## Running an Interactive Shell\n\nYou can also open up a shell inside the container and run commands there. You\ncan `exit` when you're done. Note that since your home directory is\nbind-mounted, the shell inside the container will run your shell's startup file\n(e.g. `.bashrc`).\n\n```\n$ singularity shell version-1.sif\nSingularity version-1.sif:~/singularity-tutorial/examples/xor\u003e python3 train_xor.py\n```\n\n## Running an Image on the CRC\n\nLet's try running the same image on the CRC. Log in to one of the frontends\nusing `ssh -X`. The `-X` is necessary to get the plot to appear.\n\n```bash\nssh -X yournetid@crcfe01.crc.nd.edu\n```\n\nThen download the image to your `/scratch365` directory. One gotcha is that the\n`.sif` file *must* be stored on the scratch365 device for Singularity to work.\n\n```bash\nsingularity pull /scratch365/$USER/version-1.sif library://brian/default/singularity-tutorial:version-1\n```\n\nNext, download the code to your home directory.\n\n```bash\ngit clone https://github.com/bdusell/singularity-tutorial.git ~/singularity-tutorial\n```\n\nRun the program.\n\n```bash\ncd ~/singularity-tutorial/examples/xor\nsingularity exec /scratch365/$USER/version-1.sif python3 examples/xor/train_xor.py\n```\n\nYou should get the same plot from before to show up. Note that it is not\npossible to do this using the Python installations provided by the CRC, since\nthey do not include Tk, which is required by matplotlib. I have found this\nextremely useful for making plots from data I have stored on the CRC without\nneeding to download the data to another machine.\n\n## A Beefier PyTorch Program\n\nAs an example of a program that benefits from GPU acceleration, we will be\nrunning the official\n[`word_language_model`](https://github.com/pytorch/examples/tree/master/word_language_model)\nexample PyTorch program, which I have included at\n[`examples/language-model`](examples/language-model).\nThis program trains an\n[LSTM](https://en.wikipedia.org/wiki/Long_short-term_memory)\n[language model](https://en.wikipedia.org/wiki/Language_model)\non a corpus of Wikipedia text.\n\n## Adding GPU Support\n\nIn order to add GPU support, we need to include CUDA in our image. In\nSingularity, this is delightfully simple. We just need to pick one of\n[Nvidia's official Docker images](https://hub.docker.com/r/nvidia/cuda)\nto base our image on. Again, the easiest way to install library X is often to\nGoogle \"X docker\" and pick an image from the README or tags page on Docker Hub.\n\nThe README lists several tags. They tend to indicate variants of the image that\nhave different components and different versions of things installed. Let's\npick the one that is based on CUDA 10.1, uses Ubuntu 18.04, and includes cuDNN\n(which PyTorch can leverage for highly optimized neural network operations).\nLet's also pick the `devel` version, since PyTorch needs to compile itself in\nthe container. This is the image tagged `10.1-cudnn7-devel-ubuntu18.04`. Since\nthe image comes from the nvidia/cuda repository, the full image name is\n`nvidia/cuda:10.1-cudnn7-devel-ubuntu18.04`.\n\nOur definition file now looks\n[like this](examples/language-model/version-2.def). Although we don't need it,\nI kept matplotlib for good measure.\n\n```\nBootstrap: docker\nFrom: nvidia/cuda:10.1-cudnn7-devel-ubuntu18.04\n\n%post\n    # Downloads the latest package lists (important).\n    apt-get update -y\n    # Runs apt-get while ensuring that there are no user prompts that would\n    # cause the build process to hang.\n    # python3-tk is required by matplotlib.\n    DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \\\n        python3 \\\n        python3-tk \\\n        python3-pip\n    # Reduce the size of the image by deleting the package lists we downloaded,\n    # which are useless now.\n    rm -rf /var/lib/apt/lists/*\n    # Install Python modules.\n    pip3 install torch numpy matplotlib\n```\n\nWe build the image as usual.\n\n```bash\ncd examples/language-model\nsudo singularity build version-2.sif version-2.def\n```\n\n[View the screencast](https://bdusell.github.io/singularity-tutorial/casts/version-2.html)\n\nWe run the image like before, except that we have to add the `--nv` flag to\nallow the container to access the Nvidia drivers on the host in order to use\nthe GPU. That's all we need to get GPU support working. Not bad!\n\nThis program takes a while to run. Do not run it on the CRC frontend. When I\nrun one epoch on my workstation (which has a GPU), the output looks like this:\n\n```\n$ singularity exec --nv version-2.sif python3 main.py --cuda --epochs 1\n| epoch   1 |   200/ 2983 batches | lr 20.00 | ms/batch 45.68 | loss  7.63 | ppl  2050.44\n| epoch   1 |   400/ 2983 batches | lr 20.00 | ms/batch 45.11 | loss  6.85 | ppl   945.93\n| epoch   1 |   600/ 2983 batches | lr 20.00 | ms/batch 45.03 | loss  6.48 | ppl   653.61\n| epoch   1 |   800/ 2983 batches | lr 20.00 | ms/batch 46.43 | loss  6.29 | ppl   541.05\n| epoch   1 |  1000/ 2983 batches | lr 20.00 | ms/batch 45.50 | loss  6.14 | ppl   464.91\n| epoch   1 |  1200/ 2983 batches | lr 20.00 | ms/batch 44.99 | loss  6.06 | ppl   429.36\n| epoch   1 |  1400/ 2983 batches | lr 20.00 | ms/batch 45.27 | loss  5.95 | ppl   382.01\n| epoch   1 |  1600/ 2983 batches | lr 20.00 | ms/batch 45.09 | loss  5.95 | ppl   382.31\n| epoch   1 |  1800/ 2983 batches | lr 20.00 | ms/batch 45.25 | loss  5.80 | ppl   330.43\n| epoch   1 |  2000/ 2983 batches | lr 20.00 | ms/batch 45.08 | loss  5.78 | ppl   324.42\n| epoch   1 |  2200/ 2983 batches | lr 20.00 | ms/batch 45.11 | loss  5.66 | ppl   288.16\n| epoch   1 |  2400/ 2983 batches | lr 20.00 | ms/batch 45.14 | loss  5.67 | ppl   291.00\n| epoch   1 |  2600/ 2983 batches | lr 20.00 | ms/batch 45.21 | loss  5.66 | ppl   287.51\n| epoch   1 |  2800/ 2983 batches | lr 20.00 | ms/batch 45.02 | loss  5.54 | ppl   255.68\n-----------------------------------------------------------------------------------------\n| end of epoch   1 | time: 140.54s | valid loss  5.54 | valid ppl   254.69\n-----------------------------------------------------------------------------------------\n=========================================================================================\n| End of training | test loss  5.46 | test ppl   235.49\n=========================================================================================\n```\n\n## Running a GPU Program on the CRC\n\nFinally, I will show you how to run this program on the CRC's GPU queue.\nThis image is too big to be hosted on the Singularity Library, so you need to\ncopy it from my home directory. We will address this size issue later on.\n\n```bash\ncp /afs/crc.nd.edu/user/b/bdusell1/Public/singularity-tutorial/version-2.sif /scratch365/$USER/version-2.sif\n```\n\nThen, submit a job to run this program on the GPU queue. For convenience, I've\nincluded a script to run the submission command.\n\n```bash\ncd ~/singularity-tutorial/examples/language-model\nbash submit-gpu-job-version-2.bash\n```\n\nCheck back in a while to verify that the job completed successfully. The output\nwill be written to `output-version-2.txt`.\n\nSomething you should keep in mind is that, by default, if there are multiple\nGPUs available on the system, PyTorch grabs the first one it sees (some\ntoolkits grab all of them). However, the CRC assigns each job its own GPU,\nwhich is not necessarily the one that PyTorch would pick. If PyTorch does not\nrespect this assignment, there can be contention among different jobs. You can\ncontrol which GPUs PyTorch has access to using the environment variable\n`CUDA_VISIBLE_DEVICES`, which can be set to a space-separated list of numbers.\nThe CRC now sets this environment variable automatically, and since Singularity\ninherits environment variables, you actually don't need to do anything. It's\njust something you should know about, since there is potential for abuse.\n\n## Separating Python modules from the image\n\nNow that you know the basics of how to run a GPU job on the CRC, here's a tip\nfor managing Python modules. There's a problem with our current workflow.\nEvery time we want to install a new Python library, we have to re-build the\nimage. We should only need to re-build the image when we install a package with\n`apt-get` or inherit from a different base image -- in other words, actions\nthat require root privileges. It would be nice if we could store our Python\nlibraries in the current working directory using a **package manager**, and\nrely on the image only for the basic Ubuntu/CUDA/Python environment.\n\n[Pipenv](https://github.com/pypa/pipenv) is a package manager for Python. It's\nlike the Python equivalent of npm (Node.js package manager) or gem (Ruby package\nmanager). It keeps track of the libraries your project depends on in text files\nnamed `Pipfile` and `Pipfile.lock`, which you can commit to version control in\nlieu of the massive libraries themselves. Every time you run `pipenv install\n\u003clibrary\u003e`, Pipenv will update the `Pipfile` and download the library locally.\nThe important thing is that, rather than putting the library in a system-wide\nlocation, Pipenv installs the library in a *local* directory called `.venv`.\nThe benefit of this is that the libraries are stored *with* your project, but\nthey are not part of the image. The image is merely the vehicle for running\nthem.\n\nHere is the\n[new version](examples/language-model/version-3.def)\nof our definition file:\n\n```\nBootStrap: docker\nFrom: nvidia/cuda:10.1-cudnn7-devel-ubuntu18.04\n\n%post\n    # Downloads the latest package lists (important).\n    apt-get update -y\n    # Runs apt-get while ensuring that there are no user prompts that would\n    # cause the build process to hang.\n    # python3-tk is required by matplotlib.\n    # python3-dev is needed to install some packages.\n    DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \\\n        python3 \\\n        python3-tk \\\n        python3-pip \\\n        python3-dev\n    # Reduce the size of the image by deleting the package lists we downloaded,\n    # which are useless now.\n    rm -rf /var/lib/apt/lists/*\n    # Install Pipenv.\n    pip3 install pipenv\n\n%environment\n    # Pipenv requires a certain terminal encoding.\n    export LANG=C.UTF-8\n    export LC_ALL=C.UTF-8\n    # This configures Pipenv to store the packages in the current working\n    # directory.\n    export PIPENV_VENV_IN_PROJECT=1\n```\n\nOn the CRC, download the new image.\n\n```bash\nsingularity pull /scratch365/$USER/version-3.sif library://brian/default/singularity-tutorial:version-3\n```\n\nNow we can use the container to install our Python libraries into the current\nworking directory. We do this by running `pipenv install`.\n\n```bash\nsingularity exec /scratch365/$USER/version-3.sif pipenv install torch numpy matplotlib\n```\n\n[![asciicast](https://asciinema.org/a/cywx1Ta3XpO89DvwaE0MaogDo.svg)](https://asciinema.org/a/cywx1Ta3XpO89DvwaE0MaogDo)\n\nThis may take a while. When it is finished, it will have installed the\nlibraries in a directory named `.venv`. The benefit of installing packages like\nthis is that you can install new ones without re-building the image, and you\ncan re-use the image for multiple projects. The `.sif` file is smaller too.\n\nWhen you're done, you can test it out by submitting a GPU job. If you look at\nthe script, you will see that we replace the `python3` command with `pipenv run\npython`, which runs the program inside the environment that Pipenv manages.\n\n```bash\nbash submit-gpu-job-version-3.bash\n```\n\n## Docker\n\nIf this container stuff interests you, you might be interested in\n[Docker](https://www.docker.com/)\ntoo. Docker is not available on the CRC, but it may prove useful elsewhere.\nFor example, I've used it to compile PyTorch from source before. Docker has\nits own set of idiosyncrasies, but a good place to start is the\n[Docker documentation](https://docs.docker.com/).\n\nThis would be a good time to plug my\n[dockerdev](https://github.com/bdusell/dockerdev)\nproject, which is a bash library that sets up a streamlined workflow for using\nDocker containers as development environments.\n\n## Conclusion\n\nBy now I think I have shown you that the sky is the limit when it comes to\ncontainers. Hopefully this will prove useful to your research. If you like, you\ncan show your appreciation by leaving a star on GitHub. :)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbdusell%2Fsingularity-tutorial","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbdusell%2Fsingularity-tutorial","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbdusell%2Fsingularity-tutorial/lists"}