{"id":29041858,"url":"https://github.com/abhans/archdev","last_synced_at":"2026-05-07T06:32:43.101Z","repository":{"id":295308684,"uuid":"980811261","full_name":"abhans/archdev","owner":"abhans","description":"Container that is built with Arch Linux with NVIDIA Driver \u0026 CUDA support, PyTorch and TensorFlow built in.","archived":false,"fork":false,"pushed_at":"2025-06-25T16:43:55.000Z","size":1267,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-25T17:42:13.034Z","etag":null,"topics":["archlinux","container","cuda","docker"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc0-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/abhans.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-09T18:55:22.000Z","updated_at":"2025-06-04T05:12:13.000Z","dependencies_parsed_at":"2025-05-24T20:31:09.388Z","dependency_job_id":"914f28d6-015c-4ea1-bfd5-bb879e70833c","html_url":"https://github.com/abhans/archdev","commit_stats":null,"previous_names":["abhans/archdev"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/abhans/archdev","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abhans%2Farchdev","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abhans%2Farchdev/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abhans%2Farchdev/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abhans%2Farchdev/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/abhans","download_url":"https://codeload.github.com/abhans/archdev/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abhans%2Farchdev/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262090174,"owners_count":23257124,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["archlinux","container","cuda","docker"],"created_at":"2025-06-26T15:02:55.302Z","updated_at":"2026-05-07T06:32:43.094Z","avatar_url":"https://github.com/abhans.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Arch Linux Development Environment\r\n\r\nThis setup utilizes Docker containers to setup Arch Linux with CUDA drivers.\r\n\r\nIt also comes with the `uv` **package manager** with a virtual environment setup, configured with **popular deep learning frameworks** such as `PyTorch`, `TensorFlow` and other useful libraries, one of which is `OpenCV`.\r\n\r\n## The `Dockerfile`\r\n\r\n`Dockerfile` is created to store the building process of an image, gives the ability to customize if necessary in future developments.\r\n\r\nThis setup consists of 6 main steps:\r\n\r\n1. **Setting Up the Build Arguments**\r\n2. **Initialization of Arch Linux**\r\n3. **Configuration of the User**\r\n4. **Installation of Packages**\r\n5. **Setting up CUDA \u0026 Drivers**\r\n6. **Setting Up the Environment**\r\n\r\n### 1. Setting Up the Build Arguments\r\n\r\nThe latest `archlinux` image is utilized as the base image:\r\n\r\n```Dockerfile\r\n# Arch Linux Development Environment\r\n#   - Fastfetch (at the start of each bash session)\r\n#   - Tensorflow \u0026 CUDA\r\n#   - NVIDIA Drivers\r\n#   - Python and C++ support\r\nFROM archlinux:latest\r\n```\r\n\r\n\u003e It can be pinned with its digest, which is exposed with:\r\n\u003e\r\n\u003e ```pwsh\r\n\u003e docker pull archlinux:latest\r\n\u003e docker images archlinux:latest --format '{{.Digest}}'\r\n\u003e ```\r\n\r\nTo simplify the rest of the building process, build arguments are created. This ensures a proper initialization of the environment while also making it configurable:\r\n\r\n```Dockerfile\r\n# ----------------------- BUILD ARGS ------------------------\r\n# Set the user and group IDs for the container\r\n#   This allows the container to run with the same user and group IDs as the host system\r\n#   This is useful for avoiding permission issues when mounting volumes\r\n\r\nARG USER=hans\r\nARG GUID=1000\r\nARG UID=${GUID}\r\n# Environment variables for the user\r\nENV HOME=/home/${USER}\r\nENV VENV_DIR=${HOME}/.base\r\n# Development directory\r\nARG DEV=${HOME}/.dev/\r\n```\r\n\r\n### 2. Initialization of Arch Linux\r\n\r\nFirst, the Arch Linux should be initialized with generating signature keys. System is updated and missing dependencies are installed:\r\n\r\n\u003e Locales are also generated in this step to be `en_US.UTF-8`.\r\n\r\n```Dockerfile\r\n# ------------------------ ARCH LINUX INIT \u0026 USER CONFIG ------------------------\r\n# Initialize Arch Linux\r\n#  This includes setting up the package manager, locale, and user permissions\r\n# Set the user and group IDs for the container\r\n#  This allows the container to run with the same user and group IDs as the host system\r\n\r\n# -------------------------------- [  R O O T  ] --------------------------------\r\nUSER root\r\n\r\n# Initialize Arch Linux\r\nRUN pacman-key --init \\\r\n    \u0026\u0026 pacman -Sy --noconfirm sudo \\\r\n    \u0026\u0026 pacman-key --populate archlinux \\\r\n    \u0026\u0026 pacman --needed --noconfirm -Syu \\\r\n    # Generate en_US.UTF-8 locale\r\n    \u0026\u0026 sed -i 's/#en_US.UTF-8/en_US.UTF-8/' /etc/locale.gen \\\r\n    \u0026\u0026 locale-gen \\\r\n    \u0026\u0026 pacman -Scc --noconfirm \\\r\n```\r\n\r\n### 3. Configuration of the User\r\n\r\nA new user is created, using the build arguments. The \"home\" directory is created and its permissions are configured:\r\n\r\n\u003e Created user is also set as a \"sudoer\".\r\n\r\n```Dockerfile\r\n    # Create a new user\r\n    \u0026\u0026 useradd --create-home --shell /bin/bash ${USER} \\\r\n    \u0026\u0026 usermod -aG wheel ${USER} \\\r\n    # Setting the user as a \"sudoer\"\r\n    \u0026\u0026 sed -i 's/^# %wheel/%wheel/' /etc/sudoers \\\r\n    \u0026\u0026 mkdir -p ${HOME}/.local \\\r\n    \u0026\u0026 mkdir -p ${HOME}/.cache \\\r\n    # Fix permissions for the home directory\r\n    \u0026\u0026 chown -R ${USER}:${USER} ${HOME} \\\r\n```\r\n\r\n### 4. Installation of Packages\r\n\r\nCUDA, drivers and other related packages (e.g. fastfetch, openssh, git, curl etc.) are installed. For managing Python, `uv` package manager is installed:\r\n\r\n```Dockerfile\r\n    # ------------------------ INSTALLATION (Python \u0026 Packages) ------------------------\r\n    # Install base development tools, essentials and CUDA\r\n    #  This installs essential development tools such as Git, GCC, Make, and CMake\r\n    \u0026\u0026 pacman -Sy --needed --noconfirm cmake gcc make fastfetch openssh unzip curl git vi nvim jq \\\r\n    \u0026\u0026 pacman -Sy --noconfirm nvidia cuda cudnn nccl \\\r\n    \u0026\u0026 pacman -S --noconfirm nvidia-container-toolkit opencl-nvidia \\\r\n    \u0026\u0026 pacman -Scc --noconfirm\r\n```\r\n\r\n### 5. Setting up CUDA \u0026 Drivers\r\n\r\nTo utilize GPU acceleration and parallel computation, **CUDA** must be set up and configured for deep learning frameworks such as TensorFlow, PyTorch etc.\r\n\r\n```Dockerfile\r\n# ------------------------ CUDA CONFIGURATION ------------------------\r\n# Configure CUDA for the container\r\n#  This includes setting up the CUDA toolkit and adding it to the PATH\r\n    \r\n# Add the CUDA folders to the PATH\r\n#   Adds CUDA binaries and libraries to environment variables\r\nENV PATH=/opt/cuda/bin${PATH:+:${PATH}}\r\nENV LD_LIBRARY_PATH=/usr/lib:/opt/cuda/lib64\r\n# Configure the Matplotlib temporary directory\r\nENV MPLCONFIGDIR=/tmp/matplotlib\r\n```\r\n\r\n\u003e - `/opt/cuda/bin` is appended to the system `PATH` to ensure that **CUDA command-line tools** (like `nvcc`, `cuda-gdb`, etc.) are available from \"anywhere\" in the container.\r\n\u003e - `/opt/cuda/lib64` is appended to the system `LD_LIBRARY_PATH` to ensure that at runtime, the **dynamic linker** can find CUDA’s **shared libraries** (like `libcudart.so`, `libcublas.so`, etc.), which are required by deep learning frameworks (TensorFlow, PyTorch, etc.) and CUDA-accelerated applications.\r\n\u003e - Lastly, `/tmp/matplotlib` is set as `MPLCONFIGDIR` which is the Matplotlib `config`  directory to **prevent permission errors** when Matplotlib tries to write config or cache files.\r\n\r\n#### 5.A What is CUDA?\r\n\r\n**CUDA** (Compute Unified Device Architecture) is a **parallel computing platform** and programming model developed by NVIDIA.\r\n\r\n- It allows developers to use NVIDIA GPUs for **general purpose processing (GPGPU)**, enabling significant acceleration for compute-intensive applications such as deep learning, scientific computing, and image processing.\r\n\r\n#### 5.B What are cuDNN, cuFFT, and cuBLAS?\r\n\r\n- **cuDNN**: NVIDIA **CUDA Deep Neural Network** library.\r\n  - Provides **highly optimized implementations for standard routines** such as forward and backward convolution, pooling, normalization, and activation layers for deep neural networks.\r\n- **cuFFT**: NVIDIA **CUDA Fast Fourier Transform** library.\r\n  - Delivers **GPU-accelerated FFT computations** for signal and image processing.\r\n- **cuBLAS**: NVIDIA **CUDA Basic Linear Algebra Subprograms** library.\r\n  - Offers **GPU-accelerated linear algebra operations**, such as matrix multiplication and vector operations.\r\n\r\nA detailed guide for setting up CUDA in Arch can be found at [[Arch Linux Wiki: GPGPU CUDA setup]](https://wiki.archlinux.org/title/GPGPU#CUDA)\r\n\r\n### 6. Setting Up the Environment\r\n\r\nFinally, the Python environment is set up and libraries including PyTorch, TensorFlow and OpenCV are installed. After the setup, the package data is stored as a `.lock` file for consistent builds in the future:\r\n\r\n```Dockerfile\r\n# ------------------------ ENVIRONMENT ------------------------\r\n# Sets up the environment for the container\r\n#  This includes setting a virtual environment, downlaoding packages, \r\n#  copying entrypoint scripts, and fixing permissions\r\n\r\n# Set the locale to UTF-8\r\nENV LANG=en_US.UTF-8\r\nENV LANGUAGE=en_US.UTF-8\r\n\r\n# Copy entrypoint bash script \u0026 change its' permission to executable\r\nCOPY entrypoint.sh /entrypoint.sh\r\nRUN chmod +x /entrypoint.sh \\\r\n    # Create bin directory\r\n    \u0026\u0026 mkdir -p ${HOME}/bin \\\r\n    \u0026\u0026 chown -R ${USER}:${USER} ${HOME}/bin\r\n```\r\n\r\n#### 6.1 Installing Oh My Posh\r\n\r\nTo improve the experience, popular prompt engine [[Oh My Posh]](https://ohmyposh.dev/) is installed and configured with a theme for the user. The theme file `theme.omp.json` is copied and engine is installed:\r\n\r\n```Dockerfile\r\n# Copy Oh-My-Posh theme\r\nCOPY theme.omp.json ${HOME}/bin/theme.omp.json\r\nRUN chown ${USER}:${USER} ${HOME}/bin/theme.omp.json\r\n\r\n# Copy project files to the home directory\r\nCOPY --chown=${USER}:${USER} . ${DEV}\r\n\r\n# -------------------------------- [  U S E R  ] --------------------------------\r\nUSER ${USER}\r\n\r\n# Installation \u0026 setup of Oh-My-Posh\r\nRUN curl -s https://ohmyposh.dev/install.sh | bash -s -- -d ${HOME}/bin \\\r\n    \u0026\u0026 echo 'eval \"$(oh-my-posh init bash --config $HOME/bin/theme.omp.json)\"' \u003e\u003e ${HOME}/.bashrc\r\n```\r\n\r\nThen, `uv` virtual environment `.base` is initiated and packages are installed.\r\n\r\n```Dockerfile\r\n# Set the working directory to the project directory\r\nWORKDIR ${DEV}\r\n\r\n# Append \".local/bin\" to PATH\r\n#   This ensures that binaries installed by `uv` (such as Python) are available \"system-wide\"\r\nENV PATH=\"${HOME}/.local/bin:${HOME}/bin:${PATH}\"\r\n\r\n# Install the \"uv\" package manager, Python 3.12 and create a virtual environment\r\nRUN curl -LsSf https://astral.sh/uv/install.sh | sh \\\r\n    \u0026\u0026 uv python install 3.12 \\\r\n    \u0026\u0026 uv venv --python 3.12 ${VENV_DIR} \\\r\n    \u0026\u0026 source ${VENV_DIR}/bin/activate \\\r\n    # Initialize a `uv` project (base)\r\n    \u0026\u0026 uv init --bare --python 3.12 -v -n base \\\r\n    \u0026\u0026 uv pip install --upgrade pip \\\r\n    \u0026\u0026 uv pip install --no-cache-dir torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu130 \\\r\n    \u0026\u0026 uv pip install --no-cache-dir -r requirements.txt \\\r\n    # Save the installed packages to a lock file\r\n    \u0026\u0026 uv pip compile requirements.txt -o uv.lock \\\r\n    # Clean uv cache\r\n    \u0026\u0026 uv cache clean \\\r\n    # Fixing NCLL links\r\n    \u0026\u0026 rm -rf ${VENV_DIR}/lib/python3.12/site-packages/torch/lib/../../nvidia/nccl \\\r\n    \u0026\u0026 ln -s /usr/lib/libnccl.so.2 ${VENV_DIR}/lib/python3.12/site-packages/torch/lib/libnccl.so.2\r\n```\r\n\r\n\u003e An entrypoint script `entrypoint.sh` is provided and used at startup. This script includes environment activation and runs the report `.py` script.\r\n\u003e\r\n\u003e ```Dockerfile\r\n\u003e # Fetching System information\r\n\u003e RUN echo \"fastfetch\" \u003e\u003e /home/${USER}/.bashrc\r\n\u003e\r\n\u003e # Ensure the container starts in the user's HOME directory\r\n\u003e WORKDIR ${HOME}\r\n\u003e\r\n\u003e ENTRYPOINT [\"/entrypoint.sh\"]\r\n\u003e ```\r\n\r\n## How to Use?\r\n\r\nAfter pulling the image, it can be tested with:\r\n\r\n```pwsh\r\ndocker run --gpus all -it --rm \"archlinux/latest:cuda\" bash\r\n```\r\n\r\nTo **locally store** the image, remove `--rm` flag and specify a **name** for the container:\r\n\r\n```pwsh\r\ndocker run --gpus all -it --name \u003cNAME\u003e \"archlinux/latest:cuda\" bash\r\n```\r\n\r\n### Configuring the Build Arguments\r\n\r\nYou can customize the Docker image by altering the build arguments defined in the `Dockerfile`. For example, you can change the default username, user ID, or group ID to fit your preferences or environment.\r\n\r\nTo specify custom values during the build process, use the `--build-arg` flag with `docker build`.\r\n\r\nFor example:\r\n\r\n```pwsh\r\ndocker build --build-arg USER=\u003cMY_USER_NAME\u003e -D -t \"archlinux/latest:cuda\" .\r\n```\r\n\r\nThis command sets the username to `MY_USER_NAME` in the resulting image.\r\n\r\n![Entrypoint for the launch](entrypoint.png)\r\n\r\n### Creating a User Password\r\n\r\nBy default, the user account created in the container does **not** have a password set.  \r\nIf you need to use `sudo` or perform administrative tasks inside the container, you must set a password for your user after the container starts.\r\n\r\nTo set a password, open a new terminal and run:\r\n\r\n```sh\r\ndocker exec -it \u003cNAME\u003e passwd \u003cusername\u003e\r\n```\r\n\r\nReplace `\u003cNAME\u003e` with the name of your running container and `\u003cusername\u003e` with your chosen username (default is `hans` unless changed via build arguments).\r\n\r\nYou will be prompted to enter and confirm a new password.  \r\nAfter this, you can use `sudo` as expected inside the container.\r\n\r\n\u003e For security, do **not** set a password in the Dockerfile or image itself.  \r\n\u003e Always set it at runtime as shown above.\r\n\r\n### Logging the Build Process\r\n\r\nFor curiosity and more verbose process, additional arguments can be provided to **log the process** to a `\u003cBUILD_LOG.log\u003e` file:\r\n\r\n```pwsh\r\ndocker build --build-arg USER=hans -D --progress=plain -t \"archlinux/latest:cuda\" . *\u003e \u003cBUILD_LOG.log\u003e\r\n```\r\n\r\nWhen this command is run, the build process is silent and instead logged to the `\u003cBUILD_LOG.log\u003e` file.\r\n\r\n## Reminder to User\r\n\r\nThe whole process **takes quite a long time (over 30 min)** and the resulting image is **very large (\u003e30 GB).** Currently thinking of an improvement on both areas.\r\n\r\n\u003c!--\r\n## ToDos\r\n\r\n- [x] Explain the  cuDNN, cuFFT and cuBLAS situation. Understand how it's related to the topic.\r\n- [ ] Initialize `uv` to the `.dev/` directory as a project\r\n  - Read more about projects [[here]](https://docs.astral.sh/uv/concepts/projects/)\r\n- [ ] Find a better way to check system, done by `run.py`\r\n  - Added `checkTorch` to `run.py`\r\n- [ ] Properly set up OpenCV to connect the camera.\r\n\r\n```bash\r\nuv init --bare --python 3.12 --no-cache -v\r\n```\r\n--\u003e","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabhans%2Farchdev","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fabhans%2Farchdev","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabhans%2Farchdev/lists"}