{"id":30358088,"url":"https://github.com/obeone/multi-registry-cache","last_synced_at":"2026-01-18T03:17:18.016Z","repository":{"id":206310205,"uuid":"716139145","full_name":"obeone/multi-registry-cache","owner":"obeone","description":"Multi-registry cache for Docker/containerd, optimizing image pulls using the original registry image","archived":false,"fork":false,"pushed_at":"2025-06-08T07:41:15.000Z","size":80,"stargazers_count":31,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-19T09:39:58.693Z","etag":null,"topics":["docker","docker-compose","docker-registry-cache","docker-registry-v2","registry","registry-server"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/obeone.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-11-08T14:19:38.000Z","updated_at":"2025-08-03T03:54:24.000Z","dependencies_parsed_at":null,"dependency_job_id":"b6071b21-b040-4f6f-982b-2c8d3e512383","html_url":"https://github.com/obeone/multi-registry-cache","commit_stats":{"total_commits":22,"total_committers":1,"mean_commits":22.0,"dds":0.0,"last_synced_commit":"00933397419d7106bbf7c999d38e0d2df4f93dee"},"previous_names":["obeone/multi-registry-cache"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/obeone/multi-registry-cache","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/obeone%2Fmulti-registry-cache","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/obeone%2Fmulti-registry-cache/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/obeone%2Fmulti-registry-cache/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/obeone%2Fmulti-registry-cache/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/obeone","download_url":"https://codeload.github.com/obeone/multi-registry-cache/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/obeone%2Fmulti-registry-cache/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28528034,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-18T00:39:45.795Z","status":"online","status_checked_at":"2026-01-18T02:00:07.578Z","response_time":98,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","docker-compose","docker-registry-cache","docker-registry-v2","registry","registry-server"],"created_at":"2025-08-19T09:03:02.113Z","updated_at":"2026-01-18T03:17:17.997Z","avatar_url":"https://github.com/obeone.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Multi-Registry Pull Through Cache Setup Guide 🚀\n\nWelcome to the Multi-Registry Pull Through Cache Setup Guide! This project is designed to help you create an efficient, bandwidth-saving local mirror for Docker Hub images and other container registries, including private registries. By setting up a pull-through cache, you can significantly reduce internet traffic and improve the speed of image pulls for your containerized environments. 🌐💨\n\nThis script will generate a `compose.yaml` file (same as `docker-compose.yml`). It will include one registry service for each registry mirror you wish to set up. A Traefik Proxy will be placed as a frontend for routing and providing a TLS endpoint. Additionally, a Redis service will be provided to enhance performance.\n\n## Features ✨\n\n- **Multi-Registry Support**: Set up mirrors for Docker Hub, GitHub Container Registry (GHCR), or any other container registries.\n- **Private Registry Integration**: Easily add a private registry to push and pull images from your private repositories.\n- **Bandwidth Optimization**: Save bandwidth by caching container images locally, reducing repeated downloads.\n- **Speed Improvements**: Expedite image pulls within your network, leading to faster deployments and scaling.\n- **Docker and Kubernetes Compatibility**: Seamlessly integrates with Docker and Kubernetes environments.\n- **Advanced Configuration**: Customize registry settings, including TTL and credentials, with a user-friendly setup.\n- **Simple Deployment**: Leverage Docker Compose for a straightforward setup and management of services.\n- **Traefik Proxy Integration**: Use Traefik for intelligent routing and secure TLS endpoints out of the box.\n- **Performance Enhancements**: Utilize Redis to improve caching operations and reduce latency.\n- **Environment-Friendly**: Lower external data transfer, contributing to a reduced carbon footprint.\n\n## Table of Contents 📚\n\n- [Multi-Registry Pull Through Cache Setup Guide 🚀](#multi-registry-pull-through-cache-setup-guide-)\n  - [Features ✨](#features-)\n  - [Table of Contents 📚](#table-of-contents-)\n  - [Purpose of the Project 🎯](#purpose-of-the-project-)\n  - [How to Set Up the Project 🛠️](#how-to-set-up-the-project-️)\n    - [Prerequisites](#prerequisites)\n    - [Running with Docker](#running-with-docker)\n      - [Run the Setup Script](#run-the-setup-script)\n      - [Run the Generate Script](#run-the-generate-script)\n    - [Running python files](#running-python-files)\n      - [Step-by-Step Setup](#step-by-step-setup)\n  - [Configuring Container Runtimes 🔄](#configuring-container-runtimes-)\n    - [containerd Configuration](#containerd-configuration)\n      - [nerdctl Configuration](#nerdctl-configuration)\n    - [dockerd Configuration](#dockerd-configuration)\n    - [Kubernetes Clusters (k3s, RKE, RKE2 etc)](#kubernetes-clusters-k3s-rke-rke2-etc)\n      - [k3s / RKE2](#k3s--rke2)\n      - [Other distributions](#other-distributions)\n  - [Conclusion 🎉](#conclusion-)\n\n## Purpose of the Project 🎯\n\nThe primary goal of this project is to establish a local caching service that acts as an intermediary between your Docker daemons and public container image registries. This setup is perfect for environments with multiple instances of Docker or Kubernetes clusters, where each node pulling images separately can lead to unnecessary bandwidth consumption and latency. 🐳🔁\n\nBy using a pull-through cache, you can:\n\n- Minimize external bandwidth usage\n- Accelerate image pull times\n- Reduce the load on public registries\n- Ensure consistent availability of images within your network\n- Reduce your carbon footprint\n\n## How to Set Up the Project 🛠️\n\n### Prerequisites\n\n- Docker installed on your host machine\n- `containerd` and/or `dockerd` running on your nodes\n- Access to the internet to pull initial images\n- Basic knowledge of Docker, Kubernetes, and container registries\n\n### Running with Docker\n\n#### Run the Setup Script\n\nTo execute the `setup.py` script inside the Docker container:\n\n```bash\ntouch config.yaml\ndocker run --rm -ti -v \"./config.yaml:/app/config.yaml\" obeoneorg/multi-registry-cache setup \n```\n\n#### Run the Generate Script\n\nTo trigger the `generate.py` script:\n\n```bash\ndocker run --rm -ti -v \"./config.yaml:/app/config.yaml\" -v \"./compose:/app/compose\" obeoneorg/multi-registry-cache generate\n```\n\n### Running python files\n\n#### Step-by-Step Setup\n\n1. **Clone the Repository**\n\n   ```bash\n   git clone https://github.com/obeone/multi-registry-cache.git\n   cd multi-registry-cache\n   ```\n\n2. **Set Up a Virtual Environment (Optional)**\n   You may choose to create a virtual environment to avoid affecting your global Python package setup.\n\n   ```bash\n   python3 -m venv .venv\n   source .venv/bin/activate\n   ```\n\n3. **Install Dependencies**\n   Install the required packages using pip.\n\n   ```bash\n   pip install -r requirements.txt\n   ```\n\n4. **Run the User-Friendly Setup**\n   Execute the setup script to configure your registries and generate necessary files.\n\n   ```bash\n   python setup.py\n   ```\n\n5. **Review Advanced Configuration**\n   After running the setup script, you should manually review the `config.yaml` file for advanced configurations such as TLS settings, Let's Encrypt, addition of private registries, and more. More details about this file in [CONFIG.md](CONFIG.md)\n\n   ```bash\n   nano config.yaml # or use your preferred text editor\n   ```\n\n6. **Generate Configuration Files**\n   Run the generate script to create the necessary configuration files for your setup.\n\n   ```bash\n   python generate.py\n   ```\n\n7. **Start Your Services**\n   Use Docker Compose to start your registry mirrors and the Traefik reverse proxy.\n\n   ```bash\n   cd compose\n   docker compose up -d\n   ```\n\n## Configuring Container Runtimes 🔄\n\n### containerd Configuration\n\nFor `containerd`, you'll need to modify the `config.toml` file to specify the registry mirrors.\n\n```toml\n[plugins.\"io.containerd.grpc.v1.cri\".registry.mirrors]\n  [plugins.\"io.containerd.grpc.v1.cri\".registry.mirrors.\"docker.io\"]\n    endpoint = [\"https://dockerhub.registry-cache.example.net\"]\n  [plugins.\"io.containerd.grpc.v1.cri\".registry.mirrors.\"ghcr.io\"]\n    endpoint = [\"https://ghcr.registry-cache.example.net\"]\n```\n\nAfter updating the configuration, restart `containerd`:\n\n```bash\nsudo systemctl restart containerd\n```\n\n#### nerdctl Configuration\n\nFor `nerdctl`, you'll need to create a directory per registry mirror and push content into a file:\n\n```bash\nmkdir -p /etc/containerd/certs.d/docker.io/\n```\n\nAnd create a file `/etc/containerd/certs.d/docker.io/hosts.toml` with the following content:\n\n```toml\nserver = \"https://docker.io\"\n\n[host.\"https://dockerhub.registry-cache.example.net\"]\n  capabilities = [\"pull\", \"resolve\"]\n```\n\n(Same principle in rootless mode, just modify user config)\n\n### dockerd Configuration\n\nFor `dockerd`, you can only configure a single mirror for Docker Hub. Update the `/etc/docker/daemon.json` file with the following:\n\n```json\n{\n  \"registry-mirrors\": [\"https://dockerhub.registry-cache.example.net\"]\n}\n```\n\nReload the Docker daemon to apply the changes:\n\n```bash\nsudo systemctl daemon-reload\nsudo systemctl restart docker\n```\n\n### Kubernetes Clusters (k3s, RKE, RKE2 etc)\n\n#### k3s / RKE2\n\nEdit file `/etc/rancher/(k3s|rke2)/registries.yaml` and add:\n\n```yaml\nmirrors:\n  docker.io:\n    endpoint:\n      - https://dockerhub.registry-cache.example.net\n  ghcr.io:\n    endpoint:\n      - https://ghcr.registry-cache.example.net\n```\n\n#### Other distributions\n\nFor Kubernetes clusters, you'll probably need to configure each node's container runtime to use the registry mirror. Refer to the specific documentation of your distribution for details on how to apply registry mirror configurations.\n\n## Conclusion 🎉\n\nCongratulations! You've now set up a multi-registry pull-through cache that will serve as a local mirror for your container images, including private registry configurations. Enjoy faster image pulls, reduced external bandwidth, and a more resilient container environment!\n\n---\n\nFeel free to contribute to this project by submitting issues or pull requests. For questions or support, open an issue on the project's GitHub page. Happy caching! 🐋💾\n\n---\n\n## License\n\nThis project is licensed under the [MIT License](LICENSE).\n\n**Keywords**: Docker, container registry, pull-through cache, Docker Hub mirror, private registry, containerd, dockerd, Kubernetes, k3s, RKE, RKE2, image caching, setup guide, local mirror, container image optimization.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fobeone%2Fmulti-registry-cache","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fobeone%2Fmulti-registry-cache","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fobeone%2Fmulti-registry-cache/lists"}