{"id":13717624,"url":"https://github.com/reproducible-containers/repro-get","last_synced_at":"2025-05-07T07:31:50.793Z","repository":{"id":61240932,"uuid":"549382695","full_name":"reproducible-containers/repro-get","owner":"reproducible-containers","description":"[Soft-deprecated] Reproducible apt/dnf/apk/pacman, with content-addressing","archived":true,"fork":false,"pushed_at":"2023-12-11T21:45:50.000Z","size":299,"stargazers_count":104,"open_issues_count":23,"forks_count":6,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-08-04T00:13:45.882Z","etag":null,"topics":["reproducible-builds"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/reproducible-containers.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-11T05:19:40.000Z","updated_at":"2024-07-29T10:15:40.000Z","dependencies_parsed_at":"2024-01-17T09:21:17.032Z","dependency_job_id":"264201a8-9c21-4bf3-95ab-91e6b09e2f91","html_url":"https://github.com/reproducible-containers/repro-get","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reproducible-containers%2Frepro-get","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reproducible-containers%2Frepro-get/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reproducible-containers%2Frepro-get/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reproducible-containers%2Frepro-get/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/reproducible-containers","download_url":"https://codeload.github.com/reproducible-containers/repro-get/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224573501,"owners_count":17333804,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["reproducible-builds"],"created_at":"2024-08-03T00:01:24.941Z","updated_at":"2024-11-14T05:32:17.843Z","avatar_url":"https://github.com/reproducible-containers.png","language":"Go","funding_links":[],"categories":["Point-of-use validations"],"sub_categories":["Vulnerability information exchange"],"readme":"# :warning: Soft-deprecated\n\n`repro-get` is soft-deprecated due to its complexity.\n\nConsider switching to the following alternatives:\n\n|Project                                                           |Cache location                           |Best for                             |\n|------------------------------------------------------------------|-----------------------------------------|-------------------------------------|\n|\u003chttps://github.com/reproducible-containers/repro-sources-list.sh\u003e|Distros' permanent snapshot servers (*\\1)|Debian, Ubuntu, ArchLinux            |\n|\u003chttps://github.com/reproducible-containers/repro-pkg-cache\u003e      |Your own permanent image registry        |Alpine, Fedora, Rocky, openSUSE, etc.|\n\n(\\*1): The packages can be also ephemerally cached on GitHub Actions to reduce loads on distros' snapshot servers.\nSee \u003chttps://github.com/reproducible-containers/buildkit-cache-dance\u003e.\n\n- - -\n[[⬇️ **Download]**](https://github.com/reproducible-containers/repro-get/releases)\n[[📖 **Quick start]**](#quick-start)\n[[❓**FAQs \u0026 Troubleshooting]**](#faqs)\n\n# `repro-get`: reproducible `apt`, `dnf`, `apk`, and `pacman`, with content-addressing\n\n✅ HTTP and HTTPS\n\n✅ Filesystems\n\n✅ OCI (Open Container Initiative) registries\n\n✅ IPFS\n\n`repro-get` installs a specific snapshot of packages using `SHA256SUMS`, for the sake of [reproducible builds](https://reproducible-builds.org/):\n```console\n$ cat SHA256SUMS-amd64\n35b1508eeee9c1dfba798c4c04304ef0f266990f936a51f165571edf53325cbc  pool/main/h/hello/hello_2.10-2_amd64.deb\n\n$ repro-get install SHA256SUMS-amd64\n(001/001) hello_2.10-2_amd64.deb Downloading from http://debian.notset.fr/snapshot/by-hash/SHA256/35b1508eeee9c1dfba798c4c04304ef0f266990f936a51f165571edf53325cbc\n...\nPreparing to unpack .../35b1508eeee9c1dfba798c4c04304ef0f266990f936a51f165571edf53325cbc ...\nUnpacking hello (2.10-2) ...\nSetting up hello (2.10-2) ...\n```\n\n`repro-get` supports the following distros:\n\n| Distro                  | \"Batteries included\" | Support generating Dockerfiles | Support verifying package signatures |\n| ----------------------- | -------------------- | ------------------------------ | ------------------------------------ |\n| `debian`                | ✅                   | ✅                             | [❌](https://github.com/reproducible-containers/repro-get/issues/10) |\n| `ubuntu`                | ✅                   | ❌                             | ❌                                   |\n| `fedora` (Experimental) | ✅                   | ❌                             | ✅                                   |\n| `alpine` (Experimental) | ❌                   | ❌                             | ✅                                   |\n| `arch`                  | ✅                   | ✅                             | ✅                                   |\n\n\u003cdetails\u003e\n\u003csummary\u003e \"Batteries included\" for Debian, Ubuntu, Fedora, and Arch Linux.\u003c/summary\u003e\n\n\u003cp\u003e\n\nOn Debian, the packages are fetched from the following URLs by default:\n- `http://deb.debian.org/debian/{{.Name}}` for recent packages (fast, but ephemeral)\n- `http://snapshot-cloudflare.debian.org/archive/debian/{{timeToDebianSnapshot .Epoch}}/{{.Name}}` for archived packages (slow, but persistent)\n\nOn Ubuntu: `http://launchpad.net/ubuntu/+archive/primary/+files/{{.Basename}}`\n\nOn Fedora: `https://kojipkgs.fedoraproject.org/packages/{{.Name}}`\n\nOn Arch Linux: `https://archive.archlinux.org/packages/{{.Name}}`\n\n\u003c/p\u003e\n\n\u003c/details\u003e\n\nOn other distros, the file provider has to be manually specified in the `--provider=...` flag for long-term persistence.\n\nThe following file providers are supported:\n- HTTP/HTTPS URLs, such as `http://debian.notset.fr/snapshot/by-hash/SHA256/{{.SHA256}}`\n- Filesystems, such as `file:///mnt/nfs/files/{{.Basename}}`, or `file:///mnt/nfs/blobs/{{.SHA256}}`\n- [OCI-compliant container registries](#container-registries), such as `oci://ghcr.io/USERNAME/REPO`\n- [IPFS](#ipfs) gateways, such as `http://ipfs.io/ipfs/{{.CID}}`\n\n- - -\n\u003c!-- START doctoc generated TOC please keep comment here to allow auto update --\u003e\n\u003c!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --\u003e\n\n- [Quick start](#quick-start)\n  - [Set up](#set-up)\n  - [Installing packages with the hash file](#installing-packages-with-the-hash-file)\n  - [Generating the hash file](#generating-the-hash-file)\n  - [Updating the hash file](#updating-the-hash-file)\n- [Advanced usage](#advanced-usage)\n  - [Dockerfile](#dockerfile)\n  - [Cache management](#cache-management)\n    - [Populate](#populate)\n    - [Export](#export)\n    - [Import](#import)\n    - [Clean](#clean)\n  - [Container registries](#container-registries)\n    - [Push](#push)\n    - [Pull](#pull)\n  - [IPFS](#ipfs)\n    - [Push](#push-1)\n    - [Pull](#pull-1)\n- [FAQs](#faqs)\n  - [Why do we need reproducibility?](#why-do-we-need-reproducibility)\n  - [Why not just use `snapshot.debian.org` with `apt-get`?](#why-not-just-use-snapshotdebianorg-with-apt-get)\n  - [Are container images \"bit-to-bit\" reproducible?](#are-container-images-bit-to-bit-reproducible)\n  - [Does this work with Ubuntu?](#does-this-work-with-ubuntu)\n  - [How to use HTTPS on Debian/Ubuntu?](#how-to-use-https-on-debianubuntu)\n  - [Why not use HTTPS by default on Debian/Ubuntu?](#why-not-use-https-by-default-on-debianubuntu)\n- [Acknowledgement](#acknowledgement)\n\n\u003c!-- END doctoc generated TOC please keep comment here to allow auto update --\u003e\n\n\n## Quick start\n\n### Set up\nDownload the latest binary release from https://github.com/reproducible-containers/repro-get/releases .\n\nTo install `repro-get` from source, install [Go](https://go.dev/dl/), run `make`, and `sudo make install`.\nThe recommended version of Go is written in the [`go.mod`](./go.mod) file.\n\nThe binary release can be reproduced locally by checking out the related tag and running `make artifacts.docker`.\n\n### Installing packages with the hash file\nCreate the `SHA256SUMS-amd64` file for the [`hello`](https://packages.debian.org/bullseye/amd64/hello/download) package,\nusing the information from `apt-cache show hello`:\n```\n35b1508eeee9c1dfba798c4c04304ef0f266990f936a51f165571edf53325cbc  pool/main/h/hello/hello_2.10-2_amd64.deb\n```\n\nThen run `repro-get install SHA256SUMS-amd64`:\n```console\n$ repro-get install SHA256SUMS-amd64\n(001/001) hello_2.10-2_amd64.deb Downloading from http://debian.notset.fr/snapshot/by-hash/SHA256/35b1508eeee9c1dfba798c4c04304ef0f266990f936a51f165571edf53325cbc\n...\nPreparing to unpack .../35b1508eeee9c1dfba798c4c04304ef0f266990f936a51f165571edf53325cbc ...\nUnpacking hello (2.10-2) ...\nSetting up hello (2.10-2) ...\n```\n\nSee also [Dockerfile](#dockerfile) for running `repro-get` inside containers.\n\n### Generating the hash file\n\u003e **Note**\n\u003e\n\u003e Make sure to run `apt-get update` before running `repro-get hash generate`.\n\u003e\n\u003e See also [Dockerfile](#dockerfile) for how to run `apt-get update` in a container image such as `debian:bullseye-yyyyMMdd`.\n\nTo generate the hash for all the installed packages, including the system packages:\n```bash\nrepro-get hash generate \u003eSHA256SUMS-amd64\n```\n\nTo generate the hash for specific packages:\n```bash\nrepro-get hash generate hello \u003eSHA256SUMS-amd64\n```\n\nTo generate the hash for newly installed packages:\n```bash\nrepro-get hash generate \u003eSHA256SUMS-amd64.old\napt-get install -y hello\nrepro-get hash generate --dedupe=SHA256SUMS-amd64.old \u003eSHA256SUMS-amd64\n```\n\n### Updating the hash file\n\u003e **Note**\n\u003e\n\u003e Make sure to run `apt-get update` before running `repro-get hash update`.\n\nTo update the hash file:\n```bash\nrepro-get hash update SHA256SUMS-amd64\n```\n\n## Advanced usage\n\n### Dockerfile\n\u003e **Warning**\n\u003e\n\u003e `repro-get dockerfile generate` is an experimental feature.\n\nThe following example produces an image with `gcc`, using the packages from 2021-12-20.\n```bash\n# Generate \"Dockerfile.generate-hash\" and \"Dockerfile\" in the current directory\nrepro-get --distro=debian dockerfile generate . debian:bullseye-20211220 gcc build-essential\n\n Enable BuildKit\nexport DOCKER_BUILDKIT=1\n\n# Generate \"SHA256SUMS-amd64\" file in the current directory (needed by the next step)\ndocker build --output . -f Dockerfile.generate-hash .\n\n# Build the image\ndocker build .\n```\n\nSee [`./examples/gcc`](./examples/gcc) for an example output.\n\nSee also [FAQs](#faqs) for \"bit-to-bit\" reproducibility of container images.\n\n### Cache management\nThe cache directory (`--cache`) defaults to `/var/cache/repro-get`.\n\n#### Populate\nTo populate the package files into the cache without installing them:\n```bash\nrepro-get download SHA256SUMS-amd64\n```\n\n#### Export\nTo export the cached package files to the current directory:\n```bash\nrepro-get cache export .\n```\n\n#### Import\nTo import package files in the current directory into the cache:\n```bash\nrepro-get cache import .\n```\n\n#### Clean\nTo clean the cache:\n```bash\nrepro-get cache clean\n```\n\n### Container registries\n\n`repro-get` supports downloading package files from [OCI](https://github.com/opencontainers/distribution-spec)-compliant container registries.\n\n\u003e **Note**\n\u003e\n\u003e Make sure to create a container registry credential as `~/.docker/config.json` .\n\u003e\n\u003e - [GitHub Container Registry (`ghcr.io`)](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry)\n\u003e - [Others](https://github.com/containerd/nerdctl/blob/master/docs/registry.md#using-managed-registry-services)\n\n#### Push\nTo push the package files into a container registry such as https://ghcr.io/ , use [ORAS](https://oras.land/cli/):\n```bash\nrepro-get cache export .\noras push ghcr.io/USERNAME/dpkgs:latest *.deb\n```\n\n#### Pull\nTo pull and install packages from the registry:\n```bash\nrepro-get --provider=oci://ghcr.io/USERNAME/dpkgs install SHA256SUMS-amd64\n```\n\nTips about the `oci://...` provider strings:\n- The provider string does not need contain the `:\u003cTAG\u003e@\u003cDIGEST\u003e` value, as `repro-get` ignores the container manifests.\n- Defaults to HTTPS for non-localhost registries. Use `oci+http://...` scheme to disable HTTPS.\n\n### IPFS\n\n`repro-get` also supports uploading package files to IPFS, and downloading them from IPFS via an IPFS gateway such as `http://ipfs.io/ipfs/{{.CID}}` .\n\n\u003e **Note**\n\u003e\n\u003e The `ipfs` command ([Kubo](https://github.com/ipfs/kubo)) needs to be installed for pushing (not for pulling).\n\n#### Push\n\nRun `repro-get ipfs push` to push the package files, and update the hash file to include the IPFS CIDs:\n```console\n$ cat SHA256SUMS-amd64\n35b1508eeee9c1dfba798c4c04304ef0f266990f936a51f165571edf53325cbc  pool/main/h/hello/hello_2.10-2_amd64.deb\n\n$ repro-get ipfs push SHA256SUMS-amd64\n35b1508eeee9c1dfba798c4c04304ef0f266990f936a51f165571edf53325cbc  /ipfs/QmRY19HEWeTJtRC6vAdz7rDfX3PjSMgXmd1KYi9guAACUj\n\n$ cat SHA256SUMS-amd64\n35b1508eeee9c1dfba798c4c04304ef0f266990f936a51f165571edf53325cbc  pool/main/h/hello/hello_2.10-2_amd64.deb\n35b1508eeee9c1dfba798c4c04304ef0f266990f936a51f165571edf53325cbc  /ipfs/QmRY19HEWeTJtRC6vAdz7rDfX3PjSMgXmd1KYi9guAACUj\n```\n\n#### Pull\nTo pull and install packages from IPFS:\n```bash\nrepro-get --provider=http://ipfs.io/ipfs/{{.CID}} install SHA256SUMS-amd64\n```\n\nThe hash file must contain the `...  /ipfs/...` lines.\n\nThe hash file may contain multiple CIDs for a single SHA256, but only a single CID is used for pulling.\n\n## FAQs\n### Why do we need reproducibility?\nFor supply chain security.\n\nIf a binary can be bit-to-bit reproducible by multiple independent people, the binary (and its distributor) can be considered more trustable than others.\n\nAchieving bit-to-bit reproducibility is still challenging (see below), but even \"quasi-\"reproducibility is useful for avoiding regressions that could be introduced by installing unexpected updates.\n\nSee also https://reproducible-builds.org/docs/buy-in/ .\n\n### Why not just use `snapshot.debian.org` with `apt-get`?\nAlthough it is already possible to reproduce a specific snapshot of Debian by specifying [`deb [...] http://snapshot.debian.org/archive/debian/yyyyMMddTHHmmssZ/ ... ...`](https://snapshot.debian.org/)\nin `/etc/apt/sources.list`, this will cause a huge traffic on `snapshot.debian.org` when everybody begins to make builds reproducible.\n\n`repro-get` mitigates this issue by content-addressing: A package file can be fetched from anywhere, such as HTTP(S) sites, local filesystems, OCI registries, or even IPFS, by its SHA256 (or CID) checksum.\nAlso, as the package files are verified by checksums, existing package files are not affected by potential GPG key leakage.\n\n### Are container images \"bit-to-bit\" reproducible?\nYes, with BuildKit v0.11 or later.\n\nSee [`./hack/test-dockerfile-repro.sh`](./hack/test-dockerfile-repro.sh) for testing reproducibility.\n\nHowever, it should be noted that the reproducibility is not guaranteed across different versions of BuildKit.\nThe host operating system version, filesystem configuration, etc. may affect reproducibility too.\n\n### How to use HTTPS on Debian/Ubuntu?\n```bash\nrepro-get --provider='https://deb.debian.org/debian/{{.Name}},https://debian.notset.fr/snapshot/by-hash/SHA256/{{.SHA256}}' install\n```\n\nUsing HTTPS needs the `ca-certificates` package to be installed.\nThe `ca-certificates` package is not installed by default in the [`debian`](https://hub.docker.com/_/debian) and [`ubuntu`](https://hub.docker.com/_/ubuntu)) images on Docker Hub.\n\n### Why not use HTTPS by default on Debian/Ubuntu?\nBecause `apt-get` does not use HTTPS by default, either.\nSee [an archive of `whydoesaptnotusehttps.com`](https://web.archive.org/web/20200806030606/https://whydoesaptnotusehttps.com/) for the reason.\n\n## Acknowledgement\nA huge thanks to Frédéric Pierret ([@fepitre](https://github.com/fepitre)) for maintaining the [snapshot](https://github.com/fepitre/debian-snapshot) server http://snapshot.notset.fr/ .\nAlso huge thanks to maintainers of http://snapshot.debian.org/ , https://kojipkgs.fedoraproject.org/ , and other package snapshot servers.\n`repro-get` could not be implemented without these snapshot servers.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freproducible-containers%2Frepro-get","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Freproducible-containers%2Frepro-get","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freproducible-containers%2Frepro-get/lists"}