{"id":13586299,"url":"https://github.com/devrandom/gitian-builder","last_synced_at":"2025-05-15T08:04:51.603Z","repository":{"id":1761934,"uuid":"1310080","full_name":"devrandom/gitian-builder","owner":"devrandom","description":"Build packages in a secure deterministic fashion inside a VM","archived":false,"fork":false,"pushed_at":"2024-12-20T09:35:48.000Z","size":352,"stargazers_count":409,"open_issues_count":45,"forks_count":234,"subscribers_count":32,"default_branch":"master","last_synced_at":"2025-04-14T13:07:19.774Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://gitian.org/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/devrandom.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2011-01-30T21:08:50.000Z","updated_at":"2025-03-25T19:28:34.000Z","dependencies_parsed_at":"2025-03-09T22:31:33.575Z","dependency_job_id":"4706daa5-ffe5-496e-a108-b65024d97523","html_url":"https://github.com/devrandom/gitian-builder","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devrandom%2Fgitian-builder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devrandom%2Fgitian-builder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devrandom%2Fgitian-builder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devrandom%2Fgitian-builder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/devrandom","download_url":"https://codeload.github.com/devrandom/gitian-builder/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254301422,"owners_count":22047901,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T15:05:27.611Z","updated_at":"2025-05-15T08:04:51.554Z","avatar_url":"https://github.com/devrandom.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# MAINTENANCE MODE\n\nDue to the move of Bitcoin Core to [Guix](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#building), this repository is switching to maintenance mode.  Only serious bugs (including security issues) will be considered going forward.\n\n# Gitian\n\nRead about the project goals at the [project home page](https://gitian.org/).\n\nThis package can do a deterministic build of a package inside a VM.\n\n## Deterministic build inside a VM\n\nThis performs a build inside a VM, with deterministic inputs and outputs.  If the build script takes care of all sources of non-determinism (mostly caused by timestamps), the result will always be the same.  This allows multiple independent verifiers to sign a binary with the assurance that it really came from the source they reviewed.\n\n## Prerequisites:\n\n### Arch:\n\n    sudo pacman -S python2-cheetah qemu rsync\n    sudo pacman -S lxc libvirt bridge-utils # for lxc mode\n\nFrom AUR:\n\n* [apt-cacher-ng](https://aur.archlinux.org/packages/apt-cacher-ng/) (you may have to play with permissions (chown to apt-cacher-ng) on files to get apt-cacher-ng to start)\n* [debootstrap](https://aur.archlinux.org/packages/debootstrap-git/)\n* [dpkg](https://aur.archlinux.org/packages/dpkg/)\n* [gnupg1](https://aur.archlinux.org/packages/gnupg1/)\n* [multipath-tools](https://aur.archlinux.org/packages/multipath-tools/) (for kpartx)\n\nNon-AUR packages:\n\n* [debian-archive-keyring](https://packages.debian.org/jessie/debian-archive-keyring) (for making Debian guests)\n* [ubuntu-keyring](https://packages.ubuntu.com/search?keywords=ubuntu-keyring) (for making Ubuntu guests)\n\nFrom newroco on GitHub:\n\n* [vmbuilder](https://github.com/newroco/vmbuilder)\n\nAlso, I had to modify the default /etc/sudoers file to uncomment the `secure_path` line, because vmbuilder isn't found otherwise when the `env -i ... sudo vmbuilder ...` line is executed (because the i flag resets the environment variables including the PATH).\n\n### Gentoo:\n\n    layman -a luke-jr  # needed for vmbuilder\n    sudo emerge dev-vcs/git net-misc/apt-cacher-ng app-emulation/vmbuilder dev-lang/ruby\n    sudo emerge app-emulation/qemu\n    export KVM=qemu-system-x86_64\n\n### Ubuntu:\n\nThis pulls in all pre-requisites for KVM building on Ubuntu:\n\n    sudo apt-get install git apache2 apt-cacher-ng python-vm-builder ruby qemu-utils\n\nIf you'd like to use LXC mode instead, install it as follows:\n\n    sudo apt-get install lxc\n\nIf you'd like to use docker mode instead, install it as follows:\n\n    sudo apt-get install docker-ce\n\n### Debian:\n\nSee Ubuntu, and also run the following on Debian Jessie or newer:\n\n    sudo apt-get install ubuntu-archive-keyring\n\nOn Debian Wheezy you run the same command, but you must first add backports to your system, because the package is only available in wheezy-backports.\n\n### OSX with MacPorts:\n\n    sudo port install ruby coreutils\n    export PATH=$PATH:/opt/local/libexec/gnubin  # Needed for sha256sum\n    \n### OSX with Homebrew:\n\n    brew install ruby coreutils\n    export PATH=$PATH:/opt/local/libexec/gnubin    \n\n#### VirtualBox:\n\nInstall virtualbox from http://www.virtualbox.org, and make sure `VBoxManage` is in your `$PATH`.\n\n## Debian Guests\n\nGitian supports Debian guests in addition to Ubuntu guests. Note that this doesn't mean you can allow the builders to choose to use either Debian or Ubuntu guests. The person creating the Gitian descriptor will need to choose a particular distro and suite for the guest and all builders must use that particular distro and suite, otherwise the software won't reproduce for everyone.\n\nTo create a Debian guest:\n\n    bin/make-base-vm --distro debian --suite jessie\n\nThere is currently no support for LXC Debian guests. There is just KVM support. LXC support for Debian guests is planned to be added soon.\n\nOnly Debian Jessie guests have been tested with Gitian. If you have success (or trouble) with other versions of Debian, please let us know.\n\nIf you are creating a Gitian descriptor, you can now specify a distro. If no distro is provided, the default is to assume Ubuntu. Since Ubuntu is assumed, older Gitian descriptors that don't specify a distro will still work as they always have.\n\n## Create the base VM for use in further builds\n**NOTE:** requires `sudo`, please review the script\n\n### KVM\n\n    bin/make-base-vm\n    bin/make-base-vm --arch i386\n\n### LXC\n\n    bin/make-base-vm --lxc\n    bin/make-base-vm --lxc --arch i386\n\nSet the `USE_LXC` environment variable to use `LXC` instead of `KVM`:\n\n    export USE_LXC=1\n\n### Docker\n\n    bin/make-base-vm --docker\n    bin/make-base-vm --docker --arch i386\n\nSet the `USE_DOCKER` environment variable to use `DOCKER` instead of `KVM`:\n\n    export USE_DOCKER=1\n\n### VirtualBox\n\nCommand-line `VBoxManage` must be in your `$PATH`.\n\n#### Setup:\n\n`make-base-vm` cannot yet make VirtualBox virtual machines ( _patches welcome_, it should be possible to use `VBoxManage`, boot-from-network Linux images and PXE booting to do it). So you must either get or manually create VirtualBox machines that:\n\n1. Are named `Gitian-\u003csuite\u003e-\u003carch\u003e` -- e.g. Gitian-xenial-i386 for a 32-bit, Ubuntu 16 machine.\n2. Have a booted-up snapshot named `Gitian-Clean` .  The build script resets the VM to that snapshot to get reproducible builds.\n3. Has the VM's NAT networking setup to forward port `localhost:2223` on the host machine to port `22` of the VM; e.g.:\n\n```\n    VBoxManage modifyvm Gitian-xenial-i386 --natpf1 \"guestssh,tcp,,2223,,22\"\n```\n\nThe final setup needed is to create an `ssh` key that will be used to login to the virtual machine:\n\n    ssh-keygen -t rsa -f var/id_rsa -N \"\"\n    ssh -p 2223 ubuntu@localhost 'mkdir -p .ssh \u0026\u0026 chmod 700 .ssh \u0026\u0026 cat \u003e\u003e .ssh/authorized_keys' \u003c var/id_rsa.pub\n\nThen log into the vm and copy the `ssh` keys to root's `authorized_keys` file.\n\n    ssh -p 2223 ubuntu@localhost\n    # Now in the vm\n    sudo bash\n    mkdir -p .ssh \u0026\u0026 chmod 700 .ssh \u0026\u0026 cat ~ubuntu/.ssh/authorized_keys \u003e\u003e .ssh/authorized_keys\n\nSet the `USE_VBOX` environment variable to use `VBOX` instead of `KVM`:\n\n    export USE_VBOX=1\n\n## Sanity-testing\n\nIf you have everything set-up properly, you should be able to:\n\n    PATH=$PATH:$(pwd)/libexec\n    make-clean-vm --suite xenial --arch i386\n\n    # on-target needs $DISTRO to be set to debian if using a Debian guest\n    # (when running gbuild, $DISTRO is set based on the descriptor, so this line isn't needed)\n    DISTRO=debian\n\n    # For LXC:\n    LXC_ARCH=i386 LXC_SUITE=xenial on-target ls -la\n\n    # For KVM:\n    start-target 32 xenial-i386 \u0026\n    # wait a few seconds for VM to start\n    on-target ls -la\n    stop-target\n\n## Building\n\nCopy any additional build inputs into a directory named _inputs_.\n\nThen execute the build using a `YAML` description file (can be run as non-root):\n\n    export USE_LXC=1 # LXC only\n    bin/gbuild \u003cpackage\u003e.yml\n\nor if you need to specify a commit for one of the git remotes:\n\n    bin/gbuild --commit \u003cdir\u003e=\u003chash\u003e \u003cpackage\u003e.yml\n\nThe resulting report will appear in `result/\u003cpackage\u003e-res.yml`\n\nTo sign the result, perform:\n\n    bin/gsign --signer \u003csigner\u003e --release \u003crelease-name\u003e \u003cpackage\u003e.yml\n\nWhere `\u003csigner\u003e` is your signing PGP key ID and `\u003crelease-name\u003e` is the name for the current release.  This will put the result and signature in the `sigs/\u003cpackage\u003e/\u003crelease-name\u003e`.  The `sigs/\u003cpackage\u003e` directory can be managed through git to coordinate multiple signers.\n\nAfter you've merged everybody's signatures, verify them:\n\n    bin/gverify --release \u003crelease-name\u003e \u003cpackage\u003e.yml\n\n\n## Poking around\n\n* Log files are captured to the _var_ directory\n* You can run the utilities in libexec by running `PATH=\"libexec:$PATH\"`\n* To start the target VM run `start-target 32 xenial-i386` or `start-target 64 xenial-amd64`\n* To ssh into the target run `on-target` (after setting $DISTRO to debian if using a Debian guest) or `on-target -u root`\n* On the target, the _build_ directory contains the code as it is compiled and _install_ contains intermediate libraries\n* By convention, the script in `\u003cpackage\u003e.yml` starts with any environment setup you would need to manually compile things on the target\n\nTODO:\n- disable sudo in target, just in case of a hypervisor exploit\n- tar and other archive timestamp setter\n\n## LXC tips\n\n`bin/gbuild` runs `lxc-execute` or `lxc-start`, which may require root.  If you are in the admin group, you can add the following sudoers line to prevent asking for the password every time:\n\n    %admin ALL=NOPASSWD: /usr/bin/lxc-execute\n    %admin ALL=NOPASSWD: /usr/bin/lxc-start\n\nRight now `lxc-start` is the default, but you can force `lxc-execute` (useful for Ubuntu 14.04) with:\n\n    export LXC_EXECUTE=lxc-execute\n\nRecent distributions allow lxc-execute / lxc-start to be run by non-privileged users, so you might be able to rip-out the `sudo` calls in `libexec/*`.\n\nIf you have a runaway `lxc-start` command, just use `kill -9` on it.\n\nThe machine configuration requires access to br0 and assumes that the host address is `10.0.2.2`:\n\n    sudo brctl addbr br0\n    sudo ifconfig br0 10.0.2.2/24 up\n\n## Tests\n\nNot very extensive, currently.\n\n`python -m unittest discover test`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevrandom%2Fgitian-builder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdevrandom%2Fgitian-builder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevrandom%2Fgitian-builder/lists"}