https://github.com/flashbots/yocto-manifests
Repo Manifests for the Yocto Project Build System for reproducible TEE builds
https://github.com/flashbots/yocto-manifests
Last synced: about 2 months ago
JSON representation
Repo Manifests for the Yocto Project Build System for reproducible TEE builds
- Host: GitHub
- URL: https://github.com/flashbots/yocto-manifests
- Owner: flashbots
- License: mit
- Created: 2024-06-03T13:23:55.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-01-07T14:03:44.000Z (10 months ago)
- Last Synced: 2025-01-11T15:15:20.907Z (10 months ago)
- Language: Shell
- Size: 68.4 KB
- Stars: 3
- Watchers: 11
- Forks: 1
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-tee-blockchain - flashbots/yocto-manifests - Repo Manifests for the Yocto Project Build System for reproducible TEE builds (Code Repositories / Others)
README
# yocto-manifests
This repository provides Repo manifests to setup the Yocto build system for reproducible TEE builds.
[The Yocto Project](https://docs.yoctoproject.org/singleindex.html#) allows the creation of custom linux distributions for
embedded systems, including AMD based systems. It is a collection of git
repositories known as *layers* each of which provides *recipes* to build
software packages as well as configuration information.
[Repo](https://gerrit.googlesource.com/git-repo/+/HEAD/README.md) is a tool that enables the management of many git repositories given a
single *manifest* file. Tell repo to fetch a manifest from this repository and
it will fetch the git repositories specified in the manifest and, by doing so,
setup a Yocto Project build environment for you!
## Build Environment Requirements
### System Requirements
- **Operating System**: Ubuntu 22.04 LTS (recommended for stability and reproducibility)
- **Storage**: Minimum 300GB free disk space
- Required for build artifacts and output images
- SSD storage recommended for optimal performance
- **Processor**: Minimum 16 CPU cores
- Builds are heavily parallelized
- More cores will significantly reduce build time
- **Memory**: Minimum 16GB RAM (32GB recommended)
### Recommended Setup
- Dedicated build server for consistent and uninterrupted builds
- Fast internet connection for downloading source packages
- Regular system maintenance to ensure sufficient free space
### Notes
- Clean builds typically take 2-4 hours depending on hardware
- Incremental builds are significantly faster
- Consider using build containers for consistent environments across teams
## Manifest Files
This repository contains several manifest files:
* **tdx-base.xml** - Minimal confidential computing image configuration (uses [tdx-base_yocto_build_config.env](env_files/tdx-base_yocto_build_config.env))
* **tdx-bob.xml** - Searcher node image configuration (uses [bob_yocto_build_config.env](env_files/bob_yocto_build_config.env))
* **tdx-buildernet.xml** - BuilderNet node image configuration (uses [buildernet_yocto_build_config.env](env_files/buildernet_yocto_build_config.env))
## Build Profiles
This repository supports multiple build profiles, each tailored for specific use cases and defined by its corresponding manifest file:
* **tdx-base**: Minimal confidential computing image.
See [tdx-base/README.md](config_files/tdx-base/README.md) for details.
* **tdx-bob**: Specialized image for running searcher nodes with podman support and searcher-specific configurations.
See [tdx-bob/README.md](config_files/tdx-bob/README.md) for details.
* **tdx-buildernet**: Comprehensive environment to run as part of [BuilderNet](https://buildernet.org/). Includes Lighthouse, Reth, and rbuilder.
See [tdx-buildernet/README.md](config_files/tdx-buildernet/README.md) for details.
Each profile's configuration and setup scripts are maintained in the `config_files/` directory. See the profile-specific READMEs for detailed information about features, requirements, and build configurations.
## Reproducing image measurements
**Make sure you are checked out on the correct commit (in this repo).**
Building images and generating their reproducible build measurements is as simple as installing docker and running `make measurements-`. For list of images run `make help`.
Make sure you have plenty of disk space available (200 GBs), and that your session will not time out as a fresh build can take over an hour on slow hardware.
Built images as well as measurements will be available in `./reproducible-build/artifacts-`.
## Preparing your host for non-docker builds
**If you only want to build images, see the "Building images" section above. This is only required if you want to build images on your physical host and is aimed at image developers**
1. See the [Preparing Build Host](https://docs.yoctoproject.org/singleindex.html#preparing-the-build-host)
documentation to install essential host packages on your build host. The
following command installs the host packages based on an Ubuntu distribution.
```
$ sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev xterm python3-subunit mesa-common-dev zstd liblz4-tool chrpath diffstat lz4 mtools
$ sudo locale-gen en_US.UTF-8
```
2. Install Repo tool.
If on Debian/Ubuntu, then run:
```
sudo apt-get install repo
```
Otherwise, follow these steps:
* Download the Repo script.
```
$ curl https://storage.googleapis.com/git-repo-downloads/repo > repo
```
* Make it executable.
```
$ chmod a+x repo
```
* Move it on to your system path.
```
$ sudo mv repo /usr/local/bin/
```
If it is correctly installed, you should see a Usage message when invoked
with the help flag.
```
$ repo --help
```
3. Initialize a Repo client.
* Create an empty directory to hold your working files.
```
$ mkdir -p yocto/tdx
$ cd yocto/tdx
```
* Clone the Yocto meta layer source using yocto manifest as show below.
```
$ repo init -u https://github.com/flashbots/yocto-manifests.git -b main -m .xml
# Replace with tdx-base.xml, tdx-bob.xml, or tdx-buildernet.xml depending on your needs
```
A successful initialization will end with a message stating that Repo is
initialized in your working directory. Your directory should now contain a
.repo directory where repo control files such as the manifest are stored but
you should not need to touch this directory.
To learn more about repo, look at https://source.android.com/setup/develop/repo
4. Fetch all the repositories.
```
$ repo sync
```
5. Start a branch with for development starting from the revision specified in
the manifest. This is an optional step.
```
$ repo start --all
```
6. Setup the Yocto OE Init scripts by sourcing `setup` script.
```
$ source setup
```
> **Note:** if you are building on ubuntu 24.04, yocto bitbake build will fail due to apparmor strict profile update.
>
> Here is a workaround to fix the build issue [link](https://lists.yoctoproject.org/g/docs/topic/yocto_workaround_for/106220010)
7. Build the image by using the provided `Makefile`.
Your host is now ready to build images.
> **Note:** Make sure you are exporting the env vars you want to enable/disable yocto build time configuration. \
> Please inspect the [env_files](./env_files/) and export the desired ones as env variables before trigger the build. \
> e.g. export DEBUG_TWEAKS_ENABLED=1. This builds the image in debug mode and enable root ssh access for dev/debug purposes
```
$ make build
```
8. Generate the measurements values after building the image. They will be located in measurements directory
```
$ make gen-measurements
```
> **Note:** to generate the measurements, you need to make sure that you have `python3`, `libssl-dev` and the `signify` module installed.
>
>$ sudo apt-get install python3 libssl-dev
>
>$ ln -s /usr/bin/python3 /usr/bin/python
>
>$ pip install signify
## Staying Up to Date
To pick up the latest changes for all source repositories, run:
```
$ repo sync
```
## Docker build env
There is also [poky-container](https://github.com/crops/poky-container/) as an alternative docker build environment to build your yocto projects with it.
## Alternative Build with Docker
1. Clone this repo
```
$ git clone https://github.com/flashbots/yocto-manifests.git
```
2. adjust the yocto build configuration in [env_files](https://github.com/flashbots/yocto-manifests/tree/main/env_files) for the target image you want to build
3. make the desired image using, these are the current options:
- image-base: builds a basic tdx-image and outputs the image artifacts in */reproducible-build/artifacts-base* (see [tdx-base profile](config_files/tdx-base/README.md))
- image-bob: builds the image for the BOB project with the searcher embedded ss-key and podman support and outputs the image artifacts in */reproducible-build/artifacts-bob* (see [tdx-bob profile](config_files/tdx-bob/README.md))
- image-buildernet: builds the image for with rbuilder/reth/lighthouse and outputs the image artifacts to */reproducible-build/artifacts-buildernet* (see [tdx-buildernet profile](config_files/tdx-buildernet/README.md))
```
$ make image-
```