Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/ostreedev/ostree-rs-ext
Rust library with higher level APIs on top of the core ostree API
https://github.com/ostreedev/ostree-rs-ext
ostree
Last synced: about 1 month ago
JSON representation
Rust library with higher level APIs on top of the core ostree API
- Host: GitHub
- URL: https://github.com/ostreedev/ostree-rs-ext
- Owner: ostreedev
- License: apache-2.0
- Created: 2021-04-03T14:36:27.000Z (almost 4 years ago)
- Default Branch: main
- Last Pushed: 2024-05-21T22:18:48.000Z (8 months ago)
- Last Synced: 2024-05-22T18:58:02.214Z (8 months ago)
- Topics: ostree
- Language: Rust
- Homepage:
- Size: 1.47 MB
- Stars: 68
- Watchers: 10
- Forks: 21
- Open Issues: 27
-
Metadata Files:
- Readme: README.md
- License: LICENSE-APACHE
Awesome Lists containing this project
README
# NOTE: THIS REPOSITORY IS MOVED INTO https://github.com/containers/bootc/
This repository has been merged into https://github.com/containers/bootc/
and will no longer be published as a standalone crate. Everything
below is preserved for historical context.
The future of ostree and containers/OCI will be driven by bootc.
---
# ostree-ext
Extension APIs for [ostree](https://github.com/ostreedev/ostree/) that are written in Rust, using the [Rust ostree bindings](https://crates.io/crates/ostree).
If you are writing tooling that uses ostree and Rust, this crate is intended for you.
However, while the ostree core is very stable, the APIs and data models and this crate
should be considered "slushy". An effort will be made to preserve backwards compatibility
for data written by prior versions (e.g. of tar and container serialization), but
if you choose to use this crate, please [file an issue](https://github.com/ostreedev/ostree-rs-ext/issues)
to let us know.
At the moment, the following projects are known to use this crate:
- https://github.com/containers/bootc
- https://github.com/coreos/rpm-ostree
The intention of this crate is to be where new high level ostree-related features
land. However, at this time it is kept separate from the core C library, which
is in turn separate from the [ostree-rs bindings](https://github.com/ostreedev/ostree-rs).
High level features (more on this below):
- ostree and [opencontainers/image](https://github.com/opencontainers/image-spec) bridging/integration
- Generalized tar import/export
- APIs to diff ostree commits
```mermaid
flowchart TD
ostree-rs-ext --- ostree-rs --- ostree
ostree-rs-ext --- containers-image-proxy-rs --- skopeo --- containers/image
```
For more information on the container stack, see below.
## module "tar": tar export/import
ostree's support for exporting to a tarball is lossy because it doesn't have e.g. commit
metadata. This adds a new export format that is effectively a new custom repository mode
combined with a hardlinked checkout.
This new export stream can be losslessly imported back into a different repository.
### Filesystem layout
```
.
├── etc # content is at traditional /etc, not /usr/etc
│ └── passwd
├── sysroot
│ └── ostree # ostree object store with hardlinks to destinations
│ ├── repo
│ │ └── objects
│ │ ├── 00
│ │ └── 8b
│ │ └── 7df143d91c716ecfa5fc1730022f6b421b05cedee8fd52b1fc65a96030ad52.file.xattrs
│ │ └── 7df143d91c716ecfa5fc1730022f6b421b05cedee8fd52b1fc65a96030ad52.file
│ └── xattrs # A new directory with extended attributes, hardlinked with .xattr files
│ └── 58d523efd29244331392770befa2f8bd55b3ef594532d3b8dbf94b70dc72e674
└── usr
├── bin
│ └── bash
└── lib64
└── libc.so
```
Think of this like a new ostree repository mode `tar-stream` or so, although right now it only holds a single commit.
A major distinction is the addition of special `.xattr` files; tar variants and support library differ too much for us to rely on this making it through round trips. And further, to support the webserver-in-container we need e.g. `security.selinux` to not be changed/overwritten by the container runtime.
## module "diff": Compute the difference between two ostree commits
```rust
let subdir: Option<&str> = None;
let refname = "fedora/coreos/x86_64/stable";
let diff = ostree_ext::diff::diff(repo, &format!("{}^", refname), refname, subdir)?;
```
This is used by `rpm-ostree ex apply-live`.
## module "container": Bridging between ostree and OCI/Docker images
This module contains APIs to bidirectionally map between OSTree commits and the [opencontainers](https://github.com/opencontainers)
ecosystem.
Because container images are just layers of tarballs, this builds on the [`crate::tar`] module.
This module builds on [containers-image-proxy-rs](https://github.com/containers/containers-image-proxy-rs)
and [skopeo](https://github.com/containers/skopeo), which in turn is ultimately a frontend
around the [containers/image](https://github.com/containers/image) ecosystem.
In particular, the `containers/image` library is used to fetch content from remote registries,
which allows building on top of functionality in that library, including signatures, mirroring
and in general a battle tested codebase for interacting with both OCI and Docker registries.
### Encapsulation
For existing organizations which use ostree, APIs (and a CLI) are provided to "encapsulate"
and "unencapsulate" an OSTree commit as as an OCI image.
```
$ ostree-ext-cli container encapsulate --repo=/path/to/repo exampleos/x86_64/stable docker://quay.io/exampleos/exampleos:stable
```
You can then e.g.
```
$ podman run --rm -ti --entrypoint bash quay.io/exampleos/exampleos:stable
```
Running the container directly for e.g. CI testing is one use case. But more importantly, this container image
can be pushed to any registry, and used as part of ostree-based operating system release engineering.
However, this is a very simplistic model - it currently generates a container image with a single layer, which means
every change requires redownloading that entire layer. As of recently, the underlying APIs for generating
container images support "chunked" images. But this requires coding for a specific package/build system.
A good reference code base for generating "chunked" images is [rpm-ostree compose container-encapsulate](https://coreos.github.io/rpm-ostree/container/#converting-ostree-commits-to-new-base-images). This is used to generate the current [Fedora CoreOS](https://quay.io/repository/fedora/fedora-coreos?tab=tags&tag=latest)
images.
### Unencapsulate an ostree-container directly
A primary goal of this effort is to make it fully native to an ostree-based operating system to pull a container image directly too.
The CLI offers a method to "unencapsulate" - fetch a container image in a streaming fashion and
import the embedded OSTree commit. Here, you must use a prefix scheme which defines signature verification.
- `ostree-remote-image:$remote:$imagereference`: This declares that the OSTree commit embedded in the image reference should be verified using the ostree remote config `$remote`.
- `ostree-image-signed:$imagereference`: Fetch via the containers/image stack, but require *some* signature verification (not via ostree).
- `ostree-unverified-image:$imagereference`: Don't do any signature verification
```
$ ostree-ext-cli container unencapsulate --repo=/ostree/repo ostree-remote-image:someremote:docker://quay.io/exampleos/exampleos:stable
```
But a project like rpm-ostree could hence support:
```
$ rpm-ostree rebase ostree-remote-image:someremote:quay.io/exampleos/exampleos:stable
```
(Along with the usual `rpm-ostree upgrade` knowing to pull that container image)
To emphasize this, the current high level model is that this is a one-to-one mapping - an ostree commit
can be exported (wrapped) into a container image, which will have exactly one layer. Upon import
back into an ostree repository, all container metadata except for its digested checksum will be discarded.
#### Signatures
OSTree supports GPG and ed25519 signatures natively, and it's expected by default that
when booting from a fetched container image, one verifies ostree-level signatures.
For ostree, a signing configuration is specified via an ostree remote. In order to
pair this configuration together, this library defines a "URL-like" string schema:
`ostree-remote-registry::`
A concrete instantiation might be e.g.: `ostree-remote-registry:fedora:quay.io/coreos/fedora-coreos:stable`
To parse and generate these strings, see [`OstreeImageReference`].
### Layering
A key feature of container images is support for layering. This functionality is handled
via a separate [container/store](https://docs.rs/ostree_ext/latest/ostree_ext/container/store/) module.
These APIs are also exposed via the CLI:
```
$ ostree-ext-cli container image --help
ostree-ext-cli-container-image 0.4.0-alpha.0
Commands for working with (possibly layered, non-encapsulated) container images
USAGE:
ostree-ext-cli container image
FLAGS:
-h, --help Prints help information
-V, --version Prints version information
SUBCOMMANDS:
copy Copy a pulled container image from one repo to another
deploy Perform initial deployment for a container image
help Prints this message or the help of the given subcommand(s)
list List container images
pull Pull (or update) a container image
```
## More details about ostree and containers
See [ostree-and-containers.md](ostree-and-containers.md).