Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/lexsca/prefab

Build container images faster ⚡️
https://github.com/lexsca/prefab

build-automation build-tool ci container container-image continuous-integration docker docker-image python python3

Last synced: 3 months ago
JSON representation

Build container images faster ⚡️

Host: GitHub
URL: https://github.com/lexsca/prefab
Owner: lexsca
License: mit
Created: 2020-08-31T03:20:14.000Z (over 4 years ago)
Default Branch: main
Last Pushed: 2022-11-09T03:09:46.000Z (about 2 years ago)
Last Synced: 2024-09-28T16:02:52.143Z (4 months ago)
Topics: build-automation, build-tool, ci, container, container-image, continuous-integration, docker, docker-image, python, python3
Language: Python
Homepage: https://prefab.readthedocs.io
Size: 130 KB
Stars: 5
Watchers: 2
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.rst
- License: LICENSE

Awesome Lists containing this project

README

######
Prefab
######

**Build container images faster** ⚡️

.. image:: https://imgs.xkcd.com/comics/compiling.png
:target: https://xkcd.com/license.html
:alt: https://xkcd.com/303/

.. image:: https://github.com/lexsca/prefab/actions/workflows/checks.yml/badge.svg
:target: https://github.com/lexsca/prefab/actions/workflows/checks.yml

.. image:: https://img.shields.io/docker/image-size/lexsca/prefab/dood
:target: https://hub.docker.com/r/lexsca/prefab/tags

.. image:: https://img.shields.io/pypi/v/container-prefab.svg
:target: https://pypi.org/project/container-prefab/

.. image:: https://img.shields.io/pypi/pyversions/container-prefab.svg
:target: https://pypi.org/project/container-prefab/

.. image:: https://img.shields.io/github/license/lexsca/prefab.svg
:target: https://github.com/lexsca/prefab/blob/master/LICENSE

.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
:target: https://github.com/psf/black

*Prefab* is a Python-based container image build tool that uses deterministic remote caching to reduce build times. Unlike `BuildKit `_ and the `Docker CLI `_, which use build layer caching, *Prefab* uses whole image caching based on a digest of the Dockerfile in combination with digests of specified files and directory trees. This allows *Prefab* to check for and pull cached images before resorting to building a new image.

Quickstart
==========

Look at the `example directory `_ to see how to build an example app with *Prefab*.

Installation and usage
======================

*Prefab* can be installed and run in three different ways:

#. Local Python package
#. Docker outside of Docker (DooD) container
#. Docker in Docker (DinD) container

Use whichever mode works best for the use-case(s) at hand. Each supports the same CLI arguments:

CLI arguments
-------------

usage: prefab [-h] [--config PATH] [--dry-run] [--force] [--monochrome]
[--push TARGET_NAME [TARGET_NAME ...]] [--push-all] --repo URI
--target TARGET_NAME[:TAG] [TARGET_NAME[:TAG] ...]

Build container images faster ⚡️

optional arguments:
-h, --help show this help message and exit
--config PATH, -c PATH
Target build config file to use (default: prefab.yml)
--dry-run Show how targets would be built (implies --force)
--force Force target(s) to be rebuilt
--monochrome, -m Don't colorize log messages
--push TARGET_NAME [TARGET_NAME ...], -p TARGET_NAME [TARGET_NAME ...]
Image target(s) to push to repo after building
--push-all Push all image targets to repo after building
--repo URI, -r URI Image repo to use (e.g. lexsca/prefab)
--target TARGET_NAME[:TAG] [TARGET_NAME[:TAG] ...], -t TARGET_NAME[:TAG] [TARGET_NAME[:TAG] ...]
Image target(s) to build with optional custom image tag

Local Python package
--------------------

To install *Prefab* as a local Python package::

pip install container-prefab

To run *Prefab* as a local Python package to build an push a build target::

prefab --repo repo.tld/org/project --push --target name

NOTE: Container images now hosted on Docker Hub
-----------------------------------------------

The container images below used to be hosted by Quay and are now
hosted by `Docker Hub `_
This decision was not taken lightly. Sadly, Quay has not proven to
be a reliable service. The final straw was when RedHat acquired
them and broke authentication.

Docker outside of Docker (DooD)
-------------------------------

To get the *Prefab* Docker outside of Docker (DooD) image::

docker pull lexsca/prefab:dood

To run the *Prefab* Docker outside of Docker image to build an push a build target::

docker run --rm -it -v $(/bin/pwd):/build -w /build \
-e REGISTRY_AUTH=$(jq -c . ~/.docker/config.json | base64) \
-v /var/run/docker.sock:/docker.sock \
lexsca/prefab:dood --repo repo.tld/org/project \
--push-all --target name

Docker in Docker (DinD)
-----------------------

To get the *Prefab* Docker in Docker (DinD) image::

docker pull lexsca/prefab:dind

To run the *Prefab* Docker in Docker image to build an push a build target::

docker run --rm -it -v $(/bin/pwd):/build -w /build --privileged \
-e REGISTRY_AUTH=$(jq -c . ~/.docker/config.json | base64) \
lexsca/prefab:dind --repo repo.tld/org/project \
--push-all --target name

Configuration
=============

*Prefab* uses a `YAML `_ configuration file to determine which container images to build for a given target and in which order. This configuration below is taken from the `example directory `_ in this repo.

This example ``prefab.yml`` file has two build targets, each with a Dockerfile. The ``app`` target has a dependency on the ``packages`` target, so it's built or pulled first, before building the ``app`` target. This is a simple example, but the dependency graph can be arbitrarily deep or wide for complex build targets.

``prefab.yml``
--------------

targets:

app:
dockerfile: Dockerfile.app
depends_on:
- packages
watch_files:
- app.py

packages:
dockerfile: Dockerfile.packages

When building a container image, *Prefab* populates `build arguments `_ for each build target depndency, uppercased by convention, and prefixed with ``PREFAB_`` to avoid conflicts with other build arguments.

``Dockerfile.app``
------------------

ARG PREFAB_PACKAGES

FROM $PREFAB_PACKAGES as packages

Contributing
============

Bug reports are welcome. Pull requests even more so.

Before making any changes, first ensure the development environment is functional and the extant linting and tests are passing. To start a development environment, clone or fork this source repo and follow the instructions below.

Alternatively, it's fine to create a virtual environment an install packages from ``requirements.txt`` and ``requirements-dev.txt`` files. The Python version should be 3.7 or later.

Prerequisites
-------------

#. POSIX Shell (e.g. bash)
#. Docker
#. GNU Make

Create environment
------------------

To create a development runtime environment::

$ make bootstrap

The above will create a minimal environment that will allow *Prefab* to build its development environment image. This image can be used to run linting and tests::

$ docker images lexsca/prefab:dev
REPOSITORY TAG IMAGE ID CREATED SIZE
lexsca/prefab dev ddee1cafb775 About a minute ago 429MB

Use environment
---------------

Once created, the development image can used via::

$ make shell
docker run --rm -it -v /Users/lexsca/git/prefab:/prefab -w /prefab \
-v /var/run/docker.sock:/docker.sock -e PYTHONPATH=/prefab/lib \
--entrypoint /bin/bash lexsca/prefab:dev --login -o vi
3053ae861610:/prefab# make test

This will mount the docker socket and current working directory in an environment where tests can be run, dependencies built, or a debugger invoked to aid in iterating.

The ``make test`` command should pass before attempting to submit any code changes.