Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/dnephin/compose-addons

Tools to supplement the feature set of docker-compose
https://github.com/dnephin/compose-addons

Last synced: about 1 month ago
JSON representation

Tools to supplement the feature set of docker-compose

Awesome Lists containing this project

README 

        

docker-compose addons

=====================



A set of command line tools to supplement the features already available

in docker-compose. These tools generally focus on development or testing

environment use-cases.



.. image:: https://img.shields.io/pypi/v/compose-addons.svg

    :target: https://pypi.python.org/pypi/compose-addons

    :alt: Latest PyPI version



.. image:: https://travis-ci.org/dnephin/compose-addons.svg?branch=master

    :target: https://travis-ci.org/dnephin/compose-addons

    :alt: Travis CI



.. contents::

    :backlinks: none



Install

-------



Currently the only install option is pip



.. code:: sh



    pip install compose-addons



dcao-include

------------



Given a docker-compose.yml file, fetch each configuration in the include

section and merge it into a base docker-compose.yml. If any of the included

files have include sections continue to fetch and merge each of them until

there are no more files to include.



Use Cases

~~~~~~~~~



- If you have a service-oriented architecture, where each of your services

  is developed and deployed in a separate code repo, and each has its own

  docker-compose.yml. When you want to create a full testing or development

  environment for an individual service, you need to include all the

  downstream services. Instead of duplicating the topology of each

  downstream service, you can include the ``docker-compose.yml`` from the

  downstream service. Including (instead of duplicating) this topology

  allows you to change dependencies in a single place without worrying

  about breaking the test suite of dependent services.

- If the scope of your composition can change based on the task you're

  performing. Your application might have a "core" set of services that are

  always run, and some adhoc, or administrative services that are only run

  sometimes. You can split your composition into two (or more) files.

  The core ``docker-compose.yml`` would only contain the core services. The

  ``compose-admin.yml`` would include the ``docker-compose.yml`` and add

  extra services which could link to or use volumes from the core services,

  without having to duplicate any of the service configuration.

- If your composition varies by environment (dev vs prod). Similar to the

  case above, the core ``docker-compose.yml`` would remain the same for all

  environments, but ``docker-compose-dev.yml`` could include the "core"

  services, and add additional service, like database or proxies.



Working with Includes

~~~~~~~~~~~~~~~~~~~~~



``dcao-include`` works with a configuration that is different from the

``docker-compose`` config in a few ways:



- an optional top level ``include`` key, which contains a list of urls (which

  may be local file paths, http(s) urls, or s3 paths)

- a required top level ``namespace`` key, which is used by a config to link

  to services in an included file. For example, if a config includes

  http://example.com/compositions/servicea.yaml which has a ``namespace``

  of ``servicea``, all "public" services in ``servicea.yaml`` should start

  with ``servicea.``.

- since configuration can be included from a remote url, or different

  directories, the configuration should not include anything that depends

  on the host. There should be no ``build`` keys in any service, and no

  host volumes.



Example

~~~~~~~



An example composition file with includes:



.. code:: yaml



    include:

        - http://example.com/compositions/servicea.yaml

        - http://example.com/compositions/serviceb.yaml



    namespace: core



    web:

        image: example/service_a:latest

        links: ['servicea.web', 'serviceb.api']



**servicea.yaml** might look something like this



.. code:: yaml



    namespace: servicea



    servicea.web:

        image: services/a:latest



**serviceb.yaml** might look something like this



.. code:: yaml



    namespace: serviceb



    serviceb.api:

        image: services/b:latest



Usage

~~~~~



To use ``dcao-include`` with ``docker-compose`` you have a couple options:



Use it with a pipe to stdin:



.. code:: sh



    dcao-include compose-with-includes.yml | docker-compose -f - up -d



Use it once to generate a new file:



.. code:: sh



    dcao-include -o docker-compose.yml compose-with-includes.yml

    docker-compose up -d

    docker-compose ps



dcao-namespace

--------------



Given a standard ``docker-compose.yml`` file, add a namespace key, and prefix

all instances of service names with that namespace. This command is used to

prepare a standard ``docker-compose.yml`` file for being used as an include

by ``dcao-include``. This could be considered the "export" step required

before a compose file can be included by another project.



Example

~~~~~~~



Given a ``docker-compose.yml``:



.. code:: yaml



    web:

        image: example.com/web:latest

        links: ['db']

        volumes_from: ['configs']

    db:

        image: example.com/db:latest

    configs:

        image: example.com/configs:latest



running ``dcao-namespace docker-compose.yml myservice`` would produce



.. code:: yaml



    namespace: myservice

    myservice.web:

        image: example.com/web:latest

        links: ['myservice.db:db']

        volumes_from: ['myservice.configs']

    myservice.db:

        image: example.com/db:latest

    myservice.configs:

        image: example.com/configs:latest



Usage

~~~~~



First generate the namespaced config



.. code:: sh



    dcao-namespace -o myservice.yml docker-compose.yml myservice



Next you'll want to make ``myservice.yml`` available to other services. In this

example we'll assume we're using an s3 bucket



.. code:: sh



    aws s3 cp myservice.yml s3://some-bucket/compose-registry/myservice.yml



Now we can use that configuration as an include in another service. In a

different services ``compose-with-includes.yml`` (which will be consumed by

``dcao-include``)



.. code:: sh



    include:

        - s3://some-bucket/compose-registry/myservice.yml



dcao-merge

----------



Merge ``docker-compose.yml`` configuration files by overriding values in the

base configuration with values from other files. It is used to transform a

configuration without having to duplicate any fields that should remain

consistent.



Use Cases

~~~~~~~~~



- Often in development you'll want to include code using a volume for faster

  iteration, but for testing on a CI you want to include the source in the

  container with ``ADD`` (or ``COPY``). You could use an ``overrides-dev.yml`` to add

  volumes to the configuration, and skip that step during CI.

- If the composition is running on a shared host each developer needs to use a

  different host port. This variation can be included in a file maintained by

  each developer outside of version control.

- If a ``docker-compose.yml`` contains ``build`` directives for local

  development, but needs ``image`` directives in other environments (testing,

  stage, prod, etc), merge can be used to rewrite ``build`` to ``image`` with

  the correct image tag.



Usage

~~~~~



To rewrite a configuration to use image instead of build, and remove any host

specific configuration:



.. code:: sh



    dcao-merge -o export.yml docker-compose.yml compose-overrides.yml



Where ``docker-compose.yml`` is:



.. code:: yaml



    web:

        build: .

        links: ['db']

        volumes: ['./logs:/app/logs']

    db:

        build: database/



and ``compose-overrides.yml``:



.. code:: yaml



    web:

        image: example.com/web:latest

        volumes: []

    db:

        image: example.com/db:latest



would produce an ``export.yml``



.. code:: yaml



    web:

        image: example.com/web:latest

        links: ['db']

        volumes: []

    db:

        image: example.com/db:latest