Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/starkandwayne/genesis

A BOSH Deployment Paradigm
https://github.com/starkandwayne/genesis

Last synced: 3 months ago
JSON representation

A BOSH Deployment Paradigm

Awesome Lists containing this project

README 

        
Genesis - A BOSH Deployment Paradigm

==================================



Website: https://genesisproject.io 

Slack: [genesisproject](https://join.slack.com/t/genesisprojectio/shared_invite/enQtMzk3NTIxMjY1ODk0LTQyNzFhNzkxNjhlYzY0ODAxYWFhYzIwYzMwNzY5N2Y3NDQ0ODBjMWViMzJlOGI5YWNiN2NiYzM4NDdkNzFlOWU)

The #help channel is available for any questions about Genesis. 

The #announcements channel mostly contains links and info about the monthly Genesis roadmap call, which anybody is welcome to join. 



## Genesis v2



Genesis v2 is the first version of Genesis to fully support BOSH v2. It is primarily geared

to deployments that make use of Cloud Config, and Runtime Config. The BOSH v2 CLI is also a

requirement of Genesis v2.



Genesis v2 builds upon the previous generation of Genesis, eliminating

the vast majority of YAML files all over the place, leading to confusion and

questions like "Where do I put property X - properties.yml, networking.yml, or credentials.yml?"



It also supports the next generation of Genesis deployment templates - Kits.

In the old genesis, deployment templates were pulled in once, forked from their upstream,

and likely never reconciled. With kits, you can keep upgrading the kit, pulling in

newer versions of your deployment to make life much easier down the road.



### Credential Rotation Built-in



Genesis v2 makes use of Vault as a back-end for storing credentials, most easily deployed

+ managed via the [Vault kit](https://github.com/genesis-community/vault-genesis-kit), and

the [safe CLI](https://github.com/starkandwayne/safe). Each kit will auto-generate credentials

where appropriate, ensuring that each environment has unique and secure credentials. Secrets

can be manually or automatically rotated at the drop of hat with the `genesis rotate-secrets` command.

Kits will define certain credentials as fixed, indicating that they should not be rotated

under normal circumstances, as that would have ill effects on the deployment (the CF db encryption

key for example).



### The New Tiered Architecture



In Genesis v2, the data that was previously stored in `global` and `site` by and large go away.

Most of `global` is provided via Kits. Most of `site` is now moved into Cloud Config. As a result,

we can stick most of the customization into a single file per environment, and the directory

structure has been flattened. To share information between environments, and reduce config repitition,

you can create files based on shared prefixes of the environment. For example, `us-west-1` and `us-west-2`

share both `us`, and `us-west` prefixes, and could share configuration in files named as such).



Here's what the new layout looks like:



```

.

├── ci.yml

├── us-boshlite-alpha.yml

├── us-east-dev.yml

├── us-east-prod.yml

├── us.yml # shared with us-*.yml

├── LICENSE

└── README.md

```



### Using Kits



When using a kit with Genesis v2, Genesis will automatically download the latest (or

specified) version of a kit, when you initialize your deployment repo. Any new

versions of the kit can be retrieved with `genesis download`. Each deployment environment

must have at some point in its merge-path, a `kit` section, indicating what kit + version

should be deployed. It will look something like this:



```

---

kit:

  name: jumpbox

  version: 2.0

```

To get the full benefit of CI, operators should place this in a file

that is shared across all environments, so that upgrades can be vetted in the deployment pipeline

in non-production environments, before going to production.



### Flexible Deployment Pipelines



The pipeline strategy of Genesis v2 is much more flexible than the previous approach. Operators

are able to define what environments should trigger, which should not, as well as which environments

are gateways to deploying in later environments. Stemcell management is built-in, as are locking

mechanisms to ensure that your BOSH isn't upgraded while it's in the middle of deploying something.



For a full run-down on Genesis v2 + deployment pipelines, see our [pipeline documentation](docs/PIPELINES.md)



## Installation



On Ubuntu/Debian you can install `genesis` and all its dependencies:



```

wget -q -O - https://raw.githubusercontent.com/starkandwayne/homebrew-cf/master/public.key | apt-key add -

echo "deb http://apt.starkandwayne.com stable main" | tee /etc/apt/sources.list.d/starkandwayne.list

apt-get update

apt-get install genesis -y

```



On OS X/Mac:



```

brew tap starkandwayne/cf

brew tap cloudfoundry/tap

brew install genesis spruce safe bosh-cli vault git

```



On Centos 7 / Amazon Linux



```

# Install Dependencies

sudo yum install -y perl perl-Data-Dumper perl-Time-Local perl-Time-Piece \

  perl-local-lib perl-Carp perl-PathTools perl-Digest perl-File-Temp perl-Socket



# Grab latest url from releases page

wget https://github.com/starkandwayne/genesis/releases/download/${GENESIS_VERSION}/genesis

chmod 0755 genesis

```



`genesis` requires Perl. But Perl is everywhere.



You will need to set up Git:



```

git config --global user.name "Your Name"

git config --global user.email [email protected]

```



## Using Genesis



Here are a few thoughts on the Genesis CLI.



Global options should be recognized for all commands:



  - `-D` / `--debug` - Enable debugging mode, wherein Genesis

    prints messages to standard error, detailing what tasks it is

    doing, what commands it is running, return values, etc.

  - `-T` / `--trace` - Enable debugging mode in all called

    utilities, like `spruce` and `bosh`, where available.

  - `-C PATH` - Perform all operations from `PATH` as the current

    working directory.

  - `-y` / `--yes` - Answer "yes" to all questions, automatically,

    on behalf of the user.  This means, for example, running `bosh

    -n`, instead of just `bosh`.



Initialize a Genesis repo:



```

genesis init --kit shield

genesis init --kit shield/1.2.3

```



Create a new deployment named us-west-1-sandbox:



```

genesis new us-west-1-sandbox

```



Generate a manifest for an environment:



```

# Using the live the cloud-config from the BOSH director

genesis manifest my-sandbox

# or, using the local file cached-cloud-config.yml:

genesis manifest -c cached-cloud-config.yml

```



Deploy a deployment, manually:



```

genesis deploy us-west-1-sandbox

```



Rotate credentials for a deployment:



```

genesis secrets us-west-1-sandbox

```



Summarize the current state of deployments and what kits they are

using:



```

genesis summary

```



Download version 1.3.4 of the SHIELD kit:



```

genesis download shield 1.3.4

```



Deploy the Genesis CI/CD pipeline configuration to Concourse:



```

genesis repipe

```



Describe a given pipeline, in words:



```

genesis describe

```



Draw a pretty picture of a pipeline, using `dot`, suitable for

embedding in documentation and putting on your blog:



```

genesis graph pipelines/aws

```



Embed the calling Genesis script in the Genesis repo:



```

genesis embed

```



## Transitioning to Genesis v2



Due to the sweeping changes involved in Genesis v2, it is recommended to start

with a fresh deployment repo, and migrate existing deployments slowly but surely

over to Genesis v2 using the appropriate kits, or if necessary, a `dev` kit (see

[Authoring Kits](docs/AUTHORING-KITS.md) for more info). The `genesis` command will

auto-detect if you are running inside a Genesis v1 repo, and switch to a `v1` mode,

for compatibility.



## Design Notes + Genesis Developer Resources



Genesis v2 design documentation that was previously found here has moved.



  - [Design Notes](docs/DESIGN.md)

  - [Pipelines](docs/PIPELINES.md)

  - [Authoring Kits](docs/AUTHORING-KITS.md)

  - [Environment Parameters](docs/PARAMS.md)