Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/runkitdev/cloudpack

Last synced: 29 days ago
JSON representation

Host: GitHub
URL: https://github.com/runkitdev/cloudpack
Owner: runkitdev
License: apache-2.0
Created: 2019-08-09T17:00:13.000Z (over 5 years ago)
Default Branch: master
Last Pushed: 2022-12-30T18:25:31.000Z (about 2 years ago)
Last Synced: 2024-04-14T07:56:01.816Z (9 months ago)
Language: JavaScript
Size: 137 KB
Stars: 1
Watchers: 6
Forks: 1
Open Issues: 2
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

README

# Cloudpack

Cloudpack is a tool to automate the Packer -> AMI -> LaunchTemplate pipeline. Even though it currently only supports AWS, Cloudpack is highly modular and can easily be extended to support other backends (e.g. GCP images / GCP Instance Template, Digital Ocean, etc).

# Installation

```
sudo npm install -g cloudpack
```

# Usage

```
cloudpack config.json [--var "myVar=myValue"] [--verbose] [--dry-run]
```

# Configuration

The configuration file is a JSON containing five sections:

- `auth` - Instructions on how to connect to the provider.
- `builder` - Instructions on the `Packer Build` step.
- `build_script` - Assets to include in the image (same as Packer Provisioners)
- `boot_script` - Scripts to execute during the boot.
- `launch_template` - Instructions on the LaunchTemplate to be created/updated.

## auth

The auth entry instructs Cloudpack how it should connect to the underlying cloud provider. It should contain an entry which states the auth provider backend and its corresponding value.

The auth entry is optional. When omitted, Cloudpack assumes the credentials are stored on the standard environment variables or shared configuration/credential files.

Currently supported backends are: `aws`.

### auth backend: AWS

The AWS auth backend contains three keys: `access_key`, `secret_key` and `region`. When set, Cloudpack will use the given values to authenticate with AWS (including during the Packer Build step). If any of these keys are empty or omitted, Cloudpack assumes the credentials are available on the corresponding environment variables or credentials file.

NOTE: By default, Node.js's AWS SDK does not check the shared configuration file at `~/.aws/config`. You can force Cloudpack to use this file by setting the `AWS_SDK_LOAD_CONFIG` environment variable to a truthy value.

**Example**

```json
{
"auth": {
"aws": {
"access_key": "myAccessKey",
"secret_key": "mySecretKey",
"region": "us-west-2"
}
}
}
```

## builder

The `builder` entry is required and contains the information that should be relayed to Packer during the Packer Build step. Similar to `auth`, it contains a key that represents the [builder](https://www.packer.io/docs/builders/index.html), and the corresponding values it needs to properly build the image.

Currently supported builders are: `amazon-chroot`.

### builder backend: amazon-chroot

The `amazon-chroot` backend implements the [amazon-chroot builder](https://www.packer.io/docs/builders/amazon-chroot.html). The values in this entry, alongside other parts of the Cloudpack configuration file, will be used to generate a valid [Packer template](https://www.packer.io/docs/templates/index.html#template-structure), which will then be built.

There is no need to relay the `access_key` and `secret_key` entries; those values are inferred from the `auth` section.

Resources generated on success:

- `amiId` - ID of the AMI resource generated by Packer.
- `snapshotId` - ID of the Snapshot resource generated by Packer.

NOTE: `amazon-chroot` builder requires root privileges.

**Example**:

```json
{
"builder": {
"amazon-chroot": {
"ami_name": "MyProductionApp-{{ timestamp }}",
"source_ami": "ami-0d1a13e419ec9285d",
"nvme_device_path": "/dev/nvme1n1p",
"device_path": "/dev/sdf",
"ena_support": true
}
}
}
```

## build_script

The `build_script` section contains a list of commands that should be applied to the running Packer VM before it is turned into a static image. They use the same syntax of [Packer provisioners](https://www.packer.io/docs/provisioners/index.html).

The build script execution is ordered; that is, it follows the same order declared in the configuration file.

The `build_script` entry is optional and may be omitted.

**Example**:

```json
{
"build_script": [
{
"type": "file",
"source": "~/myapp.tar",
"destination": "/deploy/myapp.tar"
},
{
"type": "shell",
"inline": [ "tar -xvf /deploy/myapp.tar" ]
}
]
}
```

## boot_script

The `boot_script` section contains a list of commands that should be executed during the boot process.

The configuration must include a `type` entry, which states *how* the boot script will be executed. It may be executed either as `rc.local`, which creates an executable `/etc/rc.local` entry, or as `user-data`, which relies on [AWS UserData](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html).

Currently only `rc.local` execution method is implemented.

The user must, then, set the `cmds` entry, which should contain a list of string commands to be executed. This list will be used to generate a shell script, which will then be used by `rc.local` or `user-data`.

The user may also specify the keys `flags` and `shebang`, which applies custom flags and shebang to the generated shell script. When omitted, Cloudpack defaults to not setting any flags, and to using the `#!/usr/bin/env bash` shebang.

The `boot_script` entry is optional. When omitted, Cloudpack assumes nothing should be executed during boot.

NOTE: The `rc.local` method works on systemd due to its `rc-local.service`, which is included by default. This service will execute `/etc/rc.local` if it is an executable file.

NOTE: The `user-data` method is currently unsupported.

**Example**:

```json
{
"boot_script": {
"type": "rc.local",
"flags": "eu",
"shebang": "#!/bin/sh",
"cmds": [
"os=$(uname)",
"echo \"I love $os\" > /tmp/foo"
]
}
}
```

Generated shell script:

```sh
#!/bin/sh
set -eu

os=$(uname)
echo "I love $os" > /tmp/foo
```

## launch_template

The `launch_template` section defines what action Cloudpack should take when the image is generated successfully by Packer.

Similar to `auth` and `builder`, `launch_template` requires the user to specify a backend.

`launch_template` is optional. When omitted, Cloudpack assumes it should do nothing with the generated image.

Currently supported backends are: `aws`.

### launch_template backend: AWS

The AWS launch template backend allows the user to create a new LaunchTemplate from scratch, or to update the values of a previously created LaunchTemplate, thus setting a new version for that LaunchTemplate.

When creating a new launch template from scratch, the user must specify the `template_name`, and `template_data` keys. The `template_data` must contain a map of values that should be set on the new template. There must exist at least one entry in this map.

When creating a new version of a previously created launch template, the user must specify the `source_template` and the `modifications` key, containing the set of changes that should be applied to that LaunchTemplate.

Moreover, when creating a new version, it is usually desirable to inherit the values of a previous version. Cloudpack supports this by enabling the `source_version` key, which is a *string* pointing to the version that should be the basis of the new template.

Resources generated on success:

- `launchTemplateId` - ID of the LaunchTemplate resource created/updated.

Note: The values within `template_data` and `modifications` use the same nomenclature the official AWS API does. Any value in there is relayed to the API. [Here's a list of valid parameters](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_RequestLaunchTemplateData.html).

**Example** (creating new template):

```json
{
"launch_template": {
"aws": {
"template_name": "MyCoolTemplate",
"template_data": {
"ImageId": "{{ runtime.amiId }}"
}
}
}
}
```

**Example** (creating new version off of existing one):

```json
{
"launch_template": {
"aws": {
"source_template": "lt-0d058a349ebcab476",
"source_version": "1",
"modifications": {
"ImageId": "{{ runtime.amiId }}"
}
}
}
}
```

# Variable support

The Cloudpack configuration file supports two types of variables: `user` variables and `runtime` variables.

User variables are prefixed with `var` and must be defined at the CLI, using the `--var` flag. All user variables defined at the configuration file must be set.

Runtime variables are prefixed with `runtime` and are set internally by Cloudpack, as soon as the requested runtime variable is resolved. Valid values for runtime variables are:

- `runtime.amiId` - Contains the ID of the AMI generated by Packer.
- `runtime.snapshotId` - Contains the ID of the snapshot generated by Packer.
- `runtime.launchTemplateId` - Contains the ID of the LaunchTemplate created/modified by the AWS LaunchTemplate backend.

NOTE: Runtime variables are only accessible after they are created. For example, `runtime.amiId` is generated during the Packer Build step, and as such cannot be used prior to that.

Packer template variables, like `{{ timestamp }}` are also supported, but they only work within the Packer build step. In other words, if you use Packer variables on `builder` or `build_script`, they will be expanded properly. Otherwise, they will keep their literal value.

# Example

Functional CLI + configuration file example

CLI command:

```
sudo cloudpack config.json --var "version=1.0"
```

`config.json`:

```json
{
"auth": {
"aws": {
"region": "us-west-2"
}
},
"builder": {
"amazon-chroot": {
"ami_name": "MyProductionApp-{{ var.version}}-{{ timestamp }}",
"source_ami": "ami-0d1a13e419ec9285d",
"nvme_device_path": "/dev/nvme1n1p",
"device_path": "/dev/sdf",
"ena_support": true
}
},
"build_script": [
{
"type": "shell",
"environment_vars": [
"FOO=bar"
],
"inline": [
"echo \"$FOO\" > /foo"
]
}
],
"boot_script": {
"type": "rc.local",
"flags": "eu",
"cmds": [
"echo 'Hello World' > /tmp/foo",
"val=$(cat /tmp/foo)",
"echo \"value is $val\" > /tmp/foo2"
]
},
"launch_template": {
"aws": {
"source_template": "lt-0d058a349ebcab476",
"source_version": "1",
"modifications": {
"ImageId": "{{ runtime.amiId }}"
}
}
}
}
```

Output:

```json
{
"amiId": "ami-032c4edf187ab5ff6",
"snapshotId": "snap-0cbbd8b0af7805603",
"launchTemplateId": "lt-0d058a349ebcab476"
}
```

# Limitations & known issues

- Packer supports multiple builders in a single template file. Cloudpack supports only one per configuration file.

- Some values accepted by the Packer build or the AWS API are not currently relayed to the corresponding APIs.