https://github.com/alphahydrae/deploy
A (magic) shell script to deploy Git repositories
https://github.com/alphahydrae/deploy
deploy deployment git shell-script
Last synced: 7 months ago
JSON representation
A (magic) shell script to deploy Git repositories
- Host: GitHub
- URL: https://github.com/alphahydrae/deploy
- Owner: AlphaHydrae
- License: mit
- Created: 2017-05-17T18:07:20.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2018-03-16T15:18:57.000Z (almost 8 years ago)
- Last Synced: 2025-04-28T12:09:52.372Z (8 months ago)
- Topics: deploy, deployment, git, shell-script
- Language: Shell
- Homepage: https://alphahydrae.github.io/deploy/
- Size: 652 KB
- Stars: 6
- Watchers: 0
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.txt
Awesome Lists containing this project
README
# deploy
**_A (magic) shell script to deploy Git repositories_**
[](https://travis-ci.org/AlphaHydrae/deploy)
[](https://badge.fury.io/js/bash-deploy)
[](https://opensource.org/licenses/MIT)
Read the [annotated source](https://alphahydrae.github.io/deploy/)
Repository: [AlphaHydrae/deploy](https://github.com/AlphaHydrae/deploy)
Shamelessly inspired by: [visionmedia/deploy](https://github.com/visionmedia/deploy)
- [Usage](#usage)
- [Installation](#installation)
- [Requirements](#requirements)
- [Configuration file](#configuration-file)
- [Single-value and multiple-value keys](#single-value-and-multiple-value-keys)
- [Environment inheritance](#environment-inheritance)
- [Hooks](#hooks)
- [Configuration file](#configuration-file-1)
- [Project](#project)
- [SSH connection](#ssh-connection)
- [Host environment variables](#host-environment-variables)
- [Command line options & environment variables](#command-line-options--environment-variables)
- [General options](#general-options)
- [Project options](#project-options)
- [SSH options](#ssh-options)
- [Self-update options](#self-update-options)
- [Command line options format](#command-line-options-format)
- [Short option format](#short-option-format)
- [Long option format](#long-option-format)
- [Invalid options](#invalid-options)
- [Changing the working directory](#changing-the-working-directory)
- [Default command](#default-command)
- [Sub-commands](#sub-commands)
- [` setup`](#env-setup)
- [` rev [rev]`](#env-rev-rev)
- [`[env] config [key]`](#env-config-key)
- [` config-all `](#env-config-all-key)
- [`[env] config-section`](#env-config-section)
- [` console [path]`](#env-console-path)
- [` cleanup`](#env-cleanup)
- [` exec `](#env-exec-cmd)
- [` list`](#env-list)
- [`update [rev]`](#update-rev)
- [Cleaning up old releases](#cleaning-up-old-releases)
## Usage
**deploy** is a shell script to deploy your Git projects through SSH.
Add a `deploy.conf` file in your project's directory.
Here's an example for a Node.js project:
# deploy.conf
[production]
repo https://github.com/me/my-app
host my.server.com
user deploy
path /var/www/app
# describe how to deploy your app
env NODE_ENV=production
deploy npm install --production
post-deploy npm start
**deploy** is language-agnostic.
For a Rails project, you could replace the last 3 lines with:
env RAILS_ENV=production
deploy bundle install --without development test
deploy rake assets:precompile
post-deploy rails server
Now run **deploy**!
```sh
deploy production setup
deploy production rev master
```
It will:
* Connect to `my.server.com` as the `deploy` user through SSH
* The `setup` command will set up a deployment structure and clone your repository
* The second `rev` command will checkout the latest version of your `master` branch and run the deployment hooks you defined (`deploy` and `post-deploy` in the configuration file)
Read on to learn what to write in the [configuration file](#configuration-file) or how to use each [sub-command](#sub-commands).
## Installation
With npm:
```sh
npm install -g bash-deploy
```
With curl:
```sh
PREFIX=/usr/local/bin \
FROM=https://raw.githubusercontent.com \
&& curl -sSLo $PREFIX/deploy \
$FROM/AlphaHydrae/deploy/master/bin/deploy \
&& chmod +x $PREFIX/deploy
```
With wget:
```sh
PREFIX=/usr/local/bin \
FROM=https://raw.githubusercontent.com \
&& wget -qO $PREFIX/deploy \
$FROM/AlphaHydrae/deploy/master/bin/deploy \
&& chmod +x $PREFIX/deploy
```
Or [download it](https://raw.githubusercontent.com/AlphaHydrae/deploy/master/bin/deploy) yourself.
## Requirements
**deploy** is a one-file bash script which requires the following commands: `cat`, `cut`, `date`, `git`, `grep`, `ls`, `mkdir`, `readlink`, `sed`, `ssh`, `tail`, `tar` and `wc`.
Most bash shells have all of those out of the box except perhaps [Git](https://git-scm.com).
It also optionally requires the `chmod`, `cp` and `mktemp` commands to update itself.
## Configuration file
**deploy** reads its main configuration from a `deploy.conf` file in the current directory.
It can also be customized with [environment variables and command line options](#command-line-options-environment-variables).
The configuration file is basically a series of sections containing key/value pairs:
# deploy.conf
[staging]
host staging.example.com
user test
# how to deploy
post-deploy ./run-test.sh
[production]
host 192.168.1.42
user root
# how to deploy
env NODE_ENV=production
deploy npm install --production
deploy npm run build
post-deploy pm2 start pm2.json
Each named section, delimited by `[name]`, represents an **environment** (i.e. a host machine) to deploy to,
in this example the *staging* and *production* environments.
Lines beginning with `#` are comments and are ignored.
Other lines are key/value pairs.
A key is a sequence of characters containing no whitespace, followed by at least one tab or space.
For example, in the line `deploy npm run build`, the key is `deploy` and its value is `npm run build`.
### Single-value and multiple-value keys
Some keys like `repo`, `host`, `port` or `user` are simple configuration properties that have one value per environment.
If multiple values are found, the last one is used.
These keys can be overriden through an environment variable of the same name in uppercase and prefixed by `DEPLOY_`.
For example, the environment variable to override the `repo` key is `DEPLOY_REPO`.
Other keys like `setup`, `deploy` or `post-deploy` are multiple-value keys used for [hooks](#hooks).
They can be present multiple time in the same environment and all their values will be used in order.
### Environment inheritance
When deploying your project to multiple environments, there will probably be some configuration properties
that are identical for multiple environments.
To avoid repetition, an environment can **inherit** from one or multiple other environments:
# deploy.conf
[common]
user dev
deploy do-stuff
deploy do-more-stuff
[secure]
user sekret
deploy do-sensitive-stuff
[production]
inherits common
inherits secure
user root
deploy just-do-it
In this example, the `production` environment inherits from `common` and `secure` (in that order).
The value of a *single-value* key like `user` will be the last value found in the environment inheritance tree.
In this case, it will be `root` for the `production` environment (the values inherited from the `common` and `secure` environments are overwritten).
A *multiple-value* key like `deploy` will include all values found in the entire inheritance tree.
In this case, it will have 4 values for the `production` environments, taken in order from the `common`, `secure` and `production` sections:
do-stuff
do-more-stuff
do-sensitive-stuff
just-do-it
For convenience in simple use cases, you can also add a **default environment** by including a nameless section in your configuration file:
# deploy.conf
[]
user dev
deploy do-stuff
deploy do-more-stuff
[development]
deploy do-it-somehow
[production]
user root
deploy do-it-seriously
All environments that have no `inherits` key automatically inherit from the default environment.
## Hooks
Hooks are user-defined commands that can be run during a *deployment phase*.
There are currently two phases defined: **setup** and **deploy**.
The *setup* phase happens when you run the `setup` command,
while the *deploy* phase happens when you run the `rev` command.
There are various hooks for each phase: some that run before, some during and some after **deploy** does its thing.
These are the currently available hooks:
pre-setup
post-setup
pre-deploy
deploy
post-deploy
Hooks are multiple-value keys that are optional and can be used as many times as you want.
They will all be run in order.
Each hook is run in a specific working directory and has access to various environment variables.
`$DEPLOY_PATH` is always exported and indicates the deployment directory
(which is not necessarily the same as the hook's working directory).
Additional user-defined variables may also be made available (see [environment variables](#host-environment-variables)).
Here's an example of how you could use hooks to cache your `node_modules` directory after every deployment
to shorten future installation times:
# restore cache (if present)
deploy tar -xzf $DEPLOY_PATH/cache.gz -C . || exit 0
# install new dependencies
deploy npm install --production
# update cache
post-deploy tar -czf $DEPLOY_PATH/cache.gz node_modules
See the [`setup`](#setup) and [`rev`](#deploy) commands to learn exactly when and where hooks are executed.
## Configuration file
### Project
* **`repo `** (or the `$DEPLOY_REPO` variable) defines the Git URL from which your repository will be cloned at setup time.
* **`path `** (or the `$DEPLOY_PATH` variable) defines the directory into which your project will be deployed on the host.
*
**`keep `** (or the `$DEPLOY_KEEP` variable) defines how many releases to keep after deploying.
Older releases will be deleted. It must be either `all` (the default), or an integer greater than zero.
If it's a number, it indicates the number of releases that should be kept **after successful deployment**.
For example, with `keep` set to 3, if there are 5 old releases deployed before running the script, 3 will be deleted
(so that 2 old releases and the new one being deployed, 3 in total, remain).
It's also possible to manually trigger cleanup with the [cleanup](#cleanup) command.
### SSH connection
Various SSH options can be specified through the configuration file or environment variables:
* **`host
`** (or the `$DEPLOY_HOST` variable) is mandatory for all environments.
It indicates which host or IP address to connect to.
* **`user `** (or the `$DEPLOY_USER` variable) specifies the user to connect as.
By default, you connect with the same name as your local user.
* **`identity `** (or the `$DEPLOY_IDENTITY` variable) specifies a file from which the private key for public key authentication is read.
* **`forward-agent yes`** (or the `$DEPLOY_FORWARD_AGENT` variable) enables agent forwarding.
* **`port `** (or the `$DEPLOY_PORT` variable) specifies the host port to connect to.
* **`tty yes`** (or the `$DEPLOY_TTY` variable) forces pseudo-terminal allocation.
Commands executed on the host through SSH will be logged to the console.
For some commands, the output may be retrieved and displayed in a more user-friendly manner.
### Host environment variables
You can define environment variables that will be exported on the host when executing hooks.
The `$DEPLOY_PATH` variable is always exported and indicates the deployment directory
configured by the user with the `path` config key or the local `$DEPLOY_PATH` variable.
* **`env =...`** defines one or multiple environment variables to export on the host when running hooks.
# deploy.conf
env FOO=BAR
env BAZ=QUX CORGE=GRAULT
* **`forward-env ...`** defines the name(s) of one or multiple local environment variables
to export on the host when running hooks.
# deploy.conf
forward-env FOO
forward-env BAR BAZ QUX
If you have a **`.env`** file in your local project directory, it will automatically be sourced.
This can be handy to create local variables that you can forward to the host.
# .env
export FOO=BAR
export YEAR=$(date "+%Y")
(Note that if you use the `-C, --chdir` command line option or the `$DEPLOY_CHDIR` variable,
the `.env` file is read *after* the working directory is changed, so setting `$DEPLOY_CHDIR`
in the file has no effect.)
If you have an environment file named after the environment, it will also be sourced.
For example, when deploying in the "production" environment, a **`.env.production`** file
would be sourced if present in the local project directory.
If both `.env` and `.env.production` are present, they are sourced in that order.
## Command line options & environment variables
The properties in the configuration file can also be specified through *command line options* or *environment variables*.
For example, the `repo` property in the configuration file can also be specified:
* With the `-r, --repo ` command line option.
* With the `$DEPLOY_REPO` environment variable.
Their precedence is as follows:
* The command line option (e.g. `-r, --repo `) always takes precedence if specified.
* Next, the environment variable (e.g. `$DEPLOY_REPO`) takes precedence over the configuration file.
* Otherwise, the value in the configuration file (e.g. `repo`) is used.
### General options
These options are valid for most sub-commands:
* **`-C, --chdir `** (or the `$DEPLOY_CHDIR` variable) changes **deploy**'s *local* working directory before loading the configuration file.
* **`--color always|never|auto`** (or the `$DEPLOY_COLOR` variable) enables/disables colors in the output of the script.
This defaults to `auto`, which only enables colors if the current terminal is interactive.
* **`-c, --config `** (or the `$DEPLOY_CONFIG` variable) allows you to set a custom path for the configuration file (defaults to `./deploy.conf`).
* **`--help`** prints usage information and exits.
* **`-v, -V, --version`** prints the current version and exits.
* **`-y, --yes`** (or the `$DEPLOY_YES` variable) will automatically accept all confirmation prompts
(only valid for the `rev`, `cleanup` and `update` sub-commands).
**Use with caution:** old releases may be deleted with the `rev` or `cleanup` sub-commands if a [`keep` option](#keep) is configured.
### Project options
These options configure where and how the project is deployed on the remote host:
* **`-r, --repo `** (or the `$DEPLOY_REPO` variable) sets the Git URL to fetch the project's source code from when deploying.
* **`-P, --path `** sets the remote path to deploy the project to on the host
(this is the path to the directory managed by **deploy**, not to the current release).
* **`-k, --keep `** (or the `$DEPLOY_KEEP` variable) changes the number of old releases that are kept
after successful deployment (see the [`keep` option](#keep)).
* **`-f, --force`** (or the `$DEPLOY_FORCE` variable) forces deployment to proceed even when you have uncommitted changes.
### SSH options
These options are valid for all commands which connect to the remote host through SSH
(`setup`, `rev`, `console`, `exec`, `list` and `cleanup`):
* **`-A, --forward-agent`** (or the `$DEPLOY_FORWARD_AGENT` variable) enables forwarding of the authentication agent connection.
**Use with caution.**
Users with the ability to bypass file permissions on the remote host (for the agent's UNIX-domain socket)
can access the local agent through the forwarded connection.
An attacker cannot obtain key material from the agent,
however they can perform operations on the keys that enable them to authenticate
using the identities loaded into the agent.
* **`-H, --host
`** (or the `$DEPLOY_HOST` variable) sets the host to connect to.
* **`-i, --identity `** (or the `$DEPLOY_IDENTITY` variable) selects a file from which the identity (private key)
for public key authentication is read.
The default is `~/.ssh/id_dsa`, `~/.ssh/id_ecdsa`, `~/.ssh/id_ed25519` and `~/.ssh/id_rsa`.
* **`-p, --port `** (or the `$DEPLOY_PORT` variable) sets the port to connect to.
* **`-t, --tty`** (or the `$DEPLOY_TTY` variable) forces pseudo-terminal allocation.
This can be used to execute arbitrary screen-based programs on a remote machine,
which can be very useful, e.g. when implementing menu services.
* **`-u, --user `** (or the `$DEPLOY_USER` variable) sets the remote user to connect as.
### Self-update options
These options can be used by **deploy** to update itself with the `update` sub-command:
* **`--update-path `** (or the `$DEPLOY_UPDATE_PATH` variable) sets the path to save the updated script when using the **update** command.
This overrides the `--update-prefix` or `$DEPLOY_UPDATE_PREFIX` options.
* **`--update-prefix `** (or the `$DEPLOY_UPDATE_PREFIX` variable) sets the base directory when using the **update** command.
The updated script will be saved in `bin/deploy` relative to this path.
(Note that setting `--update-path` or `$DEPLOY_UPDATE_PATH` overrides this option.)
## Command line options format
Note that all of **deploy**'s command line options can be used anywhere in a command.
The following 3 commands are equivalent:
* `deploy -u deploy production --keep 3 --yes master`
* `deploy -u deploy --keep 3 --yes production master`
* `deploy production master -u deploy --keep 3 --yes`
### Short option format
Some options have a short form which may be specified with **one hyphen**,
e.g. `-h` to display usage or `-f` to force deployment with uncommitted local changes.
If a short option takes a value, it may be provided either as the next argument
(e.g. `-k 5`) or with an equal sign (e.g. `-k=5`).
### Long option format
All options have a long form which may be specified with **two hyphens**,
e.g. `--help` or `--force`.
If a long option takes a value, it may be provided either as the next argument
(e.g. `--keep 5`) or with an equal sign (e.g. `--keep=5`).
### Invalid options
The following errors will cause the script to be interrupted and print an error:
* Badly formatted or unknown options (e.g. `---foo`, `-bar`, `--baz`).
* Options that take no value specified with a value (e.g. `--help=3`).
* Options that take a value specified without their value.
* Duplicate options (e.g. both `-k 5` and `--keep 3`).
### Changing the working directory
By default, **deploy** looks in the current directory to find its configuration file
and optional environment files. This can be customized by setting the `-c, --chdir `
command line option or the `$DEPLOY_CHDIR` variable.
If this option is set, **deploy** will first move to that directory before sourcing
the environment files and reading the configuration file.
## Default command
**deploy** is usually used this way:
* `deploy prod setup`
* `deploy prod list`
* `deploy prod rev master`
However, since the most often used sub-command is `rev`,
it is also the **default command**. The following 2 commands are equivalent:
* `deploy prod master`
* `deploy prod rev master`
## Sub-commands
These are the commands you will use to set up and deploy your projects.
Note that most (but not all) of **deploy**'s sub-commands require an environment to be specified before the actual command name, for example:
```sh
deploy setup # error! no environment
deploy production setup # all good
```
### ` setup`
Perform **first-time setup** tasks on the host before deployment.
You should only have to run this once before deploying the first time.
```sh
deploy production setup
```
**deploy** will create the following structure for you in the deployment directory defined by the `path` config key (or the `$DEPLOY_PATH` variable):
* `$DEPLOY_PATH/releases` will contain each deployment's files
* `$DEPLOY_PATH/repo` will be a bare clone of your Git repository
* `$DEPLOY_PATH/tmp` will be used to store temporary files during deployment
This phase has 3 hooks that are all executed in the deployment directory.
Note that this directory **must already exist** on the host and be writable by the user you connect as.
This is what **deploy** will do during setup:
* Run user-defined `pre-setup` hooks (if any).
* Create the `releases` and a `tmp` directories if they don't exist already.
* Clone the repository into the `repo` directory if it isn't already there.
* Run user-defined `post-setup` hooks (if any).
### ` rev [rev]`
**Deploy a new release** from the latest changes in the Git repository.
You must provide a Git revision, i.e. a **commit, branch or tag** to deploy,
either as the `[rev]` argument, with the `rev` config key or the `$DEPLOY_REV` variable.
If your Git repository is **private**, make sure that the deployment user has access to it.
For each deployment, a new release directory will be created in `releases` in the deployment directory.
The name of a release directory is the current date and time in the `YYYY-MM-DD-HH-MM-SS` format.
(You can list deployed releases with the [list command](#list).)
Note that **deploy** will refuse to deploy unless all your local changes are committed and pushed.
You can override this behavior with the `-f|--force` option (or the `$DEPLOY_FORCE` variable).
This is what **deploy** will do during deployment:
* If a [`keep` option](#keep) is configured, all previously deployed releases are listed,
with those to be deleted in red, and those to be kept and the current release in green, and
confirmation is asked that they may be deleted if the deployment is successful.
* User-defined pre-deploy hooks (if any) are run **in the directory of the previous release**.
You may define **pre-deploy hooks** to perform any task you might want to do before the actual deployment
(e.g. you might want to stop the currently running version or put it into maintenance mode).
* The latest changes are fetched from the repository.
* The new release directory is created and the source is extracted at the specified revision into it.
* User-defined deploy hooks (if any) are run in the new release directory.
You should define **deploy hooks** to build your application or install its dependencies at this stage.
* A symlink of the new release directory is created as `current` in the deployment directory.
* User-defined post-deploy hooks (if any) are run in the current release directory
(which is now the same as the new release directory that was just created).
You should define **post-deploy hooks** to execute or start your application at this stage.
* Old releases are deleted according to the `keep` option (if configured and any were found).
### `[env] config [key]`
Print values from your `deploy.conf` configuration file.
* If called with **no environment and no argument**, it prints the whole configuration file:
```sh
$> deploy config
[]
user dev
[production]
user root
port 222
```
* If called with **the environment and a key**, it prints the last value of that key for that environment
(only the last value is printed even for a multiple-value key):
```sh
$> deploy production config user
root
```
Exits with status 1 if no value is found for the key.
### ` config-all `
Print all values of a config key in the current environment and its inherited environments
(all values are printed even for a single-value key):
```sh
$> deploy production config-all user
dev
root
```
Exits with status 1 if no value is found for the key.
### `[env] config-section`
Print all values of a config section "as is" (with no inheritance).
If no environment is specified, the default config section is printed.
Exits with status 1 if the config section is not found (including the default one).
### ` console [path]`
Launch an interactive **ssh session** on the host.
* If an **absolute path** is specified, the session starts there.
```sh
deploy production console /var/www
```
* If **no path** is specified, the session starts in the deployment directory.
```sh
deploy production console
```
* If a **relative path** is specified, it starts there relative to the deployment directory.
```sh
deploy production console current
```
### ` cleanup`
Deletes old releases based on the [`keep` option](#keep).
The number of releases to keep is determined from the `keep` configuration file option,
the `$DEPLOY_KEEP` variable or the `-k|--keep` command line option.
The releases to delete will be listed and confirmation will be asked before they are actually deleted.
### ` exec `
**Execute** the specified command on the host.
```sh
deploy production exec ps -ef
```
### ` list`
List the **releases** that have been deployed on the host.
```sh
$> deploy production list
2016-12-24-17-45-23
2017-01-01-02-03-43
2017-04-01-00-00-00
```
### `update [rev]`
Updates **deploy** to the latest version by downloading it from the Git repository and installing it at `/usr/local/bin/deploy` (by default).
In addition to the basic requirements, this sub-command also requires `chmod`, `cp` and `mktemp` to be available in the shell.
The installation path must be a writable file or not exist.
If the installation path does not already exist, its parent must be a writable directory.
To perform the update, **deploy** will download its Git repository into a temporary directory that will be cleaned up when the update is done (or fails).
The correct revision of the script is then copied to the installation path and made executable.
## Cleaning up old releases
**deploy** creates a new release directory for each deployment.
By default, it keeps these directories forever, leaving you responsible for cleaning them up.
If given a [`keep` option](#keep), it can do it for you automatically after each deployment.
**deploy** will list the contents of the `releases` directory on the host.
If no old release is found, deployment will proceed normally.
If the number of old releases is equal to or greater than the `keep` number,
**deploy** will print the list of releases that will be deleted in red.
Old releases that will be kept will be shown in green.
During deployment, the current release being deployed is also indicated in green
(as it counts towards the `keep` number).
If there are any releases to delete, **deploy** will ask for confirmation
(unless the `-y, --yes` command line option or the `$DEPLOY_YES` variable is specified).
Responding "no" interrupts the deployment and prints a help message explaining
how to configure the `keep` number.
The actual cleanup of old releases is always performed **after successful deployment**.
A single `rm` command is executed on the host through SSH,
deleting all appropriate old release directories in one go.