https://github.com/shieldproject/shield-phalanx
A Bespoke Testing Formation for SHIELD
https://github.com/shieldproject/shield-phalanx
Last synced: 5 months ago
JSON representation
A Bespoke Testing Formation for SHIELD
- Host: GitHub
- URL: https://github.com/shieldproject/shield-phalanx
- Owner: shieldproject
- License: mit
- Created: 2017-02-04T20:22:38.000Z (over 9 years ago)
- Default Branch: master
- Last Pushed: 2025-02-04T12:57:47.000Z (over 1 year ago)
- Last Synced: 2025-02-04T13:39:56.187Z (over 1 year ago)
- Language: Shell
- Size: 104 KB
- Stars: 1
- Watchers: 8
- Forks: 3
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
SHIELD Phalanx
==============
Welcome to **Phalanx**, a bespoke BOSH release for testing the
full-stack functionality of Stark & Wayne's [SHIELD][1] backup
solution.
Methodology
-----------
The gist of Phalanx is this: spin a bunch of data systems, run a
handful of SHIELDs with different configurations, and make sure
that backups and (more importantly) restore operations work!
See It Live!
------------
Current Scope of Tests
----------------------
Phalanx currently exercises the following parts of the SHIELD
distribution.
### The `s3` Plugin
For each SHIELD, a new AWS bucket is created to be used for
storing backup archives. This is the only currently supported
means of using the `s3` plugin, so if the rest of the tests
succeed, we can bet that the `s3` plugin works for both storage
and retrieval.
Note: this does not apply to the `s3`-derived `scality` plugin.
### The `postgres` Plugin
Two databases are created, and filled with some data.
Two tests are then carried out - a backup/restore of all
databases, and a backup/restore of just one of the databases.
### The `mysql` Plugin
Two databases are created, and filled with some data.
Two tests are then carried out - a backup/restore of all
databases, and a backup/restore of just one of the databases.
### The `consul` Plugin
Two keys are created in the remote Consul Key-Value store, via the
remote HTTP API (on port 8500). Then, a backup is performed
against Consul, and the keys are altered. After the backup is
restored, we check to make sure the keys revert to their original
values.
Deploying it Yourself
---------------------
If you want to do in-depth SHIELD development, you may want to
deploy your own copy of Phalanx, on a BOSH-lite. Here's the steps
to get you started.
First, generate a manifest:
```
./templates/make_manifest
```
Then, you're going to need to provide a release tarball of the
SHIELD you want to test. From your SHIELD repository, run:
```
make release VERSION=0.9.2-rc.3 # change version as needed
cp artifacts/shield-server-linux-amd64.tar.gz \
path/to/phalanx/src/shield-rc/
```
Then, cut a new (development) BOSH release and deploy your
manifest:
```
bosh create release --force
bosh upload release
bosh -n deploy
```
Once that's done, just run the `tests` errand:
```
bosh run errand tests
```
If things don't turn out well, you may want to run the errand with
the `--keep-alive` flag, and then `bosh ssh` into the `tests/0`
instance and poke around.
Writing new Tests
-----------------
Tests exist as scripts inside of the
`/var/vcap/jobs/run-tests/tests` directory on the errand VM. All
you have to do to create a new test is add it to the `run-tests`
job spec, and create the files.
Here's a template script to get you started!
```
#!/bin/bash
source /var/vcap/jobs/run-tests/lib/functions system-name
rc=0
echo "> My New Test"
echo ">> setting up test data"
# FIXME: set up the data
echo ">> configuring SHIELD"
shield create-target <> running backup job"
wait_for_task $(shield run $(shield job foo-daily | jq -r '.uuid') | jq -r '.task_uuid')
echo ">> trashing the data"
# FIXME: trash the data by deleting stuff, changing stuff, etc.
echo ">> restoring from backups"
wait_for_task $(shield restore $(shield archives -t $(shield target foo | jq -r '.uuid') | jq -r '.[0].uuid') | jq -r '.task_uuid')
# FIXME: validate the data, post-restore,
# FIXME: setting rc= on failure.
exit $rc;
```
Future Work
-----------
The following tests need to be conceptualized and implemented.
- A test of the `consul` plugin, against a Consul Key-Value store
that requires HTTP Basic Authentication.
- A test of the `mongo` plugin, against a MongoDB instance that
requires authentication.
- Tests for all of the following plugins:
- docker-postgres
- elasticsearch
- rabbitmq-broker
- redis-broker
- scality
- xtrabackup
[1]: https://github.com/starkandwayne/shield