Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/glueops/backup-tools
This repository contains backup tools and scripts for automating the backup of HashiCorp Vault data, S3 buckets, and GitHub organizations. It includes Bash scripts and Dockerfiles to facilitate creating backups, uploading them to AWS S3, and validating Vault backups to ensure data integrity.
https://github.com/glueops/backup-tools
automation aws-s3 backup backup-automation backup-scripts backup-tools bash docker github github-backup github-organization-backup hashicorp-vault s3 scripts vault vault-backup
Last synced: 20 days ago
JSON representation
This repository contains backup tools and scripts for automating the backup of HashiCorp Vault data, S3 buckets, and GitHub organizations. It includes Bash scripts and Dockerfiles to facilitate creating backups, uploading them to AWS S3, and validating Vault backups to ensure data integrity.
- Host: GitHub
- URL: https://github.com/glueops/backup-tools
- Owner: GlueOps
- Created: 2023-05-31T18:52:45.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-12-16T10:06:46.000Z (20 days ago)
- Last Synced: 2024-12-16T13:14:25.879Z (20 days ago)
- Topics: automation, aws-s3, backup, backup-automation, backup-scripts, backup-tools, bash, docker, github, github-backup, github-organization-backup, hashicorp-vault, s3, scripts, vault, vault-backup
- Language: Shell
- Homepage:
- Size: 76.2 KB
- Stars: 1
- Watchers: 5
- Forks: 0
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Requirements for ALL backup jobs
- AWS S3 Credentials. Regardless of whether you backup google team drives or github repos, you will need these environment variables to be set:
```bash
export AWS_DEFAULT_REGION=us-west-2 # must be the same region as the bucket
export S3_BUCKET_NAME="bucket-name"
export AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXXX
export AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXX
```
# GitHub Organization backups
Following variables must be set
```bash
export GITHUB_ORG_TO_BACKUP="GlueOps" # Set this to the organization you want to backup. The GITHUB_TOKEN must have read access to all the repos in this organization.
export GITHUB_TOKEN="" # GitH needs to have read access to all repositories within the organization. We use the fine grained access tokens (beta feature)
```
- To create a GITHUB_TOKEN use the newer fine grained tokens:
- Example backup
```bash
docker build . -t backup && docker run -it backup
# Export ALL the variables required as mentioned in this README.md and then run:
backup-github
```
# Google Drive Shared Drive Backups
## note this only works for shared team drives. we do not have anything to backup personal drives
- First the drive needs to be shared to our service account: `[email protected]` with `Contributor" access
- The team drive ID can be found from the URL. Example: https://drive.google.com/drive/folders/0ZZH9DD53YuyEaYU7sqb would have a team drive ID of: `0ZZH9DD53YuyEaYU7sqb`
Following variables must be set
```bash
export RCLONE_DRIVE_SERVICE_ACCOUNT_CREDENTIALS='<>' # Get this from the IAM user in the rclone google cloud service account project and remove all newlines \n
export RCLONE_DRIVE_TEAM_DRIVE="XXXXXXXXXXXXXX" # team drive id ex. `0ZZH9DD53YuyEaYU7sqb`
```
- Example to run a download of the team drive to local
```bash
docker build . -t backup && docker run -it backup
# Export ALL the variables required as mentioned in this README.md and then run:
./gdrive-backup.sh
```