{"id":20263622,"url":"https://github.com/springerpe/microbosh-simple-backup","last_synced_at":"2026-06-07T17:32:50.997Z","repository":{"id":24509148,"uuid":"27914864","full_name":"SpringerPE/microbosh-simple-backup","owner":"SpringerPE","description":"Simple backup script for a microbosh instance","archived":false,"fork":false,"pushed_at":"2015-10-12T14:07:29.000Z","size":285,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-14T04:44:36.935Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SpringerPE.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-12-12T09:55:41.000Z","updated_at":"2017-04-06T15:17:04.000Z","dependencies_parsed_at":"2022-08-23T00:01:05.231Z","dependency_job_id":null,"html_url":"https://github.com/SpringerPE/microbosh-simple-backup","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SpringerPE%2Fmicrobosh-simple-backup","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SpringerPE%2Fmicrobosh-simple-backup/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SpringerPE%2Fmicrobosh-simple-backup/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SpringerPE%2Fmicrobosh-simple-backup/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SpringerPE","download_url":"https://codeload.github.com/SpringerPE/microbosh-simple-backup/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241731744,"owners_count":20010781,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T11:35:44.441Z","updated_at":"2026-06-07T17:32:50.954Z","avatar_url":"https://github.com/SpringerPE.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"microbosh-simple-backup\n=======================\n\nSimple backup script for a microBOSH instance\n\nAbout microbosh-simple-backup\n=============================\n\nWarning! This information is about how to perform a manual recover \nof a bosh server.\n\nAssumptions:\n\n * bosh-deployments.yml is saved and up-to-date\n * micro_bosh.yml is saved and up-to-date.\n\nIn theory the blobstore is not needed, as it is going to be populated \nfrom the release you used to deploy, but it makes it much easier \nif that data is lost, the director still thinks it is available, and you \nhave to recover from that by reconstructing the blobstore and updating \nthe blob ids in the bosh database.\n\nSetup\n=====\n\nThe script works by reading a configuration file with some variables. You can pass\nthe configuration file as an argument (`-c`), but the script is able to read\nit automatically if one exists with the same name as the program (except the suffix).\n\nSo, by creating a links to the script and multiple configuration files with\nthe same name (only changing the sufix `.sh` into `.conf`) and using different \nvariables, is possible to backup different microBOSH instances.\n\nAfter defining the configuration file and creating a link to the program,\nyou have to run the program with the argument `setup`. By doing this, the program \nwill copy the _$SSH_PUBLIC_KEY_ and create the file `/etc/sudoers.d/backup` to allow \nthe execution of some commands with sudo.\n\n```\n# ./microbosh-simple-backup-test.sh setup\n--microbosh-simple-backup-test.sh 2014-12-21 01:19:52: Creating folders ...\n--microbosh-simple-backup-test.sh 2014-12-21 01:19:52: Copying public key to 10.10.10.10 ... \n/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed\n/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys\nvcap@10.10.10.10's password: \n\nNumber of key(s) added: 1\n\nNow try logging into the machine, with:   \"ssh 'vcap@10.10.10.10'\"\nand check to make sure that only the key(s) you wanted were added.\n\n--microbosh-simple-backup-test.sh 2014-12-21 01:19:59: Creating sudoers file ...\nvcap@10.10.10.10's password: \n[sudo] password for vcap: c1oudc0w\n--microbosh-simple-backup-test.sh 2014-12-21 01:20:16: Testing connection: \nThe Monit daemon 5.2.4 uptime: 29m \n\nProcess 'nats'                      running\nProcess 'redis'                     running\nProcess 'postgres'                  running\nProcess 'powerdns'                  running\nProcess 'blobstore_nginx'           running\nProcess 'director'                  running\nProcess 'worker_1'                  running\nProcess 'worker_2'                  running\nProcess 'worker_3'                  running\nProcess 'director_scheduler'        running\nProcess 'director_nginx'            running\nProcess 'health_monitor'            running\nSystem 'system_bm-e0bcbfab-fd07-4fe7-b110-675e27a85225' running\n```\n\nRun the backup\n==============\n\n```\n# ./microbosh-simple-backup-test.sh backup\n--microbosh-simple-backup-test.sh 2014-12-21 01:30:12: Checking monit processes ... ok (13 running)\n--microbosh-simple-backup-test.sh 2014-12-21 01:30:13: Stopping bosh processes  ........ done\n--microbosh-simple-backup-test.sh 2014-12-21 01:30:30: Starting DB backup. Starting processes ......... done\n--microbosh-simple-backup-test.sh 2014-12-21 01:30:49: Dumping database bosh ...  done\n--microbosh-simple-backup-test.sh 2014-12-21 01:30:50: Stopping DB processes ..... done!\n--microbosh-simple-backup-test.sh 2014-12-21 01:31:05: Stopping bosh agent ... ok\n--microbosh-simple-backup-test.sh 2014-12-21 01:31:07: Copying files with rsync ... done!\n--microbosh-simple-backup-test.sh 2014-12-21 01:31:07: Starting bosh agent ... ok\n--microbosh-simple-backup-test.sh 2014-12-21 01:31:08: Starting monit processes ... done\n--microbosh-simple-backup-test.sh 2014-12-21 01:31:25: Cleaning temp files and copying logs ... end\n```\n\nIf something went wrong it will finish with an error (return code not 0) and\nit will show the error log. Moreover, the program only performs the backup if \nall monit services are in running state (monit summary), avoiding to create\nnon consistent backups or running two process simultaneously.\n\nThe script logs almost everything on _/var/log/scripts_ and also includes\na copy of this logfile within the output tar file.\n\nHow the backups are done\n========================\n\nThe script just performs those steps (in order) using the variables\nwhich are defined in the configuration file. Those are the steps for --offline \nmode, which means that all processes are stopped to create a consistent backup.\n\n1. Logon in the microBosh VM with the parameters from the configuration file:\n  * `# ssh $USER@$HOST`\n2. Stop all bosh jobs,daemons and processes:\n  * `# sudo /var/vcap/bosh/bin/monit stop all`\n3. Wait for the termination of all processes\n  * `# sudo /var/vcap/bosh/bin/monit summary`\n4. Stop the bosh agent\n  * `# sudo /usr/bin/sv stop agent`\n5. If `$DBS` is not empty:\n  1. Start the postgresql daemon:\n    - `# sudo /var/vcap/bosh/bin/monit start postgres`\n  2. Perform a db dump with pg_dumpall if `$DBS == _all_`\n    - `# /var/vcap/packages/postgres/bin/pg_dumpall --clean -f /var/vcap/store/postgres_$(date '+%Y%m%d%H%M%S').dump.all`\n    otherwise, it will dump looping with each db with pg_dump:\n    - `# /var/vcap/packages/postgres/bin/pg_dump --create /var/vcap/store/postgres_$(date '+%Y%m%d%H%M%S').dump.$DB`\n  3. Stop all daemons again:\n    - `# sudo /var/vcap/bosh/bin/monit stop all`\n6. From your local server: rsync /var/vcap/store using vcap user and the RSYNC_LIST\n  * `# rsync -arzhv --delete --include-from=\"RSYNC_LIST\" $USER@$HOST:/var/vcap/store/ $CACHE/`\n7. On the bosh server, start all monit services\n  * `# sudo /var/vcap/bosh/bin/monit start all`\n8. On the bosh server, start bosh agent\n  * `# sudo /usr/bin/sv start agent`\n9. On the bosh server, remove database dumps (if exists)\n  * `# rm -f /var/vcap/store/postgres_*`\n10. Optional, on your local server, create a tgz file from the cache\n  * `# tar -zcvf $OUTPUT $CACHE`\n\nRecovering bosh\n===============\n\nIf you are using the same version for the stemcell, those steps should be enough.\nOtherwise, in case of problems, there is a db dump with all the information.\n\n1. Logon using vcap (for example)\n  * `# ssh $USER@$HOST`\n2. Stop all bosh processes\n  * `# sudo /var/vcap/bosh/bin/monit stop all`\n3. Wait for finishing all processes\n  * `# sudo /var/vcap/bosh/bin/monit summary`\n4. Stop microBosh agent\n  * `# sudo /usr/bin/sv stop agent`\n5. From your local server, rsync all the data cached from the latest backup,\n  otherwise, you can copy a tgz file and uncompress it manually on the bosh\n  server:\n  * `# rsync -arzhv --delete $CACHE/vcap/store/postgres $USER@$HOST:/var/vcap/store/postgres/`\n  * `# rsync -arzhv --delete $CACHE/vcap/store/blobstore/ $USER@$HOST:/var/vcap/store/blobstore`\n  or by coping a tgz file:\n  * `# rsync -arzhv BACKUP.tgz $USER@$HOST:/var/vcap/store/`\n  and go to bosh server ...\n6. On the bosh server, start bosh agent\n  * `# sudo /usr/bin/sv start agent`\n7. Start all bosh processes\n  * `# sudo /var/vcap/bosh/bin/monit start all`\n8. On the bosh server, check that everything is started\n  * `# sudo /var/vcap/bosh/bin/monit summary`\n  * In an 2619 stemcell you will see that the system is not monitored: `System 'system_bm-e0bcbfab-fd07-4fe7-b110-675e27a85225' not monitored`. You have to start it manually: `$ sudo /var/vcap/bosh/bin/monit start system_bm-e0bcbfab-fd07-4fe7-b110-675e27a85225`. As of stemcell 2719 this will be fixed.\n8. `bosh cloudcheck` (executed from bosh-workspace) will show that the uuid of the system has changed. Thus it has to be changed in the manifest as well. `bosh status` shows the new uuid. Replace the old uuid in `{test,dev,live}/manifests/RELEASE.yml`\n9. Change the admin password: `bosh create user admin very-long-and-secure-passphrase`\n\nBy doing those operations, you should be able to recover everything,\nif that does not work, then you will need more hacks.\n\nBe carefull, because the database contains references like: vm_cid, \nuuid, etc. so depending on the disaster situation you will need to \nchange the uuid, vm_cid, the disk uuid and path on the dumped database \nbefore importing it (please have a look and compare with the \nbosh-deployments file).\n\nIn case you have a DB dump with from all postgres, in order to extract \nthe bosh database from the dump, you can type:\n\n```\n# awk '/^\\\\connect bosh/ {flag=1;next} /^\\\\connect/ {flag=0} flag { print }' \u003c postgres_*.dump.all \u003e bosh.sql\n```\n\nAfter that you can edit the file, or just import it on another DB and\nwork with both databases.\n\n```\n# /var/vcap/packages/postgres/bin/createdb bosh-backup\n# /var/vcap/packages/postgres/bin/psql -f bosh.sql bosh-backup\n# /var/vcap/packages/postgres/bin/psql\n```\n\nLicense and Authors\n===================\n\nLicense:: GPLv3\n\nAuthor:: Jose Riguera Lopez, Springer SBM (jose.riguera@springer.com)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspringerpe%2Fmicrobosh-simple-backup","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fspringerpe%2Fmicrobosh-simple-backup","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspringerpe%2Fmicrobosh-simple-backup/lists"}