{"id":13696317,"url":"https://github.com/copybird/copybird","last_synced_at":"2025-05-03T17:30:48.845Z","repository":{"id":101238270,"uuid":"192913141","full_name":"copybird/copybird","owner":"copybird","description":"Cloud-native universal backup tool for databases and files","archived":false,"fork":false,"pushed_at":"2020-02-20T06:48:09.000Z","size":2810,"stargazers_count":66,"open_issues_count":2,"forks_count":6,"subscribers_count":10,"default_branch":"master","last_synced_at":"2024-11-13T00:32:40.425Z","etag":null,"topics":["backup","clickhouse","cloudnative","database","devops","elasticsearch","etcd","kubernetes","kubernetes-operator","mariadb","mongodb","mysql","postgresql"],"latest_commit_sha":null,"homepage":"https://copybird.org","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/copybird.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-06-20T12:07:25.000Z","updated_at":"2024-10-24T15:34:42.000Z","dependencies_parsed_at":null,"dependency_job_id":"3cd673c7-956e-4223-8892-dd259c60e924","html_url":"https://github.com/copybird/copybird","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/copybird%2Fcopybird","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/copybird%2Fcopybird/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/copybird%2Fcopybird/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/copybird%2Fcopybird/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/copybird","download_url":"https://codeload.github.com/copybird/copybird/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252226621,"owners_count":21714834,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backup","clickhouse","cloudnative","database","devops","elasticsearch","etcd","kubernetes","kubernetes-operator","mariadb","mongodb","mysql","postgresql"],"created_at":"2024-08-02T18:00:38.399Z","updated_at":"2025-05-03T17:30:48.356Z","avatar_url":"https://github.com/copybird.png","language":"Go","readme":"\u003cdiv style=\"display: flex; align-items: center;\" align=\"center\"\u003e\n\u003ca href=\"https://copybird.org\"\u003e\u003cimg width=\"100px\" src=\"https://raw.githubusercontent.com/copybird/copybird/master/docs/logo.svg?sanitize=true\" alt=\"Copybird\"\u003e\u003c/a\u003e\n\u003c/div\u003e\n\n# Copybird\n\n[![Developed by Mad Devs](https://maddevs.io/badge-dark.svg)](https://maddevs.io/)\n[![Project Status: WIP – Initial development is in progress, but there has not yet been a stable, usable release suitable for the public.](https://www.repostatus.org/badges/latest/wip.svg)](https://www.repostatus.org/#wip)\n[![](https://images.microbadger.com/badges/version/copybird/copybird.svg)](https://microbadger.com/images/copybird/copybird)\n[![](https://images.microbadger.com/badges/image/copybird/copybird.svg)](https://microbadger.com/images/copybird/copybird)\n[![](https://godoc.org/github.com/copybird/copybird?status.svg)](http://godoc.org/github.com/copybird/copybird)\n[![GitHub release](https://img.shields.io/github/release/copybird/copybird/all.svg?style=flat-square)](https://github.com/copybird/copybird/releases)\n\n![](https://travis-ci.org/copybird/copybird.svg?branch=master)\n[![Coverage Status](https://coveralls.io/repos/github/copybird/copybird/badge.svg)](https://coveralls.io/github/copybird/copybird)\n[![Go Report Card](https://goreportcard.com/badge/github.com/copybird/copybird)](https://goreportcard.com/report/github.com/copybird/copybird)\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n\n## About\n\nCopybird is open-source **cloud-native** universal backup tool for databases and files. \n\nIt allows you to:\n1. Create database backup\n2. Compress backup stream\n3. Encrypt backup stream\n4. Send it to various destinations fast and secure\n5. Get notification about backup status in messagers and notification services\n6. Enjoy simple backup as a service with k8s backup controller\n\nBackup process not using local storage for temp files.\n\nlearn more at [copybird.org](https://copybird.org). Note that this repository is in Work in Progress status. Feel free to contribure. Read more about contributing below.\n\n## Databases\nCurrently Copybird supports the following databases:\n- MySQL\n- Postgres\n- MongoDB\n- Etcd (v2 and v3 API)\n\n## Compression\nCopybird compresses with the following tools:\n- gzip\n- lz4\n\n## Encryption\nCopybird uses AES-GCM for Efficient Authenticated Encryption\n\n## Output\nCopybird can deliver encrypted compressed backup to the following destinations:\n- store the file locally\n- save it on [GCP](https://cloud.google.com/‎)\n- save it on [S3](https://aws.amazon.com/s3/)\n- send over HTTP\n- send over SCP\n\n## Notification services\nCopybird currently supports the following notification services: \n\n- Slack\n- Telegram\n- AWS SES\n- AWS SQS\n- get notificatoin on email\n- Kafka\n- Nats\n- Create issue in PagerDuty\n- Pushbullet\n- RabbitMQ\n- Twilio\n- Webcallback\n\nIf you would like to add additional service, please submit an issue with feature request or add it yourself and send a Pull Request.\n\n## How to Run the tool\nThere are different ways you can use this tool: \n\n### Run locally\nFirst get the source code on your machine\n```\ngo get -u github.com/copybird/copybird\n```\nThen run it with `go run main.go` to see helpers for various optional parameters\nExample creating MySQL dump: \n```\ngo run -v main.go backup -i 'mysql::dsn=root:root@tcp(localhost:3306)/test' -o local::file=dump.sql\n```\n\n### Run with Docker\nRun `docker run copybird/copybird` to see the available optional parameters\n\n### Use Backup Custom Controller/Operator for k8s\n\nFirst create custom resource definition in your cluster: \n```\nkubectl apply -f operator/crd/crd.yaml\n```\n\nTo run the controller:\n``` \ngo run main.go operator\n```\n\nAnd then in a separate shell, create custom resource:\n```\nkubectl create -f operator/example/backup-example.yaml\n```\nAs output you get the following logs when creating, updating or deleting custom resource:\n```\nINFO[0000] Successfully constructed k8s client          \nINFO[0000] Starting Foo controller                      \nINFO[0000] Waiting for informer caches to sync          \nINFO[0001] Starting workers                             \nINFO[0001] Started workers               \n```\nYou can modify example file as you wish to get proper configuration for your jobs\n\n## Tests \n\nTo run tests against MySQL module proceed with the following commands: \n```\ndocker run --name test_mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=root -e MYSQL_DATABASE=test -d percona:latest\ndocker exec -i test_mysql mysql -uroot -proot test \u003c samples/mysql.sql\ncd modules/backup/input/mysql/\ngo test -v -cover\n```\nTo clean up after you finish with tests: \n```\ndocker kill test_mysql\ndocker rm test_mysql\n```\nTo run tests against MySQLDump module, first make sure that you have `mysqldump` binary\navailable in `$PATH` and then proceed with the following commands: \n```\ndocker run --name test_mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=root -e MYSQL_DATABASE=test -d percona:latest\ndocker exec -i test_mysql mysql -hlocalhost -uroot -proot test \u003c samples/mysql.sql\ncd modules/backup/input/mysqldump/\ngo test -v -cover\n```\nTo clean up after you finish with tests: \n```\ndocker kill test_mysql\ndocker rm test_mysql\n```\nTo run tests against Postgres module proceed with the following commands: \n```\ndocker run --name test_postgres -p 5432:5432 -e POSTGRES_PASSWORD=postgres -e POSTGRES_DB=test -d postgres:latest\ndocker exec -i test_postgres psql -U postgres test \u003c samples/postgres.sql\ncd modules/backup/input/postgresql/\ngo test -v -cover\n```\nTo clean up after you finish with tests: \n```\ndocker kill test_postgres\ndocker rm test_postgres\n```\n\n\n## Contributing\nPull requests are more than welcomed. For major changes, please open an issue first to discuss what you would like to change. \n\nBefore submission of pull request make sure you pulled recent updates, included tests for your code that covers at least the core functionality and you submitted a desciptive issue that will be fixed with your pull request. Do not forget to mention the issue in the pull request. \n\nProject started by \u003ca href=\"https://github.com/miolini\"\u003eArtem Andreenko\u003c/a\u003e, \u003ca href=\"https://github.com/gen1us2k\"\u003eAndrew Minkin\u003c/a\u003e, and \u003ca href=\"https://maddevs.io\"\u003eMad Devs.\u003c/a\u003e\n\n\u003cdiv align=\"center\"\u003e\n    \u003ch3\u003eBuilt with Mad Devs support for the community\u003c/h3\u003e\n    \u003ca href=\"https://maddevs.io\"\u003e\u003cimg height=\"100px\" src =\"docs/md-logo.png\" /\u003e\u003c/a\u003e\n\u003c/div\u003e\n\nP.S. Love distributed and mesh? \u003ca href=\"https://github.com/meshbird/meshbird\"\u003eMeshbird\u003c/a\u003e.\n\n","funding_links":[],"categories":["Integrations"],"sub_categories":["Data Transfer and Synchronization"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcopybird%2Fcopybird","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcopybird%2Fcopybird","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcopybird%2Fcopybird/lists"}