{"id":19693828,"url":"https://github.com/aiven-open/astacus","last_synced_at":"2025-04-29T10:30:41.175Z","repository":{"id":37976868,"uuid":"258482175","full_name":"Aiven-Open/astacus","owner":"Aiven-Open","description":"Clustered database backup","archived":false,"fork":false,"pushed_at":"2024-11-06T15:04:38.000Z","size":2173,"stargazers_count":36,"open_issues_count":9,"forks_count":4,"subscribers_count":63,"default_branch":"main","last_synced_at":"2024-11-06T16:19:46.714Z","etag":null,"topics":["backups","cassandra","cluster","database","m3db","restore"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Aiven-Open.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-04-24T10:35:58.000Z","updated_at":"2024-11-05T08:55:28.000Z","dependencies_parsed_at":"2024-11-06T16:19:23.154Z","dependency_job_id":"56e6e043-9494-4655-bf4f-93886edcae5e","html_url":"https://github.com/Aiven-Open/astacus","commit_stats":null,"previous_names":["aiven-open/astacus"],"tags_count":56,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aiven-Open%2Fastacus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aiven-Open%2Fastacus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aiven-Open%2Fastacus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aiven-Open%2Fastacus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Aiven-Open","download_url":"https://codeload.github.com/Aiven-Open/astacus/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224160354,"owners_count":17265965,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backups","cassandra","cluster","database","m3db","restore"],"created_at":"2024-11-11T19:18:18.967Z","updated_at":"2025-04-29T10:30:41.164Z","avatar_url":"https://github.com/Aiven-Open.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# astacus\n\n[![codecov](https://codecov.io/gh/Aiven-Open/astacus/branch/main/graph/badge.svg)](https://codecov.io/gh/Aiven-Open/astacus)\n\n\n\u003e [!CAUTION]\n\u003e This repository is no longer actively maintained.\n\nAstacus is a clustered database backup system that is meant to work with\nmultiple open-source cluster databases, such as\n[M3](https://github.com/m3db/m3/) and\n[Apache Cassandra](https://cassandra.apache.org).\n\n_My name is Maximus Backupus Astacus, Co-ordinator of the Backups of the\nCluster, Master  of the Storage Availability, loyal servant to the true\nemperor, Prunus Aivenius. Father to a failed backup, husband to a corrupted\ndata. And I will have my restore, in this runtime or the next._\n\n# Goals\n\n- Support multiple clustered database products\n- Most of the code generic\n- Product-specific code with simple, testable API\n\n- Complexities to deal with e.g. reuse of blobs with same value in the\nshared code\n    - It is needed to accomplish e.g. fast non-incremental M3 backups that are\n    essentially incremental as only commit logs change frequently\n\n- Support list of object storage backup site locations -\u003e Facilitate\n  migration from old to new during service cloud migration\n\n- Have most of the code covered by unit tests\n\n# See also\n\n- [Design overview](doc/design/overview.md)\n- [Implementation overview](doc/design/implementation.md)\n\n# Installation\n\nPlease see Dockerfile.fedora and Dockerfile.ubuntu for concrete up-to-date\nexamples, but here are the current ones:\n\n## Optional features\n\n- cassandra can be added with 'cassandra' optional:\n```\nsudo pip3 install -e '.[cassandra]'\n```\n\n\n\n## Fedora 34\n\n(as root or user with sudo access; for root, skip sudo prefix)\n\n```\nsudo dnf install -y make\nmake build-dep-fedora\nsudo python3 ./setup.py install\n```\n\n## Ubuntu 20.04\n\n(as root or user with sudo access; for root, skip sudo prefix)\n\n```\nsudo apt-get update\nsudo apt-get install -y make sudo\nmake build-dep-ubuntu\nsudo python3 ./setup.py install\n```\n\n\n# Configuration\n\nCreate astacus.conf, which specifies which database to back up, and where.\nThe configuration file format is YAML, but as it is JSON superset, JSON is\nalso fine.\n\nUnfortunately the configuration part is not particularly well documented at\nthis time, but there are some examples of file backups to\n[local directory (JSON)](examples/astacus-files-local.json),\n[local directory (YAML)](examples/astacus-files-local.yaml), [Amazon S3](examples/astacus-files-s3.json), or\n[Google GCS](examples/astacus-files-gcs.json). There is even one example of\n[backing up M3 to GCS](examples/astacus-m3-gcs.json).\n\n\n# Usage\n\n## Start the nodes\n\nStart astacus server on all nodes to be backed up, either by hand or via\ne.g. systemd:\n\n`astacus server -c \u003cpath to configuration file\u003e`\n\n## Perform backups\n\nPeriodically (e.g. from cron) call on (ideally only one node, but it\ndoesn't really matter as only one operation can run at a time):\n\n- `astacus backup` or\n- HTTP POST to http://server-address:5515/backup\n\n\n## Restore backups\n\nBackup can be restored with either\n\n- `astacus restore` or\n- HTTP POST to http://server-address:5515/restore\n\n## List backups\n\nTo see list of backups:\n\n- `astacus list` or\n- HTTP GET http://server-address:5515/list\n\n## Clean up old backups\n\nTo clean up backups based on the configured retention policy,\n\n- use `astacus cleanup` (from cronjob or CLI), or\n- HTTP POST to http://server-address:5515/cleanup\n\n\n# TODO\n\nThere is separate [TODO](TODO.md) file which tracks what is still to be done.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faiven-open%2Fastacus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faiven-open%2Fastacus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faiven-open%2Fastacus/lists"}