{"id":13626093,"url":"https://github.com/bchew/dynamodump","last_synced_at":"2025-05-14T03:03:19.997Z","repository":{"id":2088202,"uuid":"14415590","full_name":"bchew/dynamodump","owner":"bchew","description":"Simple backup and restore for Amazon DynamoDB using AWS SDK for Python (boto3)","archived":false,"fork":false,"pushed_at":"2025-04-02T01:49:52.000Z","size":362,"stargazers_count":870,"open_issues_count":4,"forks_count":289,"subscribers_count":21,"default_branch":"master","last_synced_at":"2025-04-10T18:37:45.269Z","etag":null,"topics":["aws","backup","dynamodb","hacktoberfest","python","python3","restore"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bchew.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2013-11-15T05:00:00.000Z","updated_at":"2025-04-02T01:49:55.000Z","dependencies_parsed_at":"2023-12-03T04:21:11.637Z","dependency_job_id":"626e7eab-3599-4346-9886-8c9e25d91e26","html_url":"https://github.com/bchew/dynamodump","commit_stats":{"total_commits":329,"total_committers":34,"mean_commits":9.676470588235293,"dds":0.5714285714285714,"last_synced_commit":"2e4ed57123dea55b3f8aad0e6ec09af9d6878dee"},"previous_names":[],"tags_count":55,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bchew%2Fdynamodump","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bchew%2Fdynamodump/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bchew%2Fdynamodump/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bchew%2Fdynamodump/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bchew","download_url":"https://codeload.github.com/bchew/dynamodump/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254059474,"owners_count":22007767,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","backup","dynamodb","hacktoberfest","python","python3","restore"],"created_at":"2024-08-01T21:02:09.824Z","updated_at":"2025-05-14T03:03:19.887Z","avatar_url":"https://github.com/bchew.png","language":"Python","readme":"# dynamodump\n\n![Build Status](https://github.com/bchew/dynamodump/actions/workflows/build.yml/badge.svg)\n![CodeQL Status](https://github.com/bchew/dynamodump/actions/workflows/codeql-analysis.yml/badge.svg)\n[![Docker Status](https://github.com/bchew/dynamodump/actions/workflows/docker.yml/badge.svg)](https://hub.docker.com/r/bchew/dynamodump)\n![Linting Status](https://github.com/bchew/dynamodump/actions/workflows/linting.yml/badge.svg)\n![Test Status](https://github.com/bchew/dynamodump/actions/workflows/test.yml/badge.svg)\n[![PyPI version](https://img.shields.io/pypi/v/dynamodump)](https://pypi.org/project/dynamodump)\n[![PyPI pyversions](https://img.shields.io/pypi/pyversions/dynamodump.svg)](https://pypi.org/project/dynamodump)\n![Code Style](https://img.shields.io/badge/code%20style-black-black)\n\nSimple backup and restore script for Amazon DynamoDB using AWS SDK for Python (boto3) to work similarly to mysqldump.\n\nSuitable for DynamoDB usages of smaller data volume which do not warrant the usage of AWS Data Pipeline for backup/restores/empty.\n\ndynamodump supports local DynamoDB instances as well (tested with [DynamoDB Local](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html)).\n\n## Table of Contents\n\n- [Installation](#installation)\n- [Usage](#usage)\n- [Script (unattended) usage](#script-unattended-usage)\n- [Docker CLI usage](#docker-cli-usage)\n- [AWS example](#aws-example)\n- [Local example](#local-example)\n- [Development](#development)\n\n## Installation\n\n```\npip install dynamodump\n```\n\n## Usage\n\n```\nusage: dynamodump.py [-h] [-a {zip,tar}] [-b BUCKET] [-m {backup,restore,empty}] [-r REGION] [--host HOST] [--port PORT] [--accessKey ACCESSKEY] [--secretKey SECRETKEY] [-p PROFILE] [-s SRCTABLE] [-d DESTTABLE]\n                     [--prefixSeparator PREFIXSEPARATOR] [--noSeparator] [--readCapacity READCAPACITY] [-t TAG] [--writeCapacity WRITECAPACITY] [--schemaOnly] [--dataOnly] [--noConfirm] [--skipThroughputUpdate]\n                     [--dumpPath DUMPPATH] [--billingMode {PROVISIONED,PAY_PER_REQUEST}] [--log LOG] [--limit LIMIT] [-f FILTEROPTION]\n\nSimple DynamoDB backup/restore/empty.\n\noptions:\n  -h, --help            show this help message and exit\n  -a {zip,tar}, --archive {zip,tar}\n                        Type of compressed archive to create. If unset, don't create archive\n  -b BUCKET, --bucket BUCKET\n                        S3 bucket in which to store or retrieve backups. [must already exist]\n  -m {backup,restore,empty}, --mode {backup,restore,empty}\n                        Operation to perform\n  -r REGION, --region REGION\n                        AWS region to use, e.g. 'us-west-1'. Can use any region for local testing\n  --host HOST           Host of local DynamoDB. This parameter initialises dynamodump for local DynamoDB testing [required only for local]\n  --port PORT           Port of local DynamoDB [required only for local]\n  --accessKey ACCESSKEY\n                        Access key of local DynamoDB [required only for local]\n  --secretKey SECRETKEY\n                        Secret key of local DynamoDB [required only for local]\n  -p PROFILE, --profile PROFILE\n                        AWS credentials file profile to use. Allows you to use a profile instead accessKey, secretKey authentication\n  -s SRCTABLE, --srcTable SRCTABLE\n                        Source DynamoDB table name to backup or restore from, use 'tablename*' for wildcard prefix selection or '*' for all tables. Mutually exclusive with --tag\n  -d DESTTABLE, --destTable DESTTABLE\n                        Destination DynamoDB table name to backup or restore to, use 'tablename*' for wildcard prefix selection (defaults to use '-' separator) [optional, defaults to source]\n  --prefixSeparator PREFIXSEPARATOR\n                        Specify a different prefix separator, e.g. '.' [optional]\n  --noSeparator         Overrides the use of a prefix separator for backup wildcard searches [optional]\n  --readCapacity READCAPACITY\n                        Change the temp read capacity of the DynamoDB table to backup from [optional]\n  -t TAG, --tag TAG     Tag to use for identifying tables to back up. Mutually exclusive with srcTable. Provided as KEY=VALUE\n  --writeCapacity WRITECAPACITY\n                        Change the temp write capacity of the DynamoDB table to restore to [defaults to 25, optional]\n  --schemaOnly          Backup or restore the schema only. Do not backup/restore data. Can be used with both backup and restore modes. Cannot be used with the --dataOnly [optional]\n  --dataOnly            Restore data only. Do not delete/recreate schema [optional for restore]\n  --noConfirm           Don't ask for confirmation before deleting existing schemas.\n  --skipThroughputUpdate\n                        Skip updating throughput values across tables [optional]\n  --dumpPath DUMPPATH   Directory to place and search for DynamoDB table backups (defaults to use 'dump') [optional]\n  --billingMode {PROVISIONED,PAY_PER_REQUEST}\n                        Set billing mode between PROVISIONED|PAY_PER_REQUEST (defaults to use 'PROVISIONED') [optional]\n  --log LOG             Logging level - DEBUG|INFO|WARNING|ERROR|CRITICAL [optional]\n  --limit LIMIT         Limit option for backup, will stop the back up process after number of backed up items reaches the limit [optional]\n  -f FILTEROPTION, --filterOption FILTEROPTION\n                        Filter option for backup, JSON file of which keys are ['FilterExpression', 'ExpressionAttributeNames', 'ExpressionAttributeValues']\n```\n\nBackup files are stored in a 'dump' subdirectory, and are restored from there as well by default.\n\n## Script (unattended) usage\n\nAs of v1.2.0, note that `--noConfirm` is required to perform data restores involving deletions without any confirmation.\n\n## Docker CLI usage\n\n```\ndocker run --rm -it bchew/dynamodump -h\n```\n\ndynamodump container images are also published to Amazon ECR Public and GitHub Packages:\n\n```\npublic.ecr.aws/bchew/dynamodump\nghcr.io/bchew/dynamodump\n```\n\nLinks to the registries specified:\n- [Docker Hub](https://hub.docker.com/r/bchew/dynamodump)\n- [Amazon ECR Public Gallery](https://gallery.ecr.aws/bchew/dynamodump)\n- [GitHub Packages](https://github.com/bchew/dynamodump/pkgs/container/dynamodump)\n\n## AWS example\n\nSingle table backup/restore:\n\n```\ndynamodump -m backup -r us-west-1 -s testTable\n\ndynamodump -m restore -r us-west-1 -s testTable\n```\n\nMultiple table backup/restore (assumes prefix of 'production-' of table names, use --prefixSeparator to specify a\ndifferent separator):\n\n```\ndynamodump -m backup -r us-west-1 -s production*\n\ndynamodump -m restore -r us-west-1 -s production*\n```\n\nThe above, but between different environments (e.g. production-_ tables to development-_ tables):\n\n```\ndynamodump -m backup -r us-west-1 -s production*\n\ndynamodump -m restore -r us-west-1 -s production* -d development*\n```\n\nBackup all tables and restore only data (will not delete and recreate schema):\n\n```\ndynamodump -m backup -r us-west-1 -s \"*\"\n\ndynamodump -m restore -r us-west-1 -s \"*\" --dataOnly\n```\n\nDump all table schemas and create the schemas (e.g. creating blank tables in a different AWS account):\n\n```\ndynamodump -m backup -r us-west-1 -p source_credentials -s \"*\" --schemaOnly\n\ndynamodump -m restore -r us-west-1 -p destination_credentials -s \"*\" --schemaOnly\n```\n\nBackup all tables based on AWS tag `key=value`\n\n```\ndynamodump -p profile -r us-east-1 -m backup -t KEY=VALUE\n```\n\nBackup all tables based on AWS tag, compress and store in specified S3 bucket.\n\n```\ndynamodump -p profile -r us-east-1 -m backup -a tar -b some_s3_bucket -t TAG_KEY=TAG_VALUE\n\ndynamodump -p profile -r us-east-1 -m backup -a zip -b some_s3_bucket -t TAG_KEY=TAG_VALUE\n```\n\nRestore from S3 bucket to specified destination table\n\n```\n## source_table identifies archive file in S3 bucket from which backup data is restored\ndynamodump -a tar -b some_s3_bucket -m restore -r us-east-1 -p profile -d destination_table -s source_table\n```\n\n## Local example\n\nThe following assumes your local DynamoDB is running on localhost:8000 and is accessible via 'a' as access/secret keys.\nYou must specify the host to get local behavior.\n\n```\ndynamodump -m backup -r local -s testTable --host localhost --port 8000 --accessKey a --secretKey a\n\ndynamodump -m restore -r local -s testTable --host localhost --port 8000 --accessKey a --secretKey a\n```\n\nMultiple table backup/restore as stated in the AWS examples are also available for local.\n\n## Development\n\n```\npython3 -m venv env\nsource env/bin/activate\n\n# install dev requirements\npip3 install -r requirements-dev.txt\n\n# one-time install of pre-commit hooks\npre-commit install\n```\n","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbchew%2Fdynamodump","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbchew%2Fdynamodump","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbchew%2Fdynamodump/lists"}