{"id":13512577,"url":"https://github.com/danishi/dynamodb-csv","last_synced_at":"2025-08-20T22:30:45.146Z","repository":{"id":40697535,"uuid":"462311845","full_name":"danishi/dynamodb-csv","owner":"danishi","description":"A utility that allows CSV import / export to DynamoDB on the command line","archived":false,"fork":false,"pushed_at":"2024-12-09T01:47:07.000Z","size":553,"stargazers_count":20,"open_issues_count":9,"forks_count":4,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-12-12T15:39:12.068Z","etag":null,"topics":["aws","cli","csv","dynamodb","python"],"latest_commit_sha":null,"homepage":"https://danishi.github.io/dynamodb-csv/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/danishi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"danishi","patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"lfx_crowdfunding":null,"custom":null}},"created_at":"2022-02-22T13:33:44.000Z","updated_at":"2024-12-06T16:56:15.000Z","dependencies_parsed_at":"2023-12-20T15:03:53.074Z","dependency_job_id":"73b9f92d-8b63-4ed7-8e9a-d8a489a8acd6","html_url":"https://github.com/danishi/dynamodb-csv","commit_stats":{"total_commits":176,"total_committers":3,"mean_commits":"58.666666666666664","dds":"0.14204545454545459","last_synced_commit":"0ff53cf6c714d50f437b011076070d9857368d2b"},"previous_names":[],"tags_count":30,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danishi%2Fdynamodb-csv","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danishi%2Fdynamodb-csv/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danishi%2Fdynamodb-csv/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danishi%2Fdynamodb-csv/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/danishi","download_url":"https://codeload.github.com/danishi/dynamodb-csv/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230462906,"owners_count":18229864,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","cli","csv","dynamodb","python"],"created_at":"2024-08-01T04:00:21.204Z","updated_at":"2025-08-20T22:30:45.131Z","avatar_url":"https://github.com/danishi.png","language":"Python","funding_links":["https://github.com/sponsors/danishi","https://www.buymeacoffee.com/danishi"],"categories":["Tools","Python"],"sub_categories":[],"readme":"# DynamoDB CSV utility\n\n[![Release Notes](https://img.shields.io/github/release/danishi/dynamodb-csv)](https://github.com/danishi/dynamodb-csv/releases)\n[![Contributors](https://img.shields.io/github/contributors/danishi/dynamodb-csv)](https://github.com/danishi/dynamodb-csv/contributors)\n[![Last Commit](https://img.shields.io/github/last-commit/danishi/dynamodb-csv)](https://github.com/danishi/dynamodb-csv/last-commit)\n[![Open Issues](https://img.shields.io/github/issues-raw/danishi/dynamodb-csv)](https://github.com/danishi/dynamodb-csv/issues)\n[![LRepo-size](https://img.shields.io/github/repo-size/danishi/dynamodb-csv)](https://github.com/danishi/dynamodb-csv/repo-size)\n[![Open in Dev Containers](https://img.shields.io/static/v1?label=Dev%20Containers\u0026message=Open\u0026color=blue\u0026logo=visualstudiocode)](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/danishi/dynamodb-csv)\n[![MIT](https://img.shields.io/github/license/danishi/DynamoDB-CSV)](https://github.com/danishi/dynamodb-csv/blob/master/LICENSE)\n[![ci](https://github.com/danishi/DynamoDB-CSV/actions/workflows/ci.yaml/badge.svg?branch=master)](https://github.com/danishi/DynamoDBImportCSV/actions/workflows/ci.yaml)\n[![codecov](https://codecov.io/gh/danishi/dynamodb-csv/branch/master/graph/badge.svg?token=KRA27MJN42)](https://codecov.io/gh/danishi/dynamodb-csv)\n[![Maintainability](https://api.codeclimate.com/v1/badges/c1d2a51bbd72d6198e0c/maintainability)](https://codeclimate.com/github/danishi/dynamodb-csv/maintainability)\n![Supported Python versions](https://img.shields.io/pypi/pyversions/dynamodb-csv.svg?color=%2334D058)\n[![PyPI](https://badge.fury.io/py/dynamodb-csv.svg)](https://badge.fury.io/py/dynamodb-csv)\n[![Downloads](https://static.pepy.tech/badge/dynamodb-csv)](https://pepy.tech/project/dynamodb-csv)\n[![Downloads week](https://static.pepy.tech/badge/dynamodb-csv/week)](https://pepy.tech/project/dynamodb-csv)\n[![Downloads month](https://static.pepy.tech/badge/dynamodb-csv/month)](https://pepy.tech/project/dynamodb-csv)\n[![Docker Pulls](https://img.shields.io/docker/pulls/danishi/dynamodb-csv)](https://hub.docker.com/r/danishi/dynamodb-csv)\n[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/danishi/dynamodb-csv)\n[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](https://makeapullrequest.com)\n\n[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/danishi/dynamodb-csv)\n[![\"Buy Me A Coffee\"](https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png)](https://www.buymeacoffee.com/danishi)\n\n![DynamoDBCSV](https://user-images.githubusercontent.com/56535085/159007555-e72d1c26-eb44-46ca-bc38-c752164995bf.png)\n\nA utility that allows CSV import / export to DynamoDB on the command line\n\nGive a ⭐️ if you like this tool!\n\n## Introduction\n\nI made this command because I didn't have any tools to satisfy my modest desire to make it easy to import CSV files into DynamoDB.\nWritten in a simple Python script, it's easy to parse and modify.\n\nIt works for me.\n\n![terminalizer](https://user-images.githubusercontent.com/13270461/237145047-ec815dad-1ff6-4678-baa4-fd182ee35269.gif)\n\n## Getting started 🚀\n\n### Install\n\n```shell\n$ pip install dynamodb-csv\n$ dynamodb-csv -h\nusage: dynamodb-csv [-h] [-v] [-i] [-e] [--truncate] [--move] -t [TABLE ...] [-idx INDEX] [-f FILE] [-o OUTPUT] [--ignore]\n               [--profile PROFILE]\n\nImport CSV file into DynamoDB table utilities\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -v, --version         show version\n  -i, --imp             mode import\n  -e, --exp             mode export\n  --truncate            mode truncate\n  --move                mode move\n  -t [TABLE ...], --table [TABLE ...]\n                        DynamoDB table name\n  -idx INDEX, --index INDEX\n                        DynamoDB index name\n  -f FILE, --file FILE  UTF-8 CSV file path required import mode\n  -o OUTPUT, --output OUTPUT\n                        output file path required export mode\n  --ignore              ignore import error\n  --profile PROFILE     using AWS profile\n```\n\n### Install for developer\n\n\u003cdetails\u003e\n  \u003csummary\u003eSetup and install\u003c/summary\u003e\n\n```shell\n$ python -m venv venv\n$ . venv/bin/activate\n$ python setup.py install\n$ dynamodb-csv -h\n```\n\nOr\n\n```shell\n$ python -m venv venv\n$ . venv/bin/activate\n$ pip install -r requirements-dev.txt\n$ export PYTHONPATH=`pwd`\n$ python app/main.py -h\n```\n\nFor Windows\n\n```shell\n\u003e python -m venv venv\n\u003e venv\\Scripts\\activate\n\u003e pip install -r requirements-dev.txt\n\u003e set PYTHONPATH=%cd%\n\u003e python app/main.py -h\n```\n\nOr you can use devcontainer.\n\n\u003c/details\u003e\n\n### [Use Docker image](https://hub.docker.com/r/danishi/dynamodb-csv)\n\n```shell\n$ docker run --rm -v ${PWD}/:/local danishi/dynamodb-csv:tagname -i -t my_table -f sample.csv\n```\n\nFor Windows\n\n```shell\n\u003e docker run --rm -v %cd%/:/local danishi/dynamodb-csv:tagname -i -t my_table -f sample.csv\n```\n\nOr [GitHub Packages](https://github.com/danishi/dynamodb-csv/pkgs/container/dynamodb-csv)\n\n### Create your config.ini file on current directory\n\n```ini\n[AWS]\nAWS_ACCESS_KEY_ID=your_access_key\nAWS_SECRET_ACCESS_KEY=your_secret_key\nREGION=your_dynamodb_table_region\n# Option\n#ENDPOINT_URL=http://dynamodb-local:8000\n```\n\nNot required if AWS profile is specified as a parameter.\n\n### Create your CSV and CSV spec file\n\n\u003e [!NOTE]\n\u003e Prepare a UTF-8 CSV file of the format you want to import into your DynamoDB table and a file that defines that format.\n\n#### For example\n\nPlease refer to this writing method.\n\n[sample.csv](sample.csv)\n\n```csv\nStringPK,NumberSK,DecimalValue,BooleanValue,NullValue,JsonValue,StringListValues,DecimalListValues\nfoo,1,1.23,TRUE,,\"[{\"\"string\"\" : \"\"value\"\"},{\"\"number\"\" : 100}]\",foo bar baz,10 10.1 20\nfoo,2,0.001,,,\"[{\"\"boolean\"\" : true}]\",リンゴ バナナ スイカ,10 10.1 20\nfoo,3,1,,,\"[{\"\"boolean\"\" : false}]\",,\n```\n\n[sample.csv.spec](sample.csv.spec)\n\n```ini\n# sample.csv data format specification\n\n# String : S\n# Integer : I\n# Decimal : D\n# Boolean : B (blank false)\n# Json : J\n# StringList : SL\n# StringSet : SS\n# DecimalList : DL\n# DecimalSet : DS\n\n[CSV_SPEC]\nStringPK=S\nNumberSK=I\nDecimalValue=D\nBooleanValue=B\nNullValue=S\nJsonValue=J\nStringListValues=SL\nStringSetValues=SS\nDecimalListValues=DL\nDecimalSetValues=DS\n\n# [DELIMITER_OPTION]\n# DelimiterCharacter=|\n```\n\nThe CSV_SPEC type is mapped to the [DynamoDB attribute type](https://docs.aws.amazon.com/en_us/amazondynamodb/latest/developerguide/HowItWorks.NamingRulesDataTypes.html#HowItWorks.DataTypeDescriptors) in this way.\n\n|     CSV_SPEC     | DynamoDB attribute data type | example value                                   |\n| :--------------: | :--------------------------: | :---------------------------------------------- |\n|    String : S    |            String            | `foo`                                           |\n|   Integer : I    |            Number            | `1`                                             |\n|   Decimal : D    |            Number            | `1.23`                                          |\n|   Boolean : B    |           Boolean            | `TRUE`                                          |\n|     Json : J     |             Map              | `[{\"\"string\"\" : \"\"value\"\"},{\"\"number\"\" : 100}]` |\n| StringList : SL  |             List             | `foo bar baz`                                   |\n|  StringSet : SS  |          String Set          | `foo bar baz`                                   |\n| DecimalList : DL |             List             | `10 10.1 20`                                    |\n| DecimalSet : DS  |          Number Set          | `10 10.1 20`                                    |\n\nSorry, Binary type and Binary Set type is not supported.\nNull type, look [here](https://github.com/danishi/dynamodb-csv?tab=readme-ov-file#import-options).\n\nThe default delimiter for list and set types is a space.  \nIf you want to set it, please comment out `DELIMITER_OPTION` and `DelimiterCharacter`.\n\n### Create DynamoDB table\n\n\u003e [!NOTE]\n\u003e You need to have created a DynamoDB table that meets your specifications.\n\n```shell\n$ aws dynamodb create-table --cli-input-json file://my_table.json --region ap-northeast-1\n$ aws dynamodb describe-table --table-name my_table\n{\n    \"Table\": {\n        \"AttributeDefinitions\": [\n            {\n                \"AttributeName\": \"NumberSK\",\n                \"AttributeType\": \"N\"\n            },\n            {\n                \"AttributeName\": \"StringPK\",\n                \"AttributeType\": \"S\"\n            }\n        ],\n        \"TableName\": \"my_table\",\n        \"KeySchema\": [\n            {\n                \"AttributeName\": \"StringPK\",\n                \"KeyType\": \"HASH\"\n            },\n            {\n                \"AttributeName\": \"NumberSK\",\n                \"KeyType\": \"RANGE\"\n            }\n        ],\n        \"TableStatus\": \"ACTIVE\",\n        \"CreationDateTime\": \"2022-06-26T21:19:21.767000+09:00\",\n        \"ProvisionedThroughput\": {\n            \"NumberOfDecreasesToday\": 0,\n            \"ReadCapacityUnits\": 5,\n            \"WriteCapacityUnits\": 5\n        },\n        \"TableSizeBytes\": 0,\n        \"ItemCount\": 0,\n        \"TableArn\": \"arn:aws:dynamodb:ap-northeast-1:XXXXXXXXXXX:table/my_table\",\n        \"TableId\": \"XXXXXXXX-925b-4cb1-8e3a-604158118c3f\",\n        \"GlobalSecondaryIndexes\": [\n            {\n                \"IndexName\": \"NumberSK-index\",\n                \"KeySchema\": [\n                    {\n                        \"AttributeName\": \"NumberSK\",\n                        \"KeyType\": \"HASH\"\n                    }\n                ],\n                \"Projection\": {\n                    \"ProjectionType\": \"INCLUDE\",\n                    \"NonKeyAttributes\": [\n                        \"DecimalValue\",\n                        \"JsonValue\"\n                    ]\n                },\n                \"IndexStatus\": \"ACTIVE\",\n                \"ProvisionedThroughput\": {\n                    \"NumberOfDecreasesToday\": 0,\n                    \"ReadCapacityUnits\": 5,\n                    \"WriteCapacityUnits\": 5\n                },\n                \"IndexSizeBytes\": 0,\n                \"ItemCount\": 0,\n                \"IndexArn\": \"arn:aws:dynamodb:ap-northeast-1:XXXXXXXXXXX:table/my_table/index/NumberSK-index\"\n            }\n        ]\n    }\n}\n```\n\n### CSV import into Table\n\nThis command requires a CSV spec file in the same directory.\n\n```shell\n$ dynamodb-csv -i -t my_table -f sample.csv\nplease wait my_table importing sample.csv\n300it [00:00, 19983.03it/s]\n100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 300/300 [00:07\u003c00:00, 40.97it/s]\nmy_table csv imported 300 items\n```\n\nIt is processed at high speed by batch write.\n\n#### Ignore option\n\nIf there is an error such as a key schema mismatch, you can give the option to ignore the CSV record.\n\n```shell\n$ dynamodb-csv -i -t my_table -f sample.csv --ignore\nplease wait my_table importing sample.csv\n300it [00:00, 19983.03it/s]\n100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 300/300 [00:07\u003c00:00, 40.97it/s]\nmy_table csv imported 299 items and 1 error items\n```\n\nNo batch write is done when this option is used.\n\n#### Import options\n\nBy default, if CSV has an empty value, it will be set to empty.  \nThere are options to convert this to Null or not to set the attribute itself.\n\n```ini\n[IMPORT_OPTION]\nConvertBlankToNullAttrs=NullValue,JsonValue\nConvertBlankToDropAttrs=DecimalValue\n```\n\n### Export table to CSV\n\nYou will also need to expand the same data to multiple tables.  \nTherefore, data can be exported.  \nAs with import, you need a CSV spec file.\n\n```shell\n$ dynamodb-csv -e -t my_table -o sample_exp.csv\nplease wait my_table exporting sample_exp.csv\n100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 300/300 [00:00\u003c00:00, 16666.77it/s]\nmy_table csv exported 300 items\n```\n\n#### Use index\n\n```shell\n$ dynamodb-csv -e -t my_table -idx NumberSK-index -o sample_gsi_exp.csv\n```\n\n#### Use Query\n\n```shell\n$ dynamodb-csv -e -t my_table -idx NumberSK-index -o sample_query_exp.csv\n```\n\n```ini\n# sample_query_exp.csv data format specification\n\n# Integer : I\n# String : S\n# Decimal : D\n# Json : J\n\n[QUERY_OPTION]\nPKAttribute=NumberSK\nPKAttributeValue=1\nPKAttributeType=I\n\n[CSV_SPEC]\nNumberSK=I\nStringPK=S\nDecimalValue=D\nJsonValue=J\n```\n\n##### Query options\n\n|           key           | description                              | example                                                                                                                                                                      |\n| :---------------------: | :--------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n|      `PKAttribute`      | Partition key attribute name             |                                                                                                                                                                              |\n|   `PKAttributeValue`    | Partition key attribute query value      |                                                                                                                                                                              |\n|    `PKAttributeType`    | Partition key attribute data type        |                                                                                                                                                                              |\n|      `SKAttribute`      | Sort key attribute name                  |                                                                                                                                                                              |\n|   `SKAttributeValues`   | Sort key attribute query value or values | ex. `foo` or `foo,bar`                                                                                                                                                       |\n|    `SKAttributeType`    | Sort key attribute data type             |                                                                                                                                                                              |\n| `SKAttributeExpression` | Sort key attribute query expression      | [ex.](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/customizations/dynamodb.html#dynamodb-conditions) `begins_with` `between` `eq` `gt` `gte` `lt` `lte` |\n\n```shell\n$ dynamodb-csv -e -t my_table -o sample_query_exp2.csv\n```\n\n```ini\n[QUERY_OPTION]\nPKAttribute=StringPK\nPKAttributeValue=bar\nPKAttributeType=S\nSKAttribute=NumberSK\nSKAttributeValues=50,100\nSKAttributeType=I\nSKAttributeExpression=between\n```\n\n### Table truncate\n\nAlso, since you may want to erase unnecessary data during the import experiment, we have prepared a command to discard it.\n\n```shell\n$ dynamodb-csv --truncate -t my_table\nmy_table scan 300 items\nplease wait my_table truncating\n100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 300/300 [00:07\u003c00:00, 40.95it/s]\nmy_table truncated\n```\n\n\u003e [!CAUTION]\n\u003e This operation is irreversible. Take care.\n\n### Table move\n\nMove all items from table to table.\nA table with the same schema must be prepared in advance.  \nTable items is not deleted and behaves like a copy.\n\n```shell\n$ dynamodb-csv --move -t my_table_from my_table_to\nmy_table_from scan 300 items\nplease wait my_table_to moving\n100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 300/300 [00:15\u003c00:00, 20.00it/s]\nmy_table_to moved 300 items\n```\n\n## License\n\nSee [LICENSE](LICENSE)\n\n## Special Thanks\n\n### Code contributors 🤝\n\n\u003ca href=\"https://github.com/danishi/dynamodb-csv/graphs/contributors\"\u003e\n  \u003cimg src=\"https://contrib.rocks/image?repo=danishi/dynamodb-csv\" /\u003e\n\u003c/a\u003e\n\n## Appendix\n\n### User guide\n\n- [User guide (for japanese)](https://danishi.github.io/dynamodb-csv/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdanishi%2Fdynamodb-csv","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdanishi%2Fdynamodb-csv","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdanishi%2Fdynamodb-csv/lists"}