{"id":13707070,"url":"https://github.com/zshamrock/dynocsv","last_synced_at":"2026-03-12T01:52:11.426Z","repository":{"id":144203803,"uuid":"193161042","full_name":"zshamrock/dynocsv","owner":"zshamrock","description":"Exports DynamoDB table into CSV","archived":false,"fork":false,"pushed_at":"2023-02-24T23:34:22.000Z","size":92,"stargazers_count":37,"open_issues_count":15,"forks_count":5,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-11-13T15:49:26.068Z","etag":null,"topics":["csv","dynamodb","golang"],"latest_commit_sha":null,"homepage":"https://snapcraft.io/dynocsv","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zshamrock.png","metadata":{"files":{"readme":"README.md","changelog":"changelog.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-06-21T21:16:06.000Z","updated_at":"2024-04-26T02:42:45.000Z","dependencies_parsed_at":"2023-06-26T01:31:21.111Z","dependency_job_id":null,"html_url":"https://github.com/zshamrock/dynocsv","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zshamrock%2Fdynocsv","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zshamrock%2Fdynocsv/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zshamrock%2Fdynocsv/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zshamrock%2Fdynocsv/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zshamrock","download_url":"https://codeload.github.com/zshamrock/dynocsv/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252593283,"owners_count":21773440,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csv","dynamodb","golang"],"created_at":"2024-08-02T22:01:17.717Z","updated_at":"2025-05-05T23:32:01.689Z","avatar_url":"https://github.com/zshamrock.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# dynocsv\n\nExports DynamoDB table into CSV [![Build Status](https://travis-ci.org/zshamrock/vmx.svg?branch=master)](https://travis-ci.org/zshamrock/dynocsv) [![dynocsv](https://snapcraft.io/dynocsv/badge.svg)](https://snapcraft.io/dynocsv)\n\n\u003ca href=\"https://www.producthunt.com/posts/dynocsv?utm_source=badge-featured\u0026utm_medium=badge\u0026utm_souce=badge-dynocsv\" target=\"_blank\"\u003e\u003cimg src=\"https://api.producthunt.com/widgets/embed-image/v1/featured.svg?post_id=182854\u0026theme=dark\" alt=\"dynocsv - Exports DynamoDB table into CSV | Product Hunt Embed\" style=\"width: 250px; height: 54px;\" width=\"250px\" height=\"54px\" /\u003e\u003c/a\u003e \n\n```\nNAME:\n dynocsv - Export DynamoDB table into CSV file\n\nUSAGE:\n dynocsv \n --table/-t \u003ctable\u003e \n [--columns/-c \u003ccomma separated columns\u003e] \n [--skip-columns/-sc \u003ccomma separated columns to skip\u003e] \n [--limit/-l \u003cnumber\u003e]\n [--profile/-p \u003cAWS profile\u003e]\n [--index/-i \u003cindex to query instead of table\u003e]\n [--hash \u003chash value\u003e]\n [--sort \u003csort value\u003e]\n [--sort-[gt, ge, lt, le, begins-with, between] \u003csort value\u003e]\n [--output/-o \u003coutput file name\u003e]\n\nVERSION:\n 1.1.4\n\nAUTHOR:\n (c) Aliaksandr Kazlou\n\nCOMMANDS:\n help, h Shows a list of commands or help for one command\n\nGLOBAL OPTIONS:\n --table value, -t value table to export\n --index value, -i value index to query if hash/sort are set instead of table (which is default)\n --columns value, -c value columns to export from the table, if omitted, all columns will be exported (muttaly exclusive with \"skip-columns\")\n --skip-columns value, --sc value columns skipped from export from the table, if omitted, all columns will be exported (muttaly exclusive with \"columns\")\n --limit value, -l value limit number of records returned, if not set (i.e. 0) all items are fetched (default: 0)\n --profile value, -p value AWS profile to use to connect to DynamoDB, otherwise the value from AWS_PROFILE env var is used if available, or then \"default\" if it is not set or empty\n --hash value limit query by hash value (eq/=)\n --sort value limit query by sort value (eq/=)\n --sort-gt value limit query by sort value (gt/\u003e)\n --sort-ge value limit query by sort value (ge/\u003e=)\n --sort-lt value limit query by sort value (lt/\u003c)\n --sort-le value limit query by sort value (le/\u003c=)\n --sort-begins-with value limit query by sort value (begins with)\n --sort-between value limit query by sort value (between), values are separated by comma, i.e. \"value1,value2\"\n --output value, -o value output file, or the default \u003ctable name\u003e.csv will be used\n --help, -h show help\n --version, -v print the version\n```\n\nTable of Contents\n=================\n\n* [Installation](#installation)\n* [Usage](#usage)\n* [AWS Connection](#aws-connection)\n* [Query](#query)\n* [CSV Headers](#csv-headers)\n* [Attributes Order](#attributes-order)\n* [Limits](#limits)\n\n## Installation \n \nUse the `go` command: \n \n $ go get github.com/zshamrock/dynocsv\n \nOr using `snap`: \n \n $ snap install dynocsv\n \n[![Get it from the Snap Store](https://snapcraft.io/static/images/badges/en/snap-store-black.svg)](https://snapcraft.io/dynocsv)\n\n*Note*: for Snap you would need explicitly allow `aws-config-credetials` plug which reads data from `$HOME/.aws` by \nrunning:\n```\nsnap connect dynocsv:aws-config-credentials\n```\notherwise, it fails to read the credentials and will fail with \n\u003e 2019/10/24 17:34:14 MissingRegion: could not find region configuration\n panic: MissingRegion: could not find region configuration\n\n## Usage \n \n $ dynocsv -t \u003ctable name\u003e\n \n## AWS Connection\n\nConnection to the AWS is established using profile credentials. There are 2 options to provide the AWS profile to use:\n\n1. use explicit `--profile/-p` option to set the AWS profile to use, i.e. `dynocsv -p \u003cprofile name\u003e -t \u003ctable name\u003e`\n2. set the env var `$AWS_PROFILE` before running the app, i.e. `AWS_PROFILE=\u003cprofile name\u003e dynocsv -t \u003ctable name\u003e`\n\nIf no explicit profile value is set, it looks for the env var `$AWS_PROFILE` if present or otherwise fallbacks to the `default` profile.\n\n## Query\n\nBy default `Scan` operation is run to fetch all the data.\n\nIf `--hash` and any of the `--sort` CLI arguments are provided,\n`Query` operation will be run to query the corresponding data based on the key conditions specified by `--hash` and\n`--sort` values. \n\nThe query can be run either on the table (default) or index (if `--index` argument is set).\n\n## CSV Headers\n\nAs DynamoDB is a column-based family of DBs, technically each row could have a different number of columns/attributes, \nwith different names. And also DynamoDB only allows fetching the description about attributes that are part of \ntable/index hash or sort keys. So, in that case, it might not be possible to resolve all the attribute names \n(and so CSV headers) until all the data is read first. \n\nAnd potentially the data size could be big enough to not fit in memory or be not practical to keep all data in memory. \nSo currently it keeps the first 1000 read records in memory in the assumption that all possible attributes will be seen \nup to that point. And so write the CSV headers accordingly.\n\nIf even after 1000 records the new attribute is detected the tool outputs at the end of export the headers line into \n`stdout` which you would need manually to replace with the existing CSV headers row.\n\n## Attributes Order\n\nThe attributes in the output CSV are sorted in the following order:\n\n- if `--index` CLI argument has not been provided table hash/sort keys are coming first, then all available global \nsecondary indexes' hash/sort keys (in alphabetical order by index names), and then all the rest of the attributes sorted \nalphabetically\n- if `--index` CLI argument is set, the order will be the same as above with the exception that that index's hash/sort \nkeys will come first before the table's hash/sort keys, then all the remaining other indexes' hash/sort keys, and the \nrest of the attributes sorted alphabetically\n\n## Limits\n\nCurrently, there are the following limitations:\n\n- `String`, `Boolean`, `Number`, `Map`, `StringSet`, `NumberSet` and `List` data types are supported to export the data \n from, attributes with other data type will still be present, but the value will be \"\" (empty string)\n- there is no pause or proper throttling according to the current set table's RCU, so you might need manually to \n increase the RCU value temporarily for the period of running the export\n \n## Copyright \n \nCopyright (C) 2019-2020 by Aliaksandr Kazlou. \n \ndynocsv is released under MIT License. \nSee [LICENSE](https://github.com/zshamrock/dynocsv/blob/master/LICENSE) for details. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzshamrock%2Fdynocsv","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzshamrock%2Fdynocsv","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzshamrock%2Fdynocsv/lists"}