{"id":50607730,"url":"https://github.com/pmarkert/ddbat","last_synced_at":"2026-06-06T00:30:57.578Z","repository":{"id":355292520,"uuid":"1201592445","full_name":"pmarkert/ddbat","owner":"pmarkert","description":null,"archived":false,"fork":false,"pushed_at":"2026-05-02T20:09:30.000Z","size":1579,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-02T22:14:12.179Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pmarkert.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-04T22:14:41.000Z","updated_at":"2026-05-02T20:09:32.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/pmarkert/ddbat","commit_stats":null,"previous_names":["pmarkert/ddbat"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/pmarkert/ddbat","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pmarkert%2Fddbat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pmarkert%2Fddbat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pmarkert%2Fddbat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pmarkert%2Fddbat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pmarkert","download_url":"https://codeload.github.com/pmarkert/ddbat/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pmarkert%2Fddbat/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33965591,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-05T02:00:06.157Z","response_time":120,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-06T00:30:56.892Z","updated_at":"2026-06-06T00:30:57.562Z","avatar_url":"https://github.com/pmarkert.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DDBat\n\n![DDBat logo](doc/ddbat_logo_640.png)\n\nA CLI tool for streaming DynamoDB operations. Export, import, transform, and delete items via stdio.\n\nPronounced: either \"diddy-bat\" or \"dee-dee-bat\".\n\n## Installation\n\n### Prerequisites\n\n- Node.js 18+\n- AWS credentials configured (via environment variables, AWS CLI, or IAM role)\n\n### Install from source\n\n    git clone \u003crepository-url\u003e\n    cd ddbat\n    npm install\n    npm run build\n    npm link  # makes 'ddbat' available globally\n\n## Usage\n\nAll inputs come from flags or environment variables. Status messages go to stderr; JSON data goes to stdout, making it safe to pipe between commands.\n\n    ddbat --help\n    ddbat \u003ccommand\u003e --help\n\n### Export\n\nExport all or a filtered subset of a table to JSON lines by default, or use `--format json` for a JSON array. If the command is interrupted, DDBat finishes the current DynamoDB page, closes the output cleanly, and prints a `--start-key` cursor you can use to resume.\n\n    # Export to stdout\n    ddbat export --table users\n\n    # Export to file\n    ddbat export --table users --output users.json\n\n    # Filter by partition key and sort key\n    ddbat export --table orders --pk \"customer-123\" --sk \"\u003e= 2024-01-01\"\n\n    # Filter by expression\n    ddbat export --table users --filter \"status='active' AND age\u003e18\"\n\n    # Resume from a previously printed cursor\n    ddbat export --table users --start-key '{\"userId\":\"123\",\"createdAt\":\"2024-01-01\"}'\n\n### Import\n\nImport JSON lines or a JSON array into a DynamoDB table. Input format is auto-detected by default, or you can override it with `--input-format jsonl` or `--input-format json`.\n\n    # From file\n    ddbat import --table users --input users.json\n\n    # From stdin\n    cat users.json | ddbat import --table users\n\nExamples:\n\nJSON array:\n\n    [\n      { \"userId\": \"123\", \"name\": \"Alice\" },\n      { \"userId\": \"456\", \"name\": \"Bob\" }\n    ]\n\nJSON lines:\n\n    { \"userId\": \"123\", \"name\": \"Alice\" }\n    { \"userId\": \"456\", \"name\": \"Bob\" }\n\n### Delete\n\nCount matching items, show a forward-only preview page, and prompt for a per-page action before deleting. Use `--dry-run` to preview only, `--no-count` to skip the initial count pass, `--page-size` to control how many matching items appear on each page, `--start-key` to resume from a previously printed cursor, or `--force` to skip the prompt.\n\n    # Preview without deleting\n    ddbat delete --table users --pk \"inactive\" --dry-run\n\n    # Delete with confirmation prompt (default)\n    ddbat delete --table users --pk \"inactive\"\n\n    # Skip the initial count query and preview 25 items per page\n    ddbat delete --table users --pk \"inactive\" --no-count --page-size 25\n\n    # Delete without prompt (for automation)\n    ddbat delete --table users --pk \"inactive\" --force\n\n    # Resume from a previously printed cursor\n    ddbat delete --table users --pk \"inactive\" --start-key '{\"userId\":\"123\",\"createdAt\":\"2024-01-01\"}'\n\n    # Save deleted items to a backup file before removing them\n    ddbat delete --table orders --filter \"orderDate\u003c'2023-01-01'\" --force --format full \u003e backup.json\n\nInteractive delete actions:\n\n- `all`: delete all remaining items from the current page through the end\n- `d` (or `delete`): delete the current page, then continue to the next page\n- `n` (or `next`): skip the current page and show the next page\n- `q` (or `quit`): stop without deleting the current page or any later pages\n\nIf you press Ctrl-C during export or delete, DDBat completes the current page, prints a resume cursor to stderr, and stops. For delete, quitting from the interactive prompt also prints a resume cursor when there are later pages to process.\n\n`--format` options:\n\n| Value    | stdout output              |\n| -------- | -------------------------- |\n| (none)   | nothing                    |\n| `full`   | full JSON of matched items |\n| `keys`   | JSON array of keys only    |\n| `count`  | item count as plain text   |\n| `silent` | nothing (suppress stderr)  |\n\n### Transform\n\nApply a JavaScript or TypeScript function to every item in a JSON stream. Reads from stdin and writes to stdout by default. Input format is auto-detected by default, or you can override it with `--input-format jsonl` or `--input-format json`. Output defaults to JSON lines, or use `--format json` for a JSON array. Provide either `--transform` or `--script`.\n\n#### How Transforms Work\n\n- DDBat calls your function once per item.\n- Function signature: `(item, index)`\n- You can return sync or async results.\n- Return an object to emit one item.\n- Return an array of objects to emit many items (fan-out).\n- Return `null` or `undefined` to drop the item.\n\n#### Quick Usage\n\n```bash\n# Inline script: 'item' and 'index' are in scope; return the new item\nddbat transform --script 'const { ssn, ...safe } = item; return safe'\n\n# Return null/undefined to drop an item\nddbat transform --script 'if (!item.active) return null; return item'\n\n# Load a transform from a module file\nddbat transform --transform ./migrations/normalize.js\n\n# From file, to file\nddbat transform --input data.json --transform ./migrations/add-field.js --output out.json\n```\n\n#### Transform File Template (JavaScript)\n\n```js\nexport default function (item, index) {\n  return {\n    ...item,\n    migratedAt: new Date().toISOString(),\n    position: index,\n  };\n}\n```\n\n#### Transform File Template (TypeScript)\n\n```ts\nimport type { TransformFn } from \"ddbat/transform\";\n\ntype Input = {\n  id: string;\n  email?: string;\n  active?: boolean;\n  ssn?: string;\n};\n\ntype Output = {\n  id: string;\n  email?: string;\n  migratedAt: string;\n};\n\nconst transform: TransformFn\u003cInput, Output\u003e = (item) =\u003e {\n  if (!item.active) return null;\n  const { ssn, ...safe } = item;\n  return {\n    id: safe.id,\n    email: safe.email,\n    migratedAt: new Date().toISOString(),\n  };\n};\n\nexport default transform;\n```\n\n#### Common Transform Patterns\n\n```bash\n# Rename a field\nddbat transform --script 'const { userId, ...rest } = item; return { id: userId, ...rest }'\n\n# Add default values\nddbat transform --script 'return { status: \"active\", ...item }'\n\n# Fan-out one record into multiple records\nddbat transform --script 'return (item.tags || []).map(tag =\u003e ({ ...item, tag }))'\n```\n\n#### Test a Transform Safely\n\n```bash\n# 1) Export a small sample\nddbat export --table users --filter \"status='active'\" --output sample.json\n\n# 2) Run transform locally\nddbat transform --input sample.json --transform ./migrations/normalize.js --output sample.out.json\n\n# 3) Inspect results before importing\ncat sample.out.json\n```\n\nTypeScript transform files require Node.js 22.6+ (native TypeScript support).\n\nLimitations:\n\n- Use ESM syntax (`export default`)\n- Avoid `enum`, parameter properties, decorators, and `tsconfig` path aliases\n- Compile to `.js` first if you need unsupported TS features\n\n### Filter\n\nApply a JavaScript or TypeScript predicate to every item in a JSON stream. The command shares the same input and output options as `transform`, but the function must return a boolean. Items are only emitted when the predicate returns `true`. Inline `--script` accepts either a bare expression or a full function body.\n\n#### Quick Usage\n\n```bash\n# Inline predicate expression: 'item' and 'index' are in scope\nddbat filter --script 'item.type === \"carecircle-invitation\"'\n\n# Keep only active records from a file\nddbat filter --input data.json --script 'item.active'\n\n# Load a predicate from a module file\nddbat filter -x ./filters/keep-active.js\n```\n\n#### Filter File Template (JavaScript)\n\n```js\nexport default function (item, index) {\n  return item.active \u0026\u0026 index \u003c 1000;\n}\n```\n\nFilter functions may also be async. TypeScript filter files have the same native Node.js limitations as transform files.\n\n```bash\nddbat filter --script 'item.active \u0026\u0026 item.region === \"us-east-1\"'\n```\n\nNOTE: If you use a multi-statement inline script, include an explicit `return`:\n\n```bash\nddbat filter --script 'const keep = item.active \u0026\u0026 item.region === \"us-east-1\"; return keep'\n```\n\n### Pipelines\n\nCommands compose naturally with Unix pipes:\n\n    # Copy a table\n    ddbat export --table source | ddbat import --table destination\n\n    # Copy with filtering\n    ddbat export --table users --filter \"status='active'\" | ddbat import --table active-users\n\n    # Copy with stream filtering\n    ddbat export --table users \\\n      | ddbat filter --script 'item.status === \"active\"' \\\n      | ddbat import --table active-users\n\n    # Copy with transform\n    ddbat export --table users \\\n      | ddbat transform --transform ./migrations/normalize.js \\\n      | ddbat import --table users-v2\n\n    # Multi-step transform\n    ddbat export --table users \\\n      | ddbat transform --script 'const { ssn, ...s } = item; return s' \\\n      | ddbat transform --transform ./migrations/add-timestamps.js \\\n      | ddbat import --table users-clean\n\n## Filtering\n\n`--pk`, `--sk`, `--index`, and `--filter` work on `export` and `delete`.\n\n### `--pk`\n\nPartition key value (equality only):\n\n    --pk \"user-123\"\n\n### `--sk`\n\nSort key with optional operator. Spacing is flexible:\n\n    --sk \"2024-01-01\"            # equals\n    --sk \"\u003e= 2024-01-01\"         # greater than or equal\n    --sk \"begins_with(2024-)\"    # prefix\n    --sk \"between(100,200)\"      # range\n\nSupported operators: `=`, `\u003c`, `\u003c=`, `\u003e`, `\u003e=`, `begins_with(...)`, `between(...,...)`\n\n### `--index`\n\nQuery or scan a secondary index (GSI or LSI):\n\n    --index \"StatusIndex\"\n\n### `--filter`\n\nFilter expression for non-key attributes. Values are inlined and automatically parameterized to avoid reserved word conflicts.\n\n    --filter \"status='active'\"\n    --filter \"age\u003e=18 AND status IN ('active','pending')\"\n    --filter \"attribute_exists(email)\"\n    --filter \"begins_with(title,'Intro')\"\n\nSupported: `=`, `!=`, `\u003c`, `\u003c=`, `\u003e`, `\u003e=`, `begins_with`, `contains`, `between`, `attribute_exists`, `attribute_not_exists`, `size`, `IN`, `AND`, `OR`\n\n## Environment Variables\n\n    export DDBAT_TABLE=my-table   # Default table for export, import, delete\n\n## Debugging\n\n    DEBUG=1 ddbat export --table users\n\nPrints full error stack traces to stderr.\n\n## Shell Completion\n\nDDBat supports shell completion via [Carapace](https://carapace-sh.github.io/carapace-bin/):\n\n    brew install carapace   # macOS\n    ddbat completion      # install the completion spec\n\nRestart your shell, then use Tab to complete commands and flags.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpmarkert%2Fddbat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpmarkert%2Fddbat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpmarkert%2Fddbat/lists"}