{"id":50432882,"url":"https://github.com/gulgulia17/redshift-notebook-manager","last_synced_at":"2026-05-31T15:01:57.921Z","repository":{"id":361253064,"uuid":"1253737374","full_name":"gulgulia17/redshift-notebook-manager","owner":"gulgulia17","description":"Export, import, and manage notebooks and saved queries in Amazon Redshift Query Editor V2 from the command line. Preserves folder hierarchy, supports bulk operations, and works entirely through the AWS API — no browser automation required.","archived":false,"fork":false,"pushed_at":"2026-05-29T19:14:43.000Z","size":31,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-29T21:09:22.070Z","etag":null,"topics":["amazon-redshift","aws","aws-sdk","cli","command-line-tool","data-engineering","folder-hierarchy","jupyter-notebook","notebook-export","notebook-import","python","query-editor","redshift","sigv4","sqlworkbench"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gulgulia17.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-29T19:06:16.000Z","updated_at":"2026-05-29T19:14:47.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/gulgulia17/redshift-notebook-manager","commit_stats":null,"previous_names":["gulgulia17/redshift-notebook-manager"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/gulgulia17/redshift-notebook-manager","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gulgulia17%2Fredshift-notebook-manager","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gulgulia17%2Fredshift-notebook-manager/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gulgulia17%2Fredshift-notebook-manager/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gulgulia17%2Fredshift-notebook-manager/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gulgulia17","download_url":"https://codeload.github.com/gulgulia17/redshift-notebook-manager/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gulgulia17%2Fredshift-notebook-manager/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33735663,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-31T02:00:06.040Z","response_time":95,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["amazon-redshift","aws","aws-sdk","cli","command-line-tool","data-engineering","folder-hierarchy","jupyter-notebook","notebook-export","notebook-import","python","query-editor","redshift","sigv4","sqlworkbench"],"created_at":"2026-05-31T15:01:57.142Z","updated_at":"2026-05-31T15:01:57.915Z","avatar_url":"https://github.com/gulgulia17.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Redshift Query Editor V2 Toolkit\n\nA Python CLI for managing notebooks and saved queries in Amazon Redshift Query Editor V2 — export, import, and crawl your workspace without a browser.\n\n[![Python](https://img.shields.io/badge/python-3.11%2B-blue)](https://www.python.org)\n[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)\n\n---\n\n## Overview\n\nQuery Editor V2 stores notebooks and saved queries in a private REST API (`sqlworkbench`). This toolkit authenticates to that API using the same AWS SigV4 signing the console uses, then exposes clean Python abstractions for the operations you actually need:\n\n- **Crawl** the full folder hierarchy\n- **Export** notebooks as `.ipynb` files and queries as `.sql` files\n- **Import** notebooks from a local directory, recreating the original folder structure\n- **Delete** individual queries or bulk-delete an entire folder\n\nAll without Selenium, Playwright, or browser automation of any kind.\n\n---\n\n## Requirements\n\n- Python 3.11+\n- AWS credentials with access to the `sqlworkbench` service (see [Credentials](#credentials))\n\n```bash\npip install -r requirements.txt\n```\n\n---\n\n## Credentials\n\nQuery Editor V2 uses temporary session credentials tied to the browser identity. Standard long-lived IAM credentials may not have a root folder initialised in Query Editor V2.\n\n**Recommended:** extract credentials from an active browser session.\n\n1. Open the AWS Console and navigate to Redshift Query Editor V2.\n2. Open DevTools → Network → filter for `tb/creds`.\n3. Copy the response values into `.env` at the project root:\n\n```env\nAWS_ACCESS_KEY_ID=ASIA...\nAWS_SECRET_ACCESS_KEY=...\nAWS_SESSION_TOKEN=...\nAWS_DEFAULT_REGION=ap-south-1\n```\n\nThe `.env` file is loaded automatically at startup. It is gitignored and never committed.\n\n\u003e Temporary credentials expire in approximately one hour. Re-export from the browser when requests return 403.\n\nCopy `.env.example` to get started:\n\n```bash\ncp .env.example .env\n```\n\n---\n\n## Project Structure\n\n```\n.\n├── main.py                    # CLI entry point\n├── src/\n│   ├── aws_cli.py             # SigV4-authenticated HTTP client\n│   ├── crawler.py             # Folder and resource traversal\n│   ├── exporter.py            # Notebook export (.ipynb)\n│   ├── query_exporter.py      # Query export (.sql)\n│   ├── importer.py            # Notebook import with folder recreation\n│   ├── models.py              # Folder, Notebook, Query dataclasses\n│   └── renderer.py            # Terminal tree + JSON export\n├── tests/                     # pytest test suite (90 tests, no network required)\n│   ├── test_models.py\n│   ├── test_crawler.py\n│   ├── test_exporter.py\n│   ├── test_importer.py\n│   └── test_renderer.py\n├── storage/                   # Default output directory (gitignored)\n│   └── exports/\n├── requirements.txt\n├── pyproject.toml\n└── .env.example\n```\n\n---\n\n## Usage\n\nAll commands accept `--region` (default: `ap-south-1` or `$AWS_DEFAULT_REGION`) and `--debug` for verbose output.\n\n### Crawl and display the hierarchy\n\n```bash\n# Notebooks\npython -m main\n\n# Saved queries\npython -m main --type query\n\n# Both\npython -m main --type all\n```\n\n```\n📁 Query Editor V2 — Notebooks\n├── 📁 Analytics\n│   ├── 📁 Q1\n│   │   ├── 📄 Revenue Validation\n│   │   └── 📄 Cost Analysis\n│   └── 📁 Q2\n│       └── 📄 Budget Report\n└── 📁 Archive\n    └── 📄 Legacy Notebook\n\n────────────────────────────────────────\nStatistics\n────────────────────────────────────────\n  Total folders   : 3\n  Total notebooks : 4\n  Maximum depth   : 2\n────────────────────────────────────────\n```\n\nA JSON snapshot is written to `storage/tree.json` automatically.\n\n---\n\n### Export notebooks\n\n```bash\n# Export to storage/exports/ (default)\npython -m main --export\n\n# Export to a custom path\npython -m main --export /path/to/output\n```\n\nOutput mirrors the folder hierarchy:\n\n```\nstorage/exports/\n└── Analytics/\n    ├── Q1/\n    │   ├── Revenue Validation.ipynb\n    │   └── Cost Analysis.ipynb\n    └── Q2/\n        └── Budget Report.ipynb\n```\n\n---\n\n### Export saved queries\n\n```bash\npython -m main --type query --export\n```\n\n```\nstorage/exports/\n└── Shared/\n    ├── Daily Summary.sql\n    └── Revenue Check.sql\n```\n\n---\n\n### Import notebooks\n\nImport a specific subfolder, recreating its full path in Query Editor V2:\n\n```bash\npython -m main \\\n  --import \"storage/exports/Analytics/Q1\" \\\n  --import-base \"storage/exports\"\n```\n\nThis produces the following structure in Query Editor V2:\n\n```\n📁 Analytics        ← reused if it already exists\n└── 📁 Q1           ← created inside Analytics\n    ├── 📄 Revenue Validation\n    └── 📄 Cost Analysis\n```\n\nImport the entire export directory:\n\n```bash\npython -m main \\\n  --import \"storage/exports\" \\\n  --import-base \"storage/exports\"\n```\n\nRe-running an import is safe — existing folders are reused and no duplicates are created.\n\n---\n\n### Delete saved queries\n\n```bash\n# Delete a single query\npython -m main --delete-query \u003cuuid-or-full-arn\u003e\n\n# Delete all queries (prompts for confirmation)\npython -m main --delete-all-queries\n\n# Scope deletion to a specific folder\npython -m main --delete-all-queries --folder \u003cfolder-id\u003e\n```\n\n---\n\n## Full Option Reference\n\n```\npython -m main [options]\n\nConnection:\n  --region REGION          AWS region (default: ap-south-1 or $AWS_DEFAULT_REGION)\n  --debug                  Enable verbose debug logging\n\nCrawl / Export:\n  --type {notebook,query,all}   Resource type (default: notebook)\n  --output FILE                 JSON tree output path (default: storage/tree.json)\n  --export [DIR]                Export to directory (default: storage/exports)\n  --root-folder FOLDER_ID       Override root folder ID\n\nImport:\n  --import DIR                  Import notebooks from local directory\n  --import-base DIR             Base directory for relative path reconstruction\n                                (default: storage/exports)\n  --import-root-folder ID       Target root folder (auto-discovered when omitted)\n\nDelete:\n  --delete-query ARN_OR_ID      Delete a single saved query\n  --delete-all-queries          Delete all saved queries (confirmation required)\n  --folder FOLDER_ID            Scope --delete-all-queries to a specific folder\n```\n\n---\n\n## API Endpoints\n\nThe toolkit calls the following endpoints on `https://api.sqlworkbench.\u003cregion\u003e.amazonaws.com`:\n\n| Operation | Method | Endpoint |\n|---|---|---|\n| Current user \u0026 root folder | `GET` | `/user` |\n| List child folders | `GET` | `/v2/file?actionName=folders-only` |\n| List notebooks / queries | `POST` | `/tagged-resource` |\n| Notebook metadata | `GET` | `/notebook/\u003carn\u003e` |\n| Export notebook | `GET` | `/notebook/\u003carn\u003e/export` |\n| Import notebook | `POST` | `/notebook/import/v1` |\n| Create folder | `PUT` | `/folder` |\n| Query metadata + SQL | `GET` | `/query-saved/\u003carn\u003e` |\n| Delete query | `DELETE` | `/query-saved/\u003carn\u003e` |\n\n---\n\n## Running Tests\n\nThe test suite runs entirely offline — no AWS credentials required.\n\n```bash\npip install pytest pytest-cov\npytest\n```\n\n```\n========================= 90 passed in 0.54s =========================\n```\n\nRun with coverage:\n\n```bash\npytest --cov=src --cov-report=term-missing\n```\n\n---\n\n## Notes\n\n- Folder names with special characters (apostrophes, spaces, etc.) are handled correctly in both export and import paths.\n- All output is written under `storage/` by default, which is gitignored. The directory is created automatically.\n- The `--import-base` flag controls how much of the local path is recreated in Query Editor V2. Set it to the export root to get an exact mirror of the original hierarchy.\n\n---\n\n## License\n\n[MIT](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgulgulia17%2Fredshift-notebook-manager","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgulgulia17%2Fredshift-notebook-manager","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgulgulia17%2Fredshift-notebook-manager/lists"}