{"id":29416171,"url":"https://github.com/efemaer/terrawiz","last_synced_at":"2025-07-11T19:01:59.384Z","repository":{"id":292720237,"uuid":"981725120","full_name":"efemaer/terrawiz","owner":"efemaer","description":"🧙‍♂️ TerraWiz - A blazing-fast open-source CLI tool for discovering and analyzing Terraform modules across multiple platforms.","archived":false,"fork":false,"pushed_at":"2025-07-08T18:52:20.000Z","size":166,"stargazers_count":57,"open_issues_count":3,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-08T19:51:31.988Z","etag":null,"topics":["infrastructure-as-code","npm-package","terraform","terraform-modules"],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/terrawiz","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/efemaer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-11T18:48:59.000Z","updated_at":"2025-07-08T18:52:22.000Z","dependencies_parsed_at":null,"dependency_job_id":"03468f89-646b-45f3-bde4-79fac17368b4","html_url":"https://github.com/efemaer/terrawiz","commit_stats":null,"previous_names":["efemaer/terrawiz"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/efemaer/terrawiz","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/efemaer%2Fterrawiz","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/efemaer%2Fterrawiz/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/efemaer%2Fterrawiz/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/efemaer%2Fterrawiz/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/efemaer","download_url":"https://codeload.github.com/efemaer/terrawiz/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/efemaer%2Fterrawiz/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264340001,"owners_count":23593277,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["infrastructure-as-code","npm-package","terraform","terraform-modules"],"created_at":"2025-07-11T19:01:47.485Z","updated_at":"2025-07-11T19:01:59.303Z","avatar_url":"https://github.com/efemaer.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🧙‍♂️ Terrawiz\n\n**A blazing-fast open-source CLI tool for discovering and analyzing Terraform modules across multiple platforms.**\n\n[![npm version](https://img.shields.io/npm/v/terrawiz.svg)](https://www.npmjs.com/package/terrawiz)\n[![Build Status](https://img.shields.io/badge/build-passing-brightgreen)](https://github.com/efemaer/terrawiz)\n[![TypeScript](https://img.shields.io/badge/TypeScript-007ACC?logo=typescript\u0026logoColor=white)](https://www.typescriptlang.org/)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\nTerrawiz performs source-code scanning across GitHub organizations, local directories, and other platforms to find all Terraform and Terragrunt module usage. It identifies module sources (registry, local, git), tracks versions, and generates comprehensive reports in table, JSON, or CSV formats. Perfect for infrastructure audits, dependency tracking, security reviews, and understanding your IaC module ecosystem at scale.\n\n---\n\n## ✨ Features\n\n- 🚀 **High-Performance Scanning** - Parallel processing with configurable concurrency limits\n- 🌐 **Multi-Platform Support** - GitHub organizations, local directories, and extensible for GitLab/Bitbucket\n- 🔍 **Comprehensive Discovery** - Scan entire organizations, specific repositories, or local file systems\n- 🏗️ **Terraform \u0026 Terragrunt Support** - Parse both .tf and .hcl files with source-code analysis\n- 📊 **Multi-Format Export** - JSON, CSV, and formatted table outputs\n- 🎯 **Smart Filtering** - Repository patterns, file types, and version analysis\n- 🛡️ **Built-in Rate Limiting** - Automatic GitHub API throttling and protection\n- 📈 **Detailed Analytics** - Module usage patterns, source types, and version tracking\n- ⚡ **Intuitive CLI** - URI-style source format with backward compatibility\n\n## 🚀 Quick Start\n\n### 📋 Requirements\n\n- **Node.js** 22+\n- **GitHub Token** with repository read access (for GitHub scanning)\n- **Local filesystem access** (for local directory scanning)\n\n### Installation\n\n#### Option 1: Install from NPM (Recommended)\n\n```bash\n# Install globally\nnpm install -g terrawiz\n\n# Or run directly with npx\nnpx terrawiz scan github:your-org\n```\n\n#### Option 2: Docker Container (Recommended for CI/CD)\n\n```bash\n# Pull from GitHub Container Registry\ndocker pull ghcr.io/efemaer/terrawiz:latest\n\n# Or build locally\ndocker build -t terrawiz .\n```\n\n#### Option 3: Build from Source\n\n```bash\n# Clone and build\ngit clone https://github.com/efemaer/terrawiz.git\ncd terrawiz\nnpm install \u0026\u0026 npm run build \u0026\u0026 npm link\n```\n\n### Setup\n\n#### For GitHub Scanning\n\nCreate a [GitHub Personal Access Token](https://github.com/settings/tokens) with repository read access:\n\n```bash\n# Set environment variable\nexport GITHUB_TOKEN=your_token_here\n\n# Or create .env file\necho \"GITHUB_TOKEN=your_token_here\" \u003e .env\n```\n\n#### For Local Directory Scanning\n\nNo additional setup required - just ensure you have read access to the target directory.\n\n### Basic Usage\n\n#### Command Line\n\n```bash\n# Scan GitHub organization\nterrawiz scan github:your-org\n\n# Scan local directory\nterrawiz scan local:/path/to/terraform/code\n\n# Scan specific GitHub repository\nterrawiz scan github:your-org/specific-repo\n\n# Export to CSV with performance tuning\nterrawiz scan github:your-org -f csv -e modules.csv -c 10:20\n\n# Scan relative local path\nterrawiz scan local:./infrastructure\n```\n\n#### Docker Usage\n\n```bash\n# Scan GitHub organization (with token from environment)\ndocker run --rm -e GITHUB_TOKEN=$GITHUB_TOKEN ghcr.io/efemaer/terrawiz:latest scan github:your-org\n\n# Scan local directory (mount as volume)\ndocker run --rm -v /path/to/terraform:/workspace ghcr.io/efemaer/terrawiz:latest scan local:/workspace\n\n# Export results to host machine\ndocker run --rm -v /path/to/terraform:/workspace -v $(pwd):/output \\\n  ghcr.io/efemaer/terrawiz:latest scan local:/workspace -f csv -e /output/modules.csv\n\n# Use environment file\ndocker run --rm --env-file .env ghcr.io/efemaer/terrawiz:latest scan github:your-org\n\n# Interactive debugging\ndocker run --rm -it -e GITHUB_TOKEN=$GITHUB_TOKEN \\\n  ghcr.io/efemaer/terrawiz:latest scan github:your-org --debug --limit 3\n```\n\n## 📖 CLI Reference\n\n### Core Command\n\n```bash\nterrawiz scan \u003csource\u003e [options]\n```\n\n### Sources\n\nTerrawiz uses a URI-style format to specify what to scan:\n\n| Format            | Description                          | Examples                     |\n| ----------------- | ------------------------------------ | ---------------------------- |\n| `github:org`      | Scan GitHub organization             | `github:hashicorp`           |\n| `github:org/repo` | Scan specific GitHub repository      | `github:hashicorp/terraform` |\n| `local:/path`     | Scan local directory (absolute path) | `local:/home/user/terraform` |\n| `local:./path`    | Scan local directory (relative path) | `local:./infrastructure`     |\n\n### Options\n\n#### Legacy Options (Deprecated)\n\n| Option              | Description                          | Migration             |\n| ------------------- | ------------------------------------ | --------------------- |\n| `-o, --org \u003cname\u003e`  | **[DEPRECATED]** GitHub organization | Use `github:org`      |\n| `-r, --repo \u003cname\u003e` | **[DEPRECATED]** Specific repository | Use `github:org/repo` |\n\n#### Core Options\n\n| Option                  | Description                         | Default |\n| ----------------------- | ----------------------------------- | ------- |\n| `-p, --pattern \u003cregex\u003e` | Filter repositories by name pattern | -       |\n\n#### Output Options\n\n| Option                | Description                           | Default |\n| --------------------- | ------------------------------------- | ------- |\n| `-f, --format \u003ctype\u003e` | Output format: `table`, `json`, `csv` | `table` |\n| `-e, --export \u003cfile\u003e` | Export results to file                | -       |\n\n#### Performance Options\n\n| Option                            | Description                          | Default   |\n| --------------------------------- | ------------------------------------ | --------- |\n| `-c, --concurrency \u003crepos:files\u003e` | Concurrent processing (e.g., `5:10`) | `5:10`    |\n| `--limit \u003cnumber\u003e`                | Maximum repositories to scan         | Unlimited |\n\n#### Filtering Options\n\n| Option               | Description                                 | Default       |\n| -------------------- | ------------------------------------------- | ------------- |\n| `--include-archived` | Include archived repositories (GitHub only) | Skip archived |\n| `--terraform-only`   | Scan only Terraform (.tf) files             | Both types    |\n| `--terragrunt-only`  | Scan only Terragrunt (.hcl) files           | Both types    |\n\n#### Advanced Options\n\n| Option                 | Description                                    | Default  |\n| ---------------------- | ---------------------------------------------- | -------- |\n| `--disable-rate-limit` | Disable GitHub API rate limiting (GitHub only) | Enabled  |\n| `--debug`              | Enable detailed logging                        | Disabled |\n\n## 💡 Usage Examples\n\n### GitHub Organization Analysis\n\n```bash\n# Complete organization scan\nterrawiz scan github:hashicorp\n\n# High-performance scan for large organizations\nterrawiz scan github:aws -c 15:25 -f json -e aws-modules.json\n\n# Legacy format (deprecated but still works)\nterrawiz scan -o hashicorp\n```\n\n### Targeted Scanning\n\n```bash\n# Specific GitHub repository\nterrawiz scan github:hashicorp/terraform-aws-vpc\n\n# Pattern-based filtering\nterrawiz scan github:mycompany -p \"^terraform-\" -f csv\n\n# Local directory scanning\nterrawiz scan local:/home/user/terraform-projects\nterrawiz scan local:./infrastructure\n\n# Infrastructure-specific scans\nterrawiz scan github:myorg --terraform-only -e terraform-only.json\nterrawiz scan local:./terraform --terragrunt-only\n```\n\n### Development \u0026 Debugging\n\n```bash\n# Debug mode with detailed logging\nterrawiz scan github:myorg --debug --limit 3\n\n# Conservative scanning for rate-limited tokens\nterrawiz scan github:myorg -c 2:5\n\n# Include archived repositories (GitHub only)\nterrawiz scan github:myorg --include-archived\n\n# Local development workflow\nterrawiz scan local:. --debug\n```\n\n### Local Filesystem Features\n\n#### Command Line\n```bash\n# Scan current directory\nterrawiz scan local:.\n\n# Scan with absolute path\nterrawiz scan local:/home/user/infrastructure\n\n# Export local scan results\nterrawiz scan local:./terraform -f json -e local-modules.json\n\n# Filter file types locally\nterrawiz scan local:/projects/terraform --terraform-only\n\n# High concurrency for large local directories\nterrawiz scan local:/enterprise/iac -c 1:20\n```\n\n#### Docker Examples\n```bash\n# Scan current directory\ndocker run --rm -v $(pwd):/workspace ghcr.io/efemaer/terrawiz:latest scan local:/workspace\n\n# Export to host with absolute path\ndocker run --rm -v /home/user/terraform:/workspace -v /home/user/reports:/output \\\n  ghcr.io/efemaer/terrawiz:latest scan local:/workspace -f json -e /output/modules.json\n\n# Filter file types in container\ndocker run --rm -v $(pwd):/workspace \\\n  ghcr.io/efemaer/terrawiz:latest scan local:/workspace --terraform-only\n\n# High concurrency for large directories\ndocker run --rm -v /enterprise/iac:/workspace \\\n  ghcr.io/efemaer/terrawiz:latest scan local:/workspace -c 1:20\n```\n\n**Local Filesystem Benefits:**\n\n- ⚡ **No API Rate Limits** - Scan as fast as your filesystem allows\n- 🔒 **Complete Privacy** - No external API calls or data transmission\n- 📁 **Recursive Scanning** - Automatically traverses subdirectories\n- 🔗 **Symlink Support** - Follows symbolic links with loop protection\n- 🚫 **Smart Filtering** - Skips common directories (.git, node_modules, etc.)\n- 🔧 **Development Friendly** - Perfect for CI/CD pipelines and local development\n\n## 📊 Sample Outputs\n\n### Table Format (Default)\n\n```\nInfrastructure as Code Module Usage Report\n============================\nPlatform: GitHub\nTarget: mycompany\nScope: All repositories in mycompany\nTotal modules found: 14 (11 Terraform, 3 Terragrunt)\nTotal files analyzed: 38 (31 Terraform, 7 Terragrunt)\n\nModule Summary by Source:\n\nterraform-aws-modules/vpc/aws (5 instances, type: registry)\n  Versions:\n    - ~\u003e 5.0: 3 instances\n    - ~\u003e 4.0: 2 instances\n\n./modules/networking (4 instances, type: local)\n\nterraform-aws-modules/eks/aws (2 instances, type: registry)\n  Versions:\n    - ~\u003e 19.0: 2 instances\n\ngit::https://github.com/company/terraform-modules.git//s3?ref=v2.0 (2 instances, type: git)\n  Versions:\n    - ref=v2.0: 1 instance\n    - ref=main: 1 instance\n\nartifactory.company.com/terraform/aws-rds (1 instance, type: artifactory)\n  Versions:\n    - 1.2.3: 1 instance\n\nModules by Source Type:\n  registry: 7 (50.0%)\n  local: 4 (28.6%)\n  git: 2 (14.3%)\n  artifactory: 1 (7.1%)\n\nModules by File Type:\n  terraform: 11 (78.6%)\n  terragrunt: 3 (21.4%)\n```\n\n### JSON Format\n\n```json\n{\n  \"metadata\": {\n    \"platform\": \"GitHub\",\n    \"source\": \"github:mycompany\",\n    \"target\": \"mycompany\",\n    \"scope\": \"All repositories in mycompany\",\n    \"timestamp\": \"2024-06-29T14:23:15.456Z\",\n    \"moduleCount\": 14,\n    \"fileCount\": 38,\n    \"terraformModuleCount\": 11,\n    \"terragruntModuleCount\": 3,\n    \"terraformFileCount\": 31,\n    \"terragruntFileCount\": 7\n  },\n  \"modules\": [\n    {\n      \"name\": \"vpc_main\",\n      \"source\": \"terraform-aws-modules/vpc/aws\",\n      \"sourceType\": \"registry\",\n      \"version\": \"~\u003e 5.0\",\n      \"repository\": \"infrastructure-core\",\n      \"filePath\": \"networking/vpc.tf\",\n      \"fileUrl\": \"https://github.com/mycompany/infrastructure-core/blob/main/networking/vpc.tf\",\n      \"lineNumber\": 12,\n      \"type\": \"terraform\"\n    },\n    {\n      \"name\": \"network_module\",\n      \"source\": \"./modules/networking\",\n      \"sourceType\": \"local\",\n      \"repository\": \"app-platform\",\n      \"filePath\": \"main.tf\",\n      \"fileUrl\": \"https://github.com/mycompany/app-platform/blob/main/main.tf\",\n      \"lineNumber\": 25,\n      \"type\": \"terraform\"\n    },\n    {\n      \"name\": \"s3_bucket\",\n      \"source\": \"git::https://github.com/company/terraform-modules.git//s3?ref=v2.0\",\n      \"sourceType\": \"git\",\n      \"version\": \"ref=v2.0\",\n      \"repository\": \"data-storage\",\n      \"filePath\": \"storage.tf\",\n      \"fileUrl\": \"https://github.com/mycompany/data-storage/blob/main/storage.tf\",\n      \"lineNumber\": 8,\n      \"type\": \"terraform\"\n    }\n  ],\n  \"summary\": {\n    \"terraform-aws-modules/vpc/aws\": {\n      \"count\": 5,\n      \"versions\": {\n        \"~\u003e 5.0\": 3,\n        \"~\u003e 4.0\": 2\n      }\n    },\n    \"./modules/networking\": {\n      \"count\": 4,\n      \"versions\": {}\n    }\n  }\n}\n```\n\n### CSV Format\n\n| Module                                                        | Source Type | File Type | Version   | Repository          | File Path           | Line | File Link                                                                                               |\n|---------------------------------------------------------------|-------------|-----------|-----------|---------------------|---------------------|------|---------------------------------------------------------------------------------------------------------|\n| `terraform-aws-modules/vpc/aws`                              | registry    | terraform | `~\u003e 5.0`  | infrastructure-core | `networking/vpc.tf` | 12   | https://github.com/mycompany/infrastructure-core/blob/main/networking/vpc.tf#L12                       |\n| `terraform-aws-modules/vpc/aws`                              | registry    | terraform | `~\u003e 4.0`  | legacy-infra        | `main.tf`           | 8    | https://github.com/mycompany/legacy-infra/blob/main/main.tf#L8                                         |\n| `./modules/networking`                                        | local       | terraform | -         | app-platform        | `main.tf`           | 25   | https://github.com/mycompany/app-platform/blob/main/main.tf#L25                                        |\n| `./modules/networking`                                        | local       | terragrunt| -         | service-mesh        | `terragrunt.hcl`    | 15   | https://github.com/mycompany/service-mesh/blob/main/terragrunt.hcl#L15                                 |\n| `git::github.com/company/terraform-modules.git//s3?ref=v2.0` | git         | terraform | `ref=v2.0`| data-storage        | `storage.tf`        | 8    | https://github.com/mycompany/data-storage/blob/main/storage.tf#L8                                      |\n\n## 🗺️ Roadmap\n\n### ✅ Completed\n\n- **Local Filesystem Support** - Scan local directories with full feature parity\n- **Modern CLI Interface** - URI-style source format with backward compatibility\n- **Testing Infrastructure** - Comprehensive test suite with Jest (85%+ coverage)\n- **Clean Architecture** - Domain-organized structure with proper separation\n- **High-Performance Parallel Processing** - Configurable concurrent processing\n- **Service Factory Pattern** - Extensible architecture for multiple platforms\n- **Unified Output Format** - Consistent metadata across JSON, CSV, and table formats\n- **Code Quality Standards** - ESLint, Prettier, and TypeScript with zero warnings\n\n### 🚧 In Progress\n\n- **Extended VCS Support** - GitLab, Bitbucket, and Azure DevOps integration\n\n### 🔮 Planned\n\n- **Advanced Analytics** - Dependency graphs, security scanning, compliance reporting\n- **Performance Monitoring** - Built-in metrics and optimization suggestions\n- **Web Dashboard** - Browser-based interface for scan results\n- **CI/CD Integration** - GitHub Actions, GitLab CI, and Jenkins plugins\n\n## 🏗️ Architecture\n\nTerrawiz features a clean, modular architecture:\n\n```\nsrc/\n├── index.ts              # CLI entry point with platform routing\n├── vcs/                  # VCS platform integrations\n│   ├── base.ts          # Common VCS patterns\n│   ├── factory.ts       # Service factory\n│   └── github.ts        # GitHub implementation\n├── scanners/            # Filesystem scanners\n│   └── local-filesystem.ts # Local directory scanner\n├── parsers/             # IaC file parsers\n│   ├── base.ts          # Common parsing logic\n│   ├── terraform.ts     # Terraform parser\n│   └── terragrunt.ts    # Terragrunt parser\n├── utils/               # Utilities\n│   ├── concurrent.ts    # Parallel processing\n│   ├── source-parser.ts # Source URI parsing\n│   └── file-type-detector.ts # File type utilities\n├── constants/           # Configuration constants\n│   └── index.ts         # Centralized constants\n└── services/            # Shared services\n    └── logger.ts        # Logging service\n```\n\n### Key Design Principles\n\n- **🔄 Parallel Processing** - Concurrent repository and file processing for maximum performance\n- **🎯 Type Safety** - Comprehensive TypeScript types with proper error handling\n- **🏗️ Platform-Agnostic** - Early routing with dedicated scanners for VCS vs local filesystem\n- **🔧 Extensible** - Clean architecture ready for GitLab, Bitbucket, and other VCS platforms\n- **⚡ Performance-First** - Optimized API usage with intelligent rate limiting and caching\n\n## 🧪 Module Source Detection\n\nTerrawiz automatically categorizes modules by source type:\n\n| Type            | Examples                            | Description                |\n| --------------- | ----------------------------------- | -------------------------- |\n| **registry**    | `terraform-aws-modules/vpc/aws`     | Terraform Registry modules |\n| **git**         | `github.com/org/repo.git`           | Git repositories           |\n| **local**       | `./modules/vpc`, `../shared`        | Local file paths           |\n| **artifactory** | `jfrog.io/modules/vpc`              | Artifactory-hosted modules |\n| **archive**     | `https://releases.../module.tar.gz` | Archive downloads          |\n\n## 🚀 Performance Features\n\n### Parallel Processing\n\n- **Repository-level parallelism**: Process multiple repositories simultaneously\n- **File-level parallelism**: Download and parse files concurrently within each repository\n- **Configurable limits**: Balance performance with API rate limits\n\n### Optimizations\n\n- **Smart caching**: Avoid redundant API calls for repository metadata\n- **Batch operations**: Efficient GitHub API usage patterns\n- **Progress tracking**: Real-time feedback for long-running scans\n\n### Rate Limiting\n\n- **Automatic throttling**: Built-in GitHub API rate limit protection\n- **Retry logic**: Exponential backoff for failed requests\n- **Monitoring**: Rate limit status reporting in debug mode\n\n## 🤝 Contributing\n\nWe welcome contributions! Here's how to get started:\n\n1. **Fork** the repository\n2. **Create** a feature branch: `git checkout -b feature/amazing-feature`\n3. **Develop** your changes with tests\n4. **Test** your implementation: `npm test`\n5. **Commit** your changes: `git commit -m 'Add amazing feature'`\n6. **Push** to your branch: `git push origin feature/amazing-feature`\n7. **Open** a Pull Request\n\n### Development Setup\n\n```bash\n# Install dependencies\nnpm install\n\n# Run tests\nnpm test\n\n# Type check\nnpm run type-check\n\n# Run all quality checks\nnpm run quality\n\n# Build\nnpm run build\n```\n\n## 📄 License\n\nMIT License - see [LICENSE](LICENSE) file for details.\n\n---\n\n\u003cp align=\"center\"\u003e\n  \u003cstrong\u003eBuilt with ❤️ for Infrastructure as Code teams\u003c/strong\u003e\n\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fefemaer%2Fterrawiz","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fefemaer%2Fterrawiz","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fefemaer%2Fterrawiz/lists"}