{"id":31078089,"url":"https://github.com/amandladev/python-pipeline-creator","last_synced_at":"2025-09-21T15:04:15.683Z","repository":{"id":315000007,"uuid":"1057664005","full_name":"amandladev/python-pipeline-creator","owner":"amandladev","description":"A CLI tool for creating and managing CI/CD pipelines on AWS","archived":false,"fork":false,"pushed_at":"2025-09-16T04:12:32.000Z","size":22,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-16T06:23:36.808Z","etag":null,"topics":["aws","ci-cd","code-deploy","iac","pipeline"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/amandladev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-16T03:39:27.000Z","updated_at":"2025-09-16T04:35:28.000Z","dependencies_parsed_at":"2025-09-16T06:23:40.039Z","dependency_job_id":"d5c62400-a229-4bd5-a13e-41091485c4c4","html_url":"https://github.com/amandladev/python-pipeline-creator","commit_stats":null,"previous_names":["amandladev/python-stepfun-aws-cdk"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/amandladev/python-pipeline-creator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amandladev%2Fpython-pipeline-creator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amandladev%2Fpython-pipeline-creator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amandladev%2Fpython-pipeline-creator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amandladev%2Fpython-pipeline-creator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/amandladev","download_url":"https://codeload.github.com/amandladev/python-pipeline-creator/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amandladev%2Fpython-pipeline-creator/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":275384116,"owners_count":25454910,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-16T02:00:10.229Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","ci-cd","code-deploy","iac","pipeline"],"created_at":"2025-09-16T08:02:57.797Z","updated_at":"2025-09-18T10:02:52.839Z","avatar_url":"https://github.com/amandladev.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🚀 Pipeline Creator CLI\n\n**A powerful CLI tool for creating and managing enterprise-grade CI/CD pipelines on AWS with intelligent automation.**\n\n[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)\n[![AWS CDK](https://img.shields.io/badge/AWS-CDK-orange.svg)](https://aws.amazon.com/cdk/)\n[![MIT License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)\n\n## ✨ Features\n\n- 🏗️ **Smart Pipeline Generation** - Automated AWS CodePipeline creation with CDK\n- 📦 **Multi-Language Support** - Python, Node.js, React, and more\n- 🧪 **Advanced Build Stages** - SonarQube, Snyk, Codecov, ESLint, Bandit integration\n- 📧 **Intelligent Notifications** - Slack, Email, Webhooks with smart alerting rules  \n- 📋 **Pipeline Templates** - Reusable templates for faster setup\n- 🔄 **Template Inheritance** - Extend and customize existing templates\n- ⚙️ **Interactive CLI** - User-friendly prompts and rich console output\n- 🛡️ **Security First** - Built-in security scanning and best practices\n\n## 🚀 Quick Start\n\n### Installation\n\n```bash\n# 1. Clone and setup\ngit clone \u003crepository-url\u003e\ncd pipeline_creator\n\n# 2. Create virtual environment\npython -m venv .venv\nsource .venv/bin/activate  # Mac/Linux\n# or .venv\\Scripts\\activate  # Windows\n\n# 3. Install dependencies\npip install -e .\npip install aiohttp email-validator  # For notifications\n```\n\n### Create Your First Pipeline\n\n```bash\n# 1. Initialize a new pipeline\npipeline init\n\n# 2. Use a template for quick setup (optional)\npipeline templates use react-app -P app_name=my-app\n\n# 3. Generate AWS infrastructure\npipeline generate\n\n# 4. Deploy your pipeline\npipeline deploy\n```\n\n## 📋 Complete Command Reference\n\n### Core Pipeline Commands\n\n#### `pipeline init`\nInitialize pipeline configuration in your project.\n```bash\npipeline init                    # Interactive setup\npipeline init --name my-app     # Quick setup with name\n```\n\n#### `pipeline generate`\nGenerate AWS CDK infrastructure files.\n```bash\npipeline generate               # Generate all infrastructure\npipeline generate --force      # Overwrite existing files\n```\n\n#### `pipeline deploy`\nDeploy your pipeline to AWS.\n```bash\npipeline deploy                 # Deploy with confirmation\npipeline deploy --auto-approve  # Skip confirmations\n```\n\n#### `pipeline status`\nCheck your pipeline status.\n```bash\npipeline status                 # Show current status\npipeline status --detailed     # Include execution history\n```\n\n#### `pipeline logs`\nView pipeline logs.\n```bash\npipeline logs                   # Latest execution logs\npipeline logs --build-id \u003cid\u003e  # Specific build logs\n```\n\n### 🧪 Extra Build Stages\n\n#### `pipeline add-stage`\nAdd specialized build stages to your pipeline.\n```bash\npipeline add-stage              # Interactive stage selection\npipeline add-stage sonarqube   # Add SonarQube analysis\npipeline add-stage snyk        # Add security scanning\npipeline add-stage codecov     # Add coverage reporting\npipeline add-stage docker      # Add Docker build\n```\n\n**Available Stages:**\n- 🔍 **SonarQube Cloud** - Code quality analysis\n- 🛡️ **Snyk Security** - Vulnerability scanning  \n- 📊 **Codecov** - Coverage reporting\n- 🐳 **Docker Build** - Container builds\n- 🧹 **ESLint** - JavaScript/TypeScript linting\n- 🐍 **Bandit** - Python security linting\n- ⚙️ **Custom Stages** - Define your own build steps\n\n### 📧 Notification System\n\n#### `pipeline notifications`\nIntelligent notification management with smart alerting rules.\n\n```bash\n# Setup notifications\npipeline notifications setup              # Interactive setup\npipeline notifications setup -c slack    # Setup specific channel\n\n# Check status\npipeline notifications status            # View current config\n\n# Test notifications  \npipeline notifications test              # Send test messages\n\n# Disable notifications\npipeline notifications disable           # Disable all channels\n```\n\n**Supported Channels:**\n- 📧 **Email** - HTML/text email notifications via SMTP\n- 📢 **Slack** - Rich webhook notifications with formatting\n- 🔗 **Webhooks** - Custom HTTP endpoints with JSON payloads\n\n**Smart Features:**\n- ⚡ **Failure Alerts** - Always notify on pipeline failures\n- 🎉 **Recovery Notifications** - Alert when pipeline recovers\n- 🔇 **Spam Prevention** - Intelligent rules to reduce noise\n- 📈 **Event History** - Track notification patterns\n\n### 📋 Template System\n\n#### `pipeline templates`\nPowerful template system for reusable pipeline configurations.\n\n```bash\n# List available templates\npipeline templates list                   # All templates\npipeline templates list --category api   # Filter by category\n\n# Get template information\npipeline templates info react-app        # Detailed template info\n\n# Use templates\npipeline templates use react-app \\\n  -P app_name=my-app \\\n  -P test_coverage_threshold=90\n\n# Create custom templates\npipeline templates create my-template \\\n  --description \"My custom pipeline\" \\\n  --category web-frontend \\\n  --author \"My Team\"\n\n# Template inheritance\npipeline templates extend react-app enhanced-react \\\n  --description \"Enhanced React template\"\n\n# Share templates\npipeline templates export react-app template.json\npipeline templates import-template template.json\n```\n\n**Predefined Templates:**\n- 🌐 **react-app** - Complete React application with S3 + CloudFront\n- 🔌 **python-api** - Python/FastAPI API with ECS Fargate\n- 🟨 **nodejs-api** - Node.js/Express API with containers\n\n**Template Categories:**\n- `web-frontend` - Frontend applications\n- `web-backend` - Backend services  \n- `api` - API services\n- `microservice` - Microservices\n- `mobile` - Mobile applications\n- `data-processing` - Data pipelines\n- `ml-ai` - Machine Learning workflows\n\n## 🛠️ Prerequisites\n\n- **Python 3.8+** with pip\n- **AWS CLI** configured (`aws configure`)\n- **Node.js 16+** (for AWS CDK)\n- **Docker** (optional, for containerized builds)\n\n## 📁 Project Structure\n\nAfter initialization, your project will have:\n\n```\nyour-project/\n├── pipeline.json           # Main pipeline configuration\n├── .pipeline/             \n│   ├── cdk/               # Generated CDK files\n│   ├── config.json        # Internal configuration\n│   └── templates/         # User templates (if any)\n└── buildspec.yml          # CodeBuild specification\n```\n\n## 🎯 Configuration Examples\n\n### Basic Python API Pipeline\n```json\n{\n  \"name\": \"my-python-api\",\n  \"project_type\": \"python\",\n  \"runtime\": {\n    \"language\": \"python\", \n    \"version\": \"3.11\"\n  },\n  \"build\": {\n    \"commands\": [\n      \"pip install -r requirements.txt\",\n      \"pytest tests/\",\n      \"docker build -t my-api .\"\n    ]\n  },\n  \"deploy\": {\n    \"type\": \"ecs-fargate\",\n    \"config\": {\n      \"container_port\": 8000,\n      \"cpu\": 256,\n      \"memory\": 512\n    }\n  }\n}\n```\n\n### React App with Extra Stages\n```json\n{\n  \"name\": \"my-react-app\",\n  \"project_type\": \"react\",\n  \"extra_stages\": [\n    {\n      \"name\": \"sonarqube\",\n      \"type\": \"sonarqube_cloud\",\n      \"config\": {\n        \"project_key\": \"my-sonar-key\",\n        \"sources\": \"src\"\n      }\n    },\n    {\n      \"name\": \"security-scan\", \n      \"type\": \"snyk\",\n      \"config\": {\n        \"language\": \"nodejs\",\n        \"severity_threshold\": \"high\"\n      }\n    }\n  ],\n  \"notifications\": {\n    \"enabled\": true,\n    \"channels\": [\"slack\", \"email\"],\n    \"events\": [\"pipeline_failed\", \"deploy_completed\"]\n  }\n}\n```\n\n## 🚀 Advanced Usage\n\n### Environment Variables\nSet these environment variables for enhanced functionality:\n\n```bash\nexport SONAR_TOKEN=\"your-sonar-token\"\nexport SNYK_TOKEN=\"your-snyk-token\" \nexport CODECOV_TOKEN=\"your-codecov-token\"\nexport SLACK_WEBHOOK_URL=\"your-slack-webhook\"\n```\n\n### Custom Build Stages\nAdd custom build logic to your pipeline:\n\n```json\n{\n  \"extra_stages\": [\n    {\n      \"name\": \"custom-build\",\n      \"type\": \"custom\",\n      \"config\": {\n        \"commands\": [\n          \"echo 'Running custom build steps'\",\n          \"npm run custom-script\",\n          \"python custom_script.py\"\n        ],\n        \"environment_variables\": {\n          \"CUSTOM_VAR\": \"custom_value\"\n        }\n      }\n    }\n  ]\n}\n```\n\n## 🧪 Testing\n\n```bash\n# Run all tests\npytest\n\n# Run with coverage\npytest --cov=pipeline_creator --cov-report=html\n\n# Run specific test modules\npytest tests/test_templates.py\npytest tests/test_notifications.py\n```\n\n## 🤝 Contributing\n\nWe welcome contributions! Here's how to get started:\n\n1. **Fork** the repository\n2. **Create** a feature branch (`git checkout -b feature/amazing-feature`)\n3. **Install** development dependencies (`pip install -e .[dev]`)\n4. **Write** tests for your changes\n5. **Run** tests (`pytest`)\n6. **Commit** your changes (`git commit -m 'Add amazing feature'`)\n7. **Push** to the branch (`git push origin feature/amazing-feature`)\n8. **Create** a Pull Request\n\n### Development Setup\n```bash\n# Install development dependencies\npip install -e .[dev]\n\n# Install pre-commit hooks\npre-commit install\n\n# Run linting\nflake8 pipeline_creator/\nblack pipeline_creator/\n\n# Run type checking\nmypy pipeline_creator/\n```\n\n## 📚 Documentation\n\n- [Getting Started Guide](docs/getting-started.md)\n- [Template Creation Guide](docs/templates.md)\n- [Notification Setup](docs/notifications.md)\n- [Advanced Configuration](docs/advanced.md)\n- [API Reference](docs/api.md)\n\n## ❓ Troubleshooting\n\n### Common Issues\n\n**Q: \"ModuleNotFoundError\" when running commands**  \nA: Make sure you've activated your virtual environment and installed dependencies:\n```bash\nsource .venv/bin/activate\npip install -e .\n```\n\n**Q: AWS deployment fails with permissions error**  \nA: Ensure your AWS credentials have the necessary permissions:\n```bash\naws sts get-caller-identity  # Check current identity\naws configure list          # Check configuration\n```\n\n**Q: Notifications not working**  \nA: Check your notification configuration:\n```bash\npipeline notifications status\npipeline notifications test\n```\n\n## 🎉 What's New\n\n### v0.3.0 - Template System\n- ✨ Complete template system with inheritance\n- 📋 3 predefined templates (React, Python API, Node.js API)\n- 🔄 Template import/export functionality\n- 🎯 Interactive parameter configuration\n\n### v0.2.0 - Smart Notifications  \n- 📧 Multi-channel notification support (Email, Slack, Webhooks)\n- ⚡ Smart alerting rules to reduce spam\n- 🎨 Rich HTML email templates\n- 📊 Event history and analytics\n\n### v0.1.0 - Core Features\n- 🏗️ AWS CodePipeline generation with CDK\n- 🧪 Extra build stages (SonarQube, Snyk, Codecov)\n- ⚙️ Interactive CLI with rich console output\n- 📦 Multi-language project support\n\n## 🔮 What's Next\n\nWe're continuously improving Pipeline Creator CLI! Here's our exciting roadmap:\n\n### 🎯 v0.4.0 - Multi-Environment Support (In Progress)\n- 🌍 **Environment Management** - Staging, production, and custom environments\n- 🚪 **Approval Gates** - Manual approval workflows between environments  \n- 🔄 **Progressive Deployments** - Blue-green and canary deployment strategies\n- ⚙️ **Environment-Specific Configs** - Different settings per environment\n- 🔁 **Automatic Rollbacks** - Smart rollback on deployment failures\n- 🏷️ **Environment Tagging** - Resource organization and cost tracking\n\n### 🚀 v0.5.0 - Enhanced Pipeline Intelligence\n- 📊 **Pipeline Analytics** - Performance metrics and deployment insights\n- 🤖 **AI-Powered Optimization** - Automatic pipeline improvements suggestions\n- 🔍 **Dependency Analysis** - Smart dependency tracking and updates\n- 📈 **Cost Optimization** - AWS cost analysis and recommendations\n- 🕒 **Scheduling** - Time-based deployments and maintenance windows\n\n### 🛡️ v0.6.0 - Enterprise Security \u0026 Governance\n- 🔐 **RBAC Integration** - Role-based access control with AWS IAM\n- 📋 **Compliance Templates** - SOC2, HIPAA, PCI-DSS compliant pipelines\n- 🛡️ **Secret Management** - AWS Secrets Manager and Parameter Store integration\n- 📊 **Audit Trails** - Comprehensive deployment and change logging\n- 🔒 **Policy Enforcement** - Automated policy compliance checking\n\n### 🌐 v0.7.0 - Multi-Cloud \u0026 Hybrid Support\n- ☁️ **Azure DevOps** - Azure Pipelines generation and deployment\n- 🔧 **Google Cloud Build** - GCP pipeline creation and management\n- 🏢 **On-Premise Integration** - Jenkins and GitLab CI/CD support\n- 🔗 **Multi-Cloud Deployments** - Cross-cloud deployment strategies\n- 📦 **Universal Templates** - Cloud-agnostic pipeline templates\n\n### 🎨 v0.8.0 - Developer Experience Enhancements\n- 🖥️ **Web Dashboard** - Visual pipeline builder and monitoring\n- 📱 **Mobile App** - Pipeline monitoring on the go\n- 🔌 **IDE Extensions** - VS Code, IntelliJ, and other IDE integrations\n- 🤝 **Git Integration** - Advanced Git hooks and branch strategies\n- 📝 **Documentation Generator** - Auto-generate pipeline documentation\n\n### 🧪 v0.9.0 - Advanced Testing \u0026 Quality\n- 🤖 **Automated Testing** - AI-powered test generation and execution\n- 🔍 **Performance Testing** - Load testing integration (k6, JMeter)\n- 🌐 **Cross-Browser Testing** - Selenium Grid and Playwright support\n- 📊 **Quality Gates** - Advanced quality metrics and thresholds\n- 🐛 **Bug Prevention** - Predictive analysis for common issues\n\n### 📦 v1.0.0 - Production Ready Enterprise Edition\n- 🏢 **Enterprise Templates** - Industry-specific pipeline templates\n- 📊 **Executive Dashboards** - C-level reporting and insights\n- 🎓 **Training Materials** - Comprehensive documentation and tutorials\n- 🔧 **Professional Support** - Dedicated support channels\n- 🚀 **Performance Optimization** - Enterprise-scale performance tuning\n\n## 💡 Community Requests\n\nVote for features you'd like to see next! Open an issue with the `feature-request` label:\n\n- 🐍 **Python Package Publishing** - PyPI integration for Python projects\n- 📦 **Docker Registry Support** - Private registry integration (ECR, Docker Hub)\n- 🌊 **Kubernetes Deployments** - EKS and self-managed K8s support\n- 🔄 **GitOps Integration** - ArgoCD and Flux support\n- 📧 **Microsoft Teams** - Teams notification channel\n- 🎯 **Terraform Integration** - Infrastructure as Code workflows\n- 🔒 **HashiCorp Vault** - Advanced secret management\n- 📈 **Datadog Integration** - Enhanced monitoring and alerting\n\n## 🤝 Contributing to the Roadmap\n\nWe value community input! Here's how you can influence our roadmap:\n\n1. **🗳️ Vote on Features** - Star issues you'd like prioritized\n2. **💡 Suggest Ideas** - Open feature request issues\n3. **🛠️ Contribute Code** - Submit PRs for roadmap items\n4. **🐛 Report Issues** - Help us improve current features\n5. **📖 Improve Docs** - Documentation contributions welcome\n\nJoin our [Discord Community](https://discord.gg/pipeline-creator) for roadmap discussions!\n\n## 📄 License\n\nMIT License - see the [LICENSE](LICENSE) file for details.\n\n---\n\n**Made with ❤️ by the Pipeline Creator Team**\n\n*Streamlining DevOps, one pipeline at a time.*\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Famandladev%2Fpython-pipeline-creator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Famandladev%2Fpython-pipeline-creator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Famandladev%2Fpython-pipeline-creator/lists"}