{"id":48839530,"url":"https://github.com/sourcefuse/ai-test-quality-analyzer","last_synced_at":"2026-04-15T01:03:16.657Z","repository":{"id":321569294,"uuid":"1086303502","full_name":"sourcefuse/ai-test-quality-analyzer","owner":"sourcefuse","description":null,"archived":false,"fork":false,"pushed_at":"2026-04-02T16:34:03.000Z","size":6568,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-03T04:43:42.428Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sourcefuse.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-30T08:30:53.000Z","updated_at":"2026-01-20T13:53:39.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/sourcefuse/ai-test-quality-analyzer","commit_stats":null,"previous_names":["sourcefuse/ai-test-quality-analyzer"],"tags_count":10,"template":false,"template_full_name":null,"purl":"pkg:github/sourcefuse/ai-test-quality-analyzer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcefuse%2Fai-test-quality-analyzer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcefuse%2Fai-test-quality-analyzer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcefuse%2Fai-test-quality-analyzer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcefuse%2Fai-test-quality-analyzer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sourcefuse","download_url":"https://codeload.github.com/sourcefuse/ai-test-quality-analyzer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcefuse%2Fai-test-quality-analyzer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31821686,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-14T18:05:02.291Z","status":"ssl_error","status_checked_at":"2026-04-14T18:05:01.765Z","response_time":153,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-04-15T01:03:16.378Z","updated_at":"2026-04-15T01:03:16.630Z","avatar_url":"https://github.com/sourcefuse.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AI Test Quality Analyzer\n\n[![GitHub Action](https://img.shields.io/badge/GitHub-Action-blue)](https://github.com/marketplace/actions/check-unit-test-cases-quality)\n[![Version](https://img.shields.io/badge/version-QC--V1.0-green)](https://github.com/sourcefuse/ai-test-quality-analyzer/releases)\n[![License](https://img.shields.io/badge/license-ISC-brightgreen)](LICENSE)\n\nAn intelligent GitHub Action that automatically analyzes the quality of unit test cases in your codebase by comparing them against JIRA requirements and Confluence documentation using AI-powered analysis.\n\n## 🎯 Features\n\n- **Automated Test Quality Scoring**: Generates a comprehensive quality score (0-10) for your test suite\n- **JIRA Integration**: Fetches requirements directly from JIRA tickets\n- **Confluence Integration**: Analyzes related documentation for complete context\n- **AI-Powered Analysis**: Uses AWS Bedrock with Claude for intelligent test evaluation\n- **PR Comments**: Automatically posts detailed reports as pull request comments\n- **Detailed Reporting**: Provides actionable insights and improvement recommendations\n- **Configurable Thresholds**: Set minimum quality scores for your workflow\n\n## 📊 Sample Report\n\nWhen the action runs, it generates a comprehensive report that includes:\n\n```\n## ✅ Unit Test Quality Report\n\n**JIRA Ticket:** BB-1234\n**Branch:** feature/BB-1234-new-feature\n\n### 📊 Quality Score\n\n████████░░ 8.0/10\n\n| Metric | Score |\n|--------|-------|\n| Test Quality Score | **8.0/10** |\n| Minimum Threshold | 6.0/10 |\n| Status | **PASSED** |\n```\n\n## 🚀 Quick Start\n\n### 1. Add Workflow to Your Repository\n\nCreate `.github/workflows/check-unit-testcases.yml`:\n\n```yaml\nname: Check Unit Test Cases Quality\n\non:\n  pull_request:\n    branches:\n      - dev\n    types: [opened, synchronize, reopened]\n\njobs:\n  analyze-test-quality:\n    name: Analyze Test Quality\n    runs-on: ubuntu-latest\n    \n    permissions:\n      contents: read\n      pull-requests: write\n      issues: write\n    \n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          fetch-depth: 0\n\n      - name: Extract JIRA ticket\n        id: extract_ticket\n        run: |\n          # Extract ticket from branch name or PR title\n          if [[ \"${{ github.head_ref }}\" =~ ([A-Z]+-[0-9]+) ]]; then\n            TICKET_ID=\"${BASH_REMATCH[1]}\"\n          fi\n          echo \"ticket_id=$TICKET_ID\" \u003e\u003e $GITHUB_OUTPUT\n\n      - name: Run Unit Test Quality Analysis\n        uses: sourcefuse/ai-test-quality-analyzer@QC-V1.0\n        with:\n          jira_url: ${{ secrets.UT_QUALITY_JIRA_URL }}\n          jira_email: ${{ secrets.UT_QUALITY_JIRA_EMAIL }}\n          jira_api_token: ${{ secrets.UT_QUALITY_JIRA_API_TOKEN }}\n          jira_ticket_id: ${{ steps.extract_ticket.outputs.ticket_id }}\n          confluence_url: ${{ secrets.UT_QUALITY_CONFLUENCE_URL }}\n          confluence_email: ${{ secrets.UT_QUALITY_CONFLUENCE_EMAIL }}\n          confluence_api_token: ${{ secrets.UT_QUALITY_CONFLUENCE_API_TOKEN }}\n          aws_access_key_bedrock: ${{ secrets.UT_QUALITY_AWS_ACCESS_KEY_BEDROCK }}\n          aws_secret_key_bedrock: ${{ secrets.UT_QUALITY_AWS_SECRET_KEY_BEDROCK }}\n          minimum_score: '6.0'\n```\n\n### 2. Configure Secrets\n\nAdd required secrets in your repository settings:\n\n- `UT_QUALITY_JIRA_URL` - Your JIRA instance URL\n- `UT_QUALITY_JIRA_EMAIL` - JIRA user email\n- `UT_QUALITY_JIRA_API_TOKEN` - JIRA API token\n- `UT_QUALITY_CONFLUENCE_URL` - Confluence URL\n- `UT_QUALITY_CONFLUENCE_EMAIL` - Confluence email\n- `UT_QUALITY_CONFLUENCE_API_TOKEN` - Confluence API token\n- `UT_QUALITY_AWS_ACCESS_KEY_BEDROCK` - AWS access key\n- `UT_QUALITY_AWS_SECRET_KEY_BEDROCK` - AWS secret key\n\n### 3. Create Pull Request\n\nThe action automatically triggers when you:\n- Create a PR to your configured branch\n- Include a JIRA ticket ID in branch name or PR title (e.g., `feature/BB-1234-description`)\n\n## 📋 How It Works\n\n```mermaid\ngraph LR\n    A[PR Created] --\u003e B[Extract JIRA ID]\n    B --\u003e C[Fetch JIRA Requirements]\n    C --\u003e D[Fetch Confluence Docs]\n    D --\u003e E[Clone Repository]\n    E --\u003e F[AI Analysis]\n    F --\u003e G[Generate Score]\n    G --\u003e H[Post PR Comment]\n    H --\u003e I[Upload to Confluence]\n```\n\n1. **Requirement Gathering**: Fetches requirements from JIRA and related Confluence pages\n2. **Code Analysis**: Examines test files in your repository\n3. **AI Evaluation**: Uses Claude to assess test quality against requirements\n4. **Score Generation**: Calculates quality score based on multiple factors\n5. **Report Creation**: Generates detailed analysis with recommendations\n6. **Result Posting**: Adds comment to PR and optionally uploads to Confluence\n\n## 📈 Quality Metrics\n\nThe analyzer evaluates tests across five key dimensions:\n\n| Category | Weight | Focus Areas |\n|----------|--------|-------------|\n| **Test Coverage** | 20% | Requirement coverage, missing scenarios |\n| **Test Quality** | 20% | Assertions, test structure, clarity |\n| **Edge Cases** | 20% | Error handling, boundary conditions |\n| **Assertions** | 20% | Quality and completeness of checks |\n| **Maintainability** | 20% | Code organization, documentation |\n\n## 🔧 Configuration Options\n\n### Action Inputs\n\n| Input | Required | Default | Description |\n|-------|----------|---------|-------------|\n| `jira_ticket_id` | Yes | - | JIRA ticket to analyze |\n| `minimum_score` | No | `6.0` | Minimum acceptable score |\n| `save_to_file` | No | `true` | Save results to files |\n| `upload_to_confluence` | No | `true` | Upload to Confluence |\n| `repository_branch` | No | `main` | Branch to analyze |\n\n### Environment Variables\n\nConfigure via GitHub Variables (optional):\n\n- `UT_QUALITY_JIRA_PROJECT_KEY` - Default JIRA project (default: `BB`)\n- `UT_QUALITY_CONFLUENCE_SPACE_KEY` - Default Confluence space (default: `BB`)\n- `UT_QUALITY_MINIMUM_SCORE` - Global minimum score (default: `1.0`)\n- `UT_QUALITY_CLAUDE_CODE_USE_BEDROCK` - Use Bedrock (default: `1`)\n\n## 🛠️ Setup Script\n\nFor automated setup, use the provided script:\n\n```bash\n# Clone the repository\ngit clone https://github.com/sourcefuse/ai-test-quality-analyzer.git\ncd ai-test-quality-analyzer\n\n# Create .env file with your credentials\ncp .env.example .env\n# Edit .env with your values\n\n# Run setup script\nchmod +x setup-github-secrets.sh\n./setup-github-secrets.sh\n```\n\nThe script will:\n- Verify GitHub CLI authentication\n- Configure all required secrets\n- Set up optional variables\n- Provide setup confirmation\n\n## 📝 Example Use Cases\n\n### 1. Enforce Quality Standards\n\n```yaml\n- name: Run Unit Test Quality Analysis\n  uses: sourcefuse/ai-test-quality-analyzer@QC-V1.0\n  with:\n    minimum_score: '7.0'  # Require high quality\n    \n- name: Fail if quality is low\n  if: steps.quality_check.outputs.score_passed == 'false'\n  run: exit 1\n```\n\n### 2. Different Standards per Branch\n\n```yaml\nminimum_score: ${{ \n  github.base_ref == 'main' \u0026\u0026 '8.0' ||\n  github.base_ref == 'dev' \u0026\u0026 '6.0' ||\n  '4.0'\n}}\n```\n\n### 3. Manual Analysis\n\nTrigger manually from Actions tab with custom parameters:\n- Specific JIRA ticket\n- Custom quality threshold\n- Target branch\n\n## 📊 Output Examples\n\n### PR Comment Format\n\nThe action posts a formatted comment with:\n- Visual score indicator\n- Pass/fail status\n- Score breakdown\n- Links to detailed reports\n- Improvement suggestions\n\n### Confluence Report\n\nUploaded reports include:\n- Complete requirements mapping\n- Test coverage analysis\n- Code quality assessment\n- Specific improvement recommendations\n- Example code snippets\n\n## 🔍 Troubleshooting\n\n### Common Issues\n\n| Issue | Solution |\n|-------|----------|\n| \"Cannot find JIRA ticket\" | Ensure branch/PR contains ticket ID (e.g., `BB-1234`) |\n| \"AWS authentication failed\" | Verify Bedrock credentials and region |\n| \"Confluence upload failed\" | Check API token permissions |\n| \"Score below threshold\" | Review report for improvement areas |\n\n### Debug Mode\n\nEnable detailed logging:\n\n```yaml\nenv:\n  ACTIONS_RUNNER_DEBUG: true\n  ACTIONS_STEP_DEBUG: true\n```\n\n## 🔒 Security\n\n- All credentials stored as GitHub Secrets\n- API tokens should have minimal required permissions\n- Rotate tokens every 90 days\n- Use dedicated service accounts\n- Enable GitHub secret scanning\n\n## 📦 Development\n\n### Local Testing\n\n```bash\n# Install dependencies\nnpm install\n\n# Set up environment\ncp .env.example .env\n# Add your credentials to .env\n\n# Run analysis\nnpm run start\nnpm run create-requirement-doc\nnpm run analyze-test-quality\n```\n\n### Project Structure\n\n```\n├── action.yml              # GitHub Action definition\n├── fetch-and-analyze.ts    # Main analysis orchestrator\n├── src/\n│   ├── services/          # Core services\n│   │   ├── jira.service.ts\n│   │   ├── confluence.service.ts\n│   │   ├── bedrock-ai.service.ts\n│   │   └── requirement-analyzer.service.ts\n│   ├── dtos/              # Data transfer objects\n│   ├── models/            # Data models\n│   └── utils/             # Utility functions\n├── templates/             # Workflow templates\n├── prompts/              # AI prompt templates\n└── setup-github-secrets.sh # Setup automation\n```\n\n## 🤝 Contributing\n\nWe welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.\n\n1. Fork the repository\n2. Create a feature branch\n3. Commit your changes\n4. Push to the branch\n5. Open a Pull Request\n\n## 📄 License\n\nThis project is licensed under the ISC License - see the [LICENSE](LICENSE) file for details.\n\n## 👥 Support\n\n- **Issues**: [GitHub Issues](https://github.com/sourcefuse/ai-test-quality-analyzer/issues)\n- **Discussions**: [GitHub Discussions](https://github.com/sourcefuse/ai-test-quality-analyzer/discussions)\n- **Documentation**: [Setup Guide](SETUP.md)\n\n## 🙏 Acknowledgments\n\n- Built with [AWS Bedrock](https://aws.amazon.com/bedrock/) and [Claude](https://www.anthropic.com/claude)\n- Integrates with [Atlassian JIRA](https://www.atlassian.com/software/jira) and [Confluence](https://www.atlassian.com/software/confluence)\n- Powered by [GitHub Actions](https://github.com/features/actions)\n\n## 📈 Roadmap\n\n- [ ] Support for multiple testing frameworks\n- [ ] Integration with code coverage tools\n- [ ] Custom quality rubrics\n- [ ] Performance testing analysis\n- [ ] Integration testing support\n- [ ] Multi-language support\n\n---\n\n**Created by** [Vishal Gupta](https://github.com/vishalgupta2)\n\n**Maintained by** [SourceFuse](https://github.com/sourcefuse)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsourcefuse%2Fai-test-quality-analyzer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsourcefuse%2Fai-test-quality-analyzer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsourcefuse%2Fai-test-quality-analyzer/lists"}