{"id":44104185,"url":"https://github.com/miltroj/result-companion","last_synced_at":"2026-04-02T15:37:45.507Z","repository":{"id":274337190,"uuid":"922506816","full_name":"miltroj/result-companion","owner":"miltroj","description":"Result Companion is a Python-based CLI tool designed to analyze Robot Framework output.xml results, focusing on identifying and diagnosing failing tests. It leverages LiteLLM to integrate with powerful models for root cause analysis, helping automate the troubleshooting process in test workflows","archived":false,"fork":false,"pushed_at":"2026-03-26T16:04:03.000Z","size":10206,"stargazers_count":1,"open_issues_count":6,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-03-26T20:27:11.119Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/miltroj.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-01-26T12:03:13.000Z","updated_at":"2026-03-17T19:25:15.000Z","dependencies_parsed_at":"2025-02-25T21:24:00.728Z","dependency_job_id":"3e05ab74-aefd-4cb3-842b-4085169db847","html_url":"https://github.com/miltroj/result-companion","commit_stats":null,"previous_names":["miltroj/result-companion"],"tags_count":12,"template":false,"template_full_name":null,"purl":"pkg:github/miltroj/result-companion","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miltroj%2Fresult-companion","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miltroj%2Fresult-companion/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miltroj%2Fresult-companion/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miltroj%2Fresult-companion/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/miltroj","download_url":"https://codeload.github.com/miltroj/result-companion/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miltroj%2Fresult-companion/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31309130,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-02T12:59:32.332Z","status":"ssl_error","status_checked_at":"2026-04-02T12:54:48.875Z","response_time":89,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-02-08T15:00:19.513Z","updated_at":"2026-04-02T15:37:45.499Z","avatar_url":"https://github.com/miltroj.png","language":"Python","funding_links":[],"categories":["Tools"],"sub_categories":["AI Tools"],"readme":"# Result Companion\n\n[![PyPI version](https://img.shields.io/pypi/v/result-companion)](https://pypi.org/project/result-companion/)\n[![Python versions](https://img.shields.io/pypi/pyversions/result-companion)](https://pypi.org/project/result-companion/)\n[![License](https://img.shields.io/pypi/l/result-companion)](https://github.com/miltroj/result-companion/blob/main/LICENSE)\n[![CI](https://github.com/miltroj/result-companion/actions/workflows/publish.yml/badge.svg)](https://github.com/miltroj/result-companion/actions/workflows/publish.yml)\n\n**Turn your Robot Framework test failures into instant, actionable insights with AI.**\n\n![Demo](https://raw.githubusercontent.com/miltroj/result-companion/main/assets/demo.gif)\n\n## Why Result Companion?\n\nEvery QA engineer knows the pain: A test fails. You dig through logs. You trace keywords. You hunt for that one error message buried in thousands of lines. **Hours wasted.**\n\nResult Companion changes that. It reads your `output.xml`, understands the entire test flow, and tells you exactly what went wrong and how to fix it—in seconds, not hours.\n\n## What It Does\n\n```bash\n# Before: Manual debugging for hours\nrobot tests/                     # Test fails\n# Now: Where did it fail? Why? What's the root cause?\n\n# After: Instant AI analysis\nresult-companion analyze -o output.xml   # Get answers in seconds\n```\n\nYour enhanced `log.html` now includes:\n- **Root Cause Analysis**: Pinpoints the exact keyword and reason for failure\n- **Test Flow Summary**: Understand what happened at a glance\n- **Actionable Fixes**: Specific suggestions to resolve the issue\n\nFor CI logs and pipelines, use text output. An overall failure synthesis is run by default and added to both `rc_log.html` and text output. Disable with `--no-overall-summary`:\n\n```bash\nresult-companion analyze -o output.xml --text-report rc_summary.txt\nresult-companion analyze -o output.xml --print-text-report\nresult-companion analyze -o output.xml --no-overall-summary\n```\n\n## Copilot Review Agent\n\nReplaces the manual \"which commit broke this test?\" investigation. AI cross-references Robot Framework failures with PR code changes via GitHub Copilot and posts the verdict as a PR comment:\n\n```bash\nresult-companion analyze -o output.xml --json-report rc_summary.json\nresult-companion review -s rc_summary.json --repo owner/repo --pr 65\n\n# Save to file for review/editing before posting\nresult-companion review -s rc_summary.json --repo owner/repo --pr 65 --preview -o review.md\n```\n\nSee [`examples/PR_REVIEW.md`](examples/PR_REVIEW.md) for setup, flow diagram, flags, and GitHub Actions usage.\n\n\u003cdetails\u003e\n\u003csummary\u003eExample generated PR comment — \u003ca href=\"https://github.com/miltroj/result-companion/pull/65#issuecomment-4100454015\"\u003ePR #65\u003c/a\u003e\u003c/summary\u003e\n\n## 🔍 result-companion: Test Failure Analysis\n\n**Root cause:** unclear — investigate further\n\n- **Location:** [`poc_pr_review.py:6`](https://github.com/miltroj/result-companion/blob/investigate_code_review_functionality/poc_pr_review.py#L6) — file docstring and example usage reference interactive `gh auth login` which, if executed in CI without a token, can trigger GitHub 403/forbidden responses\n- **Location:** [`poc_pr_review.py:35`](https://github.com/miltroj/result-companion/blob/investigate_code_review_functionality/poc_pr_review.py#L35) — the prompt/action builder constructs shell commands that would run `gh pr comment` without using a non-interactive token, risking authentication failures in CI\n\n## 💡 Suggested Fix\n\nReplace interactive GH auth and posting with a token-based non-interactive command:\n\n```python\naction = (\n    \"Print the review comment body only — do NOT run gh pr comment.\"\n    if preview\n    else (\n        f'echo \"$GITHUB_TOKEN\" | gh auth login --with-token \u0026\u0026 '\n        f'gh pr comment {pr_number} --repo {repo_name} --body \"\u003creview text\u003e\"'\n    )\n)\n```\n\nEnsure CI provides `GITHUB_TOKEN` secret and keep `preview=True` by default in CI invocation.\n\n\u003c/details\u003e\n\n## Quick Start\n\n### Option 1: GitHub Copilot (Easiest for Users With Copilot)\n\nAlready have GitHub Copilot? Use it directly—no API keys needed.\n\n```bash\npip install result-companion\n\n# One-time setup\nbrew install copilot-cli   # or: npm install -g @github/copilot\ncopilot -i \"/login\"            # Login when prompted, then /exit\n\n# Analyze your tests\nresult-companion analyze -o output.xml -c examples/configs/copilot_config.yaml\n```\n\nSee [Copilot setup guide](https://github.com/miltroj/result-companion/blob/main/examples/EXAMPLES.md#github-copilot-recommended-for-users-with-copilot).\n\n### Option 2: Local AI (Free, Private)\n\n```bash\npip install result-companion\n\n# Auto-setup local AI model\nresult-companion setup ollama\nresult-companion setup model deepseek-r1:1.5b\n\n# Analyze your tests\nresult-companion analyze -o output.xml -c examples/configs/ollama_config.yaml\n```\n\n### Option 3: Cloud AI ([OpenAI](https://github.com/miltroj/result-companion/blob/main/examples/EXAMPLES.md#openai), Azure, Google)\n\n```bash\npip install result-companion\n\n# Configure and run\nexport OPENAI_API_KEY=\"your-key\"\nresult-companion analyze -o output.xml -c examples/configs/openai_config.yaml\n```\n\nSupports 100+ LLM providers via [LiteLLM](https://docs.litellm.ai/docs/providers).\n\n## Real Example\n\n**Your test fails with:**\n```\nLogin Test Suite\n└── Login With Valid Credentials [FAIL]\n```\n\n**Result Companion tells you:**\n```markdown\n**Flow**\n- Open browser to login page ✓\n- Enter username \"testuser\" ✓\n- Enter password ✓\n- Click login button ✓\n- Wait for dashboard [FAILED after 10s timeout]\n\n**Failure Root Cause**\nThe keyword \"Wait Until Page Contains Element\" failed because\nelement 'id=dashboard' was not found. Server returned 503 error\nin network logs at timestamp 14:23:45.\n\n**Potential Fixes**\n- Check if backend service is running and healthy\n- Verify dashboard element selector hasn't changed\n- Increase timeout if service startup is slow\n```\n\n## Beyond Error Analysis\n\nCustomize prompts for any use case:\n\n```yaml\n# security_audit.yaml\nllm_config:\n  question_prompt: |\n    Find security issues: hardcoded passwords,\n    exposed tokens, insecure configurations...\n```\n\n```yaml\n# performance_review.yaml\nllm_config:\n  question_prompt: |\n    Identify slow operations, unnecessary waits,\n    inefficient loops...\n```\n\nSee [Custom Analysis examples](https://github.com/miltroj/result-companion/blob/main/examples/EXAMPLES.md#custom-analysis) for security audits, performance reviews, and more. The `llm_config` section also supports `chunking` prompts for large test suites.\n\n## Configuration Examples\n\nCheck [`examples/configs/`](https://github.com/miltroj/result-companion/tree/main/examples/configs) for ready-to-use configs:\n- **GitHub Copilot** (easiest for users with copilot)\n- Local Ollama setup\n- OpenAI, Azure, Google Cloud\n- Custom endpoints (Databricks, self-hosted)\n- Prompt customization for security, performance, quality reviews\n\n## Filter Tests by Tags\n\nAnalyze only the tests you care about:\n\n```bash\n# Analyze smoke tests only\nresult-companion analyze -o output.xml --include \"smoke*\"\n\n# Exclude work-in-progress tests\nresult-companion analyze -o output.xml --exclude \"wip,draft*\"\n\n# Analyze critical tests (including passes)\nresult-companion analyze -o output.xml --include \"critical*\" -i\n```\n\nOr use config file:\n```yaml\ntest_filter:\n  include_tags: [\"smoke\", \"critical*\"]\n  exclude_tags: [\"wip\", \"flaky\"]\n  include_passing: false  # Analyze failures only\n```\n\nSee [tag_filtering_config.yaml](https://github.com/miltroj/result-companion/blob/main/examples/configs/tag_filtering_config.yaml) for details.\n\n## Limitations\n\n- Text-only analysis (no screenshots/videos)\n- Large test suites processed in chunks\n- **Local models**: Need 4-8GB RAM + GPU/NPU for good performance (Apple Silicon, NVIDIA, AMD)\n\n## Contributing\n\nContributions welcome! See [CONTRIBUTING.md](https://github.com/miltroj/result-companion/blob/main/CONTRIBUTING.md) for guidelines.\n\nFor bugs or feature requests, open an issue on GitHub.\n\n## Development Setup\n\n```bash\nmake install                # install with dev dependencies\npoetry run pre-commit install  # one-time: install pre-commit hooks\n\nmake test-unit              # unit tests only\nmake test-integration      # integration tests (e2e skipped automatically)\nmake test-e2e              # e2e only (requires Copilot CLI / Ollama locally)\nmake test-integration-all  # all integration tests including e2e\n```\n\n## License\n\nApache 2.0 - See [LICENSE](https://github.com/miltroj/result-companion/blob/main/LICENSE)\n\n## Disclaimer\n\nCloud AI providers may process your test data. Local models (Ollama) keep everything private on your machine.\n\n**You are responsible for data privacy.** The creator takes no responsibility for data exposure, intellectual property leakage, or security issues. By using Result Companion, you accept all risks and ensure compliance with your organization's data policies.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiltroj%2Fresult-companion","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmiltroj%2Fresult-companion","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiltroj%2Fresult-companion/lists"}