{"id":29520993,"url":"https://github.com/code-and-sorts/promptdrifter-action","last_synced_at":"2026-05-19T09:07:05.115Z","repository":{"id":303129364,"uuid":"983157151","full_name":"Code-and-Sorts/PromptDrifter-action","owner":"Code-and-Sorts","description":"PromptDrifter Action to use the 🧭 PromptDrifter – one‑command CI guardrail that catches prompt drift and fails the build when your LLM answers change.","archived":false,"fork":false,"pushed_at":"2025-07-05T20:49:53.000Z","size":18,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-05T22:00:56.032Z","etag":null,"topics":["github-actions","llm","prompt-drift","prompt-tuning","regression-testing","testing-tools"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Code-and-Sorts.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-14T01:08:16.000Z","updated_at":"2025-07-05T20:47:09.000Z","dependencies_parsed_at":"2025-07-05T22:00:59.621Z","dependency_job_id":"4ce934d9-3b91-4508-87f8-7ca562d505aa","html_url":"https://github.com/Code-and-Sorts/PromptDrifter-action","commit_stats":null,"previous_names":["code-and-sorts/promptdrifter-action"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/Code-and-Sorts/PromptDrifter-action","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Code-and-Sorts%2FPromptDrifter-action","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Code-and-Sorts%2FPromptDrifter-action/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Code-and-Sorts%2FPromptDrifter-action/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Code-and-Sorts%2FPromptDrifter-action/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Code-and-Sorts","download_url":"https://codeload.github.com/Code-and-Sorts/PromptDrifter-action/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Code-and-Sorts%2FPromptDrifter-action/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33209493,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-19T07:54:09.561Z","status":"ssl_error","status_checked_at":"2026-05-19T07:54:08.508Z","response_time":58,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["github-actions","llm","prompt-drift","prompt-tuning","regression-testing","testing-tools"],"created_at":"2025-07-16T16:08:42.592Z","updated_at":"2026-05-19T09:07:05.110Z","avatar_url":"https://github.com/Code-and-Sorts.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./docs/img/logo.svg\" alt=\"PromptDrifter Logo\" width=\"100\"/\u003e\n\u003c/p\u003e\n\n# PromptDrifter GitHub Action\n\nDocker-based GitHub Action for running [PromptDrifter](https://github.com/Code-and-Sorts/PromptDrifter) tests in CI/CD workflows to detect prompt drift and LLM response changes.\n\n## Features\n\n- 🔍 **Drift Detection**: Catch when LLM responses change unexpectedly\n- 🚀 **CI Integration**: Fail builds when prompt drift is detected\n- 🔐 **Secure**: API keys handled via GitHub secrets\n- 📋 **Multiple Formats**: Support for various drift detection types\n- ⚡ **Fast**: Built-in caching for faster subsequent runs\n- 🌐 **Multi-Provider**: Support for OpenAI, Claude, Gemini, and more\n\n## Versioning\n\nPromptDrifter Action versions are kept in sync with the PromptDrifter CLI for clarity:\n\n- **Action v0.0.2** → **CLI v0.0.2**\n- **Action v0.0.3** → **CLI v0.0.3**\n- etc.\n\nThis ensures you know exactly which CLI version you're getting and can pin to specific, tested combinations.\n\n### Version Compatibility\n\n| Action Version | CLI Version | Status     |\n|----------------|-------------|------------|\n| `v0.0.2`       | `0.0.2`     | ✅ Current |\n\n## Usage\n\n### Basic Example\n\n```yaml\nname: PromptDrifter Tests\non: [push, pull_request]\n\njobs:\n  prompt-tests:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v4\n      - uses: Code-and-Sorts/PromptDrifter-action@v0.0.2\n        with:\n          files: 'tests/promptdrifter.yaml'\n          openai-api-key: ${{ secrets.OPENAI_API_KEY }}\n```\n\n### Advanced Example\n\n```yaml\nname: Multi-Provider Prompt Tests\non: [push, pull_request]\n\njobs:\n  prompt-tests:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v4\n      - uses: Code-and-Sorts/PromptDrifter-action@v0.0.2\n        with:\n          files: |\n            tests/openai-tests.yaml\n            tests/claude-tests.yaml\n            tests/gemini-tests.yaml\n          openai-api-key: ${{ secrets.OPENAI_API_KEY }}\n          claude-api-key: ${{ secrets.CLAUDE_API_KEY }}\n          gemini-api-key: ${{ secrets.GEMINI_API_KEY }}\n          no-cache: 'false'\n          max-concurrent: '5'\n```\n\n### Initialize Configuration\n\n```yaml\n- name: Initialize PromptDrifter\n  uses: Code-and-Sorts/PromptDrifter-action@v0.0.1\n  with:\n    command: 'init'\n    directory: './prompt-tests'\n```\n\n### Validate Configuration Files\n\n```yaml\n- name: Validate Configurations\n  uses: Code-and-Sorts/PromptDrifter-action@v0.0.1\n  with:\n    command: 'validate'\n    files: 'config/*.yaml'\n```\n\n## Inputs\n\n| Input | Description | Required | Default |\n|-------|-------------|----------|---------|\n| `command` | PromptDrifter command to run (`init`, `run`, `validate`, `test-drift-type`, `migrate`) | No | `run` |\n| `files` | Space-separated list of configuration files | Yes (for most commands) | - |\n| `directory` | Directory for init command | No | `.` |\n| `openai-api-key` | OpenAI API key | No | - |\n| `claude-api-key` | Anthropic Claude API key | No | - |\n| `gemini-api-key` | Google Gemini API key | No | - |\n| `qwen-api-key` | Qwen API key | No | - |\n| `grok-api-key` | Grok API key | No | - |\n| `deepseek-api-key` | DeepSeek API key | No | - |\n| `mistral-api-key` | Mistral API key | No | - |\n| `no-cache` | Disable response caching | No | `false` |\n| `max-concurrent` | Maximum concurrent tests | No | `10` |\n| `config-dir` | Directory containing config files | No | `.` |\n| `cache-db` | Path to cache database | No | - |\n| `drift-type` | Drift type for testing | No | - |\n| `expected` | Expected value for drift testing | No | - |\n| `actual` | Actual value for drift testing | No | - |\n| `migrate-input` | Input file for migration | No | - |\n| `migrate-output` | Output file for migration | No | - |\n\n## Outputs\n\n| Output | Description |\n|--------|-------------|\n| `result` | Result of the PromptDrifter execution |\n\n## Configuration\n\nCreate a `promptdrifter.yaml` file in your repository:\n\n```yaml\nversion: \"0.1\"\nproviders:\n  - name: openai\n    model: gpt-4o-mini\n    api_key: env:OPENAI_API_KEY\n\ntests:\n  - name: \"Basic greeting test\"\n    prompt: \"Say hello\"\n    expected: \"Hello!\"\n    assertion: exact_match\n```\n\nFor detailed configuration options, see the [PromptDrifter documentation](https://github.com/Code-and-Sorts/PromptDrifter).\n\n## Security\n\n- Store API keys in GitHub repository secrets\n- Never commit API keys to your repository\n- API keys are passed as environment variables to Docker container\n\n## Docker Action Structure\n\n```\n├── action.yml          # Action metadata (using: 'docker')\n├── Dockerfile          # Python + PromptDrifter installation\n├── entrypoint.sh       # Shell script that calls promptdrifter CLI\n└── README.md          # This file\n```\n\n## License\n\nMIT License - see [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcode-and-sorts%2Fpromptdrifter-action","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcode-and-sorts%2Fpromptdrifter-action","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcode-and-sorts%2Fpromptdrifter-action/lists"}