{"id":18073323,"url":"https://github.com/afedukov/searchtweak","last_synced_at":"2026-03-07T19:01:37.596Z","repository":{"id":259369906,"uuid":"870575809","full_name":"afedukov/searchtweak","owner":"afedukov","description":"Optimize and Enhance Your Search Quality","archived":false,"fork":false,"pushed_at":"2025-07-16T14:00:22.000Z","size":13478,"stargazers_count":13,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-07T14:53:32.212Z","etag":null,"topics":["algolia","docker","elasticsearch","labeling-tool","laravel","machine-learning-pipeline","opensearch","rest-api","search","search-engine-optimization","search-quality-evaluation","search-relevance","solr"],"latest_commit_sha":null,"homepage":"https://searchtweak.com","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/afedukov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-10T09:40:18.000Z","updated_at":"2025-07-16T14:00:27.000Z","dependencies_parsed_at":"2024-10-24T22:42:36.285Z","dependency_job_id":"57b00aba-1503-4f32-8a3e-1f8266d6c717","html_url":"https://github.com/afedukov/searchtweak","commit_stats":null,"previous_names":["afedukov/searchtweak"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/afedukov/searchtweak","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/afedukov%2Fsearchtweak","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/afedukov%2Fsearchtweak/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/afedukov%2Fsearchtweak/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/afedukov%2Fsearchtweak/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/afedukov","download_url":"https://codeload.github.com/afedukov/searchtweak/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/afedukov%2Fsearchtweak/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279016551,"owners_count":26085850,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-13T02:00:06.723Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["algolia","docker","elasticsearch","labeling-tool","laravel","machine-learning-pipeline","opensearch","rest-api","search","search-engine-optimization","search-quality-evaluation","search-relevance","solr"],"created_at":"2024-10-31T10:06:07.708Z","updated_at":"2026-03-07T19:01:37.583Z","avatar_url":"https://github.com/afedukov.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SearchTweak\n\n[![PHP 8.3+](https://img.shields.io/badge/PHP-8.3+-777BB4?logo=php\u0026logoColor=white)](https://php.net)\n[![Laravel 11](https://img.shields.io/badge/Laravel-11-FF2D20?logo=laravel\u0026logoColor=white)](https://laravel.com)\n[![License: FSL-1.1-Apache-2.0](https://img.shields.io/badge/License-FSL--1.1--Apache--2.0-blue)](LICENSE.md)\n\n**Self-hosted search relevance evaluation platform with AI Judges.** Assess search quality by running keyword queries against your search APIs, collecting human and AI relevance judgments, and calculating industry-standard IR metrics.\n\nUse it to benchmark search configurations, label training data for ML models, and track search quality over time.\n\n![Evaluation Detail](public/images/features/evaluations/evaluations1.png)\n\n## Key Features\n\n- **AI Judges** — automated relevance judging with OpenAI, Anthropic, Google, DeepSeek, xAI, Groq, Mistral, Ollama, and Custom OpenAI-compatible providers\n- **Human + AI Hybrid Flow** — people and AI judges work in one feedback model with `Single` and `Multiple (3)` strategies\n- **Judge Operations** — provider/model configuration, per-scale prompts, batch size control, logs, token usage, runtime statuses, and JSONL log export by active filters\n- **Search Evaluation** — run keyword queries against any search API, collect relevance judgments, and compute metrics automatically\n- **IR Metrics** — Precision, MAP, MRR, CG, DCG, nDCG with support for Binary, Graded, and Detail grading scales\n- **Metrics Over Time** — track how search quality changes across evaluations with historical charts\n- **Feedback Management** — assign grading tasks to team members, reuse judgments (human and AI) across evaluations, scoring guidelines\n- **Team Collaboration** — role-based access (Admin, Evaluator), team invitations, tag-based organization\n- **Real-time Updates** — live progress via WebSockets as evaluations run and grades come in\n- **REST API** — manage evaluations and models programmatically with token-based authentication\n- **Customizable Dashboard** — configurable widgets for metrics, leaderboard, and feedback activity\n\n## How It Works\n\n1. Configure a **Search Endpoint** (URL, method, headers, mapper).\n2. Configure a **Search Model** (request template with `#query#`).\n3. Create a **Search Evaluation** (keywords, scale, metrics, strategy).\n4. Collect feedback from **humans and/or AI judges**.\n5. Review metrics and compare quality over time.\n\n\u003cdetails\u003e\n\u003csummary\u003eMore screenshots\u003c/summary\u003e\n\n### Metrics Dashboard\n![Metrics Widget](public/images/features/misc/widget1.png)\n\n### Give Feedback\n![Give Feedback](public/images/features/misc/give_feedback1.png)\n\n### Search Models\n![Search Models](public/images/features/models/models1.png)\n\n\u003c/details\u003e\n\n## Tech Stack\n\n| Layer | Technology |\n|-------|-----------|\n| Backend | Laravel 11 (PHP 8.3) |\n| Frontend | Livewire 3, Alpine.js, Tailwind CSS |\n| Real-time | Laravel Reverb (WebSockets) |\n| Queue | Laravel Horizon |\n| Database | MySQL |\n| Cache/Queue/Sessions | Redis |\n| Infrastructure | Docker Compose, Traefik, Nginx, PHP-FPM |\n| Auth | Jetstream + Sanctum + Fortify |\n\n## Requirements\n\n- [Docker](https://docs.docker.com/get-docker/) and Docker Compose\n- [Make](https://www.gnu.org/software/make/) (pre-installed on macOS/Linux)\n\n## Getting Started\n\n### 1. Clone the Repository\n\n```bash\ngit clone https://github.com/afedukov/searchtweak.git\ncd searchtweak/devops\n```\n\n### 2. Setup Environment\n\n```bash\ncp .env.dist .env\n```\n\n### 3. Configure Hosts File\n\nEdit your `/etc/hosts` file (or `C:\\Windows\\System32\\drivers\\etc\\hosts` on Windows):\n```\n127.0.0.1    searchtweak.local traefik.searchtweak.local db.searchtweak.local\n```\n\n### 4. Start the Application\n\n```bash\nmake\n```\n\nThis will start all containers and run `make bootstrap` (destructive bootstrap with fresh migrations + seed + assets/docs build).\n\n### 5. Open in Browser\n\n| Service | URL |\n|---------|-----|\n| App | http://searchtweak.local |\n| Docs (VitePress) | http://searchtweak.local/docs/overview |\n| Traefik Dashboard | http://traefik.searchtweak.local |\n| phpMyAdmin | http://db.searchtweak.local |\n| MailHog | http://localhost:8025 |\n\n### Default Admin User\n\nLog in at http://searchtweak.local with:\n\n- **Email:** `admin@searchtweak.com`\n- **Password:** `12345678`\n\n## Useful Commands\n\n```bash\ncd devops\nmake start        # Start the application\nmake stop         # Stop the application\nmake bootstrap    # Bootstrap (fresh DB reset + seed + assets/docs build)\nmake bootstrap-up # Bootstrap without DB reset (safe migrate + seed)\nmake bootstrap-fresh # Explicit destructive bootstrap (same as make bootstrap)\nmake jobs         # Start Horizon (queue worker)\nmake queue-reload # Reload Horizon workers in the running queue container\nmake reverb       # Start Reverb (WebSocket server)\nmake vite         # Start Vite development server\nmake vite-prod    # Build Vite for production\nmake docs-install # Install docs dependencies\nmake docs         # Run docs dev server (http://localhost:3001/docs/overview)\nmake docs-build   # Build documentation\nmake docs-preview # Preview built documentation (http://localhost:3001/docs/overview)\nmake docs-publish # Publish built docs to public/docs\n```\n\n`make bootstrap` is intended for clean local setup. Use `make bootstrap-up` if you need to keep existing local data.\n\n## Running Tests\n\n```bash\ncd devops\nmake test                                                   # Run all tests\ndocker compose run --rm artisan test                        # Run all tests (alternative)\ndocker compose run --rm artisan test --testsuite=Unit       # Run unit tests only\ndocker compose run --rm artisan test --testsuite=Feature    # Run feature tests only\ndocker compose run --rm artisan test --filter TestName      # Run a specific test\n```\n\n## Email Setup\n\nBy default, **SearchTweak** uses [MailHog](https://github.com/mailhog/MailHog) to capture outgoing emails in development. Access the MailHog UI at http://localhost:8025.\n\n\u003cdetails\u003e\n\u003csummary\u003eConfiguring real email sending (SMTP / Amazon SES)\u003c/summary\u003e\n\nRemove the `mailhog` service from `/devops/docker-compose.yml`, then update your `.env` file:\n\n#### SMTP\n```dotenv\nMAIL_MAILER=smtp\nMAIL_HOST=smtp.your-email-provider.com\nMAIL_PORT=587\nMAIL_USERNAME=your-email@example.com\nMAIL_PASSWORD=your-email-password\nMAIL_ENCRYPTION=tls\nMAIL_FROM_ADDRESS=your-email@example.com\nMAIL_FROM_NAME=\"${APP_NAME}\"\n```\n\n#### Amazon SES\n```dotenv\nMAIL_MAILER=ses\nAWS_ACCESS_KEY_ID=your-aws-access-key\nAWS_SECRET_ACCESS_KEY=your-aws-secret-key\nAWS_SES_REGION=us-east-1\nMAIL_FROM_ADDRESS=your-email@example.com\nMAIL_FROM_NAME=\"${APP_NAME}\"\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eEnforcing email verification\u003c/summary\u003e\n\n### 1. Update the User Model\nIn `App\\Models\\User`, implement the `MustVerifyEmail` interface:\n```php\nclass User extends Authenticatable implements TaggableInterface, MustVerifyEmail\n{\n    // ...\n}\n```\n\n### 2. Enable Email Verification in Fortify\nIn `config/fortify.php`, uncomment:\n```php\n'features' =\u003e [\n    // ...\n    Features::emailVerification(),\n    // ...\n],\n```\n\n\u003c/details\u003e\n\n## Documentation\n\n- Main docs: http://searchtweak.local/docs/overview\n- API reference: http://searchtweak.local/docs/api/overview\n- Project meta docs: [DOCUMENTATION.md](DOCUMENTATION.md)\n\n## Contributing\n\nContributions are welcome! Please fork the repository and submit a pull request with your enhancements or bug fixes.\n\n## License\n\nThis project is licensed under the **Functional Source License, Version 1.1** (FSL-1.1-Apache-2.0), with an irrevocable grant to the Apache License, Version 2.0 effective on the second anniversary of the software's release.\n\nCopyright 2024-2026 Andrey Fedyukov\n\n- You may use, modify, and redistribute the software for any purpose, except in products or services that compete with the software or any other product or service we offer.\n- After two years, you may alternatively use the software under the Apache License, Version 2.0.\n\nSee the full [LICENSE](LICENSE.md) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fafedukov%2Fsearchtweak","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fafedukov%2Fsearchtweak","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fafedukov%2Fsearchtweak/lists"}