https://github.com/afedukov/searchtweak
Optimize and Enhance Your Search Quality
https://github.com/afedukov/searchtweak
algolia docker elasticsearch labeling-tool laravel machine-learning-pipeline opensearch rest-api search search-engine-optimization search-quality-evaluation search-relevance solr
Last synced: 3 months ago
JSON representation
Optimize and Enhance Your Search Quality
- Host: GitHub
- URL: https://github.com/afedukov/searchtweak
- Owner: afedukov
- License: other
- Created: 2024-10-10T09:40:18.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-07-16T14:00:22.000Z (11 months ago)
- Last Synced: 2025-09-07T14:53:32.212Z (9 months ago)
- Topics: algolia, docker, elasticsearch, labeling-tool, laravel, machine-learning-pipeline, opensearch, rest-api, search, search-engine-optimization, search-quality-evaluation, search-relevance, solr
- Language: PHP
- Homepage: https://searchtweak.com
- Size: 12.9 MB
- Stars: 13
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
# SearchTweak
[](https://php.net)
[](https://laravel.com)
[](LICENSE.md)
**Self-hosted search relevance evaluation platform with AI Judges.** Assess search quality by running keyword queries against your search APIs, collecting human and AI relevance judgments, and calculating industry-standard IR metrics.
Use it to benchmark search configurations, label training data for ML models, and track search quality over time.
## Key Features
- **AI Judges** — automated relevance judging with OpenAI, Anthropic, Google, DeepSeek, xAI, Groq, Mistral, Ollama, and Custom OpenAI-compatible providers
- **Human + AI Hybrid Flow** — people and AI judges work in one feedback model with `Single` and `Multiple (3)` strategies
- **Judge Operations** — provider/model configuration, per-scale prompts, batch size control, logs, token usage, runtime statuses, and JSONL log export by active filters
- **Search Evaluation** — run keyword queries against any search API, collect relevance judgments, and compute metrics automatically
- **IR Metrics** — Precision, MAP, MRR, CG, DCG, nDCG with support for Binary, Graded, and Detail grading scales
- **Metrics Over Time** — track how search quality changes across evaluations with historical charts
- **Feedback Management** — assign grading tasks to team members, reuse judgments (human and AI) across evaluations, scoring guidelines
- **Team Collaboration** — role-based access (Admin, Evaluator), team invitations, tag-based organization
- **Real-time Updates** — live progress via WebSockets as evaluations run and grades come in
- **REST API** — manage evaluations and models programmatically with token-based authentication
- **Customizable Dashboard** — configurable widgets for metrics, leaderboard, and feedback activity
## How It Works
1. Configure a **Search Endpoint** (URL, method, headers, mapper).
2. Configure a **Search Model** (request template with `#query#`).
3. Create a **Search Evaluation** (keywords, scale, metrics, strategy).
4. Collect feedback from **humans and/or AI judges**.
5. Review metrics and compare quality over time.
More screenshots
### Metrics Dashboard
### Give Feedback
### Search Models
## Tech Stack
| Layer | Technology |
|-------|-----------|
| Backend | Laravel 11 (PHP 8.3) |
| Frontend | Livewire 3, Alpine.js, Tailwind CSS |
| Real-time | Laravel Reverb (WebSockets) |
| Queue | Laravel Horizon |
| Database | MySQL |
| Cache/Queue/Sessions | Redis |
| Infrastructure | Docker Compose, Traefik, Nginx, PHP-FPM |
| Auth | Jetstream + Sanctum + Fortify |
## Requirements
- [Docker](https://docs.docker.com/get-docker/) and Docker Compose
- [Make](https://www.gnu.org/software/make/) (pre-installed on macOS/Linux)
## Getting Started
### 1. Clone the Repository
```bash
git clone https://github.com/afedukov/searchtweak.git
cd searchtweak/devops
```
### 2. Setup Environment
```bash
cp .env.dist .env
```
### 3. Configure Hosts File
Edit your `/etc/hosts` file (or `C:\Windows\System32\drivers\etc\hosts` on Windows):
```
127.0.0.1 searchtweak.local traefik.searchtweak.local db.searchtweak.local
```
### 4. Start the Application
```bash
make
```
This will start all containers and run `make bootstrap` (destructive bootstrap with fresh migrations + seed + assets/docs build).
### 5. Open in Browser
| Service | URL |
|---------|-----|
| App | http://searchtweak.local |
| Docs (VitePress) | http://searchtweak.local/docs/overview |
| Traefik Dashboard | http://traefik.searchtweak.local |
| phpMyAdmin | http://db.searchtweak.local |
| MailHog | http://localhost:8025 |
### Default Admin User
Log in at http://searchtweak.local with:
- **Email:** `admin@searchtweak.com`
- **Password:** `12345678`
## Useful Commands
```bash
cd devops
make start # Start the application
make stop # Stop the application
make bootstrap # Bootstrap (fresh DB reset + seed + assets/docs build)
make bootstrap-up # Bootstrap without DB reset (safe migrate + seed)
make bootstrap-fresh # Explicit destructive bootstrap (same as make bootstrap)
make jobs # Start Horizon (queue worker)
make queue-reload # Reload Horizon workers in the running queue container
make reverb # Start Reverb (WebSocket server)
make vite # Start Vite development server
make vite-prod # Build Vite for production
make docs-install # Install docs dependencies
make docs # Run docs dev server (http://localhost:3001/docs/overview)
make docs-build # Build documentation
make docs-preview # Preview built documentation (http://localhost:3001/docs/overview)
make docs-publish # Publish built docs to public/docs
```
`make bootstrap` is intended for clean local setup. Use `make bootstrap-up` if you need to keep existing local data.
## Running Tests
```bash
cd devops
make test # Run all tests
docker compose run --rm artisan test # Run all tests (alternative)
docker compose run --rm artisan test --testsuite=Unit # Run unit tests only
docker compose run --rm artisan test --testsuite=Feature # Run feature tests only
docker compose run --rm artisan test --filter TestName # Run a specific test
```
## Email Setup
By default, **SearchTweak** uses [MailHog](https://github.com/mailhog/MailHog) to capture outgoing emails in development. Access the MailHog UI at http://localhost:8025.
Configuring real email sending (SMTP / Amazon SES)
Remove the `mailhog` service from `/devops/docker-compose.yml`, then update your `.env` file:
#### SMTP
```dotenv
MAIL_MAILER=smtp
MAIL_HOST=smtp.your-email-provider.com
MAIL_PORT=587
MAIL_USERNAME=your-email@example.com
MAIL_PASSWORD=your-email-password
MAIL_ENCRYPTION=tls
MAIL_FROM_ADDRESS=your-email@example.com
MAIL_FROM_NAME="${APP_NAME}"
```
#### Amazon SES
```dotenv
MAIL_MAILER=ses
AWS_ACCESS_KEY_ID=your-aws-access-key
AWS_SECRET_ACCESS_KEY=your-aws-secret-key
AWS_SES_REGION=us-east-1
MAIL_FROM_ADDRESS=your-email@example.com
MAIL_FROM_NAME="${APP_NAME}"
```
Enforcing email verification
### 1. Update the User Model
In `App\Models\User`, implement the `MustVerifyEmail` interface:
```php
class User extends Authenticatable implements TaggableInterface, MustVerifyEmail
{
// ...
}
```
### 2. Enable Email Verification in Fortify
In `config/fortify.php`, uncomment:
```php
'features' => [
// ...
Features::emailVerification(),
// ...
],
```
## Documentation
- Main docs: http://searchtweak.local/docs/overview
- API reference: http://searchtweak.local/docs/api/overview
- Project meta docs: [DOCUMENTATION.md](DOCUMENTATION.md)
## Contributing
Contributions are welcome! Please fork the repository and submit a pull request with your enhancements or bug fixes.
## License
This project is licensed under the **Functional Source License, Version 1.1** (FSL-1.1-Apache-2.0), with an irrevocable grant to the Apache License, Version 2.0 effective on the second anniversary of the software's release.
Copyright 2024-2026 Andrey Fedyukov
- You may use, modify, and redistribute the software for any purpose, except in products or services that compete with the software or any other product or service we offer.
- After two years, you may alternatively use the software under the Apache License, Version 2.0.
See the full [LICENSE](LICENSE.md) file for details.