{"id":31698844,"url":"https://github.com/taarobravo/semantic-audio-search","last_synced_at":"2026-05-16T11:31:37.635Z","repository":{"id":318042091,"uuid":"1069802330","full_name":"TaaroBravo/semantic-audio-search","owner":"TaaroBravo","description":"Self-hosted semantic audio search engine powered by CLAP + FastAPI + Qdrant + Gradio. Search and tag sound effects or music by meaning — not just filenames.","archived":false,"fork":false,"pushed_at":"2025-10-04T18:04:38.000Z","size":32,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-04T18:33:46.662Z","etag":null,"topics":["audio-search","clap","embeddings","fastapi","gradio","huggingface","machile-learning","open-source","qdrant","self-hosted","semantic-search","sound-design"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TaaroBravo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-04T16:48:05.000Z","updated_at":"2025-10-04T18:04:41.000Z","dependencies_parsed_at":"2025-10-04T18:33:50.986Z","dependency_job_id":null,"html_url":"https://github.com/TaaroBravo/semantic-audio-search","commit_stats":null,"previous_names":["taarobravo/semantic-audio-search"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/TaaroBravo/semantic-audio-search","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TaaroBravo%2Fsemantic-audio-search","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TaaroBravo%2Fsemantic-audio-search/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TaaroBravo%2Fsemantic-audio-search/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TaaroBravo%2Fsemantic-audio-search/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TaaroBravo","download_url":"https://codeload.github.com/TaaroBravo/semantic-audio-search/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TaaroBravo%2Fsemantic-audio-search/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279000701,"owners_count":26082805,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-08T02:00:06.501Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["audio-search","clap","embeddings","fastapi","gradio","huggingface","machile-learning","open-source","qdrant","self-hosted","semantic-search","sound-design"],"created_at":"2025-10-08T19:11:12.619Z","updated_at":"2025-10-08T19:11:34.722Z","avatar_url":"https://github.com/TaaroBravo.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# Semantic Audio Search\n\n![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)\n![Python 3.11](https://img.shields.io/badge/Python-3.11%2B-green)\n![FastAPI](https://img.shields.io/badge/Backend-FastAPI-009688)\n![Frontend](https://img.shields.io/badge/UI-Gradio-orange)\n\nA self‑hosted semantic audio search stack that indexes your sound library, auto‑tags files, and lets you find and preview audio with natural‑language queries.\n\n**Stack:** FastAPI (Python) · CLAP embeddings · Qdrant vector DB · SQLite · Gradio UI · Docker\n\n---\n\n## What it does\n\n- **Semantic search:** Type things like “applause” or “bad feedback buzzer” and get the right sounds first.\n- **Auto‑metadata:** Lightweight auto‑tagging powered by text–audio similarity. Optionally merges with your own tags/descriptions.\n- **Hybrid ranking:** Combines vector similarity with keyword matches (filename, tags, description).\n- **Library sync:** Reindex or incremental rescan to keep Qdrant and SQLite in sync with your library folder.\n- **Inline preview:** Play audio directly in the UI and copy/download the file URL.\n- **Simple admin:** Edit tags/description, delete tracks, bulk update via API.\n\n---\n\n## Architecture\n\n```\n+-----------+       +------------------+         +-----------------+\n|  Gradio   | \u003c---\u003e | FastAPI (CLAP,   |  upsert |     Qdrant      |\n|   UI      |   API |   auto-tags, DB) | \u003c-----\u003e |  vector search  |\n+-----------+       +------------------+         +-----------------+\n                         |     |\n                         |     +---- SQLite (metadata)\n                         |\n                         +---- Library folder (audio files; served via /media)\n```\n\n- **Embeddings:** `laion/clap-htsat-unfused` for both audio and text.\n- **Auto‑tags:** Matches each audio file against a configurable label list via CLAP text embeddings.\n- **Metadata:** Stored in SQLite (`tracks` table) and mirrored as Qdrant payload for reranking.\n\n---\n\n## Quick start (Docker Compose)\n\n```bash\ngit clone \u003cyour-fork\u003e semantic-audio-search\ncd semantic-audio-search\ndocker compose up -d --build\n```\n\nServices:\n- API at **http://localhost:8000**\n- UI at **http://localhost:7860**\n\n\u003e The first run downloads a CLAP model and may take a few minutes.\n\n---\n\n## Configure your library\n\nBy default the compose file mounts a host folder into the API container at `/app/library`. You have two options:\n\n### A) **Local folder (recommended for first run)**\nEdit `docker-compose.yml`:\n\n```yaml\n  api:\n    volumes:\n      - ./library:/app/library   # put .wav/.mp3 here\n      - ./data:/app/data\n      - ./config:/app/config\n```\n\nCreate the folder and drop a few audio files:\n```bash\nmkdir -p library data config\ncp /path/to/sounds/*.wav library/\n```\n\n### B) **Seafile/WebDAV (optional)**\nIf you already sync your sounds with Seafile or WebDAV (via `rclone`), mount that path instead of `./library`:\n\n```yaml\n  api:\n    volumes:\n      - /mnt/seafile/semantic-audio-search/sounds:/app/library:rw\n      - ./data:/app/data\n      - ./config:/app/config\n```\n\n\u003e If you use the UI **Upload** tab, ensure the library path is a persistent bind mount; otherwise uploaded files will disappear when containers are removed.\n\n---\n\n## Environment variables\n\nThese are set in `docker-compose.yml`. Override as needed.\n\n| Variable | Default | Purpose |\n|---|---|---|\n| `QDRANT_URL` | `http://qdrant:6333` | Qdrant endpoint |\n| `COLLECTION_NAME` | `sfx` | Qdrant collection name |\n| `DB_PATH` | `/app/data/meta.sqlite3` | SQLite path (persist this) |\n| `LIBRARY_DIR` | `/app/library` | Mounted audio library |\n| `CLAP_MODEL` | `laion/clap-htsat-unfused` | Hugging Face model id |\n| `SYNONYMS_PATH` | `/app/config/synonyms.json` | Query synonyms |\n| `SIM_WEIGHT` | `0.5` | Weight for vector similarity |\n| `KW_WEIGHT` | `0.5` | Weight for keyword bonus |\n| `AUTO_TAGS` | `1` | Enable/disable auto-tags |\n| `AUTO_TAGS_OVERWRITE` | `1` | Reindex may overwrite empty fields |\n| `AUTO_TAGS_TOPK` | `5` | Max tags to pick |\n| `CAPTION_MODE` | `basic` | Basic description template |\n| `AUTO_TAGS_LABELS_PATH` | `/app/config/auto_tags_labels.json` | Custom label list (optional) |\n| `AUTO_TAGS_MIN_SIM` | `0.28` | Similarity floor before picking extra labels |\n| `AUTOTAGS_MODE` | `merge` | `merge` \\| `fill_missing` \\| `overwrite` |\n| `API_URL` (UI) | `http://api:8000` | API URL inside the Docker network |\n| `API_URL_PUBLIC` (UI) | _unset_ | Public base URL for clickable links (e.g., `http://localhost:8000`) |\n\n**Custom labels:** create `/app/config/auto_tags_labels.json` with either:\n```json\n[\"applause\",\"ui click\",\"buzzer\",\"whoosh\"]\n```\nor\n```json\n{\"labels\": [\"applause\",\"ui click\",\"buzzer\",\"whoosh\"]}\n```\n\n---\n\n## API reference\n\nBase URL: `http://localhost:8000`\n\n### `GET /healthz`\nHealth probe.\n\n### `GET /search?query=...\u0026limit=20`\nSemantic search with hybrid reranking.\nReturns: list of results `{id, filename, path, score, duration, tags, description, url}`.\n\n### `POST /upload` (multipart form)\nFields: `file` (audio), `tags` (str, optional), `description` (str, optional), `auto` (0/1).  \nBehavior depends on `AUTOTAGS_MODE`: `merge` (default), `fill_missing`, or `overwrite`.\n\n### `POST /reindex`\nFull scan of `LIBRARY_DIR`. Inserts new files and (optionally) overwrites empty metadata.\n\n### `POST /reindex-incremental`\nOnly updates changed/new files; deletes records for missing files.\n\n### `GET /media/{subpath}`\nServes audio files from the library for inline playback.\n\n### `GET /download/{track_id}`\nDownloads a file by track id with the original filename.\n\n### `GET /tracks`\nList all tracks `{id, filename, tags, description}` (paginated via `limit`/`offset`).\n\n### `PATCH /tracks/{track_id}`\nUpdate tags/description for a single track.\n\n### `DELETE /tracks/{track_id}`\nDelete a track (DB + Qdrant + file on disk if present).\n\n### `GET /tracks/export`\nExport all track metadata as JSON.\n\n### `POST /tracks/bulk`\nBulk upsert by `id` or `filename`:\n```json\n{\"items\":[{\"id\":\"...\",\"tags\":\"ui tap\",\"description\":\"short chime\"}]}\n```\n\n### `POST /annotate-missing`\nFill in missing tags/descriptions via auto‑tags.\n\n### `POST /annotate-force`\nRegenerate tags/descriptions for all rows (obeys `AUTO_TAGS_OVERWRITE`).\n\n### `POST /qdrant-gc`\nGarbage‑collect Qdrant points that don’t exist in SQLite.\n\n---\n\n## UI usage\n\n- Open **http://localhost:7860**.\n- **Search** tab:\n  - Enter a query → see results with score, tags, description.\n  - Select a row to **preview** and **download**.\n  - Edit tags/description and **Save**.\n  - **Rescan library** to ingest new files.\n- **Manage Library** tab:\n  - Upload a new file (optionally disable auto‑tagging with the checkbox).\n  - Prefer mounting a persistent library path so uploads survive container restarts.\n\n---\n\n## Performance notes\n\n- Qdrant collection uses HNSW index; you can tune `m` and `ef_construct` at creation and `hnsw_ef` at search time.\n- The API caches text embeddings (`@lru_cache`) to speed up repeated queries.\n- For larger libraries, consider running the API with multiple workers (e.g., `--workers 2`) and giving Qdrant more RAM.\n- The audio embedder truncates to ~10s mono @48kHz to keep inference fast and memory low.\n\n---\n\n## Screenshots\n\n**Home**\n![Home UI](./screenshots/home-ui.PNG)\n\n**Search UI**\n![Search UI](./screenshots/search-ui.PNG)\n\n**Manage Library**\n![Manage Library](./screenshots/manage-library-ui.PNG)\n\n**Upload**\n![Manage Library](./screenshots/upload-ui.PNG)\n\n---\n\n## Roadmap\n\n- Modern React/Next.js frontend\n- Sorting/pagination, dark mode, bulk tagging\n- Optional segment‑level embeddings (long files → playable snippets)\n- Auth + HTTPS for public demos\n\n---\n\n## Credits\n\n- **CLAP**: `laion/clap-htsat-unfused` (Hugging Face)  \n- **Qdrant**: vector database  \n- **Gradio**: quick admin UI  \n- Zenodo pretrained assets downloaded in the API Dockerfile are public resources from their respective authors.\n\n---\n\n## Security \u0026 privacy\n\n- No external calls at query time except model downloads on first run.\n- Keep your library and `data/` on trusted storage; no analytics or telemetry are collected.\n- If you expose the API publicly, put it behind a reverse proxy with HTTPS and auth.\n\n---\n\n## License\n\nMIT. See `LICENSE` for details.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftaarobravo%2Fsemantic-audio-search","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftaarobravo%2Fsemantic-audio-search","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftaarobravo%2Fsemantic-audio-search/lists"}