{"id":47425331,"url":"https://github.com/rzru/nightingale","last_synced_at":"2026-05-10T19:02:34.286Z","repository":{"id":344570418,"uuid":"1182454913","full_name":"rzru/nightingale","owner":"rzru","description":"Machine learning powered Karaoke app (with scores!) ","archived":false,"fork":false,"pushed_at":"2026-05-10T12:02:10.000Z","size":166111,"stargazers_count":1102,"open_issues_count":13,"forks_count":72,"subscribers_count":11,"default_branch":"master","last_synced_at":"2026-05-10T12:33:03.159Z","etag":null,"topics":["ai","bevy-engine","demucs","karaoke","karaoke-application","karaoke-game","machine-learning","machine-learning-algorithms","ml","party","react","rust","shadcn","tailwind","tauri","tauri-app","whisper","whisper-ai","whisperx"],"latest_commit_sha":null,"homepage":"https://nightingale.cafe","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rzru.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-15T14:48:09.000Z","updated_at":"2026-05-10T12:02:14.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/rzru/nightingale","commit_stats":null,"previous_names":["rzru/nightingale"],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/rzru/nightingale","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rzru%2Fnightingale","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rzru%2Fnightingale/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rzru%2Fnightingale/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rzru%2Fnightingale/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rzru","download_url":"https://codeload.github.com/rzru/nightingale/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rzru%2Fnightingale/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32867988,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-10T13:40:02.631Z","status":"ssl_error","status_checked_at":"2026-05-10T13:40:02.145Z","response_time":54,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","bevy-engine","demucs","karaoke","karaoke-application","karaoke-game","machine-learning","machine-learning-algorithms","ml","party","react","rust","shadcn","tailwind","tauri","tauri-app","whisper","whisper-ai","whisperx"],"created_at":"2026-03-21T23:00:37.276Z","updated_at":"2026-05-10T19:02:34.278Z","avatar_url":"https://github.com/rzru.png","language":"TypeScript","funding_links":[],"categories":["TypeScript"],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"client/src/assets/images/logo.png\" alt=\"Nightingale\" width=\"400\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  Karaoke from any song in your music library, powered by neural networks.\n\u003c/p\u003e\n\n---\n\nNightingale scans your music folder, separates lead vocals from instrumentals using the [UVR Karaoke model](https://github.com/Anjok07/ultimatevocalremovergui) (or [Demucs](https://github.com/facebookresearch/demucs)), transcribes lyrics with word-level timestamps via [WhisperX](https://github.com/m-bain/whisperX), and plays it all back with synchronized highlighting, pitch scoring, key/tempo controls, profiles, and dynamic backgrounds.\n\nShips as a single binary. No manual installation of Python, ffmpeg, or ML models required — everything is downloaded and bootstrapped automatically on first launch.\n\n## Features\n\n🎤 **Stem Separation** — isolates lead vocals from instrumentals using the UVR Karaoke model (default) or Demucs, with adjustable guide vocal volume. The karaoke model preserves backing vocals in the instrumental for a more natural sound\n\n📝 **Word-Level Lyrics** — automatic transcription with alignment, or fetched from [LRCLIB](https://lrclib.net) when available. CJK songs (Japanese / Chinese / Korean) get per-character forced alignment and romanized readings (Hepburn / pinyin / Revised Romanization) shown above each token\n\n🗣️ **Pluggable ASR Engines** — choose Whisper (default, broad language coverage) or **Parakeet v3 (experimental)** for ~25 European languages, with NeMo on CUDA and ONNX Runtime everywhere else\n\n🎼 **UltraStar Deluxe Songs (experimental)** — drop USDX song folders (`.txt` or `.usdx` plus sibling audio/vocals/instrumental/video) into your library; pitch and lyric data come from the file directly, no analyzer pass needed. See [docs/usdx](site/docs/src/usdx.md)\n\n🎯 **Pitch Scoring** — real-time microphone input with pitch detection, star ratings, and per-song scoreboards\n\n🎚️ **Key \u0026 Tempo Shifts** — adjust song key and tempo after analysis, with cached playback variants for quick retries\n\n👤 **Profiles** — create and switch between player profiles; scores are tracked per profile\n\n🎬 **Video Files** — drop video files (`.mp4`, `.mkv`, etc.) into your music folder; vocals are separated from the audio track and the original video plays as a synchronized background\n\n🌌 **Audio-Reactive Backgrounds** — 10 GPU shaders that react to your microphone in real time (Plasma, Waves, Nebula, Starfield, Sonar, Voronoi, Vortex, Metaballs, Spectrum, Oscilloscope), Pixabay video loops in 5 flavors (Nature, Underwater, Space, City, Countryside), plus source-video playback for video files\n\n🧭 **Sidebar + Library Filters** — quick filters, metadata cleanup buckets, artist/album groups, and an **Analyze All** action for bulk analysis\n\n🎙️ **Mic Mirroring** — optionally route your live mic into playback for low-latency practice and monitoring, with an adjustable monitor gain (0–200%) in Settings\n\n🎮 **Gamepad Support** — full navigation and control via gamepad (D-pad, sticks, face buttons)\n\n📺 **Adaptive UI Scaling** — scales to any resolution including 4K TVs\n\n📦 **Self-Contained** — ffmpeg, uv, Python, PyTorch, and ML packages are downloaded automatically during setup. Video backgrounds are pre-downloaded so the first session is ready to go\n\n⬆️ **In-App Updates** — auto-checks for new releases at launch, badges the sidebar avatar when one is available, and downloads and installs signed updates with one click on Linux, macOS, and Windows\n\n## Quick start\n\nDownload the latest release for your platform from the [Releases](../../releases) page and run it. On first launch, Nightingale shows setup steps, lets you pick a data folder, then installs the Python environment and ML models automatically.\n\n## Updates\n\nNightingale checks for new releases once at launch. When one is available, the sidebar avatar grows a small green dot and the **Update** entry in the dropdown menu opens a dialog with the release notes. Click **Install \u0026 Restart** and the app downloads the signed bundle, installs it, and relaunches.\n\nThe in-app updater works for Linux, macOS, and Windows. On Windows the installer runs in `passive` mode — a small progress window flashes and the app comes back automatically once the install finishes.\n\n### macOS\n\nmacOS quarantines files downloaded from the internet. Since Nightingale isn't signed with an Apple Developer ID, Gatekeeper will block it with a message like _\"app is damaged and can't be opened\"_. To fix this, remove the quarantine attribute after moving the Nightingale.app to Applications:\n\n```bash\nxattr -cr /Applications/Nightingale.app\n```\n\n### Supported formats\n\nAudio: `.mp3`, `.flac`, `.ogg`, `.wav`, `.m4a`, `.aac`, `.wma`. Video: `.mp4`, `.mkv`, `.avi`, `.webm`, `.mov`, `.m4v`. UltraStar: `.usdx`, plus `.txt` files whose contents look like USDX.\n\n## Controls\n\n### Navigation\n\n| Action           | Keyboard       | Gamepad            |\n| ---------------- | -------------- | ------------------ |\n| Move             | Arrow keys     | D-pad / Left stick |\n| Confirm / Select | Enter          | A (South)          |\n| Back / Cancel    | Escape         | B (East) / Start   |\n| Switch panel     | Tab            | —                  |\n| Search songs     | Type to filter | —                  |\n\n### Playback\n\n| Action                  | Keyboard          | Gamepad   |\n| ----------------------- | ----------------- | --------- |\n| Pause / Resume          | Space             | Start     |\n| Exit to menu            | Escape            | B (East)  |\n| Toggle guide vocals     | G                 | —         |\n| Guide volume up/down    | + / -             | —         |\n| Cycle background theme  | T                 | —         |\n| Cycle video flavor      | F                 | —         |\n| Toggle microphone       | M                 | —         |\n| Next microphone         | N                 | —         |\n| Toggle mic mirroring    | R                 | —         |\n| Toggle fullscreen       | F11               | —         |\n| Skip Intro / Skip Outro | On-screen buttons | A (South) |\n\n## How it works\n\n```mermaid\nflowchart TD\n    A[\"Audio or video file\"] --\u003e B[\"UVR Karaoke / Demucs\"]\n    A2[\"USDX bundle (.txt / .usdx)\"] --\u003e E[\"Tauri App (Rust + React)\"]\n    B --\u003e|\"vocals + instrumental\"| C[\"LRCLIB\"]\n    C --\u003e|\"synced lyrics if available\"| D[\"WhisperX or Parakeet v3 (exp.)\"]\n    D --\u003e|\"word-level alignment, CJK reading\"| E\n    E --\u003e F[\"Plays instrumental + synced lyrics with pitch scoring, key/tempo, mic mirroring, audio-reactive backgrounds\"]\n```\n\nThe analyzer runs as a persistent local process: Nightingale starts it once and talks to it over a token-authenticated loopback TCP socket using newline-delimited JSON, so per-song startup overhead (model load, CUDA init) is paid only once.\n\nAnalysis results are cached using blake3 file hashes. Re-analysis only happens if the source file changes, the user triggers it manually, or you choose to shift key/tempo and create playback variants. USDX songs skip stem separation entirely when `#VOCALS` and `#INSTRUMENTAL` are provided.\n\n## Hardware\n\nThe Python analyzer uses PyTorch and auto-detects the best backend:\n\n| Backend | Device        | Notes                                       |\n| ------- | ------------- | ------------------------------------------- |\n| CUDA    | NVIDIA GPU    | Fastest                                     |\n| MPS     | Apple Silicon | macOS; WhisperX alignment falls back to CPU |\n| CPU     | Any           | Slowest but always works                    |\n\nThe UVR Karaoke model uses ONNX Runtime and enables CUDA acceleration automatically on NVIDIA GPUs, or CoreML on Apple Silicon.\n\nA song typically takes 2–5 minutes on GPU, 10–20 minutes on CPU.\n\n## Data storage\n\nDuring setup, you can choose where Nightingale stores data (default: `~/.nightingale`). Most runtime data is stored in that selected data folder, while `config.json` and `nightingale.log` remain in `~/.nightingale`.\n\nTypical selected data folder layout:\n\n```\n\u003cselected-data-folder\u003e/\n├── cache/               # Stems, transcripts, lyrics, shifted variants, covers, playable videos\n├── songs.db             # SQLite song library and analysis metadata\n├── profiles.json        # Player profiles and scores\n├── videos/              # Cached Pixabay video backgrounds\n├── sounds/              # Sound effects (celebration)\n├── vendor/\n│   ├── ffmpeg           # Downloaded ffmpeg binary\n│   ├── uv               # Downloaded uv binary\n│   ├── python/          # Python 3.10 installed via uv\n│   ├── venv/            # Virtual environment with ML packages\n│   ├── analyzer/        # Extracted analyzer Python scripts\n│   └── .ready           # Marker indicating setup is complete\n└── models/\n    ├── torch/           # Demucs model cache\n    ├── huggingface/     # WhisperX model cache\n    └── audio_separator/ # UVR Karaoke model cache\n```\n\n`~/.nightingale/config.json` stores app settings, including the selected data folder path.\n\n### Video backgrounds\n\nPixabay video backgrounds use the [Pixabay API](https://pixabay.com/api/docs/). The API key is embedded in release builds. For development, create a `.env` file at the project root:\n\n```\nPIXABAY_API_KEY=your_key_here\n```\n\n## Building from source\n\n### Prerequisites\n\n| Tool       | Version                                                                                                               |\n| ---------- | --------------------------------------------------------------------------------------------------------------------- |\n| Rust       | 1.85+ (edition 2024)                                                                                                  |\n| Node.js    | 20+                                                                                                                   |\n| pnpm       | latest                                                                                                                |\n| Linux only | `libwebkit2gtk-4.1-dev`, `libssl-dev`, `libayatana-appindicator3-dev`, `librsvg2-dev`, `libxdo-dev`, `libasound2-dev` |\n\n### Development\n\n```bash\ngit clone \u003crepo-url\u003e nightingale\ncd nightingale\ncargo desktop dev\n```\n\n### Release build\n\n```bash\ncargo desktop build\n```\n\n## Supported platforms\n\n| Platform       | Target                      |\n| -------------- | --------------------------- |\n| Linux x86_64   | `x86_64-unknown-linux-gnu`  |\n| Linux aarch64  | `aarch64-unknown-linux-gnu` |\n| macOS ARM      | `aarch64-apple-darwin`      |\n| macOS Intel    | `x86_64-apple-darwin`       |\n| Windows x86_64 | `x86_64-pc-windows-msvc`    |\n\n## License\n\nGPL-3.0-or-later — see [LICENSE](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frzru%2Fnightingale","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frzru%2Fnightingale","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frzru%2Fnightingale/lists"}