{"id":13579845,"url":"https://github.com/rhasspy/rhasspy3","last_synced_at":"2025-04-06T04:11:00.484Z","repository":{"id":102979323,"uuid":"608865901","full_name":"rhasspy/rhasspy3","owner":"rhasspy","description":"An open source voice assistant toolkit for many human languages","archived":false,"fork":false,"pushed_at":"2023-12-26T10:47:32.000Z","size":16098,"stargazers_count":349,"open_issues_count":51,"forks_count":31,"subscribers_count":26,"default_branch":"master","last_synced_at":"2025-03-30T03:03:58.138Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rhasspy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-02T22:32:39.000Z","updated_at":"2025-03-24T08:00:07.000Z","dependencies_parsed_at":null,"dependency_job_id":"6a45751e-f15f-4dd1-9ce1-cbafe49ef925","html_url":"https://github.com/rhasspy/rhasspy3","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rhasspy%2Frhasspy3","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rhasspy%2Frhasspy3/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rhasspy%2Frhasspy3/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rhasspy%2Frhasspy3/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rhasspy","download_url":"https://codeload.github.com/rhasspy/rhasspy3/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247430870,"owners_count":20937874,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T15:01:43.939Z","updated_at":"2025-04-06T04:11:00.463Z","avatar_url":"https://github.com/rhasspy.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"![Rhasspy 3](img/banner.png)\n\n**NOTE: This is a very early developer preview!**\n\nAn open source toolkit for building voice assistants.\n\n![Voice assistant pipeline](img/pipeline.png)\n\nRhasspy focuses on:\n\n* Privacy - no data leaves your computer unless you want it to\n* Broad language support - more than just English\n* Customization - everything can be changed\n\n## Getting Started\n\n* Check out the [tutorial](docs/tutorial.md)\n* Connect Rhasspy to [Home Assistant](docs/home_assistant.md)\n   * Install the [Rhasspy 3 add-on](https://github.com/rhasspy/hassio-addons)\n* Run one or more [satellites](docs/satellite.md)\n* Join [the community](https://community.rhasspy.org/)\n\n\n## Missing Pieces\n\nThis is a developer preview, so there are lots of things missing:\n\n* A user friendly web UI\n* An automated method for installing programs/services and downloading models\n* Support for custom speech to text grammars\n* Intent systems besides Home Assistant\n* The ability to accumulate context within a pipeline\n\n\n## Core Concepts\n\n### Domains\n\nRhasspy is organized by [domain](docs/domains.md):\n\n* mic - audio input\n* wake - wake word detection\n* asr - speech to text\n* vad - voice activity detection\n* intent - intent recognition from text\n* handle - intent or text input handling\n* tts - text to speech\n* snd - audio output\n\n\n### Programs\n\nRhasspy talks to external programs using the [Wyoming protocol](docs/wyoming.md). You can add your own programs by implementing the protocol or using an [adapter](#adapters).\n\n\n### Adapters\n\n[Small scripts](docs/adapters.md) that live in `bin/` and bridge existing programs into the [Wyoming protocol](docs/wyoming.md).\n\nFor example, a speech to text program (`asr`) that accepts a WAV file and outputs text can use `asr_adapter_wav2text.py`\n\n\n### Pipelines\n\nComplete voice loop from microphone input (mic) to speaker output (snd). Stages are:\n\n1. detect (optional)\n    * Wait until wake word is detected in mic\n2. transcribe\n    * Listen until vad detects silence, then convert audio to text\n3. recognize (optional)\n    * Recognize an intent from text\n4. handle\n    * Handle an intent or text, producing a text response\n5. speak\n    * Convert handle output text to speech, and speak through snd\n\n### Servers\n\nSome programs take a while to load, so it's best to leave them running as a server. Use `bin/server_run.py` or add `--server \u003cdomain\u003e \u003cname\u003e` when running the HTTP server.\n\nSee `servers` section of `configuration.yaml` file.\n\n---\n\n\n## Supported Programs\n\n* mic\n    * [arecord](https://alsa-project.org/wiki/Main_Page)\n    * [gstreamer_udp](https://gstreamer.freedesktop.org/)\n    * [sounddevice](https://python-sounddevice.readthedocs.io)\n    * [pyaudio](https://people.csail.mit.edu/hubert/pyaudio/docs/)\n* wake \n    * [porcupine1](https://github.com/Picovoice/porcupine)\n    * [precise-lite](https://github.com/mycroftAI/mycroft-precise)\n    * [snowboy](https://github.com/Kitt-AI/snowboy)\n* vad\n    * [silero](https://github.com/snakers4/silero-vad)\n    * [webrtcvad](https://pypi.org/project/webrtcvad/)\n* asr \n    * [whisper](https://github.com/openai/whisper)\n    * [whisper-cpp](https://github.com/ggerganov/whisper.cpp/)\n    * [faster-whisper](https://github.com/guillaumekln/faster-whisper/)\n    * [vosk](https://alphacephei.com/vosk/)\n    * [coqui-stt](https://stt.readthedocs.io)\n    * [pocketsphinx](https://github.com/cmusphinx/pocketsphinx)\n* handle\n    * [home_assistant_conversation](https://www.home-assistant.io/docs/assist)\n* tts \n    * [piper](https://github.com/rhasspy/piper/)\n    * [mimic3](https://github.com/mycroftAI/mimic3)\n    * [larynx](https://github.com/rhasspy/larynx/)\n    * [coqui-tts](https://tts.readthedocs.io)\n    * [marytts](http://mary.dfki.de/)\n    * [flite](http://www.festvox.org/flite/)\n    * [festival](http://www.cstr.ed.ac.uk/projects/festival/)\n    * [espeak-ng](https://github.com/espeak-ng/espeak-ng/)\n* snd\n    * [aplay](https://alsa-project.org/wiki/Main_Page)\n    * [gstreamer_udp](https://gstreamer.freedesktop.org/)\n    \n    \n---\n\n\n## HTTP API\n\n`http://localhost:13331/\u003cendpoint\u003e`\n\nUnless overridden, the pipeline named \"default\" is used.\n\n* `/pipeline/run`\n    * Runs a full pipeline from mic to snd\n    * Produces JSON\n    * Override `pipeline` or:\n        * `wake_program`\n        * `asr_program`\n        * `intent_program`\n        * `handle_program`\n        * `tts_program`\n        * `snd_program`\n    * Skip stages with `start_after`\n        * `wake` - skip detection, body is detection name (text)\n        * `asr` - skip recording, body is transcript (text) or WAV audio\n        * `intent` - skip recognition, body is intent/not-recognized event (JSON)\n        * `handle` - skip handling, body is handle/not-handled event (JSON)\n        * `tts` - skip synthesis, body is WAV audio\n    * Stop early with `stop_after`\n        * `wake` - only detection\n        * `asr` - detection and transcription\n        * `intent` - detection, transcription, recognition\n        * `handle` - detection, transcription, recognition, handling\n        * `tts` - detection, transcription, recognition, handling, synthesis\n* `/wake/detect`\n    * Detect wake word in WAV input\n    * Produces JSON\n    * Override `wake_program` or `pipeline`\n* `/asr/transcribe`\n    * Transcribe audio from WAV input\n    * Produces JSON\n    * Override `asr_program` or `pipeline`\n* `/intent/recognize`\n    * Recognizes intent from text body (POST) or `text` (GET)\n    * Produces JSON\n    * Override `intent_program` or `pipeline`\n* `/handle/handle`\n    * Handles intent/text from body (POST) or `input` (GET)\n    * `Content-Type` must be `application/json` for intent input\n    * Override `handle_program` or `pipeline`\n* `/tts/synthesize`\n    * Synthesizes audio from text body (POST) or `text` (GET)\n    * Produces WAV audio\n    * Override `tts_program` or `pipeline`\n* `/tts/speak`\n    * Plays audio from text body (POST)  or `text` (GET)\n    * Produces JSON\n    * Override `tts_program`, `snd_program`, or `pipeline`\n* `/snd/play`\n    * Plays WAV audio via snd\n    * Override `snd_program` or `pipeline`\n* `/config`\n    * Returns JSON config\n* `/version`\n    * Returns version info\n\n\n## WebSocket API\n\n`ws://localhost:13331/\u003cendpoint\u003e`\n\nAudio streams are raw PCM in binary messages.\n\nUse the `rate`, `width`, and `channels` parameters for sample rate (hertz), width (bytes), and channel count. By default, input audio is 16Khz 16-bit mono, and output audio is 22Khz 16-bit mono.\n\nThe client can \"end\" the audio stream by sending an empty binary message.\n\n* `/pipeline/asr-tts`\n    * Run pipeline from asr (stream in) to tts (stream out)\n    * Produces JSON messages as events happen\n    * Override `pipeline` or:\n        * `asr_program`\n        * `vad_program`\n        * `handle_program`\n        * `tts_program`\n    * Use `in_rate`, `in_width`, `in_channels` for audio input format\n    * Use `out_rate`, `out_width`, `out_channels` for audio output format\n* `/wake/detect`\n    * Detect wake word from websocket audio stream\n    * Produces a JSON message when audio stream ends\n    * Override `wake_program` or `pipeline`\n* `/asr/transcribe`\n    * Transcribe a websocket audio stream\n    * Produces a JSON message when audio stream ends\n    * Override `asr_program` or `pipeline`\n* `/snd/play`\n    * Play a websocket audio stream\n    * Produces a JSON message when audio stream ends\n    * Override `snd_program` or `pipeline`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frhasspy%2Frhasspy3","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frhasspy%2Frhasspy3","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frhasspy%2Frhasspy3/lists"}