{"id":29029747,"url":"https://github.com/jeanjerome/moshi-stt-apple-installer","last_synced_at":"2025-07-29T01:33:37.870Z","repository":{"id":301057983,"uuid":"1008021311","full_name":"jeanjerome/moshi-stt-apple-installer","owner":"jeanjerome","description":"An automated installation script for deploying Kyutai's Moshi STT server on macOS Apple Silicon.","archived":false,"fork":false,"pushed_at":"2025-07-04T17:13:00.000Z","size":48,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-04T18:30:35.445Z","etag":null,"topics":["installation","kyutai","moshi","stt"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jeanjerome.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-24T22:47:12.000Z","updated_at":"2025-07-04T17:13:03.000Z","dependencies_parsed_at":"2025-06-25T00:37:42.800Z","dependency_job_id":null,"html_url":"https://github.com/jeanjerome/moshi-stt-apple-installer","commit_stats":null,"previous_names":["jeanjerome/moshi-stt-apple-installer"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jeanjerome/moshi-stt-apple-installer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeanjerome%2Fmoshi-stt-apple-installer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeanjerome%2Fmoshi-stt-apple-installer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeanjerome%2Fmoshi-stt-apple-installer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeanjerome%2Fmoshi-stt-apple-installer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jeanjerome","download_url":"https://codeload.github.com/jeanjerome/moshi-stt-apple-installer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeanjerome%2Fmoshi-stt-apple-installer/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267616577,"owners_count":24116154,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-28T02:00:09.689Z","response_time":68,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["installation","kyutai","moshi","stt"],"created_at":"2025-06-26T08:36:44.576Z","updated_at":"2025-07-29T01:33:37.864Z","avatar_url":"https://github.com/jeanjerome.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Moshi STT Server - Installation Script\n\nAn automated installation script for deploying Kyutai's Moshi STT server on macOS Apple Silicon. This project complements the official [Kyutai Delayed Streams Modeling](https://github.com/kyutai-labs/delayed-streams-modeling) repository by providing a streamlined setup process.\n\n## Overview\n\nThis installation script provides an automated setup for running Kyutai's high-performance STT (Speech-to-Text) server using:\n- **Kyutai's STT model** (`kyutai/stt-1b-en_fr-candle`) with 1B parameters supporting English and French\n- **Rust-based server** with Metal acceleration optimized for Apple Silicon\n- **Streaming inference** for real-time transcription with word-level timestamps\n- **WebSocket API** with `/api/asr-streaming` endpoint\n- **Semantic Voice Activity Detection (VAD)** capabilities\n\n\u003e **Note**: This is a deployment script based on the official [Kyutai Labs Delayed Streams Modeling](https://github.com/kyutai-labs/delayed-streams-modeling) project. For the complete research project, model details, and alternative installation methods, please visit the original repository.\n\n## Prerequisites\n\n### System Requirements\n- **macOS** with Apple Silicon (M series) - Metal acceleration support\n- **Homebrew** package manager ([install here](https://brew.sh/))\n- **Xcode Command Line Tools** (for `clang` compiler)\n- **~2GB disk space** for model weights download\n\n### Install Dependencies\n```bash\n# Install required system packages\nbrew install rust cmake python@3.12\n```\n\n**Dependencies breakdown:**\n- **Rust**: Compiles the moshi-server binary with Metal acceleration\n- **CMake**: Build system for native dependencies\n- **Python 3.12**: Required for PyO3 bindings and tokenization components\n\n## Quick Start\n\n### 1. Clone and Setup\n```bash\ngit clone https://github.com/jeanjerome/moshi-stt-apple-installer.git\n```\n\n### 2. Run the Server\n```bash\ncd moshi-stt-apple-silicon\n./scripts/install.sh\n```\n\nThe build script will:\n1. Install missing system dependencies\n2. Compile `moshi-server` with Metal support (binary installed to `~/.cargo/bin/moshi-server`)\n3. Download the STT configuration for English/French (if not already present)\n4. Start the server on port 8080\n\n### 3. Verify Installation (optional)\n\nFirst, install `uv` (Python package manager, independent of Moshi server):\n```bash\ncurl -LsSf https://astral.sh/uv/install.sh | sh\n```\n\nThen test the Moshi server with the WebSocket client:\n```bash\n# Test Moshi server using WebSocket client\nuv run test_client.py\n```\n\nExpected output:\n```\nInstalled 4 packages in 7ms\nreceived: {'type': 'Ready'}\nreceived: {'type': 'Word', 'text': 'Bonjour,', 'start_time': 1.68}\nreceived: {'type': 'Marker', 'id': 0}\nReceived marker, stopping stream.\nexiting\nTranscription: Bonjour,\n```\n\n## Usage\n\n### Starting the Server\n\nUse the provided startup script (recommended):\n```bash\n./scripts/start.sh\n```\n\nThis script initializes the required environment variables for macOS:\n- **Python runtime**: `PYTHON_SYS_EXECUTABLE`, `DYLD_LIBRARY_PATH`\n- **Library paths**: Required for PyO3 bindings and native dependencies\n- **Homebrew paths**: Ensures system libraries are found correctly\n\nOr start manually (requires setting environment variables):\n```bash\n~/.cargo/bin/moshi-server worker --config moshi-stt.toml --port 8080\n```\n\n### WebSocket Audio Transcription\n\nThe Moshi server uses WebSocket for real-time audio streaming transcription:\n\n```\nws://localhost:8080/api/asr-streaming\nHeader: kyutai-api-key: public_token\nProtocol: MessagePack binary format\n```\n\n### Audio Format Requirements\n- **Format**: WAV, 16-bit PCM\n- **Sample Rate**: 24kHz (resampled automatically)\n- **Channels**: Mono\n- **Duration**: Up to 10 minutes recommended for optimal performance\n\n### Response Format\n\nThe server sends MessagePack-encoded responses with these message types:\n\n```python\n# Connection ready\n{\"type\": \"Ready\"}\n\n# Processing step (can be ignored)\n{\"type\": \"Step\"}\n\n# Word detected with timestamp\n{\"type\": \"Word\", \"text\": \"Bonjour,\", \"start_time\": 1.68}\n\n# Word end timestamp\n{\"type\": \"EndWord\", \"stop_time\": 2.1}\n\n# Stream end marker\n{\"type\": \"Marker\", \"id\": 0}\n```\n\n### Client Example\n\nUse the provided WebSocket client:\n```bash\nuv run test_client.py\n```\n\nOr implement your own using the same MessagePack protocol for sending audio chunks and receiving transcription events.\n\nFor additional client implementations (Python, MLX), see [Kyutai's streaming client examples](https://github.com/kyutai-labs/delayed-streams-modeling/tree/main/scripts).\n\n## Configuration\n\n### Command Line Options\n\n```bash\n~/.cargo/bin/moshi-server worker [OPTIONS] --config \u003cCONFIG\u003e\n```\n\n| Option | Default | Description |\n|--------|---------|-------------|\n| `--config \u003cCONFIG\u003e` | Required | Path to TOML configuration file |\n| `--port \u003cPORT\u003e` | 8080 | Server port |\n| `--addr \u003cADDR\u003e` | 0.0.0.0 | Server address |\n| `--log \u003cLOG_LEVEL\u003e` | info | Log level (error, warn, info, debug, trace) |\n| `--cpu` | - | Force CPU mode (disable Metal acceleration) |\n| `--silent` | - | Suppress output |\n\n### Configuration File (`moshi-stt.toml`)\n\n| Setting | Value | Description |\n|---------|-------|-------------|\n| **API Path** | `/api/asr-streaming` | WebSocket endpoint |\n| **Model** | `kyutai/stt-1b-en_fr-candle` | Bilingual STT model |\n| **Languages** | EN, FR | Supported languages |\n| **Delay (tokens)** | 6 | ASR delay in tokens |\n| **Batch Size** | 64 | Processing batch size |\n| **Temperature** | 0.0 | Deterministic output |\n| **Auth Token** | `public_token` | WebSocket authentication |\n\n### Examples\n\n```bash\n# Custom port\n~/.cargo/bin/moshi-server worker --config moshi-stt.toml --port 8081\n\n# Debug logging\n~/.cargo/bin/moshi-server worker --config moshi-stt.toml --log debug\n\n# CPU-only mode\n~/.cargo/bin/moshi-server worker --config moshi-stt.toml --cpu\n```\n\n## Server Management\n\n### Manual Start\n```bash\n# If already installed\n./scripts/start.sh\n```\n\n### Stop Server\n```bash\n# Ctrl+C in terminal, or:\npkill -f moshi-server\n```\n\n### Check Resource\n```bash\n# Check resource usage\ntop -pid $(pgrep moshi-server)\n```\n\n### Restart\n```bash\npkill -f moshi-server \u0026\u0026 ./scripts/start.sh\n```\n\n## Troubleshooting\n\n### Common Issues\n\n#### Normal WebSocket Closing Error\n```\nERROR: WebSocket protocol error: Sending after closing is not allowed\n```\nThis error is normal when the client closes the connection. The server continues processing briefly after the client disconnects.\n\n#### Port Already in Use\n```bash\n# Find process using port 8080\nlsof -i :8080\n\n# Use different port\n~/.cargo/bin/moshi-server worker --config moshi-stt.toml --port 8081\n```\n\n#### Rust Compilation Error\n```bash\n# Clean and rebuild\ncargo clean\nrustup update\n./scripts/install.sh\n```\n\n#### Python/PyTorch Issues\n```bash\n# Check Python version\n/opt/homebrew/bin/python3 --version\n\n# Reinstall if needed\nbrew reinstall python@3.12\n```\n\n#### Models Not Downloaded\nThe models download automatically on first run. If this fails:\n- Check your internet connection\n- Ensure sufficient disk space (~2GB for models)\n\n### Performance\n\n| Metric | Expected Value |\n|--------|----------------|\n| **RAM Usage** | 4-8 GB |\n| **Processing Speed** | ~94ms per step |\n| **GPU Acceleration** | Metal (Apple Silicon) |\n| **Latency** | Real-time streaming with 6-token delay |\n\n**Tested on M4 Max**: Short audio clips process in a few seconds with word-level timestamps.\n\n### Limitations\n\n- **Languages**: English and French only (per model tokenizer)\n- **Installation Script**: This installer is macOS Apple Silicon specific\n- **Model Size**: Requires ~2GB disk space and 4-8GB RAM\n- **Audio Length**: Optimal performance up to 10 minutes (model training limitations) \n\n### Technology Stack\n\n- **Backend**: Rust with Candle ML framework\n- **Model**: Kyutai STT 1B parameters transformer\n- **Acceleration**: Metal Performance Shaders\n- **Protocol**: WebSocket with MessagePack binary format\n- **Audio Processing**: Real-time streaming with PCM float32\n\n## Advanced Usage\n\n### Custom Model Configuration\nEdit `moshi-stt.toml` to modify:\n- Model parameters\n- Processing delays\n- Batch sizes\n- Audio tokenizer settings\n\n## Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Make your changes\n4. Test thoroughly\n5. Submit a pull request\n\n## License\n\nThis installation script is provided as-is. The underlying components have different licenses:\n\n- **Kyutai STT Model Weights**: [CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/)\n- **Moshi Rust Server**: Apache 2.0\n- **Python Components**: MIT License\n\nFor complete license information, please refer to the [original Kyutai repository](https://github.com/kyutai-labs/delayed-streams-modeling).\n\n## Related Projects\n\n- **[Kyutai Delayed Streams Modeling](https://github.com/kyutai-labs/delayed-streams-modeling)** - Original research project and models\n- **[Kyutai Labs](https://github.com/kyutai-labs)** - AI research laboratory\n- **[Candle Framework](https://github.com/huggingface/candle)** - Rust ML framework\n- **[Moshi Project](https://github.com/kyutai-labs/moshi)** - Complete multimodal AI project\n\n## Alternative Installation Methods\n\nIf this script doesn't meet your needs, consider these alternatives from the original repo:\n\n### PyTorch (Python)\n```bash\npip install moshi\npython -m moshi.run_inference --hf-repo kyutai/stt-1b-en_fr audio.wav\n```\n\n### MLX (Apple Silicon - Alternative)\n```bash\npip install moshi-mlx\npython -m moshi_mlx.run_inference --hf-repo kyutai/stt-1b-en_fr-mlx audio.wav\n```\n\n### Manual Rust Installation\n```bash\ncargo install --features metal moshi-server\n```\n\n---\n\n**Need more help?** Open an issue or check the troubleshooting section above.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjeanjerome%2Fmoshi-stt-apple-installer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjeanjerome%2Fmoshi-stt-apple-installer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjeanjerome%2Fmoshi-stt-apple-installer/lists"}