https://github.com/endlessreform/csm_mlx
https://github.com/endlessreform/csm_mlx
Last synced: about 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/endlessreform/csm_mlx
- Owner: EndlessReform
- Created: 2025-03-14T04:41:59.000Z (2 months ago)
- Default Branch: main
- Last Pushed: 2025-03-15T06:13:57.000Z (2 months ago)
- Last Synced: 2025-03-15T06:19:11.029Z (2 months ago)
- Language: Jupyter Notebook
- Size: 1.91 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Sesame CSM 1B MLX port
Port of Sesame's [CSM](https://github.com/SesameAILabs/csm) model to [MLX](https://github.com/ml-explore/mlx) for use on Apple Silicon.
The project goal is realtime streaming inference on a MacBook.
## Installation
Clone this repo.
Get [uv](https://docs.astral.sh/uv/getting-started/installation/) if you haven't already. Then:
```bash
uv sync
```
Run the WebUI with:
```bash
uv run webui/app.py
```
Q8 quantization is available for a ~60% speedup at some loss of quality:
```bash
uv run webui/app.py --quantize q8
```
For procedural examples, check out `example.ipynb`
## FAQ
If `uv sync` fails due to sentencepiece error, run:
```bash
brew install cmake
```
## Roadmap
- [x] Safetensors conversion
- [x] Core modeling and entry point
- [x] Gradio UI (blocking, streaming will be supported below)
- [x] Streaming output (for model)
- [ ] FastRTC speech-to-speech webui
- [ ] Perf improvements
- [ ] PyPI library