{"id":13433580,"url":"https://github.com/modal-labs/quillman","last_synced_at":"2026-01-31T19:05:30.748Z","repository":{"id":154035743,"uuid":"630052479","full_name":"modal-labs/quillman","owner":"modal-labs","description":"A voice chat app ","archived":false,"fork":false,"pushed_at":"2025-05-21T20:18:49.000Z","size":4526,"stargazers_count":1153,"open_issues_count":4,"forks_count":144,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-08-10T19:45:24.413Z","etag":null,"topics":["ai","language-model","python","serverless","speech-recognition","speech-to-text"],"latest_commit_sha":null,"homepage":"https://modal.com/docs/examples/llm-voice-chat","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/modal-labs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-04-19T15:07:55.000Z","updated_at":"2025-08-10T18:56:11.000Z","dependencies_parsed_at":"2023-12-18T06:25:05.857Z","dependency_job_id":"b3eb9037-dec1-4d62-ba91-01487458cc4f","html_url":"https://github.com/modal-labs/quillman","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/modal-labs/quillman","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modal-labs%2Fquillman","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modal-labs%2Fquillman/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modal-labs%2Fquillman/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modal-labs%2Fquillman/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/modal-labs","download_url":"https://codeload.github.com/modal-labs/quillman/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modal-labs%2Fquillman/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28950363,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-31T18:30:42.805Z","status":"ssl_error","status_checked_at":"2026-01-31T18:30:19.593Z","response_time":128,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","language-model","python","serverless","speech-recognition","speech-to-text"],"created_at":"2024-07-31T02:01:29.853Z","updated_at":"2026-01-31T19:05:30.741Z","avatar_url":"https://github.com/modal-labs.png","language":"Python","funding_links":[],"categories":["Python","HarmonyOS","🤖 AI \u0026 Machine Learning"],"sub_categories":["Windows Manager"],"readme":"# QuiLLMan: Voice Chat with Moshi\n\nA complete voice chat app powered by a speech-to-speech language model and bidirectional streaming.\n\nOn the backend is Kyutai Lab's [Moshi](https://github.com/kyutai-labs/moshi) model, which will continuously listen, plan, and respond to the user. It uses the [Mimi](https://huggingface.co/kyutai/mimi) streaming encoder/decoder model to maintain an unbroken stream of audio in and out, and a [speech-text foundation model](https://huggingface.co/kyutai/moshiko-pytorch-bf16) to determine when and how to respond.\n\nThanks to bidirectional websocket streaming and use of the [Opus audio codec](https://opus-codec.org/) for compressing audio across the network, response times on good internet can be nearly instantaneous, closely matching the cadence of human speech.\n\nYou can find the demo live [here](https://modal-labs--quillman-web.modal.run/).\n\n![Quillman](https://github.com/user-attachments/assets/afda5874-8509-4f56-9f25-d734b8f1c40a)\n\nThis repo is meant to serve as a starting point for your own language model-based apps, as well as a playground for experimentation. Contributions are welcome and encouraged!\n\n[Note: this code is provided for illustration only; please remember to check the license before using any model for commercial purposes.]\n\n## File structure\n\n1. React frontend ([`src/frontend/`](./src/frontend/)), served by [`src/app.py`](./src/app.py)\n2. Moshi websocket server ([`src/moshi.py`](./src/moshi.py))\n\n## Developing locally\n\n### Requirements\n\n- `modal` installed in your current Python virtual environment (`pip install modal`)\n- A [Modal](http://modal.com/) account (`modal setup`)\n- A Modal token set up in your environment (`modal token new`)\n\n### Developing the inference module\n\nThe Moshi server is a [Modal class](https://modal.com/docs/reference/modal.Cls#modalcls) module to load the models and maintain streaming state, with a [FastAPI](https://fastapi.tiangolo.com/) http server to expose a websocket interface over the internet.\n\nTo run a [development server](https://modal.com/docs/guide/webhooks#developing-with-modal-serve) for the Moshi module, run this command from the root of the repo.\n\n```shell\nmodal serve -m src.moshi\n```\n\nIn the terminal output, you'll find a URL for creating a websocket connection.\n\nWhile the `modal serve` process is running, changes to any of the project files will be automatically applied. `Ctrl+C` will stop the app.\n\n### Testing the websocket connection\n\nFrom a seperate terminal, we can test the websocket connection directly from the command line with the `tests/moshi_client.py` client.\n\nIt requires non-standard dependencies, which can be installed with:\n\n```shell\npython -m venv venv\nsource venv/bin/activate\npip install -r requirements/requirements-dev.txt\n```\n\nWith dependencies installed, run the terminal client with:\n\n```shell\npython tests/moshi_client.py\n```\n\nAnd begin speaking! Be sure to have your microphone and speakers enabled.\n\n### Developing the http server and frontend\n\nThe http server at `src/app.py` is a second [FastAPI](https://fastapi.tiangolo.com/) app, for serving the frontend as static files.\n\nA [development server](https://modal.com/docs/guide/webhooks#developing-with-modal-serve) can be run with:\n\n```shell\nmodal serve src.app\n```\n\nSince `src/app.py` imports the `src/moshi.py` module, this also starts the Moshi websocket server.\n\nIn the terminal output, you'll find a URL that you can visit to use your app.\nWhile the `modal serve` process is running, changes to any of the project files will be automatically applied. `Ctrl+C` will stop the app.\n\nNote that for frontend changes, the browser cache may need to be cleared.\n\n### Deploying to Modal\n\nOnce you're happy with your changes, [deploy](https://modal.com/docs/guide/managing-deployments#creating-deployments) your app:\n\n```shell\nmodal deploy src.app\n```\n\nThis will deploy both the frontend server and the Moshi websocket server.\n\nNote that leaving the app deployed on Modal doesn't cost you anything! Modal apps are serverless and scale to 0 when not in use.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmodal-labs%2Fquillman","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmodal-labs%2Fquillman","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmodal-labs%2Fquillman/lists"}