{"id":43116982,"url":"https://github.com/modal-labs/seamless-chat","last_synced_at":"2026-01-31T19:05:31.927Z","repository":{"id":258005905,"uuid":"872536107","full_name":"modal-labs/seamless-chat","owner":"modal-labs","description":"a multilingual chat room 🌎","archived":false,"fork":false,"pushed_at":"2024-11-06T16:39:11.000Z","size":186,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-11-06T16:48:02.077Z","etag":null,"topics":["ai","python","serverless","translation"],"latest_commit_sha":null,"homepage":"https://seamless.modal.chat/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/modal-labs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-14T15:51:42.000Z","updated_at":"2024-11-06T16:39:15.000Z","dependencies_parsed_at":"2024-10-28T16:22:52.432Z","dependency_job_id":null,"html_url":"https://github.com/modal-labs/seamless-chat","commit_stats":null,"previous_names":["modal-labs/seamless-chat"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/modal-labs/seamless-chat","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modal-labs%2Fseamless-chat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modal-labs%2Fseamless-chat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modal-labs%2Fseamless-chat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modal-labs%2Fseamless-chat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/modal-labs","download_url":"https://codeload.github.com/modal-labs/seamless-chat/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modal-labs%2Fseamless-chat/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28950364,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-31T18:30:42.805Z","status":"ssl_error","status_checked_at":"2026-01-31T18:30:19.593Z","response_time":128,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","python","serverless","translation"],"created_at":"2026-01-31T19:05:30.491Z","updated_at":"2026-01-31T19:05:31.922Z","avatar_url":"https://github.com/modal-labs.png","language":"Python","funding_links":[],"categories":["LLMs/Multimodal Models"],"sub_categories":[],"readme":"# Seamless Chat: Run multilingual chat rooms with SeamlessM4T-V2\n\nChat with friends from around the world without speaking the same language!\n\nIntroducing [Seamless Chat](https://github.com/modal-labs/seamless-chat), a speech-to-speech chat app that lets you chat with all your friends in their language at the same time.\n\nSeamless Chat is powered by Meta's [SeamlessM4T-V2](https://github.com/facebookresearch/seamless_communication/tree/main), a state-of-the-art multilingual speech-to-speech model that supports text and speech translation for over 20 languages. Thanks to Modal's WebSocket support and distributed object stores, we're able to create a scalable, low-latency chat interface.\n\nTry out Seamless-Chat yourself [here](https://seamless.modal.chat)!\n\n\u003cdiv style=\"display: flex; justify-content: space-around;\"\u003e\n  \u003cimg src=\"https://modal-cdn.com/seamless-chat/bob.png\" alt=\"Seamless-Chat Bob\" width=\"48%\"\u003e\n  \u003cimg src=\"https://modal-cdn.com/seamless-chat/alice.png\" alt=\"Seamless-Chat Alice\" width=\"48%\"\u003e\n\u003c/div\u003e\n\n## Code Overview\n\nSeamless Chat's frontend and backend are both entirely deployed on Modal. The frontend is a standard SvelteKit app and the backend is a FastAPI server that handles WebSockets for chat connections and GPU accelerated inference. Let's take a look at each of these in more detail!\n\n### Chat Backend - SeamlessM4T on GPUs\n\nThe code that powers Seamless Chat's backend is defined with Modal's class syntax and the `@app.cls` decorator. We define methods on that class for managing model loading, as described below. This allows us to manage multiple WebSocket connections in a shared container while managing the overhead of starting new containers. We can also specify the maximum number of concurrent connections with the `allow_concurrent_inputs` property.\n\n```python\n@app.cls(\n    gpu=\"H100\",\n    allow_concurrent_inputs=10,\n    # and more\n)\nclass SeamlessM4T:\n    ...\n```\n\nWith the `@modal.build()` decorator, we download the model into our container image, and with the `@modal.enter()` decorator, we load the model into memory once the container is instantiated.\n\n```python notest\n@modal.build()\ndef build(self):\n    snapshot_download(\"facebook/seamless-m4t-v2-large\")\n\n@modal.enter()\ndef enter(self):\n    self.processor = AutoProcessor.from_pretrained(\"facebook/seamless-m4t-v2-large\")\n    self.model = SeamlessM4Tv2Model.from_pretrained(\"facebook/seamless-m4t-v2-large\").to(\"cuda\")\n```\n\n### Chat Backend - FastAPI Server\n\nModal makes it easy to define a ASGI server: just wrap the `@modal.asgi_app()` decorator around a function that returns a `FastAPI` app.\n\nThe main component of our server is our WebSocket endpoint for handling chat connections. Each socket connection needs to listen for incoming messages from the user along with outgoing messages from other users in the room. Using `asyncio`, we can handle both of these tasks concurrently, while also gracefully handling disconnections and errors.\n\n```python notest\n@app.websocket(\"/chat\")\nasync def chat(websocket: WebSocket):\n    await websocket.accept()\n\n    async def recv_loop():\n        while True:\n            # fetch incoming messages from user's websocket\n            message = await websocket.receive_json()\n            ...\n\n    async def send_loop():\n        while True:\n            # fetch outgoing messages from other users in the room\n            ...\n    try:\n        tasks = [\n            asyncio.create_task(send_loop()),\n            asyncio.create_task(recv_loop()),\n        ]\n        await asyncio.gather(*tasks)\n    except WebSocketDisconnect:\n        print(f\"Socket disconnected: {user_id}\")\n        await websocket.close(code=1000)\n    except Exception as e:\n        print(f\"Socket error: {e}\")\n        await websocket.close(code=1011)\n    finally:\n        for task in tasks:\n            task.cancel()\n        await asyncio.gather(*tasks, return_exceptions=True)\n```\n\n### Chat Backend - Distributed Queues\n\nAs our application scales up, we may have multiple socket connections across different containers, so we need some way to send and synchronize messages between users. To handle this, we can take advantage of Modal's [distributed queues](https://modal.com/docs/guide/dicts-and-queues#modal-queues).\n\nWe can define a Modal Queue to store messages, with a separate FIFO partition for each user. When a user sends a message, we append it to the partition associated with each member of the room. The `send_loop` method repeatedly fetches new messages from their partition and translates the messages into their target language. In an asynchronous context, we simply use the `.aio()` function suffix to fetch messages from the queue. Finally, we pass the messages through the translation model and send the response back to the user.\n\n```python\n\nmessage_queue = modal.Queue.from_name(\"seamless-message-queue\")\n\nasync def send_loop():\n    while True:\n        message = await message_queue.get.aio(partition=user_id)\n\n        text, audio_array = self.translate(message, src_lang, tgt_lang)\n\n        message_data = {\n            \"messageId\": message[\"message_id\"],\n            \"userId\": message[\"user_id\"],\n            \"userName\": message[\"user_name\"],\n            \"lang\": src_lang,\n            \"text\": text,\n            \"audio\": audio_array.tolist(),\n        }\n\n        await websocket.send_json(message_data)\n```\n\n### Chat Frontend - SvelteKit\n\nSeamless Chat's frontend is a simple static SvelteKit application. We define a Modal function that is called through a `web_endpoint` with the same `@modal.asgi_app()` decorator. The function simply serves the frontend's static files in the `frontend/build` directory after running `npm run build` in the `frontend` directory.\n\n```python\n@app.function(\n    mounts=[modal.Mount.from_local_dir(\"some/static/path\", remote_path=\"/assets\")],\n    # and more\n)\n@modal.asgi_app(custom_domains=[\"seamless.modal.chat\"])\ndef frontend():\n    web_app = FastAPI()\n    web_app.mount(\"/\", StaticFiles(directory=\"/assets\", html=True))\n\n    return web_app\n```\n\n## Deploy\n\nTo deploy Seamless Chat, you can simply clone the repository, compile the frontend assets and run `modal deploy`. Make sure you have the latest versions of `npm` and the modal client installed.\n\n```bash\ngit clone https://github.com/modal-labs/seamless-chat\n\ncd frontend\nnpm run build\ncd ..\n\nmodal deploy seamless.py\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmodal-labs%2Fseamless-chat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmodal-labs%2Fseamless-chat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmodal-labs%2Fseamless-chat/lists"}