{"id":16896962,"url":"https://github.com/xd009642/streamer-template","last_synced_at":"2025-03-22T09:31:49.104Z","repository":{"id":238726234,"uuid":"797367166","full_name":"xd009642/streamer-template","owner":"xd009642","description":"Part of series on bidirectional streaming audio APIs","archived":false,"fork":false,"pushed_at":"2025-03-10T12:28:32.000Z","size":930,"stargazers_count":16,"open_issues_count":13,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-18T10:12:49.533Z","etag":null,"topics":["audio","rust","streaming-api"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xd009642.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"xd009642","patreon":null,"custom":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null}},"created_at":"2024-05-07T17:34:45.000Z","updated_at":"2025-03-03T14:35:14.000Z","dependencies_parsed_at":"2024-11-30T12:21:05.072Z","dependency_job_id":"a1d1b36c-3a57-4a53-86b7-415896481551","html_url":"https://github.com/xd009642/streamer-template","commit_stats":null,"previous_names":["xd009642/streamer-template"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xd009642%2Fstreamer-template","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xd009642%2Fstreamer-template/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xd009642%2Fstreamer-template/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xd009642%2Fstreamer-template/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xd009642","download_url":"https://codeload.github.com/xd009642/streamer-template/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244937751,"owners_count":20535124,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["audio","rust","streaming-api"],"created_at":"2024-10-13T17:34:12.113Z","updated_at":"2025-03-22T09:31:49.097Z","avatar_url":"https://github.com/xd009642.png","language":"Rust","funding_links":["https://github.com/sponsors/xd009642"],"categories":[],"sub_categories":[],"readme":"# Streamer Template\n\nThis is a place to play around with some project layouts for audio streaming\nservers that provide bidirectional streaming interface to some CPU bound model.\n\nCurrently very WIP. If you come back later this might be more fleshed out!\n\n## Motivation \n\nIn world there are APIs where you stream audio or some continuous data into\nthem and insights about said data are streamed out. I've had cause to do this\na lot in my work and have people with less domain experience work on these\nsystems as well. Often the audio processing is also computationally intensive\nsuch as speech AI.\n\nSo with this project I wanted to create something laid out in a clean manner\nwhich demonstrates numerous patterns and ways of designing such APIs in a way\nthat they're testable, maintainable and hopefully performant.\n\n## The Patterns\n\nIn my background there's a few decisions to make. For processing the data via a\nmodel we have the decison on whether we:\n\n1. Process all of the incoming audio\n2. Detect segments of interest and process them (VAD/energy filtering)\n\nFor the first options there's also a choice on whether we can process segments \nconcurrently or if the result from one segment needs to be applied to the future\nsegment for various reasons i.e. smoothing/hiding seams generative outputs from\nthe audio.\n\nEnumerating these patterns and representing them all in the code is a WIP.\nCurrently, I process everything and assume no relationship between utterances.\nDayJob™ may open source some packages soon that I can reuse for some things like\nfiltering.\n\n## What's Included\n\nSo far a server and a client to stream audio into it.\nThe server just returns a count of bytes in the segment but I may add some\nmore interesting non-filler functionality. The server also adds a blocking\ndelay because neural network inferences are blocking and it can serve as a\nway of modelling how this impacts the performance of the async runtime.\n\nOpentelemetry support is also included with trace propagation via websockets!\nThis is tricky because javascript doesn't let you set HTTP headers with\nwebsocket requests so one approach APIs I've used have gone with is including\na trace-id in the first message sent over the websocket. I have replicated\nthis and handled the faffy-pain of trace propagation.\n\nTokio task metrics are now included for the main blocks in the streaming\npipeline, as well as other metrics in a Prometheus compatible endpoint.\n\n## Running\n\nFor simple running I provide a client and a server. You should be able to run\nthe server with just:\n\n```\ncargo run --release\n```\n\nFor the client you can run:\n\n```\ncargo run --release --bin client -- -i \u003cWAVE FILE\u003e\n```\n\nThis will run as fast as possible via the VAD segmented API. If you want to\nsplit to the version that splits the audio into equal chunks and puts all\naudio through the model then:\n\n```\ncargo run --release --bin client -- -i \u003cWAVE FILE\u003e --addr ws://localhost:8080/api/v1/simple\n```\n\nAdd `--real-time` to get the client to throttle chunk sending to match real-time\nand `--interim-results` to get partial results.\n\nFor a more thorough server deployment look at the docker-compose and you'll\nget goodies like prometheus and opentelemetry for metrics and traces.\n\n## The Write-up\n\nLook in the doc folder and you'll find the current published blog posts as well\nas any draft ones or ones held back because they're out of sequence. Anything\nnot yet published may change significantly and be in a draft state, I wouldn't\nrecommend reading it unless you're very desperate or fancy providing some feedback.\n\nCurrent planned writeups:\n\n1. System introduction ✔️\n2. Streaming API design ✔️\n3. Audio Decoding ✔️\n4. The Model ✔️\n5. Creating an Axum API ✔️\n6. Metrics for Streaming APIs 🚧\n7. Observability with Opentelemetry ❌\n8. Batching to improve async performance 🚧\n9. Testing Streaming Audio Systems ❌\n10. Implementing a gRPC API ❌\n\nMore things may appear as we get further and further on and I see more things\nwhere there's not a good existing knowledge base. I'm also open to suggestions\non topics.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxd009642%2Fstreamer-template","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxd009642%2Fstreamer-template","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxd009642%2Fstreamer-template/lists"}