{"id":13639805,"url":"https://github.com/nick1udwig/ursr","last_synced_at":"2026-01-17T06:50:20.214Z","repository":{"id":152426613,"uuid":"390186301","full_name":"nick1udwig/ursr","owner":"nick1udwig","description":"UrSR: Urbit Speech Recognition","archived":false,"fork":false,"pushed_at":"2021-11-24T16:32:17.000Z","size":148,"stargazers_count":16,"open_issues_count":5,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-10T12:23:37.016Z","etag":null,"topics":["hoon","speech-recognition","speech-to-text","urbit"],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nick1udwig.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2021-07-28T02:25:54.000Z","updated_at":"2023-05-05T16:32:25.000Z","dependencies_parsed_at":"2024-01-14T09:59:46.030Z","dependency_job_id":"b6bc7106-bdfd-4fdb-990e-dd3c936296f4","html_url":"https://github.com/nick1udwig/ursr","commit_stats":null,"previous_names":["nick1udwig/ursr","hosted-fornet/ursr"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nick1udwig%2Fursr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nick1udwig%2Fursr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nick1udwig%2Fursr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nick1udwig%2Fursr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nick1udwig","download_url":"https://codeload.github.com/nick1udwig/ursr/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249267825,"owners_count":21240866,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hoon","speech-recognition","speech-to-text","urbit"],"created_at":"2024-08-02T01:01:05.082Z","updated_at":"2025-04-20T01:32:32.175Z","avatar_url":"https://github.com/nick1udwig.png","language":"HTML","funding_links":[],"categories":["Archive"],"sub_categories":["Tools"],"readme":"# UrSR: Urbit Speech Recognition\n\nFor some applications, speech is more natural than text.\nUrSR brings ASR (Automatic Speech Recognition) to Urbit.\nIt consists of Gall apps and a Golang middleman, and leverages the [Mod9 ASR Engine](https://mod9.io/) to make ASR as simple as a poke.\nAside from providing the infrastructure for developing speech-enabled apps, this project also contains a demo voice notes app.\n\n\n# Usage\n\n## UrSR Demo Notebook usage example\n\n### TL;DR\n\n1. Install UrSR Client and UrSR Demo Notebook apps from `~dister-hosted-labweb`.\n1. Create a group and chat on your ship to store voice notes.\n1. Open the UrSR Demo Notebook app and fill in:\n    1. `Provider` as `~hoster-hosted-labweb`,\n    1. `+code` as your ship's `+code`,\n    1. `Chat` as the name of the chat you just made (get the final part of the URL and copy-paste).\n1. Click `Send` to start taking voice notes and `Stop` to stop!\n\n### More details\n\nTo use the UrSR Demo Notbook voice notes app, you will need a machine with a mic and to:\n\n1. Know an UrSR provider (currently `~hoster-hosted-labweb` is running a provider node).\n1. Get the UrSR Client app (which knows how to talk to the provider).\n1. Get the UrSR Demo Notebook voice notes app (which is an *aesthetic* JS frontend).\n\nThe UrSR Client and Demo Notebook apps can be installed from `~dister-hosted-labweb`.\n\nFor best experience, create a group on your ship and a `Chat` channel within that group: the UrSR Demo Notebook will post transcripts to a chat.\n\nThen, open the UrSR Demo Notebook app by clicking the tile in Grid.\n\nFor the `Provider` field, fill in\n\n```\n~hoster-hosted-labweb\n```\n\nwhich is my UrSR Provider ship.\nYou will need to enter your `+code` so the JS webapp can talk to your ship.\nAnd finally, write in the `Chat` channel name you created a moment ago.\nLook in the URL when you are in the `Chat` channel in order to get the chat name and place ONLY the chat name in this field.\nFor example, if your URL when you are in the `Chat` channel is\n\n```\nhttp://localhost:8081/apps/landscape/.../~nec/my-voice-362\n```\n\nthen you should put\n\n```\nmy-voice-362\n```\n\ninto the `Chat` field in the UrSR Demo Notebook webapp.\nThe `Chat` MUST be hosted on your ship to work!\n\n\n## Provider set up\n\nA provider needs to set up three things:\n\n1. The UrSR Provider app on the provider ship.\n1. The Mod9 ASR Engine, a TCP server that will do the actual transcription.\n1. A Golang middleman that communicates between the UrSR Provider app and the Mod9 ASR Engine.\n\n### UrSR Provider app\n\nOn your provider ship, install the UrSR Provider app from my distribution ship `~dister-hosted-labweb`.\n\n### Mod9 ASR Engine\nI recommend installing the Docker image of the Engine.\nTo do so, make sure you have installed Docker and then run\n\n```bash\ndocker pull mod9/asr\n```\n\nTo run the Engine, use\n\n```bash\ndocker run -p 9900:9900 mod9/asr engine\n```\n\nAn important caveat related to the Engine: the Engine is not free or open source software.\nThe copy pulled down from Docker, above, is a 45-day trial.\nInstructions on getting a licensed copy will appear here shortly.\n\nTo run the trial Engine without CPU throttling, start it as follows:\n\n```bash\ndocker run -p 9900:9900 mod9/asr engine --accept-license=yes\n```\n\nFor more information, please consult the [Engine documentation](https://mod9.io/).\n\n### Golang middleman\nThe Golang middleman communicates between your provider ship and the Engine.\nExecutables can be found on [Github](https://github.com/hosted-fornet/ursr), or it can be run from source.\n\n```bash\n# View usage information.\n./ursr-go -h\n\n# Run the Golang middleman with the proper flags for your set up\n#  (examples for a fakeship ~wes below):\n./ursr-go -code lapwen-fadtun-lagsyl-fadpex -engine localhost:9900 -ship localhost:8080 -ttl 0\n```\n\nNOTE: The Golang middleman will log a line like `\"Monitoring events...\"` if it has connected to your ship.\nIf you do not see this line, restart the middleman to make sure it has properly subscribed to your ship.\n\n### Whitelisting\nWhitelisting is provided by [the Whitelist library](https://github.com/hosted-fornet/whitelist).\nA provider can whitelist:\n\n* All ships,\n* Its kids,\n* A specific set of ships,\n* Ships belonging to a set of groups,\n\nor any combination thereof.\n\nBy default, all of these will be disabled.\nTo add to one of these whitelisted categories, use `%add-whitelist`; to remove, use `%remove-whitelist`.\nMore details can be found at the link above, but here are some concrete examples:\n\n```\n:: Make provider public.\n:ursr-provider \u0026whitelist-command [%add-whitelist ~[%public]]\n\n:: Remove permission from kids.\n:ursr-provider \u0026whitelist-command [%remove-whitelist ~[%kids]]\n\n:: Add specific ship(s) to whitelist.\n:ursr-provider \u0026whitelist-command [%add-whitelist [%users (silt ~[~hosted-fornet ~hosted-labweb])]]\n\n:: Remove specific ship(s) from whitelist.\n:ursr-provider \u0026whitelist-command [%remove-whitelist [%users (silt ~[~hosted-fornet])]]\n\n:: Add group to whitelist (i.e. group membership means a ship can use your provider).\n:ursr-provider \u0026whitelist-command [%remove-whitelist [%groups (silt ~[[~wisdem-hosted-labweb %homunculus]])]]\n```\n\n\n# Useful `@p`s to know\n\n```\n~hosted-fornet          :: me\n~hosted-labweb          :: also me\n~dister-hosted-labweb   :: my software distribution ship\n~hoster-hosted-labweb   :: my UrSR Provider ship\n~wisdem-hosted-labweb   :: my group hosting ship\n```\n\n\n# Questions and Feedback\n\nHit me up at `~hosted-fornet` or come chat in\n\n```\n~wisdem-hosted-labweb/homunculus\n```\n\n\n# Privacy notice\n\nIf you are sending your audio to a provider to transcribe, you should not send sensitive audio unless you trust that provider.\nThere is nothing stopping a sketchy provider from keeping your audio and your transcripts.\nIf you have sensitive audio you wish to transcribe, you should set up your own provider node and set it as the provider for your request.\n\n\n# Dev stuff\n\n## Project structure\n\nUrSR uses a client-provider model, similar to the Urbit Bitcoin app.\nProviders will need more technical skills than clients: in addition to running a Gall app, they will need to run the Mod9 ASR Engine, which transcribes the audio sent by clients, and a Golang middleman that mediates between the Provider app and the Engine.\n\nIn contrast, a client need only install the UrSR Client app and whatever application makes use of it.\nAs an example, this repo includes the UrSR Demo Notebook, a simple voice notetaking application.\nWith it, users can record from the mic on their computer to a Chat channel hosted on their ship.\n\nThe UrSR Client and Provider are distributed from my distribution ship `~dister-hosted-labweb`.\nThese Gall apps have no frontend: they just talk to each other (and, in the case of the Client, to frontend or other Gall apps).\n\nThe repository is structured as follows:\n* `go/` contains the Golang middleman that mediates between the transcription Engine and the Provider app,\n* `hoon/` contains four directories, three of which are distributed as Gall apps:\n    * `ursr-client/` contains the UrSR Client,\n    * `ursr-demo/` contains the UrSR Demo Notebook,\n    * `ursr-dev/` contains the types and marks devs will need to develop their own frontends for UrSR,\n    * `ursr-provider/` contains the UrSR Provider,\n* `scripts/` is useful for devs who wish to build the Gall apps in this repo.\n\n\n## Protocol\n\n### Starting a job\n\nTo start a transcription job, poke the UrSR Client running on your ship.\nThe poke type is `ursr-payload`, which includes a `job-id=@ud`, and an `action`, here, `action=[%client-start-job =options provider=@p]`.\nThe `job-id` is an `@ud` and *must be unique* to your job: recommended practice is to use a large random number for each job.\nThe `provider` is a ship running the UrSR Provider app to which your audio data will be sent for transcription.\nThe `options` are settings for the transcription: documentation of these options can be found at [mod9.io](https://mod9.io).\nA recommended, basic set of options is\n\n```json\n{\n    \"command\": \"recognize\",\n    \"format\": \"raw\",\n    \"encoding\": \"pcm_s16le\",\n    \"rate\": 16000,\n    \"transcript-formatted\": true\n}\n```\n\nfor audio streamed from a microphone at 16kHz.\n16kHz audio is recommended for use with the default transcription model used by the Engine: other rates will be resampled internally.\n\n### Subscribing for replies\n\nThe UrSR Client will relay audio to the Provider and relay replies back to the caller.\nTo receive these replies, subscribe to the UrSR Client at the path `/frontend-path/[job-id]`, with the same job ID used to start the job.\nNote that the `job-id` here must be formatted in the Urbit pretty-printed manner, so that, e.g., `1000000` is rendered as `1.000.000`.\n\nReplies are `ursr-payload`-type, with `action` either `relay-reply` or `job-done`.\n`job-done`, as indicated by the name, means the transcription has finished.\n`relay-reply` contains some of the fields the Engine replies with, including `final`, `result-index`, `transcript`, and, if the `transcript-formatted` option is set to `true`, `transcript-formatted`.\n\n### Sending audio\n\nAudio is `poke`d to the UrSR Client using the `ursr-payload` type.\nIn addition to the `job-id`, the `action` passed should be `relay-audio`, with field `audio`, an `int16` array (for `encoding: pcm_s16le` `option` field).\n\n\n# Acknowledgements\n\nYou can find the grant proposal for this project [here](https://urbit.org/grants/speech-recognition).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnick1udwig%2Fursr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnick1udwig%2Fursr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnick1udwig%2Fursr/lists"}