{"id":16689223,"url":"https://github.com/sskorol/vosk-api-gpu","last_synced_at":"2025-03-23T14:31:38.877Z","repository":{"id":40303895,"uuid":"365037736","full_name":"sskorol/vosk-api-gpu","owner":"sskorol","description":"Vosk ASR Docker images with GPU for Jetson boards, PCs, M1 laptops and GPC","archived":false,"fork":false,"pushed_at":"2022-05-16T09:19:40.000Z","size":362,"stargazers_count":42,"open_issues_count":0,"forks_count":8,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-18T21:16:52.403Z","etag":null,"topics":["asr","cuda","docker","gcp","gpu","jetson","jetson-nano","jetson-xavier-nx","m1","nvidia","nvidia-docker","vosk","vosk-api"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sskorol.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-05-06T21:08:23.000Z","updated_at":"2025-02-24T22:13:00.000Z","dependencies_parsed_at":"2022-07-29T21:39:09.708Z","dependency_job_id":null,"html_url":"https://github.com/sskorol/vosk-api-gpu","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sskorol%2Fvosk-api-gpu","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sskorol%2Fvosk-api-gpu/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sskorol%2Fvosk-api-gpu/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sskorol%2Fvosk-api-gpu/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sskorol","download_url":"https://codeload.github.com/sskorol/vosk-api-gpu/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245115950,"owners_count":20563260,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asr","cuda","docker","gcp","gpu","jetson","jetson-nano","jetson-xavier-nx","m1","nvidia","nvidia-docker","vosk","vosk-api"],"created_at":"2024-10-12T15:47:33.093Z","updated_at":"2025-03-23T14:31:38.322Z","avatar_url":"https://github.com/sskorol.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Vosk API - Docker/GPU\n\n[Vosk](https://github.com/alphacep/vosk-api) docker images with GPU for Jetson Nano / Xavier NX boards and PCs with NVIDIA cards.\n\n### Usage\n\nPull an existing [image](https://hub.docker.com/r/sskorol/vosk-api) with a required tag.\n\n```shell\ndocker pull sskorol/vosk-api:$TAG\n```\n\nUse it as a baseline in your app's Dockerfile:\n\n```shell\nFROM sskorol/vosk-api:$TAG\n```\n\n### Build prerequisites\n\n- You have to enable [nvidia runtime](https://github.com/dusty-nv/jetson-containers#docker-default-runtime) before building the images.\n- In the case of Jetson boards, your JetPack should match at least 32.5 version (0.3.32 images were built against 32.6.1).\n- For PCs make sure you met the following [prerequisites](https://medium.com/geekculture/installing-cudnn-and-cuda-toolkit-on-ubuntu-20-04-for-machine-learning-tasks-f41985fcf9b2).\n\n### Building\n\nClone sources and check a build file help:\n\n```shell\ngit clone https://github.com/sskorol/vosk-api-gpu.git\ncd vosk-api-gpu\n```\n\n#### Jetson boards\n\nRun a build script with the required args depending on your platform, e.g.:\n\n```shell\ncd jetson \u0026\u0026 ./build.sh -m nano -i ml -t 0.3.37\n```\n\nYou can check the available NVIDIA base image tags [here](https://ngc.nvidia.com/catalog/containers/nvidia:l4t-base) and [here](https://ngc.nvidia.com/catalog/containers/nvidia:l4t-ml). \n\n#### PCs\n\nTo build images for PC, use the following script:\n\n```shell\ncd pc \u0026\u0026 ./build.sh -c 11.3.1-devel-ubuntu20.04 -t 0.3.37\n```\n\nHere, you have to provide a base cuda image tag and the output container's tag. You can read more by running the script with `-h` flag.\n\nThis script will build 2 images: base and a sample Vosk server.\n\n#### Windows 11 with WSL2\n\n- Follow the official [instructions](https://docs.docker.com/desktop/windows/wsl) to install Docker Desktop.\n- Make sure you fully accomplished the [GPU part](https://docs.docker.com/desktop/windows/wsl/#gpu-support) of the above guide.\n- Either use an existing image or build a new one following [PCs](https://github.com/sskorol/vosk-api-gpu#pcs) part of this README.\n- Follow the [Running](https://github.com/sskorol/vosk-api-gpu#running) part of this README to test your recording.\n\n#### Apple M1\n\nTo build images (w/o GPU) for Apple M1, use the following script:\n\n```shell\ncd m1 \u0026\u0026 ./build.sh -t 0.3.37\n```\n\nTo build Kaldi and Vosk API locally (w/o Docker), use the following script (thanks to @aivoicesystems):\n\n```shell\ncd m1 \u0026\u0026 ./build-local.sh\n```\n\nNote that there's a required software check when you start this script. If you see missing requirements, chances are you'll need to install the following packages:\n\n```shell\nbrew install autoconf cmake automake libtool\n```\n\nAlso note that this script installs Vosk API globally. If you want to use it for a specific project, just activate virtual env before running the script.\n\n#### GCP\n\nTo test images on GCP with NVIDIA Tesla T4, use the following steps:\n\n- Install [terraform](https://learn.hashicorp.com/tutorials/terraform/install-cli)\n- Create a new project on GCP\n- Install and init [gcloud-cli](https://cloud.google.com/sdk/docs/install-sdk)\n- Deploy a new Compute Engine instance with the following commands:\n\n```shell\ncd gcp \u0026\u0026 terraform init \u0026\u0026 terraform apply\n```\nNote that you'll be prompted to type your GCP project name and zone. When a new instance is deployed, you can now ssh to it:\n\n```shell\ngcloud compute ssh --project $PROJECT_NAME --zone $ZONE gpu\n```\n\nClone this repo and build Vosk images on a relatively powerful machine:\n\n```shell\ngit clone https://github.com/sskorol/vosk-api-gpu \u0026\u0026 cd vosk-api-gpu/gcp\n./build.sh -c 11.3.1-devel-ubuntu20.04 -t 0.3.37 -m vosk-model-en-us-0.22\n```\n\nSee build script's help for more details regarding input arguments.\n\n### Running\n\nThe following script will start docker-compose, install requirements and run a simple test:\n\n```shell\n./test.sh $TAG $WAV_FILE\n```\n\n- Pass a newly built image tag as an argument.\n- You have to download and extract a required [model](https://alphacephei.com/vosk/models) into `./model` folder first, unless you use a GCP instance.\n- Pass your own recording as a second argument. `en.wav` is used by default.\n\n### Important notes\n\n- Jetson Nano won't work with the latest large model due to high memory requirements (at least 8Gb RAM).\n- Jetson Xavier **will** work with the latest large model if you remove `rnnlm` folder from `model`.\n- Make sure you have at least Docker (20.10.6) and Compose (1.29.1) versions.\n- Your host's CUDA version must match the container's as they share the same runtime. Jetson images were built with CUDA 10.1. As per the desktop version: CUDA 11.3.1 was used.\n- If you plan to use `rnnlm`, make sure you allocated at least 12Gb of RAM to your Docker instance (16Gb is optimal).\n- In case of GCP usage, there's a know issue with K80 instance. Seems like it has an outdated architecture. So it's recommended to take at least NVIDIA T4.\n- Not all the models are adopted for GPU-usage, e.g. in RU model, you have to manually patch configs to make it work (it's done automatically for GCP instance):\n  - remove `min-active` flag from `model/conf/model.conf`\n  - copy/paste `ivector.conf` from big EN model\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsskorol%2Fvosk-api-gpu","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsskorol%2Fvosk-api-gpu","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsskorol%2Fvosk-api-gpu/lists"}