{"id":23104776,"url":"https://github.com/mayniklas/whisper_api","last_synced_at":"2025-08-16T16:32:14.348Z","repository":{"id":109061420,"uuid":"603471094","full_name":"MayNiklas/whisper_api","owner":"MayNiklas","description":"A simple API to access whisper. ","archived":false,"fork":false,"pushed_at":"2024-10-31T17:08:31.000Z","size":311,"stargazers_count":8,"open_issues_count":3,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-10-31T17:31:04.308Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MayNiklas.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-18T16:05:42.000Z","updated_at":"2024-10-31T16:34:58.000Z","dependencies_parsed_at":"2023-10-25T21:35:37.348Z","dependency_job_id":"cc57c379-4b89-43a1-b11a-5f80fb39ccb4","html_url":"https://github.com/MayNiklas/whisper_api","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MayNiklas%2Fwhisper_api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MayNiklas%2Fwhisper_api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MayNiklas%2Fwhisper_api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MayNiklas%2Fwhisper_api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MayNiklas","download_url":"https://codeload.github.com/MayNiklas/whisper_api/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230044652,"owners_count":18164194,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-17T00:36:19.235Z","updated_at":"2024-12-17T00:36:19.767Z","avatar_url":"https://github.com/MayNiklas.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Whisper API\n\nA simple API to access whisper for speech to text transcription.\n\nIt simplifies offloading the heavy lifting of using Whisper to a central GPU server, which can be accessed by multiple people.\n\n## Features\n\n* Transcribes audio files to text using OpenAI Whisper\n\n### Simple frontend\n* Includes a simple static frontend to transcribe audio files (`/`)\n* Includes a interactive API documentation using the Swagger UI (`/docs`)\n* Implements a task queue to handle multiple requests (first in, first out)\n\n### Resource friendly GPU support\n* Uses GPU acceleration if available\n* Supports loading the model into VRAM on startup OR on first request\n* Supports unloading the model after a certain time of inactivity\n\n### Privacy focussed\n* Stateless: to prioritize data privacy, the API only stores data in RAM. Audio files are stored using tempfile and are deleted after processing.\n* Logs don't contain any transcribed text and transcription ids are obfuscated\n* Results are deleted from RAM after a given time\n\n## Setup recommendations\n\nThis service performs the best, when it is run on a server with a GPU. For using the high-quality models, I recommend using a GPU with at least 12GB of VRAM. The RTX 3060 12GB is most likely the cheapest option for this task.\n\nThis service is optimized for a multi user environment. I will discuss 2 setups:\n\n### Personal setup\n\nWhen you are the only user of this service, you can run it on your local network. This way you can access the service from any device in your network. Use a VPN to access the service from outside your network.\n\n### SMB \u0026 research setup\n\nWhen hosting this service in a more professional environment, we should consider the following:\n\n* should the service be accessible from outside the network?\n* who should be able to access the service?\n\nIf only users on your local network should be able to access the service and everyone in your network should be able to access it, you can run the service on a server in your network without any further configuration.\n\nIf you need to implement access control, I suggest the following:\n\n* use a reverse proxy to terminate SSL\n* use oauth2 to only allow users which belong to a certain group to access the service\n\nMy setup uses the following software:\n\n* NGINX as a reverse proxy\n* Keycloak as an identity provider\n* oauth2_proxy to handle oauth2 authentication and session tokens\n\nIn case you have some questions about the setup or software, feel free to reach out!\n\n## How to deploy\n\n### Linux - docker\n\nPre-requisites:\n\n1. have [Docker](https://docs.docker.com/engine/install/) installed\n2. install [NVIDIA CUDA](https://developer.nvidia.com/cuda-downloads?target_os=Linux) (if you want to use GPU acceleration)\n3. install [NVidia Container Toolkit](https://github.com/NVIDIA/nvidia-container-toolkit) (if you want to use GPU acceleration)\n\nCreate the following `compose.yaml` file:\n\n```yaml\nservices:\n  whisperAPI:\n    image: ghcr.io/mayniklas/whisper_api:latest\n    ports:\n      - \"3001:3001\"\n    environment:\n      - PORT=3001\n      - LOAD_MODEL_ON_STARTUP=1\n      # - UNLOAD_MODEL_AFTER_S=300\n      # - DEVELOP_MODE=0\n    deploy:\n      resources:\n        reservations:\n          devices:\n            - driver: nvidia\n              capabilities: [gpu]\n```\n\nWhen nop using GPU acceleration, remove the `deploy` section from the `compose.yaml` file.\n\nRun the following commands:\n\n```bash\ndocker compose up -d\n```\n\nYou can also use `docker` directly:\n\n```bash\ndocker run -d -p 3001:3001 --gpus all ghcr.io/mayniklas/whisper_api:latest\n```\n\n### Linux\n\nPre-requisites:\n\n1. install [NVIDIA CUDA](https://developer.nvidia.com/cuda-downloads?target_os=Linux)\n2. install ffmpeg (e.g. `sudo apt install ffmpeg`)\n\nSince project is a well packaged python project, you don't have to worry about any project specific installation steps.\n\n1. Create a virtual environment\n2. Install this project in the virtual environment\n3. Create a systemd service that runs the server\n\n### NixOS\n\nSince I'm personally using NixOS, I created a module that is available through this `flake.nix`.\n\nAdd the following input to your `flake.nix`:\n\n```nix\n{\n  inputs = {\n    whisper_api.url = \"github:MayNiklas/whisper_api\";\n  };\n}\n```\n\nImport the module in your `configuration.nix` and use it:\n\n```nix\n{ pkgs, config, lib, whisper_api, ... }: {\n\n  imports = [ whisper_api.nixosModules.whisper_api ];\n\n  services.whisper_api = {\n    enable = true;\n    withCUDA = true;\n    loadModelOnStartup = true;\n    # unloadModelAfterSeconds = 300;\n    listen = \"0.0.0.0\";\n    openFirewall = true;\n    environment = { };\n  };\n\n}\n```\n\n## Development\n\n### Linux\n\nPre-requisites:\n\n1. install [NVIDIA CUDA](https://developer.nvidia.com/cuda-downloads?target_os=Linux)\n2. install ffmpeg (e.g. `sudo apt install ffmpeg`)\n\n```bash\n# clone the repository\ngit clone https://github.com/MayNiklas/whisper_api.git\n\n# change into the directory\ncd whisper_api\n\n# create a virtual environment\npython3 -m venv .venv\nsource .venv/bin/activate\n\n# prepare the environment\npip3 install -e '.[dev]'\n\n# run the server from within the virtual environment\ncd src/\nuvicorn whisper_api:app --reload --host 127.0.0.1 --port 3001\n\n# alternatively, you can use the following command to run the server\nexport PORT=3001\nexport LISTEN=127.0.0.1\nwhisper_api\n```\n\n### NixOS\n\n```bash\n# clone the repository\ngit clone https://github.com/MayNiklas/whisper_api.git\n\n# change into the directory\ncd whisper_api\n\n# run the server via nix (using CUDA)\nnix run .#whisper_api_withCUDA\n\n# enter the development shell providing the necessary environment\nnix develop .#withCUDA\n```\n\n## Settings\n\n| parameter                          | description                                                                               | possible values                                  | default           |\n|------------------------------------|-------------------------------------------------------------------------------------------|--------------------------------------------------|-------------------|\n| `PORT`                             | Port the API is available under                                                           | any number of port interval                      | 3001              |\n| `LISTEN`                           | Address the API is available under                                                        | any IP or domain you own                         | 127.0.0.1         |\n| `LOAD_MODEL_ON_STARTUP`            | If model shall be loaded on startup                                                       | `1` (yes) or `0` (no)                            | 1                 |\n| `DEVELOP_MODE`                     | Develop mode defaults to smallest model to save time                                      | `1` (yes) or `0` (no)                            | 0                 |\n| `UNLOAD_MODEL_AFTER_S`             | If set the model gets unloaded after inactivity of t seconds, unset means no unload       | any int (0 for instant unload)                   | 'unset'           |\n| `DELETE_RESULTS_AFTER_M`           | Time after which results are deleted from internal storage                                | any int                                          | 60                |\n| `REFRESH_EXPIRATION_TIME_ON_USAGE` | If result is used expand lifetime                                                         | `1` (yes) or `0` (no)                            | 1                 |\n| `RUN_RESULT_EXPIRY_CHECK_M`        | Interval in which timeout checks shall be executed                                        | any int (0 enables lazy timeout)                 | 5                 |\n| `USE_GPU_IF_AVAILABLE`             | If GPU shall be used when available                                                       | `1` (yes) or `0` (no)                            | 1                 |\n| `MAX_MODEL`                        | Max model to be used for decoding, unset means best possible                              | name of official model                           | 'unset'           |\n| `MAX_TASK_QUEUE_SIZE`              | The limit of tasks that can be queued in the decoder at the same time before rejection    | any int                                          | 128               |\n| `CPU_FALLBACK_MODEL`               | The fallback when `MAX_MODEL` is not set and CPU mode is needed                           | name of official model                           | medium            |\n| `LOG_DIR`                          | The directory to store log-file(s) in \"\" means 'this directory', dir is created if needed | wanted directory name or empty str               | \"data/\"           |\n| `LOG_FILE`                         | The name of the log file                                                                  | arbitrary filename                               | whisper_api.log   |\n| `LOG_LEVEL_CONSOLE`                | The name of the log file                                                                  | arbitrary filename                               | whisper_api.log   |\n| `LOG_PRIVACY_MODE`                 | Don't display full task uuids and other sensitive data in the logs                        | `1` (yes) or `0` (no)                            | 1                 |\n| `LOG_LEVEL_FILE`                   | Level of logging for the file                                                             | \"DEBUG\", \"INFO\", \"WARNING\", \"ERROR\", \"CRITICAL\") | \"INFO\"            |\n| `LOG_FORMAT`                       | Format of the log messages                                                                | any valid log message format                     | \\*see below\\*     |\n| `LOG_DATE_FORMAT`                  | Format of the date in log messages                                                        | any valid date format                            | \"%d.%m. %H:%M:%S\" |\n| `LOG_ROTATION_WHEN`                | Specifies when log rotation should occur                                                  | \"S\", \"M\", \"H\", \"D\", \"W0\"-\"W6\", \"midnight\"        | \"H\"               |\n| `LOG_ROTATION_INTERVAL`            | Interval at which log rotation should occur                                               | any int                                          | 2                 |\n| `LOG_ROTATION_BACKUP_COUNT`        | Number of backup log files to keep                                                        | any int                                          | 48                |\n| `AUTHORIZED_MAILS`                 | Mail-addresses which are authorized to access special routes (whitespace separated)       | any int                                          | 48                |\n\nThe log format is: `\"[{asctime}] [{levelname}][{processName}][{threadName}][{module}.{funcName}] {message}\"`, using `{` as format specifier.\nAll logging parameters follow pythons [logging](https://docs.python.org/3/library/logging.html) and the [RotatingFileHandler](https://docs.python.org/3/library/logging.handlers.html#timedrotatingfilehandler) specification.\n\n#### LOG_AUTHORIZED_MAILS\n\nThe API provides a `/logs` route. That route provides all logs for download.\nThe verification is done based on the `'X-Email'` field in the request headers.\nA valid input would be: `LOG_AUTHORIZED_MAILS=\"nik@example.com chris@example.com\"`.\nRequests from localhost are currently always permitted (want an env-option to disable it? - make an issue).\n\nOther privileged routes may come in the future.\n\n#### Note\n\nThe system will automatically try to use the GPU and the best possible model when `USE_GPU_IF_AVAILABLE` and `MAX_MODEL` are not set.\n\n###### CPU Mode\n\n`MAX_MODEL` must be set when CUDA is not available or explicitly disabled via `USE_GPU_IF_AVAILABLE`.\n`CPU_FALLBACK_MODEL` is the fallback when GPU Mode shall use max-model but CPU shall be limited due to reduced performance.\n\n##### Warning\n\nIf `UNLOAD_MODEL_AFTER_S` is set to `0` the model will not only be unloaded nearly instantly, it internally also results in busy waiting!\nAll ints are assumed to be unsigned.\n\n```bash\n# enable development mode -\u003e use small models\nexport DEVELOP_MODE=1\n```\n\n## Projects being used\n\n* [OpenAI Whisper](https://github.com/openai/whisper)\n* [PyTorch](https://pytorch.org/)\n* [FastAPI](https://fastapi.tiangolo.com/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmayniklas%2Fwhisper_api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmayniklas%2Fwhisper_api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmayniklas%2Fwhisper_api/lists"}