{"id":13910596,"url":"https://github.com/ahmedbesbes/audiolizr","last_synced_at":"2026-03-04T11:03:43.032Z","repository":{"id":64918412,"uuid":"574674641","full_name":"ahmedbesbes/audiolizr","owner":"ahmedbesbes","description":"A bentoML-powered API to transcribe audio and make sense of it","archived":false,"fork":false,"pushed_at":"2022-12-21T08:35:16.000Z","size":11096,"stargazers_count":39,"open_issues_count":2,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-08T13:12:18.863Z","etag":null,"topics":["bentoml","bentoml-service","docker","nlp","openai","openai-whisper","pytube","speech-recognition","t5","torch","transformers"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ahmedbesbes.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-12-05T20:44:58.000Z","updated_at":"2025-03-05T21:29:55.000Z","dependencies_parsed_at":"2023-01-30T03:01:03.493Z","dependency_job_id":null,"html_url":"https://github.com/ahmedbesbes/audiolizr","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ahmedbesbes/audiolizr","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahmedbesbes%2Faudiolizr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahmedbesbes%2Faudiolizr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahmedbesbes%2Faudiolizr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahmedbesbes%2Faudiolizr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ahmedbesbes","download_url":"https://codeload.github.com/ahmedbesbes/audiolizr/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahmedbesbes%2Faudiolizr/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30078421,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-04T08:01:56.766Z","status":"ssl_error","status_checked_at":"2026-03-04T08:00:42.919Z","response_time":59,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bentoml","bentoml-service","docker","nlp","openai","openai-whisper","pytube","speech-recognition","t5","torch","transformers"],"created_at":"2024-08-07T00:01:36.404Z","updated_at":"2026-03-04T11:03:43.004Z","avatar_url":"https://github.com/ahmedbesbes.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# audiolizr\n\nAudiolizr (***audio*** and ***analyzer***) is an API built and deployed with BentoML to transcribe Youtube videos and extract the following metadata: \n\n- keywords and topics using the Yake algorithm\n- a generated summary using the T5 Transformer model\n- named entities (people's name, locations, products, organizations, etc.) using spaCy\n\nThis API can be used to provide a summary and additional information to understand any youtube video or audio content.  \n\nThis service is deployed on AWS EC2 on a GPU-powered [g4dn.xlarge](https://aws.amazon.com/fr/ec2/instance-types/g4/) instance. (see deployment section for details)\n\n\n\u003cimg src=\"./images/audiolizr.png\"\u003e\n\n\n### Demo\n\n\nI've used audiolizr to process this (interesting) TEDx short video\n\n[![IMAGE ALT TEXT HERE](https://img.youtube.com/vi/cyZYzTk37d8/0.jpg)](https://www.youtube.com/watch?v=cyZYzTk37d8)\n\nHere a demo of audiolizr in Streamlit.\n\nhttps://user-images.githubusercontent.com/6267065/208546956-a9e97789-2933-40b8-8081-db585b5f5c4d.mov\n\nAs shown in the following diagram, the video moves through different runners to\n\n1. download the audio\n2. transcribe the audio into text\n3. extract keywords\n4. extract named entities\n5. summarize the text\n\nnote: runners 1 and 2 are executed sequentially and runners 3, 4 and 5 are executed concurrently\n\n\u003cimg src=\"./images/demo.png\"\u003e\n\nHere's the JSON output that you'd get at the end of the pipeline:\n\n```json\n{\n  \"transcript\": \"How much do you get paid? Don't answer that out loud. But put a number in your head. Now, how much do you think the person sitting next to you gets paid? It turns out that pay transparency, sharing salaries openly across a company, makes for a better workplace for both the employee and for the organization. You see, keeping salary secret leads to what economists call information asymmetry. This is a situation where in a negotiation, one party has loads more information than the other. And in hiring or promotion or annual raise discussions, an employer can use that secrecy to save a lot of money. Imagine how much better you could negotiate for a raise if you knew everybody's salary. Now, I realized that letting people know what you make might feel uncomfortable, but isn't it less uncomfortable than always wondering if you're being discriminated against, or if your wife or your daughter or your sister is being paid unfairly? Openness remains the best way to ensure fairness. And pay transparency does that.\",\n  \"metadata\": {\n    \"keywords\": [\n      [\n        \"pay transparency\",\n        0.19212871128874295\n      ],\n      [\n        \"information\",\n        0.27056834789491807\n      ],\n      [\n        \"salary\",\n        0.27854871121244107\n      ],\n      [\n        \"raise\",\n        0.2849695469393418\n      ],\n      [\n        \"uncomfortable\",\n        0.2997431150212997\n      ],\n      [\n        \"paid unfairly\",\n        0.3407466612452491\n      ],\n      [\n        \"information asymmetry\",\n        0.4356297000199267\n      ],\n      [\n        \"call information asymmetry\",\n        0.4715329551423117\n      ],\n      [\n        \"sharing salaries openly\",\n        0.47429797071055785\n      ],\n      [\n        \"call information\",\n        0.49492941874515994\n      ]\n    ],\n    \"entities\": [\n      {\n        \"entity_text\": \"one\",\n        \"entity_label\": \"CARDINAL\",\n        \"start\": 438,\n        \"end\": 441\n      },\n      {\n        \"entity_text\": \"annual\",\n        \"entity_label\": \"DATE\",\n        \"start\": 521,\n        \"end\": 527\n      }\n    ],\n    \"summary\": \"If you know everybody's salary, you can save a lot of money. And in hiring or promotion or annual raise discussions, an employer can use that secrecy to save money. Pay transparency, sharing salaries openly across companies, makes for better workplaces for both the employee and for the organization. I realized that letting people know what you make might feel uncomfortable, but isn't it less uncomfortable than always wondering whether your wife or your daughter is being paid unfairly?\"\n  }\n}\n```\n### Dependencies\n\n- [pytube](https://github.com/pytube/pytube) \n- [whisper](https://github.com/openai/whisper) (from OpenAI)\n- [Yake](https://github.com/LIAAD/yake) \n- [spaCy](https://github.com/explosion/spaCy)\n- [transformers](https://github.com/huggingface/transformers)\n\n### Run locally\n\nRun the following commands to start a fresh environment with the needed dependencies:\n\n```\ncd audiolizr/\npipenv install \npipenv shell\n\n# install whisper with pip\npip install git+https://github.com/openai/whisper.git\n# install spacy language model\npython -m spacy download en_core_web_md \n```\n\nTo serve the API locally, run the following command. \n\n```\ncd src/\nbentoml serve service:svc --reload\n```\n\nTo serve the API in production mode (and enable multiple api workers), run the following command (keep `--api-workers` low to avoid hammering the RAM)\n\n```\ncd src/\nbentoml serve service:svc --production --api-workers 2\n```\n\nIf everything works as expected, build the bento to prepare the deployment:\n\n```\ncd src/\nbentoml build\n```\n\nHere's what you'll see when it's done:\n\n```\n[nltk_data] Downloading package punkt to\n[nltk_data]     /Users/ahmedbesbes/nltk_data...\n[nltk_data]   Package punkt is already up-to-date!\nBuilding BentoML service \"speech_to_text_pipeline:m57a6etzlg4imhqa\" from build context \"/Users/ahmedbesbes/Documents/perso/whisper/src\".\n\n██████╗░███████╗███╗░░██╗████████╗░█████╗░███╗░░░███╗██╗░░░░░\n██╔══██╗██╔════╝████╗░██║╚══██╔══╝██╔══██╗████╗░████║██║░░░░░\n██████╦╝█████╗░░██╔██╗██║░░░██║░░░██║░░██║██╔████╔██║██║░░░░░\n██╔══██╗██╔══╝░░██║╚████║░░░██║░░░██║░░██║██║╚██╔╝██║██║░░░░░\n██████╦╝███████╗██║░╚███║░░░██║░░░╚█████╔╝██║░╚═╝░██║███████╗\n╚═════╝░╚══════╝╚═╝░░╚══╝░░░╚═╝░░░░╚════╝░╚═╝░░░░░╚═╝╚══════╝\n\nSuccessfully built Bento(tag=\"speech_to_text_pipeline:m57a6etzlg4imhqa\").\n```\n\nWhen a bento is created, build a Docker image from it with this command:\n\n```shell\nbentoml containerize speech_to_text_pipeline:m57a6etzlg4imhqa\n```\n\nThis will run multiple steps to build the docker image:\n\n```shell\nBuilding OCI-compliant image for speech_to_text_pipeline:m57a6etzlg4imhqa with docker\n\n[+] Building 30.7s (20/20) FINISHED\n =\u003e [internal] load build definition from Dockerfile                                                                                                                         0.0s\n =\u003e =\u003e transferring dockerfile: 2.52kB                                                                                                                                       0.0s\n =\u003e [internal] load .dockerignore                                                                                                                                            0.0s\n =\u003e =\u003e transferring context: 2B                                                                                                                                              0.0s\n =\u003e [internal] load metadata for docker.io/nvidia/cuda:11.6.2-cudnn8-runtime-ubuntu20.04                                                                                     1.1s\n =\u003e [base-container  1/15] FROM docker.io/nvidia/cuda:11.6.2-cudnn8-runtime-ubuntu20.04@sha256:812fe80b7123467f5d6c746bd5d7cbd3b96f385c3c6a57a532b21617ad433858              0.0s\n =\u003e [internal] load build context                                                                                                                                            0.0s\n =\u003e =\u003e transferring context: 24.82kB                                                                                                                                         0.0s\n =\u003e CACHED [base-container  2/15] RUN rm -f /etc/apt/apt.conf.d/docker-clean; echo 'Binary::apt::APT::Keep-Downloaded-Packages \"true\";' \u003e /etc/apt/apt.conf.d/keep-cache     0.0s\n =\u003e CACHED [base-container  3/15] RUN --mount=type=cache,target=/var/lib/apt --mount=type=cache,target=/var/cache/apt set -eux \u0026\u0026     apt-get update -y \u0026\u0026     apt-get inst  0.0s\n =\u003e CACHED [base-container  4/15] RUN --mount=type=cache,target=/var/lib/apt --mount=type=cache,target=/var/cache/apt     set -eux \u0026\u0026     apt-get install -y --no-install-r  0.0s\n =\u003e CACHED [base-container  5/15] RUN ln -sf /usr/bin/python3.9 /usr/bin/python3 \u0026\u0026     ln -sf /usr/bin/pip3.9 /usr/bin/pip3                                                 0.0s\n =\u003e CACHED [base-container  6/15] RUN curl -O https://bootstrap.pypa.io/get-pip.py \u0026\u0026     python3 get-pip.py \u0026\u0026     rm -rf get-pip.py                                        0.0s\n =\u003e CACHED [base-container  7/15] RUN groupadd -g 1034 -o bentoml \u0026\u0026 useradd -m -u 1034 -g 1034 -o -r bentoml                                                                0.0s\n =\u003e CACHED [base-container  8/15] RUN mkdir /home/bentoml/bento \u0026\u0026 chown bentoml:bentoml /home/bentoml/bento -R                                                              0.0s\n =\u003e CACHED [base-container  9/15] WORKDIR /home/bentoml/bento                                                                                                                0.0s\n =\u003e [base-container 10/15] COPY --chown=bentoml:bentoml . ./                                                                                                                 0.0s\n =\u003e [base-container 11/15] RUN --mount=type=cache,target=/root/.cache/pip bash -euxo pipefail /home/bentoml/bento/env/python/install.sh                                     20.6s\n =\u003e [base-container 12/15] RUN chmod +x /home/bentoml/bento/env/docker/setup_script                                                                                          0.2s\n =\u003e [base-container 13/15] RUN /home/bentoml/bento/env/docker/setup_script                                                                                                   6.1s\n =\u003e [base-container 14/15] RUN rm -rf /var/lib/{apt,cache,log}                                                                                                               0.2s\n =\u003e [base-container 15/15] RUN chmod +x /home/bentoml/bento/env/docker/entrypoint.sh                                                                                         0.2s\n =\u003e exporting to image                                                                                                                                                       2.2s\n =\u003e =\u003e exporting layers                                                                                                                                                      2.2s\n =\u003e =\u003e writing image sha256:a08cdb9a3818379359500fe78c0af72a38516cde471e8f6ca1c26e177a7f99c1                                                                                 0.0s\n =\u003e =\u003e naming to docker.io/library/speech_to_text_pipeline:m57a6etzlg4imhqa                                                                                                  0.0s\nSuccessfully built Bento container for \"speech_to_text_pipeline\" with tag(s) \"speech_to_text_pipeline:m57a6etzlg4imhqa\"\nTo run your newly built Bento container, use 'speech_to_text_pipeline:m57a6etzlg4imhqa' as a tag and pass it to 'docker run'. For example:\n    docker run -it --rm -p 3000:3000 speech_to_text_pipeline:m57a6etzlg4imhqa serve --production\n```\n\nRun the API from the Docker image (with port forward) and check that everything's running fine from the host\n\n```\ndocker run -it --rm -p 3000:3000 speech_to_text_pipeline:m57a6etzlg4imhqa serve --production --api-workers 2\n```\n\nHead over http://localhost:3000 to try out the API\n\n### Deploy to EC2\n\nTo deploy the API on AWS, you need to follow these steps:\n\n**1. Install the required tools**\n\n- Install the AWS CLI and configure an AWS account (see AMI) with granted permissions to Cloudformation, Lambda, API Gateway and ECR. Here's the installation [guide](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)\n- install bentoctl : `pip install bentoctl` (already installed in the pipenv environment)\n- install Terraform. This is a tool for building, configuring and managing infrastructure with code: follow these [instructions](https://developer.hashicorp.com/terraform/downloads?product_intent=terraform) depending on your OS\n\n\n**2. Setup the deployment config**\n\nTo deploy the BentoML service on AWS, we will use bentoctl, a tool that helps deploy any machine learning model on any cloud infrastructure. \n\nbentoctl uses Terraform under the hood. \n\nSince we're going to deploy the service on EC2, we first need to install the AWS EC2 operator to generate and apply the Terraform files\n\n```shell\nbentoctl operator install aws-ec2\n```\n\nThe deployment configuration must be detailed in a `deployment_config.yaml` file (this one is contained in the `bentoctl` folder)\n\nThis file contains details about the instance type, the AMI ID, the region, etc.\n\n```yaml\napi_version: v1\nname: audiolizr-bentoml\noperator:\n  name: aws-ec2\ntemplate: terraform\nspec:\n  region: eu-west-3\n  instance_type: g4dn.xlarge\n  # points to Deep Learning AMI GPU PyTorch 1.12.0 (Ubuntu 20.04) 20220913 AMI\n  ami_id: ami-0aed2a32a2ea85e67\n  enable_gpus: true\n```\n\n\n**3. Generate the Terraform files**\n\nSimply run\n\n```shell\nbentoctl generate -f deployment_config.yaml\n```\n\nThis will generates the main.tf and bentoctl.tfvars\n\n\n\u003cimg src=\"./images/tfvars.png\"\u003e\n\n**4. Build the Docker image and push it to ECR**\n\nThe image upload may take time depending on your bandwith\n\n```shell\nbentoctl build -b speech_to_text_pipeline:dzunp5dzo2uhmhqa\n```\n\u003cimg src=\"./images/docker-build.png\"\u003e\n\n**5. Apply the Terraform file to deploy to AWS EC2**\n\nIn this step, we apply the Terraform file to deploy the docker image on an EC2.\n\nThis step fires up an instance and configures all the needed installation\n\n\u003cimg src=\"./images/deployment_1.png\"\u003e\n\nWhen everything is up and running (~ takes approximatively 10 minutes), you will be prompted with a your API URL. \n\n\u003cimg src=\"./images/deployment_2.png\"\u003e \n\n🎉 !\n\n\n**6. Test the API**\n\nOnce the API is deployed, you can try it out either from the browser or via a python client.\n\nLet's query it from an ipython terminal and compare the inference time with my locally running smae API (with no GPU)\n\nBy upload an audo file \n\n- local: 15.9s\n- aws: 4.57s **(+3 times faster)**\n\n\u003cimg src=\"./images/file_endpoint.png\"\u003e \n\n\nBy downloading a youtube video\n\n- local: 22.9s\n- aws: 10.7s (**~2 times faster**)\n\n\u003cimg src=\"./images/youtube_endpoint.png\"\u003e \n\n7. Remove the service when you don't need it anymore\n\n```shell\nbentoctl destroy -f deployment_config.yaml\n```\n\n\u003cimg src=\"./images/destroy_1.png\"\u003e \n\u003cimg src=\"./images/destroy_2.png\"\u003e \n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fahmedbesbes%2Faudiolizr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fahmedbesbes%2Faudiolizr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fahmedbesbes%2Faudiolizr/lists"}