{"id":34847552,"url":"https://github.com/roshaan0/ai-audio-transcriber","last_synced_at":"2026-04-16T05:04:44.880Z","repository":{"id":329581763,"uuid":"1120089802","full_name":"roshaan0/ai-audio-transcriber","owner":"roshaan0","description":"End-to-end AI audio transcription, speaker diarization, and summarization pipeline with Urdu/English support using fine-tuned Whisper (LoRA).","archived":false,"fork":false,"pushed_at":"2025-12-20T14:05:37.000Z","size":15,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-12-22T17:18:36.296Z","etag":null,"topics":["ai","audio-processing","bart","machine-learning","nlp","pytorch","speech-to-text","streamlit","whisper"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/roshaan0.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-20T13:19:02.000Z","updated_at":"2025-12-20T14:05:41.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/roshaan0/ai-audio-transcriber","commit_stats":null,"previous_names":["roshaan0/ai-audio-transcriber"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/roshaan0/ai-audio-transcriber","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roshaan0%2Fai-audio-transcriber","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roshaan0%2Fai-audio-transcriber/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roshaan0%2Fai-audio-transcriber/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roshaan0%2Fai-audio-transcriber/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/roshaan0","download_url":"https://codeload.github.com/roshaan0/ai-audio-transcriber/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roshaan0%2Fai-audio-transcriber/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31872036,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-15T15:24:51.572Z","status":"online","status_checked_at":"2026-04-16T02:00:06.042Z","response_time":69,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","audio-processing","bart","machine-learning","nlp","pytorch","speech-to-text","streamlit","whisper"],"created_at":"2025-12-25T18:52:01.915Z","updated_at":"2026-04-16T05:04:44.868Z","avatar_url":"https://github.com/roshaan0.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AI Audio Transcriber \u0026 Analyzer (Urdu / English)\n\nAn end-to-end AI application for transcribing, diarizing, and summarizing mixed-language (Urdu/English) audio and video files.  \nThe system is optimized to run locally on consumer hardware and includes a fine-tuned Whisper model using LoRA to improve Urdu script accuracy.\n\n---\n\n## Overview\n\nAutomatic speech recognition systems often struggle with low-resource languages such as Urdu, frequently misclassifying them as Hindi or producing incorrect scripts.  \nThis project addresses that limitation by fine-tuning OpenAI Whisper using parameter-efficient techniques and integrating it into a full-stack AI pipeline.\n\nThe application supports speaker identification, mixed-language transcription, and automatic summarization through a unified web interface.\n\n---\n\n## Features\n\n- Audio and video upload support (mp4, wav, mkv)\n- Speaker diarization (identifying who spoke when)\n- Speech-to-text transcription using OpenAI Whisper\n- Mixed-language handling for Urdu and English\n- Correct Urdu Nastaliq script output\n- Automatic text summarization using BART\n- Interactive web interface built with Streamlit\n- Fully local execution on consumer GPUs (tested on RTX 4050)\n\n---\n\n## System Architecture\n\nAudio / Video Input\n-\u003e\nSpeaker Diarization (Pyannote)\n-\u003e\nSpeech Transcription (Whisper + LoRA Fine-Tuning)\n-\u003e\nText Summarization (BART)\n-\u003e\nStreamlit Web Interface\n\n\n---\n## Technology Stack\n\n### Programming Language\n- Python 3.10+\n\n### Frameworks and Libraries\n- PyTorch\n- Hugging Face Transformers\n- PEFT (LoRA)\n- Pyannote.audio\n- Streamlit\n- FFmpeg\n- Pydub\n\n---\n\n## Models Used\n\n### Transcription Model\n- Model: openai/whisper-small  \n- Architecture: Transformer-based encoder-decoder  \n- Customization: Fine-tuned using Low-Rank Adaptation (LoRA)  \n- Objective: Improve Urdu transcription accuracy and reduce Hindi misclassification\n\n### Speaker Diarization Model\n- Model: pyannote/speaker-diarization-3.1  \n- Purpose: Detect speaker boundaries and assign speaker labels\n\n### Summarization Model\n- Model: facebook/bart-large-cnn  \n- Purpose: Generate concise summaries from long transcriptions\n\n---\n\n## Installation and Setup\n\n### 1. Clone the Repository\n\ngit clone https://github.com/roshaan0/ai-audio-transcriber.git\ncd ai-audio-transcriber\n\n### 2. Install Dependencies\npip install -r requirements.txt\n\n### 3. Hugging Face Token Configuration\n\nThis project uses gated Hugging Face models (Pyannote).\nA Hugging Face read-access token is required.\n\nSteps\n\nCreate a Hugging Face account.\n\nGenerate a read-access token.\n\nCreate a file named hf_token.txt in the project root.\n\nPaste your token inside the file.\n\n### 4. Run the Application\nstreamlit run app.py\n\n## Accuracy and Performance Notes\n\nThis project is a proof-of-concept focused on efficiency rather than maximum accuracy.  \nApproximately 65 percent transcription accuracy was achieved using limited training data and consumer-grade hardware.\n\nThe architecture is fully scalable and supports higher accuracy with additional data and compute resources.\n\n---\n\n## Project Scope\n\nThis project demonstrates full-stack AI engineering, including:\n\n- Model fine-tuning workflows\n- Backend pipeline integration\n- Audio preprocessing\n- Web-based frontend development\n\nThe project was developed for academic and research purposes.\n\n---\n\n## Author\n\nRoshaan Ali  \nAI and Machine Learning Enthusiast\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Froshaan0%2Fai-audio-transcriber","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Froshaan0%2Fai-audio-transcriber","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Froshaan0%2Fai-audio-transcriber/lists"}