{"id":16780176,"url":"https://github.com/dev-khant/tell-what-a-video-does","last_synced_at":"2025-06-16T19:09:16.305Z","repository":{"id":188056115,"uuid":"677344447","full_name":"Dev-Khant/tell-what-a-video-does","owner":"Dev-Khant","description":"Know what a youtube video is about with the help of LLMs.","archived":false,"fork":false,"pushed_at":"2023-09-15T13:11:15.000Z","size":30,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-08T03:43:24.369Z","etag":null,"topics":["huggingface","llm","streamlit"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Dev-Khant.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-11T10:37:55.000Z","updated_at":"2024-12-25T10:00:50.000Z","dependencies_parsed_at":null,"dependency_job_id":"6a48aacc-9283-4e8d-8f03-703463299b42","html_url":"https://github.com/Dev-Khant/tell-what-a-video-does","commit_stats":null,"previous_names":["dev-khant/tell-what-a-video-does"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Dev-Khant/tell-what-a-video-does","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dev-Khant%2Ftell-what-a-video-does","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dev-Khant%2Ftell-what-a-video-does/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dev-Khant%2Ftell-what-a-video-does/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dev-Khant%2Ftell-what-a-video-does/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Dev-Khant","download_url":"https://codeload.github.com/Dev-Khant/tell-what-a-video-does/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dev-Khant%2Ftell-what-a-video-does/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260221556,"owners_count":22976867,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["huggingface","llm","streamlit"],"created_at":"2024-10-13T07:34:17.307Z","updated_at":"2025-06-16T19:09:16.287Z","avatar_url":"https://github.com/Dev-Khant.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Video Understanding and Q\u0026A Tool\n\nThis project allows you to input a YouTube video link, and it provides a comprehensive understanding of the video's content through audio transcription and image captioning. **LLM** is used to combine audio and video context. Additionally, you can ask questions and it will provide responses according to video content 🚀\n\n## Features ✨\n\n👉 **Video Understanding**: The tool utilizes the **Transformer** model for audio transcription, converting spoken words into textual format. It also employs image captioning techniques to extract text from images within the video. **Image embeddings** are also used to compare images and only use images unique for extracting info. Video and Audio are processed **parallelly**.\n\n👉 **Question \u0026 Answer**: Users can ask questions about the video's content. The tool leverages the power of **Chromadb** as a vector database to provide accurate and contextually relevant answers.\n\n## How to Use ⚙️\n\n• Clone this repository: `git clone https://github.com/Dev-Khant/tell-what-a-video-does.git`\n\n• Install the required dependencies: `pip install -r requirements.txt`\n\n• Run the streamlit app: `streamlit run app.py`\n\n• Provide **YouTube video** with your **OpenAI token**, **Huggingface token**, **SerpAPI token**\n\n## Technical 🖥️\n\n• [Hugging Face](https://huggingface.co/): Utilized to access the OpenAI Whisper model for audio transcription.\n\n• [SerpApi](https://serpapi.com/): Used it to access Google Lens API for getting image information.\n\n• [Streamlit](https://streamlit.io/): Used to create the interactive web interface for the project.\n\n• [Chromadb](https://www.trychroma.com/): The vector database used for storing and retrieving Q\u0026A information.\n\n## Work in Progress 🚧\n\n1. Add Weaviate and let the user select their VectorDB.\n2. Internet access to chatbot.\n3. Option to upload video.\n4. Store video explanations so they can be used later.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdev-khant%2Ftell-what-a-video-does","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdev-khant%2Ftell-what-a-video-does","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdev-khant%2Ftell-what-a-video-does/lists"}