{"id":18035598,"url":"https://github.com/adesoji1/youtubesummaryai","last_synced_at":"2025-04-04T23:24:47.966Z","repository":{"id":243024072,"uuid":"807737631","full_name":"Adesoji1/YouTubeSummaryAI","owner":"Adesoji1","description":"Python script for YouTube summary.  The service should summarize an YouTube video by url. It should works for long video and for different languages.","archived":false,"fork":false,"pushed_at":"2024-06-06T08:41:17.000Z","size":21073,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-10T07:45:52.542Z","etag":null,"topics":["cuda","googleapi","python3","speech-recognition","transformers","youtube-api-v3","youtube-dl"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Adesoji1.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-29T17:13:53.000Z","updated_at":"2024-06-13T04:08:05.000Z","dependencies_parsed_at":"2024-06-06T10:09:34.198Z","dependency_job_id":null,"html_url":"https://github.com/Adesoji1/YouTubeSummaryAI","commit_stats":null,"previous_names":["adesoji1/youtubesummaryai"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Adesoji1%2FYouTubeSummaryAI","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Adesoji1%2FYouTubeSummaryAI/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Adesoji1%2FYouTubeSummaryAI/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Adesoji1%2FYouTubeSummaryAI/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Adesoji1","download_url":"https://codeload.github.com/Adesoji1/YouTubeSummaryAI/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247263643,"owners_count":20910443,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cuda","googleapi","python3","speech-recognition","transformers","youtube-api-v3","youtube-dl"],"created_at":"2024-10-30T12:08:51.421Z","updated_at":"2025-04-04T23:24:47.937Z","avatar_url":"https://github.com/Adesoji1.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"export CUDA_LAUNCH_BLOCKING=1\n\n\npython your_script.py\n\n\n# YouTube Video Summarizer\n\nThis repository contains Python scripts to summarize YouTube videos using two different approaches. The scripts extract transcripts from YouTube videos and use pre-trained AI models to generate concise summaries. The resulting summaries are saved in a CSV file along with the corresponding YouTube video URLs.\n\n## Approaches\n\n### Approach 1: YouTube Transcript API and BART Summarization\n\n1. **Fetch Transcript**: Uses the `YouTubeTranscriptApi` to fetch video transcripts.\n2. **Preprocessing**: Cleans the transcript text.\n3. **Summarization**: Uses the BART model (`facebook/bart-large-cnn`) to summarize the cleaned transcript.\n4. **CSV Export**: Saves the summary and video URL to a CSV file.\n\n```python\nimport os\nimport csv\nimport re\nimport torch\nimport time\nfrom dotenv import load_dotenv\nfrom googleapiclient.discovery import build\nfrom transformers import pipeline\nfrom youtube_transcript_api import YouTubeTranscriptApi\n\n# Load environment variables\nload_dotenv()\n\n# Set up YouTube API\nyoutube_api_key = os.getenv('YOUTUBE_API_KEY')\nyoutube = build('youtube', 'v3', developerKey=youtube_api_key)\n\ndef setup_summarizer(retries=3):\n    for attempt in range(retries):\n        try:\n            device = 0 if torch.cuda.is_available() else -1\n            summarizer = pipeline(\"summarization\", model=\"facebook/bart-large-cnn\", device=device)\n            return summarizer\n        except RuntimeError as e:\n            if \"CUDA-capable device(s) is/are busy or unavailable\" in str(e):\n                torch.cuda.empty_cache()\n                time.sleep(10)\n    return pipeline(\"summarization\", model=\"facebook/bart-large-cnn\", device=-1)\n\nsummarizer = setup_summarizer()\n\ndef get_video_id(url):\n    video_id = None\n    if 'youtube.com/watch?v=' in url:\n        video_id = url.split('v=')[1]\n    elif 'youtu.be/' in url:\n        video_id = url.split('youtu.be/')[1]\n    elif 'youtube.com/embed/' in url:\n        video_id = url.split('embed/')[1]\n    else:\n        raise ValueError(\"Invalid YouTube URL format\")\n    \n    if '\u0026' in video_id:\n        video_id = video_id.split('\u0026')[0]\n    \n    return video_id\n\ndef get_transcript(video_id):\n    try:\n        transcript_list = YouTubeTranscriptApi.get_transcript(video_id)\n        transcript = ' '.join([t['text'] for t in transcript_list])\n        return transcript\n    except Exception as e:\n        return None\n\ndef preprocess_text(text):\n    return re.sub(r'[^\\w\\s]', '', text)\n\ndef summarize_text(text):\n    summary = summarizer(text, max_length=200, min_length=60, do_sample=False)\n    return summary[0]['summary_text']\n\ndef summarize_video(url):\n    try:\n        video_id = get_video_id(url)\n    except ValueError as e:\n        return str(e)\n    \n    transcript = get_transcript(video_id)\n    if transcript:\n        cleaned_text = preprocess_text(transcript)\n        summary = summarize_text(cleaned_text)\n        return summary\n    else:\n        return \"No transcript available or error in fetching transcript.\"\n\ndef save_summary_to_csv(video_url, summary):\n    file_exists = os.path.isfile('video_summaries.csv')\n    with open('video_summaries.csv', mode='a', newline='', encoding='utf-8') as file:\n        writer = csv.writer(file)\n        if not file_exists:\n            writer.writerow(['YouTube Link', 'Summary'])\n        writer.writerow([video_url, summary])\n\n# Example usage\ntry:\n    video_url = 'https://youtu.be/rULO-Axqs40'\n    summary = summarize_video(video_url)\n    print(\"Summary:\", summary)\n    save_summary_to_csv(video_url, summary)\nexcept ValueError as e:\n    print(e)\n```\n\n### Approach 2: Audio Transcription and T5 Summarization\n\n1. **Download Audio**: Uses `yt-dlp` to download the audio of a YouTube video.\n2. **Transcription**: Uses the `speech_recognition` library to transcribe the audio.\n3. **Summarization**: Uses the T5 model (`google-t5/t5-small`) to summarize the transcript.\n4. **CSV Export**: Saves the summary and video URL to a CSV file.\n\n```python\nimport re\nimport yt_dlp as youtube_dl\nimport speech_recognition as sr\nfrom transformers import pipeline\nfrom googleapiclient.discovery import build\nfrom time import sleep\nimport torch\nimport pandas as pd\n\nAPI_KEY = 'YOUR_API_KEY'\nyoutube = build('youtube', 'v3', developerKey=API_KEY)\n\ndef get_video_details(video_id):\n    request = youtube.videos().list(part='snippet', id=video_id)\n    response = request.execute()\n    return response['items'][0]['snippet']['title']\n\ndef download_audio(video_url):\n    ydl_opts = {\n        'format': 'bestaudio/best',\n        'postprocessors': [{\n            'key': 'FFmpegExtractAudio',\n            'preferredcodec': 'wav',\n            'preferredquality': '192',\n        }],\n        'outtmpl': 'downloaded_audio.%(ext)s',\n        'quiet': True,\n        'cookiefile': 'cookies.txt',\n    }\n    with youtube_dl.YoutubeDL(ydl_opts) as ydl:\n        ydl.download([video_url])\n    return 'downloaded_audio.wav'\n\ndef download_audio_with_retries(video_url, max_retries=3):\n    attempt = 0\n    while attempt \u003c max_retries:\n        try:\n            return download_audio(video_url)\n        except youtube_dl.utils.DownloadError as e:\n            if 'HTTP Error 429' in str(e):\n                sleep(10)\n                attempt += 1\n            else:\n                raise e\n    raise Exception(\"Failed to download audio after retries\")\n\ndef transcribe_audio(audio_file):\n    recognizer = sr.Recognizer()\n    with sr.AudioFile(audio_file) as source:\n        audio_data = recognizer.record(source)\n        try:\n            text = recognizer.recognize_google(audio_data, language='en-US')\n            return text\n        except sr.UnknownValueError:\n            return \"Audio could not be understood\"\n\ndef summarize_text(text):\n    device = 0 if torch.cuda.is_available() else -1\n    summarizer = pipeline(\"summarization\", model=\"google-t5/t5-small\", device=device)\n    summary = summarizer(text, max_length=300, min_length=60, do_sample=False)\n    return summary[0]['summary_text']\n\ndef extract_video_id(video_url):\n    patterns = [\n        r'youtu\\.be\\/([^#\\\u0026\\?]{11})',\n        r'watch\\?v=([^#\\\u0026\\?]{11})',\n        r'embed\\/([^#\\\u0026\\?]{11})'\n    ]\n\n    for pattern in patterns:\n        match = re.search(pattern, video_url)\n        if match:\n            return match.group(1)\n    \n    raise ValueError(\"Invalid YouTube URL or video ID format\")\n\ndef main(video_url):\n    try:\n        video_id = extract_video_id(video_url)\n    except ValueError as e:\n        print(e)\n        return\n\n    title = get_video_details(video_id)\n    audio_file = download_audio_with_retries(video_url)\n    transcript = transcribe_audio(audio_file)\n    summary = summarize_text(transcript)\n    \n    print(\"Video Title:\", title)\n    print(\"Summary:\", summary)\n\n    results = {\n        \"YouTube Link\": [video_url],\n        \"Summary\": [summary]\n    }\n    df = pd.DataFrame(results)\n    df.to_csv('youtube_summaries.csv', index=False)\n\nif __name__ == \"__main__\":\n    video_url = input(\"Enter YouTube video URL: \")\n    main(video_url)\n```\n\n## Future Work\n\n1. **Application Development**: Build a web application or a desktop application that allows users to input YouTube URLs and get summaries directly.\n2. **Language Support**: Enhance the script to support multiple languages for transcription and summarization.\n3. **Batch Processing**: Add functionality to process multiple videos at once.\n4. **Error Handling**: Improve error handling and logging mechanisms for more robust and user-friendly interactions.\n5. **Real-time Summarization**: Explore possibilities for real-time video summarization during live streams.\n6. **User Interface**: Develop a graphical user interface (GUI) for non-technical users.\n7. **Performance Optimization**: Optimize the scripts for faster execution and lower resource consumption.\n\n## Requirements\n\n- Python 3.11+\n- `torch`\n- `transformers`\n- `youtube_transcript_api`\n- `google-api-python-client`\n- `yt-dlp`\n- `speechrecognition`\n- `pandas`\n- `ffmpeg`\n\n## Installation\n\n1. Clone the repository:\n\n```bash\ngit clone https://github.com/yourusername/YouTubeVideoSummarizer.git\ncd YouTubeVideoSummarizer\n```\n\n2. Install the required dependencies:\n\n```bash\npip install -r requirements.txt\n```\n\n3. Set up your environment variables in a `.env` file:\n\n```\nYOUTUBE_API_KEY=YOUR_API_KEY\n```\n\n4. Run the script:\n\n```bash\npython script_name.py\n```\n\nReplace `script_name.py` with the appropriate script you want to run.\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n---\n\nThis README provides an overview of the YouTube video summarization project, details on the two approaches used, and outlines future work for further development and enhancement\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadesoji1%2Fyoutubesummaryai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadesoji1%2Fyoutubesummaryai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadesoji1%2Fyoutubesummaryai/lists"}