{"id":31214344,"url":"https://github.com/justcodeit7/ytdl_summarizer","last_synced_at":"2025-09-21T09:34:56.422Z","repository":{"id":314076482,"uuid":"1054000408","full_name":"JustCodeIt7/YTDL_Summarizer","owner":"JustCodeIt7","description":null,"archived":false,"fork":false,"pushed_at":"2025-09-10T10:32:50.000Z","size":44,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-10T13:22:54.427Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JustCodeIt7.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-10T08:20:08.000Z","updated_at":"2025-09-10T10:32:54.000Z","dependencies_parsed_at":"2025-09-10T13:27:10.384Z","dependency_job_id":"eae09d8f-3603-47ef-bab0-464fdde9ba86","html_url":"https://github.com/JustCodeIt7/YTDL_Summarizer","commit_stats":null,"previous_names":["justcodeit7/ytdl_summarizer"],"tags_count":null,"template":false,"template_full_name":"JustCodeIt7/Python_Template","purl":"pkg:github/JustCodeIt7/YTDL_Summarizer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JustCodeIt7%2FYTDL_Summarizer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JustCodeIt7%2FYTDL_Summarizer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JustCodeIt7%2FYTDL_Summarizer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JustCodeIt7%2FYTDL_Summarizer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JustCodeIt7","download_url":"https://codeload.github.com/JustCodeIt7/YTDL_Summarizer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JustCodeIt7%2FYTDL_Summarizer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":276223192,"owners_count":25605793,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-21T02:00:07.055Z","response_time":72,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-09-21T09:34:53.092Z","updated_at":"2025-09-21T09:34:56.417Z","avatar_url":"https://github.com/JustCodeIt7.png","language":"Python","readme":"# YouTube Video Summarizer with Chapters (CLI + Web)\n\nA Python tool to:\n\n- Download a YouTube video's audio\n- Transcribe it with Whisper (local via faster-whisper or via OpenAI Whisper API)\n- Detect topical shifts to create timestamped chapters\n- Generate an executive summary (200–300 words) and key takeaways using OpenAI via LangChain\n- Export JSON and Markdown\n- Use as a CLI or a Streamlit web app\n\n## Stack\n\n- Python\n- yt-dlp for audio download (requires `ffmpeg` installed)\n- Transcription:\n  - Local: faster-whisper (preferred for speed and accuracy)\n  - API: OpenAI Whisper API\n- LangChain + OpenAI for LLM summarization and titling\n- Streamlit for simple web UI\n\n## Features\n\n- Input: YouTube URL\n- Download audio, transcribe into segments with timestamps\n- Generate:\n  - Executive summary (200–300 words)\n  - 5–12 chapter ranges with titles and timestamps (topic-shift detection on transcript windows)\n  - Bullet-point key takeaways\n- Export JSON and Markdown; display in web UI\n- Constraints handled:\n  - Long videos batched for summarization (map-reduce over transcript chunks)\n  - Model choice (local `faster-whisper` vs. OpenAI Whisper API)\n  - Automatic language detection (from Whisper)\n- Acceptance criteria:\n  - CLI: `python summarize.py \u003cyoutube_url\u003e`\n  - Web: `/summarize?url=...` shows summary and chapters (Streamlit accepts path segments; query param triggers auto-run)\n  - Accurate timestamps aligned to transcript segments/windows\n\n## Installation\n\n1. System requirement: ffmpeg\n\n- macOS (brew): `brew install ffmpeg`\n- Ubuntu/Debian: `sudo apt-get install -y ffmpeg`\n- Windows: Install from https://ffmpeg.org/download.html and add to PATH\n\n2. Python environment\n\n```bash\npython -m venv .venv\nsource .venv/bin/activate  # on Windows: .venv\\Scripts\\activate\npip install -r requirements.txt\n```\n\n3. OpenAI credentials\n\n- Set your API key in the environment:\n  - macOS/Linux: `export OPENAI_API_KEY=your_key`\n  - Windows (PowerShell): `$Env:OPENAI_API_KEY = \"your_key\"`\n\nOptionally set your preferred chat model:\n\n```bash\nexport OPENAI_LLM_MODEL=gpt-4o-mini\n```\n\n## Usage\n\n### CLI\n\n```bash\npython summarize.py \"https://www.youtube.com/watch?v=VIDEO_ID\"\n```\n\nOptional flags:\n\n- `--transcriber {faster-whisper|openai}` (default: faster-whisper)\n- `--whisper-model small` (for faster-whisper: tiny/base/small/medium/large-v3 or a local path)\n- `--device {auto|cpu|cuda}` and `--compute-type {auto|int8|int8_float16|float16|float32}`\n- `--llm-model gpt-4o-mini` (OpenAI chat model)\n- `--min-chapters 6 --max-chapters 10`\n- `--language \u003clang_code\u003e` to force language (e.g., `en`, `es`) if needed\n- `--outdir outputs`\n- `--keep-audio` to keep the downloaded audio file\n\nOutputs are saved under `outputs/\u003cvideo-slug\u003e/summary.json` and `summary.md`.\n\n### Web (Streamlit)\n\n```bash\nstreamlit run streamlit_app.py\n```\n\nOpen:\n\n- http://localhost:8501/\n- For auto-run via URL: http://localhost:8501/summarize?url=https://www.youtube.com/watch?v=VIDEO_ID\n\nNotes:\n\n- Streamlit serves one app at a single route; `/summarize` is accepted as a path segment and the app reads the `url` query param to auto-run.\n\n## How it works\n\n1. Download audio with `yt-dlp` (m4a) and extract metadata such as title, duration, and video ID.\n\n2. Transcribe:\n\n- Local (default): `faster-whisper` with VAD filtering; returns segments with timestamps and detected language.\n- OpenAI Whisper API: Uses verbose JSON to retrieve segments with timestamps.\n\n3. Chapter detection:\n\n- Transcript segments are merged into ~30s windows to smooth noise.\n- TF-IDF vectors measure cosine similarity between adjacent windows.\n- Low-similarity points become boundaries indicating topical shifts.\n- The algorithm picks top-k boundaries to produce 5–12 chapters, ensuring spacing and segment alignment.\n- Chapter start/end timestamps align with nearest transcript windows.\n\n4. Summarization:\n\n- Map-reduce over transcript chunks with LangChain + OpenAI:\n  - Map: chunk summaries (bulleted)\n  - Reduce: cohesive 200–300 word executive summary\n- Key takeaways: extract from chunks then consolidate to 6–10 bullets.\n- Chapter titling: LLM generates a concise title for each chapter based on its text.\n\n5. Export:\n\n- JSON: metadata, transcript segments, chapters, summary, and key takeaways.\n- Markdown: nicely formatted report with deep links to YouTube timestamps.\n\n## JSON Schema (high-level)\n\n```json\n{\n  \"metadata\": {\n    \"video_url\": \"...\",\n    \"title\": \"...\",\n    \"uploader\": \"...\",\n    \"video_id\": \"...\",\n    \"duration\": 1234,\n    \"watch_base_url\": \"https://www.youtube.com/watch?v=ID\"\n  },\n  \"transcript\": {\n    \"language\": \"en\",\n    \"segments\": [\n      {\n        \"start\": 0.0,\n        \"end\": 4.2,\n        \"start_timecode\": \"00:00:00\",\n        \"end_timecode\": \"00:00:04\",\n        \"text\": \"...\"\n      }\n    ]\n  },\n  \"summary\": {\n    \"executive_summary\": \"...\",\n    \"key_takeaways\": [\"...\", \"...\"]\n  },\n  \"chapters\": [\n    {\n      \"idx\": 0,\n      \"title\": \"Intro\",\n      \"start\": 0.0,\n      \"end\": 120.5,\n      \"start_timecode\": \"00:00:00\",\n      \"end_timecode\": \"00:02:00\",\n      \"url\": \"https://www.youtube.com/watch?v=ID\u0026t=0s\"\n    }\n  ],\n  \"artifacts\": {\n    \"audio_path\": \"/path/to/file.m4a\"\n  }\n}\n```\n\n## Notes and Tips\n\n- Large videos: The tool chunks transcripts to keep LLM context under limits; it then synthesizes a final summary and key takeaways.\n- Faster-whisper models: choose `tiny`/`base` for speed, `small`/`medium` for balance, `large-v3` for best accuracy (slower).\n- GPU: If you have CUDA, set `--device cuda` for faster-whisper.\n- Timestamps: Chapter start times link directly to `\u0026t=SECONDS` on the YouTube watch URL.\n- If you see ffmpeg errors, ensure ffmpeg is installed and in PATH.\n\n## License\n\nMIT\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjustcodeit7%2Fytdl_summarizer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjustcodeit7%2Fytdl_summarizer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjustcodeit7%2Fytdl_summarizer/lists"}