{"id":49702980,"url":"https://github.com/martinopiaggi/summarize","last_synced_at":"2026-05-24T21:00:44.424Z","repository":{"id":235513202,"uuid":"705280147","full_name":"martinopiaggi/summarize","owner":"martinopiaggi","description":"Video transcript AI summarization from multiple sources (YouTube, X, Instagram, TikTok, Reddit, Facebook, Google Drive, Dropbox, and local files).","archived":false,"fork":false,"pushed_at":"2026-05-17T14:49:46.000Z","size":1212,"stargazers_count":178,"open_issues_count":2,"forks_count":25,"subscribers_count":4,"default_branch":"main","last_synced_at":"2026-05-17T16:50:51.229Z","etag":null,"topics":["ai","cobalt","docker","downloader","instagram-downloader","notebooklm","notebooklm-alternative","streamlit","summarization","summarize","summarizer-ai","transcription","youtube-downloader"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/martinopiaggi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":["martinopiaggi"],"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"lfx_crowdfunding":null,"polar":null,"buy_me_a_coffee":null,"thanks_dev":null,"custom":null}},"created_at":"2023-10-15T15:19:47.000Z","updated_at":"2026-05-17T14:49:49.000Z","dependencies_parsed_at":"2024-05-09T22:26:09.924Z","dependency_job_id":"505884b4-9163-43e3-ab49-621c61e3923b","html_url":"https://github.com/martinopiaggi/summarize","commit_stats":null,"previous_names":["martinopiaggi/summarize"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/martinopiaggi/summarize","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/martinopiaggi%2Fsummarize","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/martinopiaggi%2Fsummarize/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/martinopiaggi%2Fsummarize/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/martinopiaggi%2Fsummarize/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/martinopiaggi","download_url":"https://codeload.github.com/martinopiaggi/summarize/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/martinopiaggi%2Fsummarize/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33450402,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-24T19:21:36.376Z","status":"ssl_error","status_checked_at":"2026-05-24T19:21:10.562Z","response_time":57,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","cobalt","docker","downloader","instagram-downloader","notebooklm","notebooklm-alternative","streamlit","summarization","summarize","summarizer-ai","transcription","youtube-downloader"],"created_at":"2026-05-08T08:00:42.628Z","updated_at":"2026-05-24T21:00:44.413Z","avatar_url":"https://github.com/martinopiaggi.png","language":"Python","funding_links":["https://github.com/sponsors/martinopiaggi"],"categories":["Python"],"sub_categories":[],"readme":"# Video Transcript Summarizer\n\n\u003cp align=\"center\"\u003e\n    \u003cimg alt=\"sample\" src=\"./summarize_sample.gif\"\u003e\n\u003c/p\u003e\n\nTranscribe and summarize videos from YouTube, Instagram, TikTok, Twitter, Reddit, Facebook, Google Drive, Dropbox, and local files.\n\nWorks with any OpenAI-compatible LLM provider, including locally hosted endpoints.\n\n## Interfaces\n\n| Interface | Command |\n|-----------|---------|\n| CLI | `python -m summarizer --source \u003csource\u003e` |\n| Streamlit GUI | `python -m streamlit run app.py` |\n| Docker | `docker compose up -d` -\u003e `http://localhost:8501` |\n| Agent skill | [`.agent/skills/summarize/SKILL.md`](./.agent/skills/summarize/SKILL.md) for agent access to the CLI |\n\n## How It Works\n\n```\n               +--------------------+\n               |  Video URL/Path    |\n               +---------+----------+\n                         |\n                         v\n               +---------+----------+\n               |    Source Type?    |\n               +---------+----------+\n                         |\n                         v\n               +---------+----------+\n               | Transcript Cache   |------\u003e HIT -------+\n               +---------+----------+                   |\n                         | MISS                         |\n       +-----------------+-------------+                |\n       |                 |             |                |\n       |          IG/TikTok/X    Local File            |\n    YouTube       Reddit/FB      Google Drive          |\n       |          other URLs      Dropbox               |\n       |                 |             |                |\n       v            +----+-----+       |                |\n+------+----------+ | yt-dlp / |       |                |\n| Captions Exist? | | Cobalt   |       |                |\n+----+----+-------+ +----+-----+       |                |\n    Yes   No             |             |                |\n     |    +--------------+--------+----+                |\n     |                            |                     |\n     |                            v                     |\n     |                   +--------+--------+            |\n     |                   |     Whisper     |            |\n     |                   |    endpoint?    |            |\n     |                   +--------+--------+            |\n     |                            |                     |\n     |                +-----------+-----------+         |\n     |                |                       |         |\n     |           Cloud Whisper          Local Whisper   |\n     |                |                       |         |\n     |                +----------+------------+         |\n     |                           |                      |\n     +---------------------------+                      |\n                                 |                      |\n                           store in cache               |\n                                 |                      |\n                                 +----------------------+\n                                 |\n                            Transcript\n                                 |\n                                 v\n                    +------------+----------+\n summarizer.yaml -\u003e |    Prompt + LLM       |\n prompts.json    -\u003e |    Merge              |\n .env            -\u003e +------------+----------+\n                                 |\n                                 v\n                          +------+-------+\n                          |    Output    |\n                          +--------------+\n```\n\n- `summarizer.yaml`: Local provider settings (`base_url`, `model`, `chunk-size`) and defaults such as `output-language`. Generate it with `python -m summarizer --init-config` or copy [`summarizer.example.yaml`](./summarizer.example.yaml).\n- [`.env`](./.env): API keys matched by URL keyword\n- [`prompts.json`](./summarizer/prompts.json): Summary style templates\n\n**Notes:**\n- Transcripts are **cached in memory** by default (keyed by SHA-256 of source + config). Re-summarizing the same video with a different style or provider skips transcription entirely. The cache lives in process memory and clears on exit. Disable with `cache-transcript: false` in `summarizer.yaml`.\n- Cloud Whisper uses **Groq Cloud API** and requires a Groq API key\n- Non-YouTube social URLs use **yt-dlp first** for Instagram, TikTok, X/Twitter, Reddit, and Facebook. Cobalt is still used as the fallback downloader for other HTTP video URLs.\n- The Docker image does **not** include Local Whisper and is aimed at lightweight VPS deployment\n\n## Installation and Usage\n\n**Step 0 - CLI installation:**\n\n```bash\ngit clone https://github.com/martinopiaggi/summarize.git\ncd summarize\npip install -e .\n```\n\n**Step 1 - Run the CLI:**\n\n```bash\npython -m summarizer --source \"https://youtube.com/watch?v=VIDEO_ID\"\n```\n\nThe summary is saved to `summaries/watch_YYYYMMDD_HHMMSS.md`.\n\n### Streamlit GUI\n\n```bash\npython -m streamlit run app.py\n```\n\nVisit port 8501.\n\n### Docker\n\n```bash\ngit clone https://github.com/martinopiaggi/summarize.git\ncd summarize\n# Create .env with your API keys and summarizer.yaml with your providers, then:\ndocker compose up -d\n```\n\nOpen `http://localhost:8501` for the GUI. Summaries are saved to `./summaries/`.\nThe CLI and GUI both read the same local `summarizer.yaml`.\n\nCLI via Docker: `docker compose run --rm summarizer python -m summarizer --source \"URL\"`\n\nCobalt standalone: `docker compose -f docker-compose.cobalt.yml up -d`\n\n## Configuration\n\n### Providers (`summarizer.yaml`)\n\nDefine your LLM providers and defaults. CLI flags override everything. See [`summarizer.example.yaml`](./summarizer.example.yaml) for a complete starter file.\n\n```yaml\n# example of summarizer.yaml\ndefault_provider: gemini\n\nproviders:\n  gemini:\n    base_url: https://generativelanguage.googleapis.com/v1beta/openai\n    model: gemini-flash-lite-latest\n    chunk-size: 128000\n\n  groq:\n    base_url: https://api.groq.com/openai/v1\n    model: openai/gpt-oss-20b\n\n  ollama:\n    base_url: http://localhost:11434/v1\n    model: qwen3:8b\n\n  openrouter:\n    base_url: https://openrouter.ai/api/v1\n    model: openai/gpt-oss-20b\n\n# example of a second config on the same provider\n  openrouter120:\n    base_url: https://openrouter.ai/api/v1\n    model: openai/gpt-oss-120b\n\n  openai:\n    base_url: https://api.openai.com/v1\n    model: gpt-5.5\n\n  nvidia:\n    base_url: https://integrate.api.nvidia.com/v1\n    model: nvidia/nemotron-3-nano-omni-30b-a3b-reasoning\n\n  perplexity:\n    base_url: https://openrouter.ai/api/v1\n    model: perplexity/sonar\n    chunk-size: 128000\n\ndefaults:\n  prompt-type: Questions and answers\n  chunk-size: 10000\n  parallel-calls: 30\n  max-tokens: 4096\n  output-language: auto\n  audio-speed: 1.0\n  use-proxy: false\n  output-dir: summaries\n  cache-transcript: true\n```\n\n`output-language` controls the language of the generated summary. Use `auto`, `none`, or an empty value to leave the model prompt unchanged. Use a human-readable language name such as `Italian`, `Spanish`, `German`, or `Japanese` to force summaries into that language.\n\n### API Keys (`.env`)\n\n```ini\n# Required for Cloud Whisper transcription\ngroq = gsk_YOUR_KEY\n\n# LLM providers (choose one or more)\nopenai = sk-proj-YOUR_KEY\ngenerativelanguage = YOUR_GOOGLE_KEY\ndeepseek = YOUR_DEEPSEEK_KEY\nopenrouter = YOUR_OPENROUTER_KEY\nperplexity = YOUR_PERPLEXITY_KEY\nhyperbolic = YOUR_HYPERBOLIC_KEY\nNVIDIA_API_KEY = YOUR_NVIDIA_KEY\n\n# Optional: Webshare credentials\n# Used only when `defaults.use-proxy: true` or `--use-proxy` is enabled\nWEBSHARE_PROXY_USERNAME = YOUR_WEBSHARE_USERNAME\nWEBSHARE_PROXY_PASSWORD = YOUR_WEBSHARE_PASSWORD\n\n# Optional: yt-dlp authentication for social platforms\n# Cookies are preferred for 2FA accounts. Use a Netscape cookies.txt export.\nINSTAGRAM_COOKIES_FILE = C:\\path\\to\\instagram-cookies.txt\nYTDLP_COOKIES_FILE = C:\\path\\to\\cookies.txt\n\n# Optional: Instagram username/password login when cookies are not used\nINSTAGRAM_USER = YOUR_INSTAGRAM_USERNAME\nINSTAGRAM_PASS = YOUR_INSTAGRAM_PASSWORD\n```\n\nIf you pass an endpoint URL with `--base-url`, the API key is matched from `.env` by URL keyword. For example, `https://generativelanguage.googleapis.com/...` matches `generativelanguage`.\n\n### Prompts ([`prompts.json`](./summarizer/prompts.json))\n\nUse with `--prompt-type` in the CLI or select it from the dropdown in the web interface.\nAdd custom styles by editing [`prompts.json`](./summarizer/prompts.json). Use `{text}` as the transcript placeholder.\n\n### CLI Examples\n\nWith a configured local `summarizer.yaml`, the CLI is simple:\n\n```bash\n# Uses the default provider from YAML\npython -m summarizer --source \"https://youtube.com/watch?v=VIDEO_ID\"\n\n# Specify a provider\npython -m summarizer --source \"https://youtube.com/watch?v=VIDEO_ID\" --provider groq\n\n# Fact-check claims with Perplexity (use the Summarize skill for AI agents)\npython -m summarizer \\\n  --source \"https://youtube.com/watch?v=VIDEO_ID\" \\\n  --base-url \"https://api.perplexity.ai\" \\\n  --model \"sonar-pro\" \\\n  --prompt-type \"Fact Checker\"\n\n# Extract key insights\npython -m summarizer \\\n  --source \"https://youtube.com/watch?v=VIDEO_ID\" \\\n  --provider gemini \\\n  --prompt-type \"Distill Wisdom\"\n\n# Generate a Mermaid diagram\npython -m summarizer \\\n  --source \"https://youtube.com/watch?v=VIDEO_ID\" \\\n  --provider openrouter \\\n  --prompt-type \"Mermaid Diagram\"\n\n# Use NVIDIA NIM\npython -m summarizer \\\n  --source \"https://youtube.com/watch?v=VIDEO_ID\" \\\n  --provider nvidia\n\n# Multiple videos\npython -m summarizer --source \"URL1\" \"URL2\" \"URL3\"\n\n# Local files\npython -m summarizer --type \"Local File\" --source \"./lecture.mp4\"\n\n# Speed up audio before Whisper (faster, may reduce accuracy)\npython -m summarizer --source \"URL\" --force-download --audio-speed 2.0\n\n# Aggressive speed-up (supported)\npython -m summarizer --source \"URL\" --force-download --audio-speed 5.0\n\n# Force YouTube audio download and show detailed progress\npython -m summarizer \\\n  --source \"https://youtube.com/watch?v=VIDEO_ID\" \\\n  --force-download \\\n  -v\n\n# Non-YouTube social URL\n# Instagram, TikTok, X/Twitter, Reddit, and Facebook use yt-dlp first.\n# Other HTTP video URLs fall back to Cobalt.\npython -m summarizer --type \"Video URL\" --source \"https://www.instagram.com/reel/...\"\n\n# Let captions/transcription choose the language automatically (default)\npython -m summarizer --source \"URL\" --language \"auto\"\n\n# Lock YouTube captions or transcription to a specific language\npython -m summarizer --source \"URL\" --prompt-type \"Distill Wisdom\" --language \"it\"\n\n# Write the summary in a specific language\npython -m summarizer --source \"URL\" --output-language \"Italian\"\n```\n\nWithout YAML, pass `--base-url` and `--model` explicitly:\n\n```bash\npython -m summarizer \\\n  --source \"https://youtube.com/watch?v=VIDEO_ID\" \\\n  --base-url \"https://generativelanguage.googleapis.com/v1beta/openai\" \\\n  --model \"gemini-2.5-flash-lite\"\n```\n\n#### CLI Reference\n\n| Flag | Description | Default |\n|------|-------------|---------|\n| `--source` | Video URLs or file paths (multiple allowed) | Required |\n| `--provider` | Provider name from YAML | `default_provider` |\n| `--base-url` | API endpoint (overrides provider) | From YAML |\n| `--model` | Model identifier (overrides provider) | From YAML |\n| `--api-key` | API key (overrides [`.env`](./.env)) | - |\n| `--type` | `YouTube Video`, `Video URL`, `Local File`, `Google Drive Video Link`, `Dropbox Video Link`, `TXT` | `YouTube Video` |\n| `--prompt-type` | Summary style | `Questions and answers` |\n| `--chunk-size` | Input text chunk size in characters | `10000` |\n| `--force-download` | Skip captions and download audio instead | `False` |\n| `--transcription` | `Cloud Whisper` (Groq API) or `Local Whisper` (local) | `Cloud Whisper` |\n| `--whisper-model` | `tiny`, `base`, `small`, `medium`, `large` | `tiny` |\n| `--audio-speed` | Pre-transcription playback speed | `1.0` |\n| `--language` | `auto` picks the first available YouTube caption track and lets Whisper detect language; explicit codes stay strict | `auto` |\n| `--output-language` | Language for the generated summary; use `auto`, `none`, or empty to leave the prompt unchanged | `auto` |\n| `--parallel-calls` | Concurrent API requests | `30` |\n| `--max-tokens` | Max output tokens per chunk | `4096` |\n| `--cobalt-url` | Cobalt base URL for fallback downloads after YouTube/yt-dlp handlers do not match or fail | `http://localhost:9000` |\n| `--output-dir` | Output directory | `summaries` |\n| `--no-save` | Print only, no file output | `False` |\n| `--verbose`, `-v` | Detailed output | `False` |\n\nUse `--verbose` to see detailed status output during config loading, downloads, transcription, and summarization.\n\n## Extra\n\n### Local Whisper\n\nRuns transcription on your machine instead of using Groq Cloud Whisper. This removes the Groq API requirement, but CPU-only runs are much slower.\n\n```bash\n# Add Local Whisper support\npip install -e .[whisper]\n\n# Optional: install CUDA-enabled PyTorch for GPU acceleration\npip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124\n\n# Use it\npython -m summarizer --source \"URL\" --force-download --transcription \"Local Whisper\" --whisper-model \"small\"\n```\n\nIf you only need CPU transcription, `pip install -e .[whisper]` is enough.\n\n**Why not in Docker?** The Docker image installs the core app only. It does not include `openai-whisper` or GPU-oriented PyTorch because this project targets lightweight VPS deployments, where GPUs are usually unavailable. In Docker, Cloud Whisper is the practical default. Use Local Whisper on the host machine if you have the hardware for it.\n\nModel sizes: `tiny` (fastest) / `base` / `small` / `medium` / `large` (most accurate). GPU detection is automatic when PyTorch can see a CUDA device.\n\n### Proxy Setup\n\nProxy support matters in three separate places:\n\n1. The Python app, when fetching YouTube transcripts or downloading YouTube audio with `pytubefix`\n2. The Python app, when downloading supported social URLs with `yt-dlp`\n3. The Cobalt container, when it connects to upstream CDNs/providers\n\nFor the Python app, this repo expects **Webshare** credentials:\n\n1. Add credentials to [`.env`](./.env):\n\n```ini\nWEBSHARE_PROXY_USERNAME = YOUR_WEBSHARE_USERNAME\nWEBSHARE_PROXY_PASSWORD = YOUR_WEBSHARE_PASSWORD\n```\n\n2. If you want `pytubefix` audio downloads to use that proxy, enable it in `summarizer.yaml`:\n\n```yaml\ndefaults:\n  use-proxy: true\n```\n\nNotes:\n\n- YouTube transcript fetching uses Webshare automatically when those credentials are present.\n- `defaults.use-proxy: true` affects `pytubefix` and `yt-dlp` audio downloads.\n- yt-dlp authentication is independent from Webshare proxy settings.\n- Host-specific cookie files such as `INSTAGRAM_COOKIES_FILE` win over `YTDLP_COOKIES_FILE`.\n- Cookie-file authentication wins over username/password authentication.\n\nFor the Cobalt container, the proxy is configured separately. That sits outside the Python app, but this repo includes a working example:\n\n- [docker-compose.yml](./docker-compose.yml) is the default full-stack setup\n- [docker-compose.cobalt.yml](./docker-compose.cobalt.yml) runs only Cobalt\n- [docker-compose.proxy.yml](./docker-compose.proxy.yml) adds `./cobalt.proxy.env` to the `cobalt` service\n- [cobalt.proxy.env.example](./cobalt.proxy.env.example) is the template\n- `cobalt.proxy.env` is your local, ignored secrets file\n\nDocker examples:\n\n```powershell\n# Cobalt only, no proxy\ndocker compose -f docker-compose.cobalt.yml up -d\n\n# Cobalt only, with proxy\ndocker compose -f docker-compose.cobalt.yml -f docker-compose.proxy.yml up -d\n\n# Full stack, no proxy\ndocker compose up -d\n\n# Full stack, with proxy\ndocker compose -f docker-compose.yml -f docker-compose.proxy.yml up -d\n```\n\n## License\n\nThis project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmartinopiaggi%2Fsummarize","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmartinopiaggi%2Fsummarize","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmartinopiaggi%2Fsummarize/lists"}