{"id":40321801,"url":"https://github.com/ai-mindset/distil","last_synced_at":"2026-01-20T07:30:28.941Z","repository":{"id":328544358,"uuid":"1115938784","full_name":"ai-mindset/distil","owner":"ai-mindset","description":"⚗️ Distil - LLM-powered weekly digests from RSS feeds, arXiv, PubMed, YouTube etc.","archived":false,"fork":false,"pushed_at":"2026-01-01T09:56:05.000Z","size":142,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-01-03T08:08:37.509Z","etag":null,"topics":["arxiv","cli","digest","fasthtml","llm","pubmed","python","research","rss","summarisation","weekly-digest","youtube-transcripts"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ai-mindset.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-13T21:28:20.000Z","updated_at":"2026-01-01T09:56:09.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ai-mindset/distil","commit_stats":null,"previous_names":["ai-mindset/distil"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ai-mindset/distil","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ai-mindset%2Fdistil","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ai-mindset%2Fdistil/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ai-mindset%2Fdistil/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ai-mindset%2Fdistil/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ai-mindset","download_url":"https://codeload.github.com/ai-mindset/distil/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ai-mindset%2Fdistil/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28598128,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-20T02:08:49.799Z","status":"ssl_error","status_checked_at":"2026-01-20T02:08:44.148Z","response_time":117,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arxiv","cli","digest","fasthtml","llm","pubmed","python","research","rss","summarisation","weekly-digest","youtube-transcripts"],"created_at":"2026-01-20T07:30:28.296Z","updated_at":"2026-01-20T07:30:28.929Z","avatar_url":"https://github.com/ai-mindset.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ⚗️ Distil\n\nIntelligent content distillation tool. Aggregates content from RSS feeds and YouTube transcripts, filters by relevance, and generates executive summaries using local or cloud LLMs with real-time streaming and batch processing. Perfect for research, industry analysis, or staying current in any domain.\n\n## Quick Start\n\n**Requirements:** Python 3.13+ (check with `python --version`)\n\n**Step 1: Install uv (Python package manager)**\n\n**Windows:**\n```powershell\npowershell -ExecutionPolicy ByPass -c \"irm https://astral.sh/uv/install.ps1 | iex\"\n```\n\n**macOS/Linux:**\n```bash\ncurl -LsSf https://astral.sh/uv/install.sh | sh\n```\n\n\u003e **Note:** Restart your terminal after installation to update your PATH.\n\n**Step 2: Run distil (auto-installs Ollama and models)**\n```bash\ngit clone https://github.com/ai-mindset/distil.git \u0026\u0026 cd distil\nuv run distil run\n```\n\nDistil automatically installs Ollama and downloads models as needed.\n\n## Prerequisites\n\n- Python 3.13+ (check with `python --version`)\n- Internet connection for initial setup\n- *Everything else (uv, Ollama, models) is automatically installed*\n\n## Installation\n\n**From source (development):**\n```bash\ngit clone https://github.com/ai-mindset/distil.git\ncd distil\nuv run distil run  # Automatically sets up everything on first run\n```\n\n**From package (when published):**\n```bash\nuv tool install distil\ndistil run  # Automatically sets up Ollama and models\n```\n\n## Configuration\n\nCopy `config.toml` to your working directory and edit:\n\n```toml\n[llm]\nmodel = \"ollama/qwen2.5:3b\"  # Local (free) — or \"anthropic/claude-sonnet-4-20250514\" (requires API key)\n\n[output]\ndirectory = \"~/distils\"        # Where to save distils\nreading_time_minutes = 5       # Target reading time\n\n[domain]\nfocus = \"drug discovery, pharmacology, AI/ML for therapeutics\"  # Customize for your domain\n\n[[feeds]]\nurl = \"https://rss.arxiv.org/rss/cs.ai\"\nname = \"arXiv AI\"\nkeywords = [\"drug\", \"molecule\", \"protein\", \"binding\"]  # Only items matching these\nmax_items = 5\n# pattern = \"(?i)(biotech|drug|pharma)\"  # Optional: regex pattern for advanced filtering\n```\n\nFor cloud LLMs, set your API key:\n\n**macOS/Linux:**\n```bash\nexport ANTHROPIC_API_KEY=\"sk-...\"\n```\n\n**Windows (PowerShell):**\n```powershell\n$env:ANTHROPIC_API_KEY=\"sk-...\"\n```\n\n**Windows (Command Prompt):**\n```cmd\nset ANTHROPIC_API_KEY=sk-...\n```\n\n## Usage\n\n**Web UI (recommended):**\n```bash\nuv run distil serve  # Auto-installs dependencies if needed\n```\nOpens browser at `http://localhost:5001`. Click \"Fetch Items\" to preview, then \"Generate Distil\".\n\n**CLI:**\n```bash\nuv run distil run              # Generate distil with defaults (auto-installs Ollama if needed)\nuv run distil run --days 3     # Last 3 days only\n```\n\n## Output\n\nDistils are saved to `history/distil-YYYY-MM-DD_HHMM.md` in your current directory. View past distils at `http://localhost:5001/history`.\n\n## Adding Sources\n\nEdit `config.toml` to add feeds:\n\n```toml\n[[feeds]]\nurl = \"https://example.com/rss\"\nname = \"My Feed\"\nkeywords = [\"relevant\", \"terms\"]      # Optional: filter by keywords\nmax_items = 25                        # Optional: limit items\n# pattern = \"(?i)(regex|pattern)\"     # Optional: regex for advanced filtering\n\n# YouTube playlists/channels also supported\n[[feeds]]\nurl = \"https://youtube.com/@channel\"\nname = \"YouTube Channel\"\nmax_items = 10\n```\n\n## Features\n\n- **⚡ Zero-Config Setup**: Automatically installs Ollama and downloads models on first run\n- **🌍 Cross-Platform**: Works on Linux, macOS, and Windows\n- **📺 YouTube Support**: Extract transcripts from YouTube videos and playlists\n- **🌙 Dark Mode**: Comprehensive dark/light theme with localStorage persistence\n- **📊 Feed Health Monitoring**: Real-time status tracking with detailed health reports\n- **🔄 Batch Processing**: Intelligent batching prevents LLM context window limits\n- **📈 Real-time Streaming**: Server-Sent Events (SSE) for live progress updates\n- **🚀 Smart Port Management**: Automatic port conflict detection and resolution\n- **📁 History Management**: View and manage past distils with web interface\n- **🎯 Advanced Filtering**: Keyword and regex pattern matching for precise content selection\n\n## Troubleshooting\n\n| Issue | Solution |\n|-------|----------|\n| \"Connection refused\" from Ollama | Distil will automatically start Ollama server; if issues persist, check system logs |\n| Slow generation | Reduce `max_items` per feed, or use fewer feeds |\n| Missing items | Check `keywords` aren't too restrictive |\n| Web app stuck at \"Fetching...\" | Check feed URLs are accessible; see feed health report |\n| Timeout errors | System now uses batch processing to prevent this |\n| Windows Ollama setup | Manual download required from https://ollama.com/download (auto-install not supported) |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fai-mindset%2Fdistil","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fai-mindset%2Fdistil","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fai-mindset%2Fdistil/lists"}