{"id":30343859,"url":"https://github.com/programminginblood/next-youtube-analytics-scraper","last_synced_at":"2026-02-14T18:01:21.782Z","repository":{"id":305357406,"uuid":"973696604","full_name":"ProgrammingInBlood/next-youtube-analytics-scraper","owner":"ProgrammingInBlood","description":null,"archived":false,"fork":false,"pushed_at":"2025-07-19T15:39:34.000Z","size":321,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-12T10:50:53.292Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://next-youtube-analytics-scraper.vercel.app","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ProgrammingInBlood.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-27T14:49:23.000Z","updated_at":"2025-07-19T15:39:38.000Z","dependencies_parsed_at":"2025-07-19T19:27:36.267Z","dependency_job_id":"0440b397-55cc-4d5a-8b98-ff2498564739","html_url":"https://github.com/ProgrammingInBlood/next-youtube-analytics-scraper","commit_stats":null,"previous_names":["programminginblood/next-youtube-analytics-scraper"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ProgrammingInBlood/next-youtube-analytics-scraper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ProgrammingInBlood%2Fnext-youtube-analytics-scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ProgrammingInBlood%2Fnext-youtube-analytics-scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ProgrammingInBlood%2Fnext-youtube-analytics-scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ProgrammingInBlood%2Fnext-youtube-analytics-scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ProgrammingInBlood","download_url":"https://codeload.github.com/ProgrammingInBlood/next-youtube-analytics-scraper/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ProgrammingInBlood%2Fnext-youtube-analytics-scraper/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29451890,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-14T15:52:44.973Z","status":"ssl_error","status_checked_at":"2026-02-14T15:52:11.208Z","response_time":53,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-18T11:51:34.891Z","updated_at":"2026-02-14T18:01:21.766Z","avatar_url":"https://github.com/ProgrammingInBlood.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 📺 YouTube Live Chat Aggregator\n\nA production-grade web application that lets you enter up to **3 YouTube Live Video URLs** and view:\n\n- 🧵 Combined **live chat** from all videos in one unified comment box.\n- 💬 Each message includes **username**, **channel name**, and **chat message**.\n- 📊 Real-time **likes** and **live viewer count** for each video.\n\n---\n\n## ⚙️ Tech Stack\n\n| Layer        | Tech              |\n|--------------|-------------------|\n| Frontend     | Next.js (App Router + TypeScript) |\n| Backend      | Bun + ElysiaJS    |\n| Scraping     | Puppeteer Core    |\n| Communication| REST API (future: WebSockets) |\n| Deployment   | Vercel (Frontend), Railway/Bun (Backend) |\n\n---\n\n## 🧩 Features\n\n- ✅ Input up to 3 YouTube Live URLs.\n- ✅ Real-time scraping of:\n  - 🔴 Live chat (via YouTube internal API)\n  - 👍 Likes\n  - 👁️ Viewer count\n- ✅ Uses Puppeteer to mimic real browser behavior and extract required tokens.\n- ✅ ElysiaJS backend with Bun for speed and modern API structure.\n- ✅ Minimal, modern UI with React/Next.js.\n- ✅ Compatible with Android Termux using puppeteer-core.\n\n---\n\n## 📁 Folder Structure\n\n```\nproject-root/\n├── backend/           # Bun + ElysiaJS backend\n│   ├── scraping/      # Puppeteer scraping logic\n│   ├── api/           # Route handlers\n│   └── utils/         # Helpers and response formatters\n├── app/               # Next.js frontend (App Router)\n│   ├── components/    # UI components\n│   └── lib/           # Frontend API utils\n└── README.md\n```\n\n---\n\n## 🚀 Getting Started\n\n### 1. Clone the repo\n\n```bash\ngit clone https://github.com/your-username/youtube-live-aggregator\ncd youtube-live-aggregator\n```\n\n### 2. Install dependencies\n\n```bash\n# Option 1: Use the setup script (recommended)\nchmod +x setup.sh\n./setup.sh\n\n# Option 2: Manual installation\n# Frontend\nnpm install\n\n# Backend\ncd backend\nbun install\ncd ..\n```\n\n### 3. Run in development\n\n```bash\n# Option 1: Run both frontend and backend with one command\nnpm run dev:all\n\n# Option 2: Run separately\n# Start backend (in one terminal)\ncd backend\nbun run dev\n\n# Start frontend (in another terminal)\nnpm run dev\n```\n\n### 4. Open the application\n\nVisit `http://192.168.0.243:3000` in your browser.\n\n---\n\n## 📱 Running on Android with Termux\n\nThe application can be run on Android devices using Termux. We've implemented puppeteer-core support to connect to an existing Chromium installation rather than bundling a browser.\n\n### Termux Setup\n\n1. Install Termux from F-Droid (not Google Play Store version)\n2. Open Termux and run the setup script:\n\n```bash\ncd backend\nchmod +x termux-setup.sh\n./termux-setup.sh\n```\n\n3. Once setup is complete, you need to run three separate Termux sessions:\n\n**Session 1: Start Chromium**\n```bash\nproot-distro login alpine -- chromium-browser --headless --disable-gpu --remote-debugging-port=9222\n```\n\n**Session 2: Start Backend**\n```bash\ncd backend\nbun run start\n```\n\n**Session 3: Start Frontend**\n```bash\nnpm run dev\n```\n\n4. Access the application at `http://192.168.0.243:3000` in your Android browser\n\n### How It Works on Termux\n\nInstead of bundling Chromium (which is problematic on Android), we:\n1. Use proot-distro to create an Alpine Linux container\n2. Install Chromium in the container\n3. Connect to the Chromium instance via remote debugging with puppeteer-core\n4. Allow the application to run on resource-constrained Android devices\n\nThis implementation is based on the techniques from [puppeteer-on-termux](https://github.com/rishabhrpg/puppeteer-on-termux).\n\n---\n\n## 🧠 How It Works\n\n1. User enters up to 3 YouTube Live video links.\n2. Frontend sends request to the backend.\n3. Backend uses Puppeteer to open each video URL.\n4. Puppeteer:\n   - Extracts apiKey, clientVersion, and continuation token.\n   - Hits YouTube internal APIs to fetch live chat and metadata.\n5. All chat messages are combined and sent back to the frontend.\n6. Frontend displays messages in a live comment box with real-time updates.\n\n---\n\n## 🔐 Security\n\n- ✅ URL validation to prevent invalid input or XSS.\n- ✅ Puppeteer sandboxing.\n- ✅ Rate limiting and error fallback.\n- ✅ Graceful recovery for unavailable streams or expired tokens.\n\n---\n\n## 📈 Enhancements\n\n- 💡 **Performance**\n  - Cache continuation tokens in memory.\n  - Use async streaming APIs.\n  - Reduce Puppeteer launch overhead with a shared browser pool.\n- 💡 **Features**\n  - WebSocket support for real-time updates.\n  - User authentication to save favorite streams.\n  - Chat filtering and search.\n  - Custom themes and appearance options.\n\n---\n\n## 📝 License\n\nMIT License.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprogramminginblood%2Fnext-youtube-analytics-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprogramminginblood%2Fnext-youtube-analytics-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprogramminginblood%2Fnext-youtube-analytics-scraper/lists"}