https://github.com/anskm/youtube-chat-extension
AI-powered Chrome extension for chatting with YouTube videos. Features smart routing for 95%+ cost savings.
https://github.com/anskm/youtube-chat-extension
ai chatbot chrome-extension gemini-api javascript youtube
Last synced: 2 months ago
JSON representation
AI-powered Chrome extension for chatting with YouTube videos. Features smart routing for 95%+ cost savings.
- Host: GitHub
- URL: https://github.com/anskm/youtube-chat-extension
- Owner: AnsKM
- License: mit
- Created: 2025-06-10T11:14:12.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-06-14T05:08:10.000Z (about 1 year ago)
- Last Synced: 2025-06-26T03:03:30.873Z (about 1 year ago)
- Topics: ai, chatbot, chrome-extension, gemini-api, javascript, youtube
- Language: JavaScript
- Size: 11.1 MB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# YouTube Chat Assistant - Smart Edition
An AI-powered Chrome extension that enables intelligent conversations with YouTube videos using multiple AI models including Google's Gemini 2.5 Flash and DeepSeek R1 Distilled (free via OpenRouter).
## 🚀 Features
### Core Functionality
- 💬 **AI-Powered Chat**: Have natural conversations about any YouTube video
- 🤖 **Multiple AI Models**: Choose between Gemini 2.5 Flash or DeepSeek R1 Distilled
- 📝 **Full Transcript Access**: Leverages large context windows for complete video understanding
- ⏱️ **Timestamp Navigation**: Click timestamps to jump to specific moments
- 💾 **Conversation Memory**: Saves chat history per video
- 🎨 **Clean UI**: Minimal, non-intrusive interface
### New in v2.1: Multi-Model Support
- 🔄 **Model Selection**: Easy switching between Gemini and DeepSeek models
- 🆓 **Free DeepSeek Access**: Use DeepSeek R1 Distilled for free via OpenRouter
- ⚡ **Fast Response Generation**: Distilled model provides quick responses
- 🔑 **Dual API Key Support**: Manage separate keys for each provider
### Smart Cost Optimization (v2.0)
- 🧠 **Smart Query Router**: Automatically selects optimal processing strategy
- 💰 **95%+ Cost Reduction**: From $0.50 to $0.02 per video (with Gemini)
- ⚡ **Context Caching**: 75% additional savings on repeated queries
- 📊 **Cost Tracking**: Real-time monitoring of API usage and savings
- 🎯 **Intelligent Chunking**: RAG system for efficient context selection
## 📋 Table of Contents
- [Installation](#-installation)
- [Configuration](#-configuration)
- [Usage](#-usage)
- [Cost Optimization](#-cost-optimization)
- [Development](#-development)
- [Architecture](#-architecture)
- [Contributing](#-contributing)
- [Troubleshooting](#-troubleshooting)
- [Connection Fixes](#-connection-fixes)
## 🔧 Installation
### From Chrome Web Store
1. Visit [Chrome Web Store Link] (pending approval)
2. Click "Add to Chrome"
3. Follow the prompts
### From Source (Development)
1. **Clone the repository**
```bash
git clone https://github.com/yourusername/youtube-chat-extension.git
cd youtube-chat-extension
```
2. **Install dependencies**
```bash
npm install
```
3. **Build the extension**
```bash
npm run build
```
4. **Load in Chrome**
- Open Chrome and go to `chrome://extensions/`
- Enable "Developer mode"
- Click "Load unpacked"
- Select the `dist` folder
## ⚙️ Configuration
### API Key Setup
#### For Gemini 2.5 Flash (Default)
1. Get a Gemini API key from [Google AI Studio](https://makersuite.google.com/app/apikey)
2. Click the extension icon in Chrome
3. Select "Gemini 2.5 Flash" from the model dropdown
4. Enter your Gemini API key in the popup
5. The key is stored securely in Chrome's encrypted storage
#### For DeepSeek R1 Distilled (Free Alternative)
1. Get an OpenRouter API key from [OpenRouter](https://openrouter.ai/keys)
2. Click the extension icon in Chrome
3. Select "DeepSeek R1 Distilled (Free)" from the model dropdown
4. Enter your OpenRouter API key in the popup
5. Enjoy free AI responses during the promotion period!
### Model Selection
Switch between models anytime:
- **Gemini 2.5 Flash**: Fast, efficient, 1M token context window
- **DeepSeek R1 Distilled**: Free via OpenRouter, fast reasoning model, 64K token context
### Extension Settings
Configure in the popup menu:
- **Model Selection**: Choose your preferred AI model
- **Theme**: Auto/Light/Dark
- **Language**: Transcript language preference
- **Cost Tracking**: Enable/disable cost monitoring (Gemini only)
- **Cache Settings**: Configure caching behavior
## 💡 Usage
### Basic Chat
1. Navigate to any YouTube video
2. Click the extension icon or wait for auto-load
3. Type your question in the chat box
4. Press Enter or click Send
### Advanced Features
#### Timestamp Queries
```
"What did they say at 15:30?"
"Explain the concept around 10 minutes in"
```
#### Summary Requests
```
"Summarize the key points"
"What are the main takeaways?"
```
#### Specific Information
```
"How do they implement the algorithm?"
"What tools were recommended?"
```
### Keyboard Shortcuts
- `Ctrl/Cmd + Shift + Y`: Toggle chat window
- `Esc`: Minimize chat
- `Ctrl/Cmd + Enter`: Send message
### History Panel
- Click 📚 to open your chat history
- Search through previous conversations
- Load any previous chat to continue
- Delete conversations you no longer need
- History is saved per video automatically
## 💰 Cost Optimization
### How It Works
The Smart Query Router automatically optimizes costs based on:
1. **Video Length Detection**
- Short videos (<30 min): Direct caching for instant responses
- Medium videos (30-180 min): Smart RAG for 85-90% savings
- Long videos (>3 hours): Aggressive RAG for 95%+ savings
2. **Query Classification**
- Timestamp queries: Fetches only relevant segments
- Summary queries: Uses efficient summarization strategies
- Specific queries: Targets exact information needed
3. **Context Caching**
- Gemini's context caching provides 75% additional savings
- Frequently accessed segments are cached automatically
- Cache persists for 30 minutes per video segment
### Cost Breakdown
| Video Length | Without Optimization | With Smart Router | Savings |
|--------------|---------------------|-------------------|----------|
| 10 minutes | $0.03 | $0.0075 | 75% |
| 1 hour | $0.15 | $0.015 | 90% |
| 3+ hours | $0.50+ | $0.025 | 95%+ |
### Monitoring Your Usage
- Click the extension icon to see real-time cost tracking
- View cumulative savings in the popup
- Export usage reports for analysis
- Set spending alerts if needed
## 🎯 Key Features Explained
### Smart Query Router (v2.0)
- **Automatic Strategy Selection**: Chooses optimal processing based on video length
- **Query Understanding**: Classifies queries to minimize token usage
- **Intelligent Chunking**: Only sends relevant transcript segments
- **Context Caching**: Reuses cached content for 75% savings
- **Real-time Monitoring**: Track costs and savings in console
### Conversation Memory
- The assistant remembers your conversation context
- Last 3 exchanges are included for continuity
- Each video maintains its own conversation history
### Transcript Support
- Automatically fetches video transcripts using multiple methods:
- DOM scraping from YouTube's transcript panel
- YouTube Player API
- Page data extraction
- TimedText API fallback
- Handles videos without transcripts gracefully
- Supports multiple languages
### Smart Dynamic Responses
- **Intelligent length adaptation** - matches response to query complexity
- Simple questions get direct answers (1-3 sentences)
- Complex questions get structured, organized responses
- Markdown formatting only when it enhances clarity
- Natural conversation flow without unnecessary padding
- References specific video timestamps when relevant
- **3000 token limit** ensures complete responses without cutoffs
### Copy Feature
- Hover over any AI response to reveal the copy button (📋)
- Click to copy the plain text to your clipboard
- Visual feedback (✅) confirms successful copy
- Works with all response formats and lengths
### Export Feature
- Click the 📥 Export button in the chat header
- Choose your preferred format:
- **Markdown (.md)**: Best for documentation, includes rich formatting
- **JSON (.json)**: Structured data for developers and analysis
- **Plain Text (.txt)**: Simple, readable format for basic needs
- Exports include video metadata (title, URL, date)
- Filenames are timestamped for easy organization
## 🛠️ Development
### Project Structure
```
youtube_chat_extension/
├── manifest.json # Extension manifest V3
├── src/ # Source code
│ ├── background/ # Service worker and API handling
│ │ ├── service-worker.js # Core background script
│ │ └── smart-api-handler.js # Smart routing integration
│ ├── content/ # Content scripts and UI
│ │ ├── content-script.js # Main extension logic
│ │ ├── transcript-fetcher.js # YouTube transcript extraction
│ │ └── styles.css # UI styling
│ ├── popup/ # Extension popup
│ │ ├── popup.html # Settings interface
│ │ ├── popup.js # API key management
│ │ └── popup.css # Popup styling
│ ├── smart-router/ # Cost optimization system
│ │ ├── smart-query-router.js # Main routing logic
│ │ ├── query-classifier.js # Query analysis
│ │ ├── cache-manager.js # Gemini context caching
│ │ ├── simple-rag.js # Basic RAG implementation
│ │ └── enhanced-rag.js # Advanced RAG with embeddings
│ └── utils/ # Utility modules
├── assets/ # Static assets
├── docs/ # Documentation
├── demo/ # Interactive demos
└── tools/ # Development utilities
```
See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for detailed organization.
### Key Technologies
- Chrome Extension Manifest V3
- Gemini 2.5 Flash Preview API (`models/gemini-2.5-flash-preview-05-20`)
- Chrome Storage API for persistence
- Vanilla JavaScript with ES6+ features
- CSS with dark mode support
### API Configuration
The extension uses:
- **Model**: `gemini-2.5-flash-preview-05-20`
- **Context window**: 1 million tokens
- **Max output tokens**: 3500 (with 3000 soft limit in prompt)
- **Temperature**: 0.7 (balanced creativity)
### Testing
To test changes:
1. Make your modifications
2. Go to `chrome://extensions/`
3. Click the refresh icon on the extension card
4. Reload any YouTube tabs
For debugging:
- Check the console for logs
- Use `window.ytChatExtension` to access the extension instance
- Run test scripts from `/debugging/test-fixes.js`
## 🐛 Troubleshooting
### Common Issues
1. **"Connection not configured"**
- Click the extension icon and add your API key
- Reload the YouTube page
2. **No transcript available**
- Some videos don't have captions
- The assistant can still help based on title and context
3. **Chat history not saving**
- Check Chrome storage quota
- Ensure the extension has proper permissions
4. **History panel issues**
- Panel should be hidden by default
- Use 📚 button to toggle visibility
- × button closes the panel
## 🚀 Recent Updates (June 7, 2025)
### Latest Updates
- 📋 **Copy to clipboard** button for all AI responses
- 🎯 **Increased token limit** to 3500 (3000 soft limit)
- 🎨 **Minimal UI/UX Design** - clean, distraction-free interface
- 📦 **Solid colors** with subtle shadows for depth
- ⚡ **Fast, simple animations** (0.2s transitions)
- 💬 **Clean message bubbles** with simple hover states
- 🔍 **Simple search input** with focus indicators
- ✨ **Smooth entrance** without distracting effects
- 🌗 **Elegant dark mode** with proper contrast
- 📱 **Responsive design** for all screen sizes
### Completed Features
- ✅ Full transcript fetching (4 methods)
- ✅ Conversation memory and context
- ✅ Chat persistence per video
- ✅ History panel with search
- ✅ New chat functionality
- ✅ Clear chat option
- ✅ Dark mode support
- ✅ Fixed history panel visibility issues
- ✅ Enhanced response formatting
- ✅ Dynamic response length based on query
- ✅ Timestamp support with click-to-seek
- ✅ Premium UI/UX redesign
### Known Issues (Fixed)
- ~~History panel showing by default~~ ✅ Fixed
- ~~Close button not working~~ ✅ Fixed
- ~~Chat not saving to history~~ ✅ Fixed
- ~~Response cut-off issues~~ ✅ Fixed with dynamic length
## 🗺️ Architecture
### Smart Query Router System
The extension uses a sophisticated routing system to minimize costs:
```javascript
// Automatic strategy selection based on video length
if (videoLength < 30) {
// Direct caching with 75% savings
strategy = 'DIRECT_CACHE';
} else if (videoLength < 180) {
// Smart RAG with 85-90% savings
strategy = 'SMART_RAG';
} else {
// Aggressive optimization for 95%+ savings
strategy = 'AGGRESSIVE_RAG';
}
```
### Console Debugging
Enable detailed logging to see the router in action:
```javascript
// In Developer Console (F12)
window.DEBUG_SMART_ROUTER = true;
```
You'll see:
- Video analysis and token counts
- Query classification results
- Strategy selection reasoning
- Cost calculations and savings
- Cache hit/miss statistics
## 🗺️ Future Enhancements
- [x] ~~Timestamp support - click to jump to video position~~ ✅ Completed
- [x] ~~Smart Query Router for 95%+ cost reduction~~ ✅ Completed
- [x] ~~Context caching integration~~ ✅ Completed
- [x] ~~Export conversations (Markdown/JSON/Text)~~ ✅ Completed
- [ ] Multi-language UI support
- [ ] YouTube Shorts support
- [ ] Voice input/output
- [ ] Custom prompts/templates
- [ ] Floating widget mode
- [ ] Chat with multiple videos
- [ ] Integration with YouTube playlists
## 🔧 Connection Fixes
### Issue: Subsequent Questions Not Working
If you experience issues where the first question works but subsequent questions fail, this has been resolved with the following improvements:
#### ✅ Connection Issues Fixed:
1. **Service Worker Lifecycle** - Chrome service workers become inactive after 30 seconds
2. **Connection Health Monitoring** - Automatic detection of connection loss
3. **Retry Logic** - Exponential backoff retry for failed requests
4. **UI State Management** - Proper input field enabling/disabling
5. **Error Recovery** - Graceful handling of connection errors
## ⏱️ Timestamp Validation Fixes
### Issue: Hallucinated/Invalid Timestamps
The AI was generating timestamps that don't exist in the actual video (e.g., [15:30] for an 11-minute video).
#### ✅ Timestamp Issues Fixed:
1. **Enhanced Prompt Engineering** - AI receives explicit video duration constraints
2. **Multi-Layer Validation** - Timestamps validated against video duration and transcript
3. **Intelligent Replacement** - Invalid timestamps become descriptive text ("early in the video")
4. **Transcript Integration** - Only timestamps from actual transcript are referenced
5. **Visual Feedback** - Users see when invalid timestamps are detected and replaced
#### 🎯 How It Works:
- **Before**: AI generates `[15:30]` for 11-minute video ❌
- **After**: AI uses `"around the middle"` or actual transcript timestamps ✅
#### 🔍 Testing the Timestamp Fixes:
Load the test script in browser console on any YouTube video:
```javascript
// Copy and paste test-timestamp-validation.js into console, then run:
runTimestampValidationTests()
```
#### 🔍 Testing the Connection Fixes:
Load the test script in your browser console on any YouTube video page:
```javascript
// Copy and paste test-connection-fix.js into console, then run:
runAllTests()
```
#### 📊 Connection Status Indicator:
- 🟢 **Green dot**: Connected and ready
- 🟡 **Yellow dot**: Attempting to reconnect
- 🔴 **Red dot**: Connection lost
#### 🛠️ Manual Testing:
1. Ask a question and wait for response
2. Wait 30+ seconds, then ask another question
3. Reload the extension and test again
4. Try rapid-fire questions to test race condition handling
#### 🐛 If Issues Persist:
1. Check browser console for error messages
2. Verify your Gemini API key is set correctly
3. Refresh the YouTube page
4. Reload the extension in `chrome://extensions/`
## 🤝 Contributing
Contributions are welcome! Please:
1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Test thoroughly (including connection fixes)
5. Submit a Pull Request
## 📄 License
MIT License - see LICENSE file for details
## 🙏 Acknowledgments
- Built with Google's Gemini 2.5 Flash Preview
- Inspired by the need for better video comprehension tools
- Thanks to all contributors and testers
---
**Status**: 🟢 Production Ready - All core features implemented and tested
**Model**: Using `models/gemini-2.5-flash-preview-05-20` with 1M token context