{"id":23760659,"url":"https://github.com/morrismorrison/ytclipper-go","last_synced_at":"2026-05-07T19:15:42.785Z","repository":{"id":270261370,"uuid":"909202063","full_name":"MorrisMorrison/ytclipper-go","owner":"MorrisMorrison","description":"go rewrite of ytclipper","archived":false,"fork":false,"pushed_at":"2025-08-01T21:48:41.000Z","size":13616,"stargazers_count":1,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-01T23:39:29.467Z","etag":null,"topics":["echo","ffmpeg","golang","youtube","yt-dlp"],"latest_commit_sha":null,"homepage":"https://ytclipper-go.mormor.online/","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MorrisMorrison.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-12-28T02:28:24.000Z","updated_at":"2025-08-01T21:48:44.000Z","dependencies_parsed_at":null,"dependency_job_id":"cc7d4d8c-7f73-423f-a0e6-a716fb7ebddf","html_url":"https://github.com/MorrisMorrison/ytclipper-go","commit_stats":null,"previous_names":["morrismorrison/ytclipper-go"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/MorrisMorrison/ytclipper-go","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MorrisMorrison%2Fytclipper-go","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MorrisMorrison%2Fytclipper-go/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MorrisMorrison%2Fytclipper-go/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MorrisMorrison%2Fytclipper-go/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MorrisMorrison","download_url":"https://codeload.github.com/MorrisMorrison/ytclipper-go/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MorrisMorrison%2Fytclipper-go/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32752148,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-07T02:14:30.463Z","status":"ssl_error","status_checked_at":"2026-05-07T02:14:29.405Z","response_time":62,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["echo","ffmpeg","golang","youtube","yt-dlp"],"created_at":"2024-12-31T20:34:02.188Z","updated_at":"2026-05-07T19:15:42.780Z","avatar_url":"https://github.com/MorrisMorrison.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ytclipper-go\n[![Build and Test (Development)](https://github.com/MorrisMorrison/ytclipper-go/actions/workflows/build_dev.yml/badge.svg?branch=main)](https://github.com/MorrisMorrison/ytclipper-go/actions/workflows/build_dev.yml)\n[![main](https://github.com/MorrisMorrison/ytclipper-go/actions/workflows/build_and_deploy_prod.yml/badge.svg?branch=main)](https://github.com/MorrisMorrison/ytclipper-go/actions/workflows/build_and_deploy_prod.yml)\n\nA high-performance web application for creating and downloading video clips from YouTube videos. This is the Go version of [ytclipper](https://github.com/MorrisMorrison/ytclipper), built with modern web technologies and designed for production use.\n\n\nhttps://github.com/user-attachments/assets/8ab2d567-0ca7-44c6-9c76-07203b2fd986\n\n\n\n## Features\n\n### Core Functionality\n- **YouTube Video Clipping**: Extract specific segments from YouTube videos with precise start/end timestamps\n- **Format Selection**: Support for multiple video formats and quality options available from YouTube\n- **Video Preview**: Built-in video player for previewing YouTube videos before clipping\n- **Asynchronous Processing**: Job-based processing system with real-time status tracking\n- **Progress Tracking**: Real-time progress updates for clip creation jobs\n\n### Production Features\n- **Rate Limiting**: Built-in rate limiting to prevent abuse and ensure fair usage\n- **Automatic Cleanup**: Scheduled cleanup of old clips and jobs to manage storage efficiently\n- **Health Monitoring**: Health check endpoint for monitoring and load balancing\n- **Cookie Monitoring**: Dual validation system with expiration tracking and API functional testing\n- **Smart Notifications**: Proactive alerts via ntfy when cookies need attention\n- **Simple Bot Detection Bypass**: Cookie-based authentication with automatic fallback\n- **Intelligent Fallback**: Dual-tier strategy with user agent rotation and anti-detection headers\n- **Responsive UI**: Clean, dark-themed web interface optimized for all devices\n\n### Usage\n1. Enter a YouTube URL\n2. Select your desired video format and quality\n3. Specify start and end times in flexible format (`34`, `1:28`, `1:09:24`)\n4. Track progress and download your clip when ready\n\n## Built With\n- [Go](https://golang.org/)\n- [Echo](https://echo.labstack.com/) - High-performance web framework for Go\n- [yt-dlp](https://github.com/yt-dlp/yt-dlp) - A YouTube downloader with additional features\n- [FFmpeg](https://ffmpeg.org/) - Tool for handling multimedia data\n- [Video.js](https://videojs.com/) - HTML5 video player\n- [Toastr](https://github.com/CodeSeven/toastr) - Notification library\n\n---\n\n## Quick Start\n\n### Prerequisites\n- **Go 1.21+**: [Download Go](https://golang.org/dl/)\n- **Python 3.8+**: [Download Python](https://www.python.org/downloads/)\n- **FFmpeg**: [Download FFmpeg](https://ffmpeg.org/download.html)\n\n### Installation\n1. **Clone the repository**:\n   ```bash\n   git clone https://github.com/MorrisMorrison/ytclipper-go.git\n   cd ytclipper-go\n   ```\n\n2. **Install dependencies**:\n   ```bash\n   go mod tidy\n   ```\n\n3. **Install Python dependencies**:\n   ```bash\n   pip install yt-dlp certifi requests urllib3\n   ```\n\n4. **Run the application**:\n   ```bash\n   go run main.go\n   ```\n\n5. **Access the application**:\n   Open your browser and navigate to `http://localhost:8080`\n\n### Development Commands\n- **Run application**: `go run main.go` or `make run`\n- **Run tests**: `make test`\n- **Run end-to-end tests**: `make e2e`\n- **Run fast E2E tests (CI-friendly)**: `make e2e-fast`\n- **Run realistic E2E tests**: `make e2e-realistic`\n- **Build for production**: `make build-prod`\n- **Download static assets**: `make download-static`\n\n## Docker Deployment\n\n### Using Makefile (Recommended)\n\n#### Single Container\n```bash\n# Build and run (rebuilds only on changes)\nmake docker-run\n\n# Stop and remove container\nmake docker-stop\n\n# Restart container\nmake docker-restart\n\n# Build image only\nmake docker-build\n```\n\n#### Docker Compose\n```bash\n# Start services (recommended for production)\nmake compose-up\n\n# Stop services\nmake compose-down\n\n# Restart services\nmake compose-restart\n\n# View logs\nmake compose-logs\n```\n\n### Manual Docker Commands\n```bash\n# Build the Docker image\ndocker build -t ytclipper .\n\n# Run the container\ndocker run -d -e PORT=8080 -p 8080:8080 ytclipper\n```\n\n### Docker Compose Configuration\nThe included `docker-compose.yml` provides:\n- **Port mapping**: 8080:8080\n- **Volume mounting**: `./videos:/app/videos` for persistent storage\n- **Environment variables**: Production-ready configuration\n- **Restart policy**: `unless-stopped` for reliability\n\n## API Endpoints\n\nThe application provides a REST API for programmatic access:\n\n| Method | Endpoint | Description |\n|--------|----------|-------------|\n| `POST` | `/api/v1/clip` | Create a new clip job |\n| `GET` | `/api/v1/clip` | Download completed clip |\n| `GET` | `/api/v1/jobs/status` | Check job status |\n| `GET` | `/api/v1/video/duration` | Get video duration |\n| `GET` | `/api/v1/video/formats` | Get available video formats |\n| `GET` | `/health` | Health check endpoint |\n\n## Configuration\n\nThe application can be configured using environment variables:\n\n### Server Configuration\n| Variable | Description | Default |\n|----------|-------------|---------|\n| `YTCLIPPER_PORT` | Server port | `8080` |\n| `YTCLIPPER_DEBUG` | Enable debug mode | `true` |\n\n### Rate Limiting\n| Variable | Description | Default |\n|----------|-------------|---------|\n| `YTCLIPPER_RATE_LIMITER_RATE` | Requests per second | `5` |\n| `YTCLIPPER_RATE_LIMITER_BURST` | Maximum burst requests | `20` |\n| `YTCLIPPER_RATE_LIMITER_EXPIRES_IN_MINUTES` | Token expiration (minutes) | `1` |\n\n### Video Processing\n| Variable | Description | Default |\n|----------|-------------|---------|\n| `YTCLIPPER_PORT_CLIP_SIZE_LIMIT_IN_MB` | Maximum clip size (MB) | `300` |\n| `YTCLIPPER_YT_DLP_COMMAND_TIMEOUT_IN_SECONDS` | yt-dlp timeout (seconds) | `60` |\n| `YTCLIPPER_YT_DLP_EXTRACTOR_RETRIES` | Number of retry attempts | `3` |\n| `YTCLIPPER_YT_DLP_SLEEP_INTERVAL` | Base sleep interval (seconds) | `2` |\n| `YTCLIPPER_YT_DLP_ENABLE_USER_AGENT_ROTATION` | Enable rotating user agents | `true` |\n\n### Anti-Bot Detection\n| Variable | Description | Default |\n|----------|-------------|---------|\n| `YTCLIPPER_YT_DLP_USER_AGENT` | Custom user agent (overrides rotation) | `` |\n| `YTCLIPPER_YT_DLP_COOKIES_FILE` | Path to cookies file (optional) | `` |\n| `YTCLIPPER_YT_DLP_COOKIES_CONTENT` | Cookie content as string (optional) | `` |\n| `YTCLIPPER_YT_DLP_PROXY` | Proxy server (optional) | `` |\n\n### Cleanup Scheduler\n| Variable | Description | Default |\n|----------|-------------|---------|\n| `YTCLIPPER_CLIP_CLEANUP_SCHEDULER_ENABLED` | Enable automatic cleanup | `true` |\n| `YTCLIPPER_CLIP_CLEANUP_SCHEDULER_INTERVAL_IN_MINUTES` | Cleanup interval (minutes) | `5` |\n| `YTCLIPPER_CLIP_CLEANUP_SCHEDULER_CLIP_DIRECTORY_PATH` | Directory to clean | `./videos` |\n\n### Cookie Monitoring\n| Variable | Description | Default |\n|----------|-------------|---------|\n| `YTCLIPPER_COOKIE_MONITOR_ENABLED` | Enable cookie expiration monitoring | `true` |\n| `YTCLIPPER_COOKIE_MONITOR_INTERVAL_HOURS` | Check interval (hours) | `24` |\n| `YTCLIPPER_COOKIE_MONITOR_WARNING_THRESHOLD_DAYS` | Warning threshold (days) | `30` |\n| `YTCLIPPER_COOKIE_MONITOR_URGENT_THRESHOLD_DAYS` | Urgent threshold (days) | `7` |\n| `YTCLIPPER_COOKIE_MONITOR_NTFY_TOPIC` | Ntfy topic for cookie alerts | `ytclipper-cookies` |\n| `YTCLIPPER_COOKIE_MONITOR_API_VALIDATION_ENABLED` | Enable API functional validation | `true` |\n| `YTCLIPPER_COOKIE_MONITOR_API_VALIDATION_INTERVAL_HOURS` | API validation interval (hours) | `6` |\n| `YTCLIPPER_COOKIE_MONITOR_TEST_VIDEO_URL` | YouTube URL for API testing | `https://www.youtube.com/watch?v=dQw4w9WgXcQ` |\n| `YTCLIPPER_COOKIE_MONITOR_API_VALIDATION_TIMEOUT_SECS` | API validation timeout (seconds) | `30` |\n\n\u003e 📚 **Detailed Cookie Monitoring Guide**: See [docs/COOKIE_MONITORING.md](docs/COOKIE_MONITORING.md) for comprehensive setup instructions, notification examples, and troubleshooting.\n\n### Ntfy Notifications (Global)\n| Variable | Description | Default |\n|----------|-------------|---------|\n| `YTCLIPPER_NTFY_ENABLED` | Enable ntfy notifications | `false` |\n| `YTCLIPPER_NTFY_SERVER_URL` | Ntfy server URL | `` |\n\n### Auth \n| Variable | Description | Default |\n|----------|-------------|---------|\n| `YTCLIPPER_AUTH_USERNAME` | Basic Auth Username | `` |\n| `YTCLIPPER_AUTH_PASSWORD` | Basic Auth Password | `` |\n\n## Architecture\n\n### System Components\n- **Web Server**: Echo-based HTTP server with middleware for rate limiting and logging\n- **Job Queue**: In-memory job management system with concurrent-safe operations\n- **Video Processor**: yt-dlp and FFmpeg integration for video downloading and clipping\n- **Scheduler**: Background cleanup service for automatic file management\n- **Static Assets**: Responsive web UI with real-time progress tracking\n\n### Data Flow\n1. User submits clip request via web interface\n2. System validates input and creates job\n3. Job queue processes request asynchronously\n4. yt-dlp downloads video segment\n5. FFmpeg processes and optimizes clip\n6. User downloads completed clip\n7. Scheduler automatically cleans up old files\n\n## Security\n\n- **Rate Limiting**: Prevents abuse with configurable request limits\n- **Input Validation**: Sanitizes all user inputs and URL parameters\n- **File Management**: Automatic cleanup prevents disk space exhaustion\n- **Error Handling**: Graceful error handling without exposing internal details\n- **Minimal Configuration**: Optional cookie/proxy support with no credentials stored by default\n- **Basic Auth**: Optional Basic Auth support when configuration is set\n\n## YouTube Bot Detection Bypass\n\nThe application uses a **simplified cookie-based authentication approach** with automatic fallback to maximize compatibility:\n\n### **Primary Strategy: Cookie Authentication**\n**Cookie Content**: Use `YTCLIPPER_YT_DLP_COOKIES_CONTENT` environment variable to provide cookie data directly  \n**Cookie File**: Use `YTCLIPPER_YT_DLP_COOKIES_FILE` for cookie file path (alternative to content)  \n**Proxy Support**: Use `YTCLIPPER_YT_DLP_PROXY` for proxy server when needed  \n**High Success Rate**: Provides authentication for restricted content when configured  \n\n### **Fallback Strategy: Anti-Detection Headers**\n**No Authentication Required**: Works without cookies when authentication is not available  \n**User Agent Rotation**: 6 modern browser user agents automatically rotated  \n**Enhanced Headers**: Browser-like HTTP headers for authenticity  \n**Automatic Retry**: Retry with different user agent on failure  \n\n### Quick Setup (Default Configuration)\n```bash\n# Default configuration (no setup required)\nexport YTCLIPPER_YT_DLP_ENABLE_USER_AGENT_ROTATION=true\nexport YTCLIPPER_YT_DLP_SLEEP_INTERVAL=2\nexport YTCLIPPER_YT_DLP_EXTRACTOR_RETRIES=3\n\n# Option 1: Use cookie content directly (recommended)\nexport YTCLIPPER_YT_DLP_COOKIES_CONTENT=\".youtube.com\tTRUE\t/\tFALSE\t1704067200\tVISITOR_INFO1_LIVE\txyz123\"\n\n# Option 2: Use cookie file path\nexport YTCLIPPER_YT_DLP_COOKIES_FILE=/path/to/cookies.txt\n\n# Optional: Add proxy for geographic restrictions\nexport YTCLIPPER_YT_DLP_PROXY=http://proxy-server:port\n```\n\n### **Cookie Expiration Monitoring**\n**Automatic Monitoring**: Tracks cookie expiration dates and sends notifications  \n**Configurable Thresholds**: Warning (30 days) and urgent (7 days) alerts  \n**Ntfy Integration**: Push notifications to your devices via ntfy.sh or self-hosted server  \n**Proactive Management**: Prevents service disruption by alerting before expiration  \n\n```bash\n# Enable cookie monitoring with notifications\nexport YTCLIPPER_COOKIE_MONITOR_ENABLED=true\nexport YTCLIPPER_NTFY_ENABLED=true\nexport YTCLIPPER_NTFY_SERVER_URL=\"https://ntfy.sh\"\nexport YTCLIPPER_COOKIE_MONITOR_NTFY_TOPIC=\"ytclipper-alerts\"\n```\n\n### How It Works\n1. **Primary Strategy**: Uses cookie authentication if cookies are configured via environment variables\n2. **Automatic Fallback**: Falls back to anti-detection headers if cookie authentication fails\n3. **Cookie Monitoring**: Automatically tracks cookie health and sends expiration alerts\n4. **No User Interaction**: Runs completely automatically with environment variables\n5. **Simple Configuration**: Two-tier approach with straightforward setup\n\n## Monitoring\n\n- **Health Checks**: `/health` endpoint for load balancer integration\n- **Logging**: Structured logging with configurable levels\n- **Metrics**: Job processing statistics and performance metrics\n- **Error Tracking**: Comprehensive error logging and tracking\n\n---\n\n## Contributing\n\n1. **Fork the repository**\n2. **Create a feature branch**: `git checkout -b feature/amazing-feature`\n3. **Make your changes**: Follow the existing code style and patterns\n4. **Run tests**: `make test \u0026\u0026 make e2e`\n5. **Test with Docker**: `make docker-run` to verify containerized deployment\n6. **Commit your changes**: `git commit -m 'Add amazing feature'`\n7. **Push to the branch**: `git push origin feature/amazing-feature`\n8. **Open a Pull Request**\n\n### Development Guidelines\n- Follow Go best practices and idioms\n- Add tests for new functionality\n- Update documentation for API changes\n- Test both local and Docker deployments\n- Ensure all CI checks pass\n\n### Available Make Commands\n| Command | Description |\n|---------|-------------|\n| `make run` | Run application locally |\n| `make test` | Run unit tests |\n| `make e2e` | Run end-to-end tests |\n| `make build` | Build binary |\n| `make build-prod` | Production build (tests + assets + build) |\n| `make download-static` | Download static assets |\n| `make docker-build` | Build Docker image |\n| `make docker-run` | Build and run container |\n| `make docker-stop` | Stop and remove container |\n| `make docker-restart` | Restart container |\n| `make compose-up` | Start with Docker Compose |\n| `make compose-down` | Stop Docker Compose services |\n| `make compose-restart` | Restart Docker Compose services |\n| `make compose-logs` | View Docker Compose logs |\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## Acknowledgments\n\n- [yt-dlp](https://github.com/yt-dlp/yt-dlp) for powerful YouTube downloading capabilities\n- [FFmpeg](https://ffmpeg.org/) for video processing\n- [Echo](https://echo.labstack.com/) for the excellent web framework\n- The original [ytclipper](https://github.com/MorrisMorrison/ytclipper) project\n\n## Roadmap\n\n- [x] Automatically delete downloaded videos\n- [x] Track created/finished time of jobs\n- [x] Kill suspended jobs after specified timeout\n- [x] Save clips in correct file format\n- [ ] Add worker pool for better concurrency\n- [ ] Add WebSocket support for real-time updates\n- [ ] Implement user authentication and quotas\n- [ ] Add support for batch processing\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmorrismorrison%2Fytclipper-go","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmorrismorrison%2Fytclipper-go","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmorrismorrison%2Fytclipper-go/lists"}