{"id":15779549,"url":"https://github.com/jim60105/backup-dl","last_synced_at":"2026-04-29T23:32:10.063Z","repository":{"id":107553317,"uuid":"371286601","full_name":"jim60105/backup-dl","owner":"jim60105","description":"Check Youtube channels, playlists, and backup videos to Azure Blob Storage (.NET 8 in Docker)","archived":false,"fork":false,"pushed_at":"2026-01-22T18:27:40.000Z","size":214,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-01-23T11:32:10.137Z","etag":null,"topics":["azure-storage","docker","docker-image","youtube","youtube-dl","youtube-dl-wrapper"],"latest_commit_sha":null,"homepage":"https://blog.maki0419.com/2021/06/backup-dl.html","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jim60105.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2021-05-27T07:36:12.000Z","updated_at":"2026-01-22T18:27:44.000Z","dependencies_parsed_at":"2023-12-29T06:32:41.726Z","dependency_job_id":"917a7e67-23f4-4a39-996f-bc708c408634","html_url":"https://github.com/jim60105/backup-dl","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jim60105/backup-dl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jim60105%2Fbackup-dl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jim60105%2Fbackup-dl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jim60105%2Fbackup-dl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jim60105%2Fbackup-dl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jim60105","download_url":"https://codeload.github.com/jim60105/backup-dl/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jim60105%2Fbackup-dl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32448399,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-29T22:27:22.272Z","status":"ssl_error","status_checked_at":"2026-04-29T22:10:49.234Z","response_time":110,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure-storage","docker","docker-image","youtube","youtube-dl","youtube-dl-wrapper"],"created_at":"2024-10-04T18:20:27.793Z","updated_at":"2026-04-29T23:32:10.055Z","avatar_url":"https://github.com/jim60105.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# backup-dl\n\n[![CodeFactor](https://www.codefactor.io/repository/github/jim60105/backup-dl/badge?style=for-the-badge)](https://www.codefactor.io/repository/github/jim60105/backup-dl)  \n![License](https://img.shields.io/github/license/jim60105/backup-dl?style=for-the-badge)\n![.NET Core](https://img.shields.io/static/v1?style=for-the-badge\u0026message=.NET+Core\u0026color=512BD4\u0026logo=.NET\u0026logoColor=FFFFFF\u0026label=)\n![Microsoft Azure](https://img.shields.io/static/v1?style=for-the-badge\u0026message=Microsoft+Azure\u0026color=0078D4\u0026logo=Microsoft+Azure\u0026logoColor=FFFFFF\u0026label=)\n![Podman](https://img.shields.io/static/v1?style=for-the-badge\u0026message=Podman\u0026color=892CA0\u0026logo=Podman\u0026logoColor=FFFFFF\u0026label=)\n![YouTube](https://img.shields.io/static/v1?style=for-the-badge\u0026message=YouTube\u0026color=FF0000\u0026logo=YouTube\u0026logoColor=FFFFFF\u0026label=)\n\n**Automated YouTube Video Archiver for Azure Blob Storage**\n\nA .NET 8.0 console application that monitors YouTube channels and playlists, automatically downloading new videos and uploading them to Azure Blob Storage with Archive tier optimization. Built as a containerized Linux application, it integrates seamlessly with automated recording workflows.\n\n## Features\n\n### Intelligent Video Download\n\n- **Powered by yt-dlp**: Leverages the actively maintained YouTube downloader with enhanced features\n- **Smart Archive Management**: Uses `archive.txt` to track downloads and prevent duplicates\n- **Multi-Source Support**: Monitor multiple YouTube channels and playlists simultaneously\n- **Optimal Format Selection**: Downloads non-DASH formats for best quality (avoids [Dynamic Adaptive Streaming over HTTP](https://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_over_HTTP) which is optimized for streaming, not archival)\n- **Configurable Format String**: Customize video format selection via environment variables\n- **Rate Limiting**: Control maximum downloads per execution to manage bandwidth\n- **Transcoding Guard**: Only downloads videos older than specified threshold to avoid incomplete transcodes\n\n### Advanced Post-Processing\n\n- **FFmpeg Integration**: Professional video processing pipeline\n- **Thumbnail Embedding**: Attaches cover image directly into video container\n- **Rich Metadata**: Embeds title, artist, date, and full description\n- **Unified Container**: Outputs all videos in MKV format for consistency\n\n### Cloud Storage Optimization\n\n- **Azure Blob Storage Integration**: Direct upload with Azure SDK\n- **Archive Tier Storage**: Automatically applies [Archive access tier](https://learn.microsoft.com/en-us/azure/storage/blobs/access-tiers-overview) for cost-effective long-term storage\n- **Incremental Progress Tracking**: Updates archive index after each successful upload\n\n### Performance \u0026 Reliability\n\n- **Asynchronous Architecture**: .NET 8.0 async/await patterns throughout\n- **Parallel Processing**: Each downloaded video immediately enters post-processing and upload pipeline\n- **Multi-Threaded Execution**: Maximizes resource utilization across all available cores\n- **Progress Persistence**: Tracks completed uploads in real-time; interruptions don't cause re-downloads\n- **Resume Capability**: Automatically detects and uploads unfinished videos from previous runs\n\n## Installation\n\nPull the latest container image from your preferred registry:\n\n```bash\n# GitHub Container Registry\npodman pull ghcr.io/jim60105/backup-dl:latest\n\n# Quay.io\npodman pull quay.io/jim60105/backup-dl:latest\n\n# Docker Hub\npodman pull docker.io/jim60105/backup-dl:latest\n```\n\n## Configuration\n\n### Required Environment Variables\n\n| Variable | Description |\n|----------|-------------|\n| `AZURE_STORAGE_CONNECTION_STRING_VTUBER` | Azure Blob Storage connection string ([view documentation](https://learn.microsoft.com/en-us/azure/storage/common/storage-account-keys-manage?tabs=azure-portal#view-account-access-keys)) |\n| `CHANNELS_IN_ARRAY` | JSON array of YouTube channel or playlist URLs to monitor |\n\n### Optional Environment Variables\n\n| Variable | Default | Description |\n|----------|---------|-------------|\n| `MAX_DOWNLOAD` | `10` | Maximum number of videos to download per execution |\n| `DATE_BEFORE` | `2` | Only download videos older than N days (prevents incomplete transcodes) |\n| `FORMAT` | `(bv*+ba/b)[protocol^=http][protocol!*=dash]` | yt-dlp format selector ([format selection syntax](https://github.com/yt-dlp/yt-dlp#format-selection)) |\n| `SCYNCHRONOUS` | - | If set, runs in synchronous mode (use only on single-core systems) |\n\n### Volume Mounts\n\n| Host Path | Container Path | Description |\n|-----------|----------------|-------------|\n| `cookies.txt` | `/app/cookies.txt` | Optional YouTube authentication cookies for private/members-only content |\n\n## Usage\n\n### Podman\n\nStore your Azure Storage connection string in an environment variable:\n\n```bash\nexport AZURE_STORAGE_CONNECTION_STRING_VTUBER=\"DefaultEndpointsProtocol=https;AccountName=...\"\n```\n\nRun the container with your configuration:\n\n```bash\npodman run --rm \\\n  --env CHANNELS_IN_ARRAY='[\"https://www.youtube.com/@channelname\", \"https://www.youtube.com/playlist?list=PLxxxxx\"]' \\\n  --env AZURE_STORAGE_CONNECTION_STRING_VTUBER \\\n  --env MAX_DOWNLOAD=\"10\" \\\n  ghcr.io/jim60105/backup-dl:latest\n```\n\nWith authentication cookies:\n\n```bash\npodman run --rm \\\n  --env CHANNELS_IN_ARRAY='[\"https://www.youtube.com/@channelname\"]' \\\n  --env AZURE_STORAGE_CONNECTION_STRING_VTUBER \\\n  --volume ./cookies.txt:/app/cookies.txt:ro \\\n  ghcr.io/jim60105/backup-dl:latest\n```\n\n### Kubernetes (Helm)\n\nDeploy as a scheduled CronJob:\n\n```bash\ngit clone --depth=1 https://github.com/jim60105/backup-dl.git\ncd backup-dl/helm\n\n# Edit values.yaml with your configuration\nvim values.yaml\n\n# Deploy to cluster\nkubectl create namespace backup-dl\nhelm install backup-dl . --namespace backup-dl\n```\n\nThe Helm chart configures:\n\n- CronJob schedule (default: daily at 1 AM)\n- SecurityContext with non-root user\n- ConfigMap for cookies.txt\n- Environment variable configuration\n\n## Technical Details\n\n- **Runtime**: .NET 8.0 with self-contained, trimmed deployment\n- **Container Base**: `mcr.microsoft.com/dotnet/runtime-deps:8.0`\n- **Architecture**: linux/amd64\n- **User**: Non-root (UID 1654) with OpenShift compatibility\n- **CI/CD**: GitHub Actions with multi-registry publishing and Trivy security scanning\n\n## License\n\n\u003e **Dependencies:**  \n\u003e - Xabe.FFmpeg: Licensed under Agreement for non-commercial use  \n\u003e - YoutubeDLSharp: BSD 3-Clause License  \n\u003e - yt-dlp: Unlicensed\n\n\u003cimg src=\"https://github.com/jim60105/backup-dl/assets/16995691/c15741ac-04f9-44e3-b97a-32ecb731c823\" alt=\"gplv3\" width=\"300\" /\u003e\n\n[GNU GENERAL PUBLIC LICENSE Version 3](LICENSE)\n\nCopyright (C) 2021 Jim Chen \u003cJim@ChenJ.im\u003e.\n\nThis program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.\n\nThis program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License along with this program. If not, see \u003chttps://www.gnu.org/licenses/\u003e.\n\n## Related Resources\n\n- [Blog Post: Building a YouTube Backup Server](https://blog.maki0419.com/2020/11/docker-youtube-dl-auto-recording-live-dl.html) (Traditional Chinese)\n- [yt-dlp Documentation](https://github.com/yt-dlp/yt-dlp)\n- [Azure Blob Storage Tiers](https://learn.microsoft.com/en-us/azure/storage/blobs/access-tiers-overview)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjim60105%2Fbackup-dl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjim60105%2Fbackup-dl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjim60105%2Fbackup-dl/lists"}