{"id":23693339,"url":"https://github.com/alyetama/internetarchive-youtube","last_synced_at":"2025-10-03T14:19:42.297Z","repository":{"id":39467710,"uuid":"498274014","full_name":"Alyetama/internetarchive-youtube","owner":"Alyetama","description":"🚀 GitHub Action and CLI tool to archive YouTube channels by uploading the channel's videos to archive.org","archived":false,"fork":false,"pushed_at":"2025-08-09T20:26:36.000Z","size":93,"stargazers_count":20,"open_issues_count":2,"forks_count":20,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-16T04:24:28.555Z","etag":null,"topics":["archive","bot","cli","github-actions","internetarchive","wayback-machine","youtube"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Alyetama.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-05-31T09:41:54.000Z","updated_at":"2025-08-09T20:26:40.000Z","dependencies_parsed_at":"2024-01-15T16:23:50.870Z","dependency_job_id":"f45d3ad0-d222-4b79-aaa1-62ba53ab9f04","html_url":"https://github.com/Alyetama/internetarchive-youtube","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Alyetama/internetarchive-youtube","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Alyetama%2Finternetarchive-youtube","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Alyetama%2Finternetarchive-youtube/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Alyetama%2Finternetarchive-youtube/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Alyetama%2Finternetarchive-youtube/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Alyetama","download_url":"https://codeload.github.com/Alyetama/internetarchive-youtube/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Alyetama%2Finternetarchive-youtube/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278173668,"owners_count":25942355,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-03T02:00:06.070Z","response_time":53,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["archive","bot","cli","github-actions","internetarchive","wayback-machine","youtube"],"created_at":"2024-12-30T03:51:31.784Z","updated_at":"2025-10-03T14:19:42.253Z","avatar_url":"https://github.com/Alyetama.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Internetarchive-YouTube\n\n[![Poetry-build](https://github.com/Alyetama/internetarchive-youtube/actions/workflows/poetry-build.yml/badge.svg)](https://github.com/Alyetama/internetarchive-youtube/actions/workflows/poetry-build.yml) [![Supported Python versions](https://img.shields.io/badge/Python-%3E=3.7-blue.svg)](https://www.python.org/downloads/) [![PEP8](https://img.shields.io/badge/Code%20style-PEP%208-orange.svg)](https://www.python.org/dev/peps/pep-0008/) \n\n🚀 GitHub Action and CLI tool to archive YouTube channels by automatically uploading an entire YouTube channel to [archive.org](https://archive.org) in few clicks.\n\n## 📌 Global Requirements\n\n- All you need is an [Internet Archive account](https://archive.org/account/signup).\n\n## 🔧 Usage\n\n- ⚡️ To use this tool as a GitHub Action, jump to [GitHub Action: Getting Started](\u003c#%EF%B8%8F-github-action-getting-started\u003e \"GitHub Action: Getting Started\").\n- 🧑‍💻 To use this tool as a command line interface (CLI), jump to [CLI: Getting Started](\u003c#-cli-getting-started\u003e \"CLI: Getting Started\").\n\n---\n\n### ⚡️ GitHub Action: Getting Started\n\n\u003cdetails\u003e\n  \u003csummary\u003eUsing internetarchive-youtube as a GitHub Action instructions\u003c/summary\u003e\n\n1. **[Fork this repository](https://github.com/Alyetama/yt-archive-sync/fork).**\n\n2. **Enable the workflows in your fork.**\n\n\u003cimg src=\"https://i.imgur.com/J1udGei.jpeg\"  width=\"720\" alt=\"\"\u003e \n\u003cimg src=\"https://i.imgur.com/WhyFjWy.jpeg\"  width=\"720\" alt=\"\"\u003e \n\n3. **[Create a backend database (or JSON bin)](\u003c#%EF%B8%8F-creating-a-backend-database\u003e \"Creating a backend database\").**\n\n4. **Add your *Archive.org* credentials to the repository's actions secrets:**\n  - `ARCHIVE_USER_EMAIL`\n  - `ARCHIVE_PASSWORD`\n\n5. **Add a list of the channels you want to archive as a `CHANNELS` secret to the repository's actions secrets:**\n\nThe `CHANNELS` secret should be formatted like this example:\n\n```YAML\nCHANNEL_NAME: CHANNEL_URL\nFOO: FOO_CHANNEL_URL\nFOOBAR: FOOBAR_CHANNEL_URL\nSOME_CHANNEL: SOME_CHANNEL_URL\n```\n\nDon't add any quotes around the name or the URL, and make sure to keep one space between the colon and the URL.\n\n6. **Add the database secret(s) to the repository's *Actions* secrets:**\n\nIf you picked **option 1 (MongoDB)**, add this secret:\n  - `MONGODB_CONNECTION_STRING`\nThe value of the secret is the database conneciton string.\n\nIf you picked **option 2 (JSON bin)**, add this additional secret:\n  - `JSONBIN_KEY`  \nThe value of this secret is the *MASTER KEY* token you copied from JSONbin.\n\n7. (optional) You can add command line options other than the defaults by creating a secret called `CLI_OPTIONS` and adding the options to the secret. See the [CLI: Getting Started](\u003c#-cli-getting-started\u003e \"CLI: Getting Started\") for a list of all the available options.\n\n8. **Run the workflow under `Actions` manually, or wait for it to run automatically every 6 hours.**\n\nThat's it! 🎉\n\n\u003c/details\u003e\n\n\n### 🧑‍💻 CLI: Getting Started\n\n\u003cdetails\u003e\n  \u003csummary\u003eUsing internetarchive-youtube as a CLI tool instructions\u003c/summary\u003e\n\n#### Requirements:\n\n- 🐍 [Python\u003e=3.7](https://www.python.org/downloads/)\n\n#### ⬇️ Installation:\n\n```sh\npip install internetarchive-youtube\n```\n\nThen login to internetarchive:\n\n```sh\nia configure\n```\n\n#### 🗃️ Backend database:\n\n- [Create a backend database (or JSON bin)](\u003c#%EF%B8%8F-creating-a-backend-database\u003e \"Creating a backend database\") to track the download/upload overall progress.\n\n- If you choose **MongoDB**, export the connection string as an environment variable:\n\n```sh\nexport MONGODB_CONNECTION_STRING=mongodb://username:password@host:port\n\n# or add it to your shell configuration file:\necho \"MONGODB_CONNECTION_STRING=$MONGODB_CONNECTION_STRING\" \u003e\u003e \"$HOME/.$(basename $SHELL)rc\"\nsource \"$HOME/.$(basename $SHELL)rc\"\n```\n\n- If you choose **JSONBin**, export the master key as an environment variable:\n\n```sh\nexport JSONBIN_KEY=xxxxxxxxxxxxxxxxx\n\n# or add it to your shell configuration file:\necho \"JSONBIN_KEY=$JSONBIN_KEY\" \u003e\u003e \"$HOME/.$(basename $SHELL)rc\"\nsource \"$HOME/.$(basename $SHELL)rc\"\n```\n\n#### ⌨️ Usage:\n\n```\nusage: ia-yt [-h] [-p PRIORITIZE] [-s SKIP_LIST] [-f] [-t TIMEOUT] [-n] [-a] [-c CHANNELS_FILE] [-S] [-C] [-m] [-T THREADS] [-k] [-i IGNORE_VIDEO_IDS]\n\noptions:\n  -h, --help            show this help message and exit\n  -p PRIORITIZE, --prioritize PRIORITIZE\n                        Comma-separated list of channel names to prioritize when processing videos.\n  -s SKIP_LIST, --skip-list SKIP_LIST\n                        Comma-separated list of channel names to skip.\n  -f, --force-refresh   Refresh the database after every video (Can slow down the workflow significantly, but is useful when running multiple concurrent\n                        jobs).\n  -t TIMEOUT, --timeout TIMEOUT\n                        Kill the job after n hours (default: 5.5).\n  -n, --no-logs         Don't print any log messages.\n  -a, --add-channel     Add a channel interactively to the list of channels to archive.\n  -c CHANNELS_FILE, --channels-file CHANNELS_FILE\n                        Path to the channels list file to use if the environment variable `CHANNELS` is not set (default: ~/.yt_channels.txt).\n  -S, --show-channels   Show the list of channels in the channels file.\n  -C, --create-collection\n                        Creates/appends to the backend database from the channels list.\n  -m, --multithreading  Enables processing multiple videos concurrently.\n  -T THREADS, --threads THREADS\n                        Number of threads to use when multithreading is enabled. Defaults to the optimal maximum number of workers.\n  -k, --keep-failed-uploads\n                        Keep the files of failed uploads on the local disk.\n  -i IGNORE_VIDEO_IDS, --ignore-video-ids IGNORE_VIDEO_IDS\n                        Comma-separated list or a path to a file containing a list of video ids to ignore.\n```\n\n\u003c/details\u003e\n\n---\n\n## 🏗️ Creating A Backend Database\n\n\u003cdetails\u003e\n  \u003csummary\u003eCreating A Backend Database instructions\u003c/summary\u003e\n\n  **NOTICE: The `JSONBIN` option will not work at the moment because jsonbin.io changed their API recently. Please use MongoDB for now until the next release.**\n\n- **Option 1:**  MongoDB (recommended).\n  - Self-hosted (see: [Alyetama/quick-MongoDB](https://github.com/Alyetama/quick-MongoDB) or [dockerhub image](https://hub.docker.com/_/mongo)).\n  - Free cloud database on [Atlas](https://www.mongodb.com/database/free).\n- **Option 2:** JSON bin (if you want a quick start).\n  - Sign up to JSONBin [here](https://jsonbin.io/login).\n  - Click on `VIEW MASTER KEY`, then copy the key.\n  \n\u003c/details\u003e\n\n\n## 📝 Notes\n\n- Information about the `MONGODB_CONNECTION_STRING` can be found [here](https://www.mongodb.com/docs/manual/reference/connection-string/).\n- Jobs can run for a maximum of 6 hours, so if you're archiving a large channel, the job might die, but it will resume in a new job when it's scheduled to run.\n- Instead of raw text, you can pass a file path or a file URL with a list of channels formatted as `CHANNEL_NAME: CHANNEL_URL`. You can also pass raw text or a file of the channels in JSON format `{\"CHANNEL_NAME\": \"CHANNEL_URL\"}`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falyetama%2Finternetarchive-youtube","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falyetama%2Finternetarchive-youtube","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falyetama%2Finternetarchive-youtube/lists"}