{"id":22703921,"url":"https://github.com/lynkos/downloader","last_synced_at":"2025-10-14T14:13:53.042Z","repository":{"id":216736594,"uuid":"742193363","full_name":"lynkos/downloader","owner":"lynkos","description":"Basic web scraper to download media from websites. Supports .pdf generation and vertical image stacking; useful for downloading manga, comics, etc.","archived":false,"fork":false,"pushed_at":"2025-07-27T21:19:00.000Z","size":70,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-06T00:34:12.089Z","etag":null,"topics":["python","python-script","python3","scraper","scraping","scrapping","script","web-scraper","web-scraping","web-scraping-python","web-scrapping"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lynkos.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-12T00:11:11.000Z","updated_at":"2025-07-27T21:19:04.000Z","dependencies_parsed_at":"2024-05-05T04:26:28.889Z","dependency_job_id":"0d3cfb93-9c0c-4ea5-9ad6-6d3fe175f5f7","html_url":"https://github.com/lynkos/downloader","commit_stats":null,"previous_names":["lynkos/downloader"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/lynkos/downloader","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lynkos%2Fdownloader","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lynkos%2Fdownloader/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lynkos%2Fdownloader/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lynkos%2Fdownloader/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lynkos","download_url":"https://codeload.github.com/lynkos/downloader/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lynkos%2Fdownloader/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279019125,"owners_count":26086679,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-14T02:00:06.444Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["python","python-script","python3","scraper","scraping","scrapping","script","web-scraper","web-scraping","web-scraping-python","web-scrapping"],"created_at":"2024-12-10T08:13:11.943Z","updated_at":"2025-10-14T14:13:53.030Z","avatar_url":"https://github.com/lynkos.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Basic Web Scraper\nPython script to download images and mp3 files from [some] websites.\n\nDownloaded images can be concatenated and saved as a single pdf or a(n) [vertically stacked] image; useful for downloading manga, comics, etc.\n\n(Fun Fact: This project was originally created to help someone else download mp3 files from Minecraft wiki!)\n\n## Requirements\n- [x] [Anaconda](https://docs.continuum.io/free/anaconda/install) **OR** [Miniconda](https://docs.conda.io/projects/miniconda/en/latest)\n\u003e [!TIP]\n\u003e If you have trouble deciding between Anaconda and Miniconda, please refer to the table below:\n\u003e \u003ctable\u003e\n\u003e  \u003cthead\u003e\n\u003e   \u003ctr\u003e\n\u003e    \u003cth\u003e\u003ccenter\u003eAnaconda\u003c/center\u003e\u003c/th\u003e\n\u003e    \u003cth\u003e\u003ccenter\u003eMiniconda\u003c/center\u003e\u003c/th\u003e\n\u003e   \u003c/tr\u003e\n\u003e  \u003c/thead\u003e\n\u003e  \u003ctbody\u003e\n\u003e   \u003ctr\u003e\n\u003e    \u003ctd\u003eNew to conda and/or Python\u003c/td\u003e\n\u003e    \u003ctd\u003eFamiliar with conda and/or Python\u003c/td\u003e\n\u003e   \u003c/tr\u003e\n\u003e   \u003ctr\u003e\n\u003e    \u003ctd\u003eNot familiar with using terminal and prefer GUI\u003c/td\u003e\n\u003e    \u003ctd\u003eComfortable using terminal\u003c/td\u003e\n\u003e   \u003c/tr\u003e\n\u003e   \u003ctr\u003e\n\u003e    \u003ctd\u003eLike the convenience of having Python and 1,500+ scientific packages automatically installed at once\u003c/td\u003e\n\u003e    \u003ctd\u003eWant fast access to Python and the conda commands and plan to sort out the other programs later\u003c/td\u003e\n\u003e   \u003c/tr\u003e\n\u003e   \u003ctr\u003e\n\u003e    \u003ctd\u003eHave the time and space (a few minutes and 3 GB)\u003c/td\u003e\n\u003e    \u003ctd\u003eDon't have the time or space to install 1,500+ packages\u003c/td\u003e\n\u003e   \u003c/tr\u003e\n\u003e   \u003ctr\u003e\n\u003e    \u003ctd\u003eDon't want to individually install each package\u003c/td\u003e\n\u003e    \u003ctd\u003eDon't mind individually installing each package\u003c/td\u003e\n\u003e   \u003c/tr\u003e\n\u003e  \u003c/tbody\u003e\n\u003e \u003c/table\u003e\n\u003e\n\u003e Typing out entire Conda commands can sometimes be tedious, so I wrote a shell script ([`conda_shortcuts.sh` on GitHub Gist](https://gist.github.com/lynkos/7a4ce7f9e38bb56174360648461a3dc8)) to define shortcuts for commonly used Conda commands.\n\u003e \u003cdetails\u003e\n\u003e   \u003csummary\u003eExample: Delete/remove a conda environment named \u003ccode\u003etest_env\u003c/code\u003e\u003c/summary\u003e\n\u003e\n\u003e * Shortcut command\n\u003e     ```\n\u003e     rmenv test_env\n\u003e     ```\n\u003e * Manually typing out the entire command\n\u003e     ```sh\n\u003e     conda env remove -n test_env \u0026\u0026 rm -rf $(conda info --base)/envs/test_env\n\u003e     ```\n\u003e\n\u003e The shortcut has 80.8% fewer characters!\n\u003e \u003c/details\u003e\n\n## Installation\n1. Open terminal\n   * Mac\n      * Press \u003ckbd\u003e⌘\u003c/kbd\u003e + \u003ckbd\u003eSpace\u003c/kbd\u003e keys\n      * Type `terminal` **OR** (if applicable) your preferred terminal (e.g. `iterm`, etc.)\n      * Press \u003ckbd\u003eEnter\u003c/kbd\u003e key\n   * Linux\n      * Press \u003ckbd\u003eCtrl\u003c/kbd\u003e + \u003ckbd\u003eAlt\u003c/kbd\u003e + \u003ckbd\u003eT\u003c/kbd\u003e keys\n   * Windows\n      * Click search box **OR** open Start menu\n      * Type `cmd`\n      * Click `Command Prompt`, `Open`, **OR** (if needed) `Run as administrator`\n2. Verify that conda is installed\n   ```\n   conda --version\n   ```\n3. Ensure conda is up to date\n   ```\n   conda update conda\n   ```\n4. If necessary, enter the directory where you want the repository (`downloader`) to be cloned\n   ```sh\n   cd path/to/directory\n   ```\n5. Clone the repository (`downloader`), then enter its directory\n   ```sh\n   git clone https://github.com/lynkos/downloader \u0026\u0026 cd downloader\n   ```\n6. Create a conda virtual environment from [`environment.yml`](environment.yml)\n   ```\n   conda env create -f environment.yml\n   ```\n7. Activate the virtual environment (`dl_env`)\n   ```\n   conda activate dl_env\n   ```\n8. Confirm that the virtual environment (`dl_env`) is active\n   * If active, the virtual environment's name should be in parentheses `()` or brackets `[]` before your command prompt, e.g.\n      ```\n      (dl_env) $\n      ```\n   * If necessary, see which environments are available and/or currently active (active environment denoted with asterisk `*`)\n      ```\n      conda info --envs\n      ```\n      **OR**\n      ```\n      conda env list\n      ```\n\n## Usage\n### Terminal\n1. Run downloader(s)\n   * [`ImageDownloader.py`](src/ImageDownloader.py)\n     ```\n     python src/ImageDownloader.py\n     ```\n   * [`MP3Downloader.py`](src/MP3Downloader.py)\n     ```\n     python src/MP3Downloader.py\n     ```\n2. Deactivate virtual environment (`dl_env`) when finished\n   ```\n   conda deactivate\n   ```\n\n### [Visual Studio Code](https://code.visualstudio.com/download)\n1. Open the Command Palette in Visual Studio Code with the relevant keyboard shortcut\n    * Mac: \u003ckbd\u003e⌘\u003c/kbd\u003e + \u003ckbd\u003eShift\u003c/kbd\u003e + \u003ckbd\u003eP\u003c/kbd\u003e\n    * PC: \u003ckbd\u003eCtrl\u003c/kbd\u003e + \u003ckbd\u003eShift\u003c/kbd\u003e + \u003ckbd\u003eP\u003c/kbd\u003e\n2. Search and select `Python: Select Interpreter`\n3. Select the virtual environment (`dl_env`)\n4. Open [`ImageDownloader.py`](src/ImageDownloader.py) or [`MP3Downloader.py`](src/MP3Downloader.py)\n5. Run file by clicking `▷` (i.e. `Play` button) in the upper-right corner\n6. Deactivate virtual environment (`dl_env`) when finished\n   ```\n   conda deactivate\n   ```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flynkos%2Fdownloader","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flynkos%2Fdownloader","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flynkos%2Fdownloader/lists"}