{"id":15598882,"url":"https://github.com/jofaval/album-scraper","last_synced_at":"2025-07-15T22:08:56.114Z","repository":{"id":63973027,"uuid":"570921246","full_name":"jofaval/album-scraper","owner":"jofaval","description":"Modularized python script for webscraping of images. Download an Album of Chapters blazingly fast","archived":false,"fork":false,"pushed_at":"2023-07-03T21:48:43.000Z","size":473,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-29T12:15:02.848Z","etag":null,"topics":["album","automation","images","multiprocessing","multithreading","python","scraper","scraping","webscraping"],"latest_commit_sha":null,"homepage":"https://jofaval.github.io/album-scraper/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jofaval.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-26T15:08:35.000Z","updated_at":"2023-03-07T08:06:56.000Z","dependencies_parsed_at":"2024-12-09T17:13:42.320Z","dependency_job_id":"ea64ff59-adf5-44e7-b5f9-41b6ae306ba6","html_url":"https://github.com/jofaval/album-scraper","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jofaval/album-scraper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jofaval%2Falbum-scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jofaval%2Falbum-scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jofaval%2Falbum-scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jofaval%2Falbum-scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jofaval","download_url":"https://codeload.github.com/jofaval/album-scraper/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jofaval%2Falbum-scraper/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265463902,"owners_count":23770315,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["album","automation","images","multiprocessing","multithreading","python","scraper","scraping","webscraping"],"created_at":"2024-10-03T01:41:48.322Z","updated_at":"2025-07-15T22:08:56.068Z","avatar_url":"https://github.com/jofaval.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Album scraper\n\nModularized python script for webscraping images.\n\nYou can watch the changelog at [CHANGELOG](./CHANGELOG.md), it's an overall view, not greatly detailed.\n\n## Contents\n\n1. [Description](#description)\n1. [Motivation](#motivation)\n1. [History](#history)\n1. [How does it work?](#how-does-it-work)\n   1. [Requirements](#requirements)\n   1. [Installation](#installation)\n   1. [Create a module](#create-a-module)\n   1. [Executing a module](#executing-a-module)\n1. [What pages does it work on?](#what-pages-does-it-work-on)\n1. [Legal notice](#legal-notice)\n\n## Description\n\n[Back to the contents](#contents)\n\nScrape the images from a website into an album formatting (file wise).\n\nIt's not perfect, it's straight to the point. It (somewhat) complies with Pylint, where useful\n\n## Motivation\n\n[Back to the contents](#contents)\n\nIt's really annoying to download and organize those images, for whatever that reason may be, and doing it all that work manually, too much mental overhead, what if you skipped one image? how do you check it? they're unsorted, name may be the default (filename on the host), will you manually rename them? Jow many will you miss while doing so? And many more...\n\nWhy act like a robot when a robot can do it for you?\n\n## History\n\n[Back to the contents](#contents)\n\nStarted more than a year ago, _at the time the project started, 2022_, and I applied little modifications here and there, iterating, and I finally decided to try and encapsulate it for a modular usage.\n\nIt started as an assignment, from which I was reminded of the existence of webscraping as a technique, and an opportunity to automate one routine I had arose. I expanded upon it until I've decided to publish it on open-source it.\n\n## How does it work?\n\n[Back to the contents](#contents)\n\n### Requirements\n\n[Back to the contents](#contents)\n\n- Python 3.8.4 or higher\n  - not tested on lower, but it should work on \u003e= 3.6.x\n- Some decent internet speed, It worked nicely on 20 MB/s\n\nLibraries:\n\n- Pip\n\nOr manually install the following:\n\nLook at the [requirements.txt](./requirements.txt) file.\n\n### Installation\n\n[Back to the contents](#contents)\n\n```bash\npython -m pip install\n# or pip3 install if you're on linux/unix systems\n```\n\n```bash\npip install -r requirements.txt # pip3 for any unix system\n```\n\nsince python will be installed by default pip3\n\n### Create a module\n\n[Back to the contents](#contents)\n\nFolow the [example](./modules/example/).\n\n### Executing a module\n\n[Back to the contents](#contents)\n\nTo execute it, simply create an `__init__.py` at the root for the [`modules`](./modules/) folder. And \"export\" the desired init function to execute, the `__main__.py` will handle the rest:\n\n```python\nfrom .example import init\n```\n\nAnd run the project, at the root:\n\n```bash\npython . # python3 for unix/linux\n# or run the project folder\npython album-scraper\n```\n\nOr watch out for the CLI, which might make it easier, and with more options\n\n## What pages does it work on?\n\n[Back to the contents](#contents)\n\nThose without pagination, and with all the links you want to scrape visible at the homepage\n\n![Pages example](./images/base.excalidraw.png)\n_Page layouts, Image designed with [excalidraw](https://excalidraw.com/)_\n\n## Legal notice\n\n[Back to the contents](#contents)\n\nI am not endorsing any illegal activities, images still hold their licensing and ownership by it's rightful author(s). If the robots.txt does not allow the officially legal webscraping of the website, any unrightful, mischievous or illegal act will still be illegal, and not my responsibility.\n\nUse this scripts at your own responsibility.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjofaval%2Falbum-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjofaval%2Falbum-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjofaval%2Falbum-scraper/lists"}