{"id":22330434,"url":"https://github.com/tuvimen/wordpress-madara-scraper","last_synced_at":"2026-05-05T08:37:42.541Z","repository":{"id":223941030,"uuid":"761970455","full_name":"TUVIMEN/wordpress-madara-scraper","owner":"TUVIMEN","description":"A bash script for scraping image focused wordpress madara extension sites","archived":false,"fork":false,"pushed_at":"2025-06-04T17:11:49.000Z","size":45,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-07-23T06:45:00.462Z","etag":null,"topics":["bash","image-hoarding","json","reliq","scraper","wordpress-madara"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TUVIMEN.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-02-22T20:41:13.000Z","updated_at":"2025-06-04T17:11:51.000Z","dependencies_parsed_at":"2024-05-30T22:09:10.548Z","dependency_job_id":"260ab922-9932-4a99-9b6b-705aa4b78925","html_url":"https://github.com/TUVIMEN/wordpress-madara-scraper","commit_stats":null,"previous_names":["tuvimen/wordpress_madara","tuvimen/wordpress-madara-scraper"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/TUVIMEN/wordpress-madara-scraper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TUVIMEN%2Fwordpress-madara-scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TUVIMEN%2Fwordpress-madara-scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TUVIMEN%2Fwordpress-madara-scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TUVIMEN%2Fwordpress-madara-scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TUVIMEN","download_url":"https://codeload.github.com/TUVIMEN/wordpress-madara-scraper/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TUVIMEN%2Fwordpress-madara-scraper/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32642278,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-04T10:08:07.713Z","status":"online","status_checked_at":"2026-05-05T02:00:06.033Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bash","image-hoarding","json","reliq","scraper","wordpress-madara"],"created_at":"2024-12-04T04:06:53.353Z","updated_at":"2026-05-05T08:37:42.524Z","avatar_url":"https://github.com/TUVIMEN.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# wordpress-madara-scraper\n\nA bash script for scraping image focused madara wordpress in json.\n\n## Requirements\n\n - [reliq](https://github.com/TUVIMEN/reliq)\n - [jq](https://github.com/stedolan/jq)\n\n## Installation\n\n    install -m 755 wordpress-madara-scraper /usr/bin\n\n## Json format\n\nHere's example of [comics](comics-example.json).\n\n## Structure\n\nThere are two sets of options that define what will be downloaded, and are divided into those that download metadata and those thad download images.\n\n### Metadata\n\nIs downloaded by `-p`, `-c`, `-l`, `--full-comic` and `--full-pages`. Files created by them are named by the md5 hash of their urls.\n\n`-p` takes `LINK` argument and outputs a list of urls to comics. This might be used to get all of the comics from the website, category or an artist.\n\n`-c` takes `FILE` argument from which it reads urls to comics and saves them in json files.\n\n`-l` takes `FILE` argument from which it reads urls to chapters and saves the list of urls to their images to files.\n\n`--full-comic` takes `LINK` argument and downloads comic and its chapters creating a directory for its chapters named with its name with '_' character at the end.\n\n`--full-pages` takes `LINK` argument and downloads all comics from pages using `--full-comic`.\n\nExample structure created by `--full-pages`:\n\n    0001c692d6cadaa3c692412bc0ac51fe\n    0001c692d6cadaa3c692412bc0ac51fe_/\n        02c8e3f630d0cd48f13515f65a91fe3e\n        0ba18e4d9db640693a8584b01983b451\n        0df4a828f07137e21f585aa29375b223\n    008216d512f75bcb86e2a08c4df7ae8c\n    008216d512f75bcb86e2a08c4df7ae8c_/\n        091bf018a3e41cb974c20be4901ba89a\n        4e35d40ad644114a17e2995b30aa52fb\n\n### Images\n\nThese options are meant for consumption purposes only, and are just a practical simplification of Metadata. Files created by them are named by their names with `/` character translated to `|`.\n\n`--download-chapter` takes `LINK` as argument and downloads the images of the chapter\n`--download-comic` takes `LINK` as argument and downloads the comic, its chapters and their images.\n`--download-pages` takes `LINK` as argument and downloads all comics from pages using `--download-comic`\n\nExample structure created by `--download-pages`:\n\n    +99 Wooden stick manhwa\n    +99 Wooden stick manhwa_/\n        Chapter 1/\n            ch_0_1.jpg\n            ch_0_2.jpg\n            ch_0_3.jpg\n        Chapter 89.5/\n            45.webp\n            46.webp\n    My School Life Pretending To Be a Worthless Person\n    My School Life Pretending To Be a Worthless Person_/\n        Chapter 1/\n            ch_0_1.jpg\n            ch_0_2.jpg\n            ch_0_3.jpg\n        Chapter 59/\n            13.webp\n            14.webp\n            15.webp\n\n## Tested sites\n\n    https://manhwatop.com/\n    https://www.nightcomic.com/\n    https://shibamanga.com/\n    https://topmanhua.com/\n\n## Usage\n\n    wordpress-madara-scraper [OPTIONS]...\n\nDownload the images of the chapter, comic, genre and the whole site\n\n    wordpress-madara-scraper --download-chapter 'https://manhwatop.com/manga/love-hug/chapter-233/'\n    wordpress-madara-scraper --download-comic 'https://manhwatop.com/manga/love-hug/'\n    wordpress-madara-scraper --download-pages 'https://manhwatop.com/manga-genre/magical-genre/'\n    wordpress-madara-scraper --download-pages 'https://manhwatop.com/'\n\nDownload the metadata of comic and the whole page\n\n    wordpress-madara-scraper --full-comic 'https://nightcomic.com/manga/versatile-mage/'\n    wordpress-madara-scraper --full-pages 'https://nightcomic.com/new/'\n\nDownload links to comics into FILE\n\n    wordpress-madara-scraper -p 'https://www.topmanhua.com' \u003e FILE\n\nDownload comics from links in FILE using 4 threads into DIR, it will create json files named by md5 hash of their links\n\n    wordpress-madara-scraper -d DIR -t 4 -c FILE\n\nDownload images links from chapters in comics FILE into FILES named by md5 hash of their links\n\n    wordpress-madara-scraper -l FILE\n\nGet some help\n\n    wordpress-madara-scraper -h\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftuvimen%2Fwordpress-madara-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftuvimen%2Fwordpress-madara-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftuvimen%2Fwordpress-madara-scraper/lists"}