{"id":15368748,"url":"https://github.com/domenic/worm-scraper","last_synced_at":"2025-04-08T12:05:10.343Z","repository":{"id":31562874,"uuid":"35127602","full_name":"domenic/worm-scraper","owner":"domenic","description":"Scrapes the web serial Worm, its sequel Ward, and the bridge series Glow-worm into an ebook format","archived":false,"fork":false,"pushed_at":"2025-01-19T11:55:25.000Z","size":2250,"stargazers_count":222,"open_issues_count":4,"forks_count":48,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-04T22:05:06.804Z","etag":null,"topics":["ebook","ebook-downloader","epub","glow-worm","parahumans","ward","worm"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/domenic.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-05-05T22:43:28.000Z","updated_at":"2025-04-04T08:18:31.000Z","dependencies_parsed_at":"2025-01-25T23:06:06.952Z","dependency_job_id":"27b41148-9991-4a7c-8983-1962e3051527","html_url":"https://github.com/domenic/worm-scraper","commit_stats":{"total_commits":330,"total_committers":8,"mean_commits":41.25,"dds":"0.021212121212121238","last_synced_commit":"030e31f86e431851c4932b66b441d61b8abb16d8"},"previous_names":[],"tags_count":33,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/domenic%2Fworm-scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/domenic%2Fworm-scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/domenic%2Fworm-scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/domenic%2Fworm-scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/domenic","download_url":"https://codeload.github.com/domenic/worm-scraper/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247838441,"owners_count":21004580,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ebook","ebook-downloader","epub","glow-worm","parahumans","ward","worm"],"created_at":"2024-10-01T13:31:21.560Z","updated_at":"2025-04-08T12:05:10.318Z","avatar_url":"https://github.com/domenic.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# _Worm_ Scraper\n\nScrapes the web serial [_Worm_](https://parahumans.wordpress.com/), its sequel [_Ward_](https://www.parahumans.net/), and the bridge series [_Glow-worm_](https://www.parahumans.net/2017/10/21/glow-worm-0-1/) into an ebook format.\n\n## How to use\n\nFirst you'll need a modern version of [Node.js](https://nodejs.org/en/). The earliest version tested is v20.16.0.\n\nThen, open a terminal ([Mac documentation](http://blog.teamtreehouse.com/introduction-to-the-mac-os-x-command-line), [Windows documentation](http://www.howtogeek.com/235101/10-ways-to-open-the-command-prompt-in-windows-10/)) and install the program by typing\n\n```bash\nnpm install -g worm-scraper\n```\n\nThis will take a while as it downloads this program and its dependencies from the internet. Once it's done, try to run it, by typing:\n\n```bash\nworm-scraper --help\n```\n\nIf this outputs some help documentation, then the installation process went smoothly. You can move on to assemble the ebook by typing\n\n```bash\nworm-scraper\n```\n\nThis will take a while, but will eventually produce a `Worm.epub` file!\n\nIf you'd like to get _Ward_ instead of _Worm_, use `--book=ward`, e.g.\n\n```bash\nworm-scraper --book=ward\n```\n\nSimilarly, for _Glow-worm_:\n\n```bash\nworm-scraper --book=glow-worm\n```\n\n## Reading EPUBs on Amazon Kindle\n\nEPUBs are not the native format for Amazon Kindle devices and apps. However, you can send them to your Kindle library by following [Amazon's instructions](https://www.amazon.com/gp/help/customer/display.html?nodeId=G5WYD9SAF7PGXRNA).\n\n## Chapter titles\n\nThe original chapter titles, i.e. the ones that appear as heading at the top of each chapter published online, are not very book-like. They vary wildly, e.g. \"Gestation 1.1\", \"Daybreak – 1.2\", \"Interlude 1\", \"Flare – Interlude 2\", \"Interlude 10.y\", \"Interlude 10.5 (Bonus)\", \"Interlude 14.5 (Bonus Interlude)\".\n\nBy default, `worm-scraper` simplifies the titles to be just numbers (\"1\", \"2\", \"3\"), with interludes denoted via Roman numerals (\"Interlude I\", \"Interlude II\", ...)—or left as simply \"Interlude\" if an arc contains a single interlude.\n\nIf you want the original chapter titles, you can pass the following option:\n\n```bash\nworm-scraper --chapter-titles=original\n```\n\nThere's a third option, which is to have the interludes (and _Ward_'s epilogues) include character names. Samples of this format include \"Interlude: Danny\" or \"Interlude: Armsmaster\". Use\n\n```bash\nworm-scraper --chapter-titles=character-names\n```\n\nfor this. _This can be a slight spoiler_, because the reading experience of many interludes relies on you gradually discovering who the main character is and how they relate to what you've seen before. It can also spoil you on which characters survive, if you look ahead in the table of contents.\n\nThis style is _sort of_ aligned with how the interludes are presented in the table of contents [for _Worm_](https://parahumans.wordpress.com/table-of-contents/) and [for _Ward_](https://www.parahumans.net/table-of-contents/). But even those are inconsistent, and `worm-scraper` departs from the table of contents names in several cases. `worm-scraper` generally tries to pick the name name by which the character is first referred to in the chapter, to minimize the spoiler effect, but sometimes takes influence from the original tables of contents, or the names chosen by [the Fandom wiki](https://worm.fandom.com/wiki/Chapter_List).\n\nYou can see all the chosen character-name titles in the [`book-data/`](./book-data/) directory's files. If you strongly disagree with a choice made, please file an issue.\n\n## Text fixups\n\nThis project makes a lot of fixups to the original text, mostly around typos, punctuation, capitalization, and consistency. You can get a more specific idea of what these are via the code: there's [`convert-worker.js`](lib/convert-worker.js), where some things are handled generally, and the [`substitutions/` directory](./substitutions/), for one-off fixes.\n\nThis process is designed to be extensible, so if you notice any problems with the original text that you think should be fixed, file an issue to let me know, and we can update the fixup code so that the resulting ebook is improved. (Or better yet, send a pull request!)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdomenic%2Fworm-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdomenic%2Fworm-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdomenic%2Fworm-scraper/lists"}