{"id":13491587,"url":"https://github.com/soruly/anilist-crawler","last_synced_at":"2025-06-19T02:34:24.787Z","repository":{"id":10997204,"uuid":"77286279","full_name":"soruly/anilist-crawler","owner":"soruly","description":"Crawl data from anilist API and store in MariaDB.","archived":false,"fork":false,"pushed_at":"2025-04-28T15:00:07.000Z","size":583,"stargazers_count":44,"open_issues_count":0,"forks_count":4,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-28T16:21:49.364Z","etag":null,"topics":["anilist","anime","crawler"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/soruly.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":["soruly"],"patreon":"soruly","open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":"https://www.paypal.me/soruly/"}},"created_at":"2016-12-24T14:17:53.000Z","updated_at":"2025-04-28T15:00:11.000Z","dependencies_parsed_at":"2023-12-16T16:30:48.485Z","dependency_job_id":"4474e14b-f99f-410b-9604-dc794eb02c86","html_url":"https://github.com/soruly/anilist-crawler","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/soruly/anilist-crawler","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soruly%2Fanilist-crawler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soruly%2Fanilist-crawler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soruly%2Fanilist-crawler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soruly%2Fanilist-crawler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/soruly","download_url":"https://codeload.github.com/soruly/anilist-crawler/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soruly%2Fanilist-crawler/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260672721,"owners_count":23044849,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anilist","anime","crawler"],"created_at":"2024-07-31T19:00:58.389Z","updated_at":"2025-06-19T02:34:19.768Z","avatar_url":"https://github.com/soruly.png","language":"JavaScript","funding_links":["https://github.com/sponsors/soruly","https://patreon.com/soruly","https://www.paypal.me/soruly/"],"categories":["JavaScript","Downloaders"],"sub_categories":[],"readme":"# anilist-crawler\r\n\r\n[![License](https://img.shields.io/github/license/soruly/anilist-crawler.svg?style=flat-square)](https://github.com/soruly/anilist-crawler/blob/master/LICENSE)\r\n[![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/soruly/anilist-crawler/node.js.yml?style=flat-square)](https://github.com/soruly/anilist-crawler/actions)\r\n\r\nCrawl data from [AniList](https://anilist.co/home) API and store as json in file system, MariaDB, or elasticsearch.\r\n\r\n## Requirements\r\n\r\n- Node.js \u003e= 20.12\r\n- MariaDB \u003e= 10.5 (optional)\r\n- elasticsearch \u003e= 7.0 (optional)\r\n\r\n## How to use\r\n\r\n1. Clone this repository\r\n\r\n2. `npm install`\r\n\r\n3. copy `.env.example` and rename to `.env`\r\n\r\n4. Configure `.env` for your mariaDB, or elasticsearch, leave any of DB_HOST, ES_HOST, FS_DIR empty if you don't need it\r\n\r\n## Examples\r\n\r\nFetch anime ID 123\r\n\r\n`node index.js --anime 123`\r\n\r\nFetch all anime in page 240\r\n\r\n`node index.js --page 240`\r\n\r\nFetch all anime from page 240 to 244 (inclusive)\r\n\r\n`node index.js --page 240-244`\r\n\r\nFetch all anime from page 240 to the last page\r\n\r\n`node index.js --page 240-`\r\n\r\nSometimes anime would be deleted from AniList, but it still exists locally in your database. You can use `--clean` to get a clean copy every time you start crawling.\r\n\r\n`node index.js --clean --page 240-`\r\n\r\nFor details of AniList API please visit https://github.com/AniList/ApiV2-GraphQL-Docs/\r\n\r\nYou can try the interactive query tool here. https://anilist.co/graphiql\r\n\r\n## Notes\r\n\r\n- API request limit exceed (HTTP 429) has not been handled yet. With 60 requests/min per IP, it is unlikely to hit the limit with complex query.\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsoruly%2Fanilist-crawler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsoruly%2Fanilist-crawler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsoruly%2Fanilist-crawler/lists"}