{"id":21332000,"url":"https://github.com/quantumbagel/infinitescrape","last_synced_at":"2025-03-16T00:44:26.114Z","repository":{"id":223350667,"uuid":"760070566","full_name":"quantumbagel/InfiniteScrape","owner":"quantumbagel","description":"Quickly and easily scrape Neal Agarwal's InfiniteCraft at 20-25 crafts/sec. Now part of FiniteCraft","archived":false,"fork":false,"pushed_at":"2024-02-25T15:27:00.000Z","size":273,"stargazers_count":3,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-22T13:51:31.218Z","etag":null,"topics":["infinite-craft","infinite-craft-hack","proxy-checker","proxy-scraper","python","python3","python3-proxy","quantumbagel","scraper"],"latest_commit_sha":null,"homepage":"https://quantumbagel.github.io/FiniteCraft","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/quantumbagel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-02-19T18:19:34.000Z","updated_at":"2024-03-02T19:02:11.000Z","dependencies_parsed_at":"2024-02-25T16:33:45.963Z","dependency_job_id":"f6826dd6-8f03-45a1-8177-900ff6e77788","html_url":"https://github.com/quantumbagel/InfiniteScrape","commit_stats":null,"previous_names":["quantumbagel/infinite_scrape","quantumbagel/infinitescrape"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quantumbagel%2FInfiniteScrape","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quantumbagel%2FInfiniteScrape/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quantumbagel%2FInfiniteScrape/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quantumbagel%2FInfiniteScrape/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/quantumbagel","download_url":"https://codeload.github.com/quantumbagel/InfiniteScrape/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243809843,"owners_count":20351407,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["infinite-craft","infinite-craft-hack","proxy-checker","proxy-scraper","python","python3","python3-proxy","quantumbagel","scraper"],"created_at":"2024-11-21T22:45:39.787Z","updated_at":"2025-03-16T00:44:26.091Z","avatar_url":"https://github.com/quantumbagel.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# InfiniteScrape\n\n_Here, have my proxy code [@Pixelz22](https://github.com/Pixelz22)_\n\n## What's this?\n\nThis code is meant to be a smart auto-rescheduling multithreaded scraper for Neal Agarwal's Infinite Craft\n(https://neal.fun/infinite-craft).\nI have been able to get it to reach upwards of 25 crafts/sec with just 5 workers running.\nWith actually good proxies, this could conceivably scale to hundreds, if not thousands, of crafts per second.\n\nCurrently, this project only has two folders: `backend` and `legacy`. \n\n`legacy` is me screwing around with proxies and web scraping, and shouldn't really be used.\n\n`backend` is an incredibly good backend with high scalability. \n\n\nHere's some features of my backend:\n\n* Threadsafe scheduling\n* MongoDB support\n* No queue.Queue() lol\n* Automatic proxy ranking and scheduling\n* Plain awesome\n\n## Why would you spend your time on this?\n\n\n [@Pixelz22](https://github.com/Pixelz22) decided it was a good idea to compete with me. \n\nHere's his repository: https://github.com/Pixelz22/InfiniteCraft\n\n\n## Some stuff I want to add\n\n- MongoDB -\u003e neal.fun encoder\n- actual algorithm\n- Dockerfile\n- Tyler\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquantumbagel%2Finfinitescrape","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fquantumbagel%2Finfinitescrape","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquantumbagel%2Finfinitescrape/lists"}