https://github.com/quantumbagel/infinitescrape
Quickly and easily scrape Neal Agarwal's InfiniteCraft at 20-25 crafts/sec. Now part of FiniteCraft
https://github.com/quantumbagel/infinitescrape
infinite-craft infinite-craft-hack proxy-checker proxy-scraper python python3 python3-proxy quantumbagel scraper
Last synced: about 1 month ago
JSON representation
Quickly and easily scrape Neal Agarwal's InfiniteCraft at 20-25 crafts/sec. Now part of FiniteCraft
- Host: GitHub
- URL: https://github.com/quantumbagel/infinitescrape
- Owner: quantumbagel
- Created: 2024-02-19T18:19:34.000Z (about 1 year ago)
- Default Branch: master
- Last Pushed: 2024-02-25T15:27:00.000Z (about 1 year ago)
- Last Synced: 2025-01-22T13:51:31.218Z (3 months ago)
- Topics: infinite-craft, infinite-craft-hack, proxy-checker, proxy-scraper, python, python3, python3-proxy, quantumbagel, scraper
- Language: Python
- Homepage: https://quantumbagel.github.io/FiniteCraft
- Size: 267 KB
- Stars: 3
- Watchers: 1
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# InfiniteScrape
_Here, have my proxy code [@Pixelz22](https://github.com/Pixelz22)_
## What's this?
This code is meant to be a smart auto-rescheduling multithreaded scraper for Neal Agarwal's Infinite Craft
(https://neal.fun/infinite-craft).
I have been able to get it to reach upwards of 25 crafts/sec with just 5 workers running.
With actually good proxies, this could conceivably scale to hundreds, if not thousands, of crafts per second.
Currently, this project only has two folders: `backend` and `legacy`.
`legacy` is me screwing around with proxies and web scraping, and shouldn't really be used.
`backend` is an incredibly good backend with high scalability.
Here's some features of my backend:
* Threadsafe scheduling
* MongoDB support
* No queue.Queue() lol
* Automatic proxy ranking and scheduling
* Plain awesome
## Why would you spend your time on this?
[@Pixelz22](https://github.com/Pixelz22) decided it was a good idea to compete with me.
Here's his repository: https://github.com/Pixelz22/InfiniteCraft
## Some stuff I want to add
- MongoDB -> neal.fun encoder
- actual algorithm
- Dockerfile
- Tyler