https://github.com/wyattowalsh/proxywhirl

rotating proxy system
https://github.com/wyattowalsh/proxywhirl

data data-extraction dataextraction proxy proxy-checker proxy-list proxy-scraper proxy-server proxypool python python3 rotating-proxy sqlite sqlite3 web-data-extraction

Last synced: 7 days ago
JSON representation

rotating proxy system

• Host: GitHub
• URL: https://github.com/wyattowalsh/proxywhirl
• Owner: wyattowalsh
• License: mit
• Created: 2025-05-02T07:50:00.000Z (10 months ago)
• Default Branch: main
• Last Pushed: 2026-03-02T14:42:25.000Z (8 days ago)
• Last Synced: 2026-03-02T16:32:08.088Z (8 days ago)
• Topics: data, data-extraction, dataextraction, proxy, proxy-checker, proxy-list, proxy-scraper, proxy-server, proxypool, python, python3, rotating-proxy, sqlite, sqlite3, web-data-extraction
• Language: HTML
• Homepage: https://proxywhirl.com
• Size: 1.1 GB
• Stars: 0
• Watchers: 1
• Forks: 1
• Open Issues: 8
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • Contributing: CONTRIBUTING.md
  • Funding: .github/FUNDING.yml
  • License: LICENSE
  • Agents: AGENTS.md

Awesome Lists containing this project

README

          


  

  

  





🌀 Intelligent proxy rotation that just works





[![PyPI](https://img.shields.io/pypi/v/proxywhirl?style=for-the-badge&logo=pypi&logoColor=white&color=3b82f6&labelColor=1e293b)](https://pypi.org/project/proxywhirl/)

 

[![Downloads](https://img.shields.io/pypi/dm/proxywhirl?style=for-the-badge&logo=download&logoColor=white&color=22c55e&labelColor=1e293b)](https://pypi.org/project/proxywhirl/)

 

[![Python](https://img.shields.io/badge/3.9+-a855f7?style=for-the-badge&logo=python&logoColor=white&labelColor=1e293b)](https://python.org)










[ 
 📖 Docs 
 ](https://www.proxywhirl.com/docs/)  

[ 
 🚀 Examples 
 ](examples.ipynb)  

[ 
 💬 Discussions 
 ](https://github.com/wyattowalsh/proxywhirl/discussions)






---




## ⚡ 30-Second Setup

```bash

pip install proxywhirl

```

```python

from proxywhirl import ProxyWhirl

rotator = ProxyWhirl(proxies=["http://p1:8080", "http://p2:8080"])

response = rotator.get("https://api.example.com/data")

# Dead proxies auto-ejected ✓ | Slow ones deprioritized ✓ | Fast ones favored ✓

```




---






## 🎯 Why ProxyWhirl?










9 Strategies




Round-robin, weighted, geo-targeted, performance-based & more














114 Sources




Auto-fetch from built-in providers with validation














Self-Healing




Auto-eject dead proxies, circuit breakers, recovery














Blazing Fast




Async-first, <3μs selection, zero blocking










---




## 🔄 Rotation Strategies

```python

# Switch strategies on the fly

rotator = ProxyWhirl(strategy="performance-based")

rotator.set_strategy("geo-targeted", preferences={"US": [...], "EU": [...]})

```

| Strategy | Best For |

|:---------|:---------|

| `round-robin` | Even distribution |

| `random` | Unpredictable patterns |

| `weighted` | Favor reliable proxies |

| `least-used` | Even load balance |

| `performance-based` | Lowest latency |

| `geo-targeted` | Regional routing |

| `session-persistence` | Sticky sessions |

| `cost-aware` | Budget optimization |

| `composite` | Custom pipelines |




---




## 🎣 Auto-Fetch Proxies

```python

from proxywhirl import ProxyFetcher

# Grab 300+ validated proxies in seconds

proxies = await ProxyFetcher().fetch_all(validate=True)

```




---




## 🖥️ Multiple Interfaces

**Python API**

```python

rotator.get(url)

rotator.post(url, json=data)

async with AsyncProxyWhirl() as async_rotator:

    await async_rotator.get(url)

```

**CLI**

```bash

proxywhirl fetch

proxywhirl pool list

proxywhirl health

```

**REST API**

```bash

docker-compose up -d

curl localhost:8000/api/v1/proxies

```




---






## 📚 Learn More

[ 
   📖 Full Documentation   
 ](https://www.proxywhirl.com/docs/)   

[ 
   🎯 Strategy Guide   
 ](docs/source/guides/advanced-strategies.md)   

[ 
   🤖 MCP Server   
 ](docs/source/guides/mcp-server.md)   

[ 
   📓 Examples   
 ](examples.ipynb)





---




[![Star History](https://api.star-history.com/svg?repos=wyattowalsh/proxywhirl&type=Date&theme=dark)](https://star-history.com/#wyattowalsh/proxywhirl&Date)




**[GitHub](https://github.com/wyattowalsh/proxywhirl)** · **[Issues](https://github.com/wyattowalsh/proxywhirl/issues)** · **[MIT License](LICENSE)**

Built with ❤️ for ethical web automation