https://github.com/vdaubry/tor-privoxy
Setup rotating proxies with Tor and Privoxy
https://github.com/vdaubry/tor-privoxy
Last synced: 9 months ago
JSON representation
Setup rotating proxies with Tor and Privoxy
- Host: GitHub
- URL: https://github.com/vdaubry/tor-privoxy
- Owner: vdaubry
- License: mit
- Created: 2015-02-06T15:48:56.000Z (almost 11 years ago)
- Default Branch: master
- Last Pushed: 2015-02-18T11:18:56.000Z (almost 11 years ago)
- Last Synced: 2023-03-29T05:43:00.717Z (almost 3 years ago)
- Language: Ruby
- Homepage:
- Size: 801 KB
- Stars: 65
- Watchers: 4
- Forks: 12
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
Multiple rotating Tor proxy
===========================
__Goal__ : Use lots of different IP addresses for multi-threaded Mechanize requests.
__How it works__ : A God script to launch many instances of Tor. Run a privoxy instance for each Tor instance to translate HTTP proxy to SOCKS proxy. HAproxy is used to distribute requests in a round-robin fashion
```
<-> Privoxy 1 <-> Tor proxy 1
Client <----> HAproxy <-> Privoxy 2 <-> Tor proxy 2
<-> Privoxy n <-> Tor proxy n
```
Demo
----
Usage
-----
Install Tor, Privoxy, HAProxy:
apt-get install tor privoxy haproxy
To run multiple Tor circuits :
```bash
# Run 10 Tor circuit
god -c god/all.rb 10
```
Get a different IP for each Mechanize request :
```ruby
agent = Mechanize.new
agent.set_proxy("127.0.0.1", 5566)
puts agent.get("http://bot.whatismyipaddress.com").content
```
Changing circuit
----------------
Each Tor instance is set to build a new circuit (new IP) every 60sec, see : https://www.torproject.org/docs/tor-manual.html.en#NewCircuitPeriod
You can force a Tor instance to build a new circuit using the [Tor gem](https://github.com/bendiken/tor-ruby)
Carefull : the ```signal``` method to send a ```newnym``` to Tor is only available on the master branch (06/02/2015)
To get the control port of the tor instance that served the current request we added a custom header ```X-Servedby``` in the response.
ex:
```ruby
control_port = page.header["x-servedby"].to_i
Tor::Controller.connect(:port => control_port) do |tor|
tor.authenticate
tor.signal("newnym")
sleep 10
end
```
We sleep 10sec because Tor delays newnym signal : https://trac.torproject.org/projects/tor/ticket/394
In the Tor logs the delay seems random between 1 and 10sec
Warning : This only works for HTTP request, intercepting encrypted connections (HTTPS) isn't supported by Privoxy (see : http://www.privoxy.org/user-manual/config.html#ACCEPT-INTERCEPTED-REQUESTS)
Hashed control password :
-------------------------
> Tor will accept connections on this port and allow those connections to control the Tor process using the
> Tor Control Protocol (described in control-spec.txt). Note: unless you also specify one or more of
> HashedControlPassword or CookieAuthentication, setting this option will cause Tor to allow any process on
> the local host to control it.
If you specify a ControlPort without a HashedControlPassword or CookieAuthentication you'll get a warning in the logs (well deserved ;)
In order to specify a control password, generate a hash with :
```tor --hash-password password```
Use the hash when starting Tor :
```--HashedControlPassword \"16:3E49D6163CCA95F2605B339E07F753C8F567DE4200E33FDF4CC6B84E44\"```
And when authenticating use your password :
```ruby
Tor::Controller.connect(:port => 50001) do |tor|
tor.authenticate("password")
end
```
Credits :
---------
Original idea by :
* http://blog.databigbang.com/running-your-own-anonymous-rotating-proxies/
* https://github.com/mattes/rotating-proxy