{"id":21168817,"url":"https://github.com/tbhaxor/proxyforge","last_synced_at":"2025-12-29T20:39:30.178Z","repository":{"id":196731087,"uuid":"676154142","full_name":"tbhaxor/ProxyForge","owner":"tbhaxor","description":"Solve IP Throttling with Rotating Proxies","archived":false,"fork":false,"pushed_at":"2023-10-16T10:26:08.000Z","size":51,"stargazers_count":21,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-21T10:30:54.391Z","etag":null,"topics":["digitalocean","digitalocean-droplets","hacktoberfest","hacktoberfest-2023","http-proxy","networking","proxy","proxy-server","squid-proxy"],"latest_commit_sha":null,"homepage":"https://proxyforge.tbhaxor.com/","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tbhaxor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-08T14:50:10.000Z","updated_at":"2025-01-09T06:35:12.000Z","dependencies_parsed_at":null,"dependency_job_id":"8088e2c7-9735-445c-9479-74d94f553d52","html_url":"https://github.com/tbhaxor/ProxyForge","commit_stats":null,"previous_names":["tbhaxor/proxyforge"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tbhaxor%2FProxyForge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tbhaxor%2FProxyForge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tbhaxor%2FProxyForge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tbhaxor%2FProxyForge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tbhaxor","download_url":"https://codeload.github.com/tbhaxor/ProxyForge/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243615631,"owners_count":20319733,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["digitalocean","digitalocean-droplets","hacktoberfest","hacktoberfest-2023","http-proxy","networking","proxy","proxy-server","squid-proxy"],"created_at":"2024-11-20T15:17:58.535Z","updated_at":"2025-12-29T20:39:25.139Z","avatar_url":"https://github.com/tbhaxor.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Proxy Forge\n\nProxy Forge is a free and open-source tool that helps you to set up OSI level 4 proxies with rotating IPs on the Digital Ocean platform. It is a powerful and innovative solution that addresses one of the most pressing challenges faced by developers and businesses in today's digital landscape: IP throttling and blacklisting. Proxy Forge empowers you to conquer these obstacles by providing a seamless, reliable, and easy-to-deploy solution.\n\nIn simpler terms, Proxy Forge helps you to create a pool of rotating IP addresses that you can use to access the internet. This can be useful for a variety of purposes, such as avoiding IP bans, scraping websites, and testing web applications.\n\nProxy Forge is a powerful tool that can be used by both developers and businesses. It is easy to use and deploy, and it provides a number of features that make it a valuable tool for anyone who needs to access the internet anonymously or avoid IP bans.\n\n## Features\n\n- **Rotating IPs:** Automatically rotates IP addresses from a set of deployed droplets to avoid detection and blacklisting.\n- **Terraform Scripts:** Provides Terraform scripts for straightforward setup and management.\n- **Scalability:** Easily scale your proxy fleet up or down to handle your specific needs.\n- **Customizable Configuration:** Fine-tune proxy settings to match your requirements. (check [below](#customizations))\n- **IP Throttling Avoidance:** Effectively bypass IP throttling mechanisms used by websites and services.\n- **Detailed Logging:** Comprehensive logging for monitoring and troubleshooting.\n- **Open Source:** Proxy Forge is open-source, allowing for community contributions and customization.\n\n## Motivation\n\nI had to find a solution for one of my clients where certain eCommerce websites were supposed to be scraped but only allowed IP addresses from the UK region. I purchased a VPC from a cloud provider in the UK, but I later discovered that the target website implements IP throttling, which temporarily disables it. As a result, I built this solution manually (no Terraform scripts) and without load balancers. \n\nI've been learning Terraform since then, and automated its setup for you. Now it takes less than 5 minutes to complete.\n\n\n## Why DigitalOcean?\n\nThere are many reasons why you might choose to use DigitalOcean over other cloud providers. Some of the key benefits include:\n\n- **Simplicity:** DigitalOcean is known for its simple and user-friendly interface. It is easy to get started with DigitalOcean, even if you are new to cloud computing.\n- **Affordability:** DigitalOcean offers competitive pricing for its cloud services. Additionally, it provides a substantial 2 TB of free bandwidth each month.\n- **Performance:** DigitalOcean uses high-performance hardware for its Droplets, so you can be sure that your applications will run smoothly.\n- **Reliability:** DigitalOcean has a strong track record of reliability and uptime.\n- **Community:** DigitalOcean has a large and active community of users and developers. This means that there is a wealth of resources available to help you get the most out of your DigitalOcean account.\n\n## Proof of Concept\n\n\u003ccenter\u003e\n    \n[![asciicast](https://asciinema.org/a/0XtgYX87bPJgwORDU9jKDKsrB.svg)](https://asciinema.org/a/0XtgYX87bPJgwORDU9jKDKsrB)\n\n\u003c/center\u003e\n\n## Up and Running\n\n### Prerequisites\n\nBefore using Proxy Forge, ensure you have the following prerequisites in place:\n\n1. [Digital Ocean Account](https://www.digitalocean.com/?refcode=a7587e994b7e\u0026utm_campaign=Referral_Invite\u0026utm_medium=Referral_Program\u0026utm_source=badge): If you do not have an account, I would recommend creating it through my referral to get **200 USD credits for 60 days**. In this time you can test this solution.\n\n2. [Terraform](https://docs.digitalocean.com/reference/terraform/getting-started/) installed on your local machine\n\n### Steps\n\n1.  Clone the repository\n\n    ```console\n    git clone --depth=1 --branch=main https://github.com/tbhaxor/ProxyForge.git proxyforge\n    cd proxyforge\n    ```\n\n2.  Create a file for terraform variables.\n\n    ```console\n    touch terraform.tfvars\n    ```\n\n    \u003e **Note** Skip it, if you wish to provide the variables and values from [environment config](https://developer.hashicorp.com/terraform/cli/config/environment-variables#tf_var_name).\n\n3.  Setup at least `token` and `region` in the file. See [variables](#variables) section below.\n\n4.  Initialize the provider, plan and apply the changes\n\n    ```console\n    terraform init\n    terraform plan\n    terraform apply\n    ```\n\n5.  Wait for some time for squid proxy to setup and load balancer to initialize routing.\n6.  (Optional) Test the deployment\n\n    ```sh\n    LOAD_BALANCER_IP=$(terraform output -raw lb-ip)\n    SQUID_USERNAME=proxyforge # assuming you did not change squid-credentials.username in the tfvars\n    SQUID_PASSWORD=proxyforge # assuming you did not change squid-credentials.password in the tfvars\n\n    while true; do curl -x \"http://$SQUID_USERNAME:$SQUID_PASSWORD@$LOAD_BALANCER_IP:80\" https://ifconfig.me; echo; done\n    ```\n\n### Variables\n\n|    Variable Name    |                        Default                         | Description                                                                                                                                                                                                                                                                                                                             |\n| :-----------------: | :----------------------------------------------------: | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n|       `token`       |                          N/A                           | **Required** API token with read/write permissions. [see more](https://docs.digitalocean.com/reference/api/create-personal-access-token/)                                                                                                                                                                                               |\n|      `region`       |                          N/A                           | **Required** Datacenter region to deploy all the resources. [see more](https://docs.digitalocean.com/products/platform/availability-matrix/)                                                                                                                                                                                            |\n|  `ssh-fingerprint`  |                        _`null`_                        | SSH fingerprint id for droplets to use. If this is ommited, it will send you one-time-password on the email. Can be obtained from [security tab](https://i.imgur.com/TNTj7D8.png) of [accounts](https://cloud.digitalocean.com/account/security) page. [see also](https://docs.digitalocean.com/products/droplets/how-to/add-ssh-keys/) |\n|      `prefix`       |                          _pf_                          | A prefix to quickly identify proxy-forge resources.                                                                                                                                                                                                                                                                                     |\n|    `slave-count`    |                          _2_                           | Number of instances on which squid proxy will be installed.                                                                                                                                                                                                                                                                             |\n|     `lb-count`      |                          _1_                           | Number of master nodes to setup for load balancer, min `1` is required.                                                                                                                                                                                                                                                                 |\n|     `tag-name`      |                  _proxy-forge-slave_                   | Tag name to group slave droplets.                                                                                                                                                                                                                                                                                                       |\n|   `droplet-size`    |            _{ slave = \"s-1vcpu-1gb-amd\" }_             | Droplet size to use.                                                                                                                                                                                                                                                                                                                    |\n|      `project`      |                     _Proxy Forge_                      | Name of the project to associate all the resources.                                                                                                                                                                                                                                                                                     |\n| `squid-credentials` | _{ password = \"proxyforge\", username = \"proxyforge\" }_ | Squid proxy HTTP basic authentication.credentials                                                                                                                                                                                                                                                                                       |\n\n## Customizations\n\nConfiguring ProxyForge to suit your specific needs is a straightforward process. All you need to do is update the settings in the `terraform.tfvars` file to align with your requirements. This file serves as a central configuration hub, allowing you to tailor ProxyForge precisely to your preferences without diving deep into complex setup procedures.\n\n\u003e **Note** After you choose to make any changes from below, make sure to apply it on digitalocean\n\u003e\n\u003e ```console\n\u003e terraform plan\n\u003e terraform apply\n\u003e ```\n\n### Change the Squid Credentials\n\n```terraform\nsquid-credentials = {\n    username = \"YOUR NEW USERNAME\"\n    password = \"Y0ur53cr37P@55W0rD\"\n}\n```\n\n### Increase or Decrease Proxy Droplets\n\n```terraform\nslave-count = 5\n```\n\n## Future Plans\n\n- Add a user friendly dashboard for administration of squid accounts\n- Make a master service to destroy and spin-up new droplet in same region\n- Multiple regions support\n\n## License\n\nProxy Forge is licensed under the **MIT License**. See the [LICENSE](LICENSE) file for details.\n\n## Contact Me\n\nThank you for choosing Proxy Forge! I hope this tool helps you overcome IP throttling and blacklisting challenges, making your web-related tasks smoother and more efficient. If you have any questions or encounter issues, feel free to reach out to me. Happy proxying!\n\nWebsite: https://tbhaxor.com \u003cbr /\u003e\nEmail Address: tbhaxor@gmail.com \u003cbr /\u003e\nLinkedIn: [@tbhaxor](https://linkedin.com/in/tbhaxor/) \u003cbr /\u003e\nTwitter: [@tbhaxor](https://x.com/@tbhaxor) \u003cbr /\u003e\nReddit: [@tbhaxor](https://www.reddit.com/user/tbhaxor) \u003cbr /\u003e\nDiscord: @tbhaxor.com \u003cbr /\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftbhaxor%2Fproxyforge","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftbhaxor%2Fproxyforge","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftbhaxor%2Fproxyforge/lists"}