https://github.com/thewebscraping/tls-requests
TLS Requests is a powerful Python library for secure HTTP requests, offering browser-like TLS client, fingerprinting, anti-bot page bypass, and high performance.
https://github.com/thewebscraping/tls-requests
anti-bot anti-bot-detection anti-bot-page cf-clearance cloudflare-bypass cloudflare-scraper crawling-python python-crawler python-scraper python-spider python-tls-client python-web-crawler python-web-scraper python-web-scraping scraping-python tls-client web-crawler-python web-scraping-api web-scraping-python web-spider
Last synced: 3 months ago
JSON representation
TLS Requests is a powerful Python library for secure HTTP requests, offering browser-like TLS client, fingerprinting, anti-bot page bypass, and high performance.
- Host: GitHub
- URL: https://github.com/thewebscraping/tls-requests
- Owner: thewebscraping
- License: mit
- Created: 2024-12-04T08:40:49.000Z (7 months ago)
- Default Branch: main
- Last Pushed: 2025-03-09T03:05:25.000Z (4 months ago)
- Last Synced: 2025-04-09T23:14:17.850Z (3 months ago)
- Topics: anti-bot, anti-bot-detection, anti-bot-page, cf-clearance, cloudflare-bypass, cloudflare-scraper, crawling-python, python-crawler, python-scraper, python-spider, python-tls-client, python-web-crawler, python-web-scraper, python-web-scraping, scraping-python, tls-client, web-crawler-python, web-scraping-api, web-scraping-python, web-spider
- Language: Python
- Homepage: https://thewebscraping.github.io/tls-requests/
- Size: 3.68 MB
- Stars: 42
- Watchers: 1
- Forks: 3
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# TLS Requests
[](https://github.com/thewebscraping/tls-requests/blob/main/LICENSE)
[](https://github.com/thewebscraping/tls-requests/actions/workflows/ci.yml)
[](https://pypi.org/project/wrapper-tls-requests/)
[](https://github.com/thewebscraping/tls-requests)
[](https://thewebscraping.github.io/tls-requests/)
TLS Requests is a powerful Python library for secure HTTP requests, offering browser-like TLS client, fingerprinting, anti-bot page bypass, and high performance.
* * *
**Installation**
----------------
To install the library, you can choose between two methods:
#### **1\. Install via PyPI:**
```shell
pip install wrapper-tls-requests
```
#### **2\. Install via GitHub Repository:**
```shell
pip install git+https://github.com/thewebscraping/tls-requests.git
```
**Quick Start**
---------------
Start using TLS Requests with just a few lines of code:
```pycon
>>> import tls_requests
>>> r = tls_requests.get("https://httpbin.org/get")
>>> r
>>> r.status_code
200
```
**Introduction**
----------------
**TLS Requests** is a cutting-edge HTTP client for Python, offering a feature-rich,
highly configurable alternative to the popular [`requests`](https://github.com/psf/requests) library.
Built on top of [`tls-client`](https://github.com/bogdanfinn/tls-client),
it combines ease of use with advanced functionality for secure networking.
**Acknowledgment**: A big thank you to all contributors for their support!
### **Key Benefits**
* **Bypass TLS Fingerprinting:** Mimic browser-like behaviors to navigate sophisticated anti-bot systems.
* **Customizable TLS Client:** Select specific TLS fingerprints to meet your needs.
* **Ideal for Developers:** Build scrapers, API clients, or other custom networking tools effortlessly.
**Why Use TLS Requests?**
-------------------------
Modern websites increasingly use **TLS Fingerprinting** and anti-bot tools like Cloudflare Bot Fight Mode to block web crawlers.
**TLS Requests** bypass these obstacles by mimicking browser-like TLS behaviors,
making it easy to scrape data or interact with websites that use sophisticated anti-bot measures.
### Unlocking Cloudflare Bot Fight Mode
**Example Code:**
```pycon
>>> import tls_requests
>>> r = tls_requests.get('https://www.coingecko.com/')
>>> r
```
**Key Features**
----------------
### **Enhanced Capabilities**
* **Browser-like TLS Fingerprinting**: Enables secure and reliable browser-mimicking connections.
* **High-Performance Backend**: Built on a Go-based HTTP backend for speed and efficiency.
* **Synchronous & Asynchronous Support**: Seamlessly switch between synchronous and asynchronous requests.
* **Protocol Support**: Fully compatible with HTTP/1.1 and HTTP/2.
* **Strict Timeouts**: Reliable timeout management for precise control over request durations.
### **Additional Features**
* **Internationalized Domain & URL Support**: Handles non-ASCII URLs effortlessly.
* **Cookie Management**: Ensures session-based cookie persistence.
* **Authentication**: Native support for Basic and Function authentication.
* **Content Decoding**: Automatic handling of gzip and brotli-encoded responses.
* **Hooks**: Perfect for logging, monitoring, tracing, or pre/post-processing requests and responses.
* **Unicode Support**: Effortlessly process Unicode response bodies.
* **File Uploads**: Simplified multipart file upload support.
* **Proxy Configuration**: Supports Socks5, HTTP, and HTTPS proxies for enhanced privacy.
**Documentation**
-----------------
Explore the full capabilities of TLS Requests in the documentation:
* **[Quickstart Guide](https://thewebscraping.github.io/tls-requests/quickstart/)**: A beginner-friendly guide.
* **[Advanced Topics](https://thewebscraping.github.io/tls-requests/advanced/client/)**: Learn to leverage specialized features.
* **[Async Support](https://thewebscraping.github.io/tls-requests/advanced/async_client/)**: Handle high-concurrency scenarios.
* **Custom TLS Configurations**:
* **[Wrapper TLS Client](https://thewebscraping.github.io/tls-requests/tls/)**
* **[TLS Client Profiles](https://thewebscraping.github.io/tls-requests/tls/profiles/)**
* **[Custom TLS Configurations](https://thewebscraping.github.io/tls-requests/tls/configuration/)**
Read the documentation: [**thewebscraping.github.io/tls-requests/**](https://thewebscraping.github.io/tls-requests/)
**Report Issues**
-----------------
Found a bug? Please [open an issue](https://github.com/thewebscraping/tls-requests/issues/).
By reporting an issue you help improve the project.
**Credits**
-----------------
Special thanks to [bogdanfinn](https://github.com/bogdanfinn/) for creating the awesome [tls-client](https://github.com/bogdanfinn/tls-client).