https://github.com/dholth/httpx-pycurl
Rough draft pycurl transport for httpx as a proof of concept.
https://github.com/dholth/httpx-pycurl
Last synced: about 1 month ago
JSON representation
Rough draft pycurl transport for httpx as a proof of concept.
- Host: GitHub
- URL: https://github.com/dholth/httpx-pycurl
- Owner: dholth
- License: mit
- Created: 2026-03-31T20:59:28.000Z (3 months ago)
- Default Branch: main
- Last Pushed: 2026-05-03T00:20:02.000Z (about 2 months ago)
- Last Synced: 2026-05-03T02:32:15.337Z (about 2 months ago)
- Language: Python
- Homepage:
- Size: 79.1 KB
- Stars: 2
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
- Agents: AGENTS.md
Awesome Lists containing this project
README
# httpx-pycurl
`httpx-pycurl` provides an `httpx` transport that executes requests with
`pycurl`. It combines the [goodness of curl](https://everything.curl.dev/) with
the familiar `httpx` API, including support for `http/2` and even non-http
protocols built into `curl`. `AsyncPyCurlTransport` performs better than
`httpx`'s default `AsyncHttpTransport` with `http2=True`, taking about 78% of
the time to issue 128 requests in parallel. Under heavier usage `httpx-pycurl`
appears to pull further ahead of alternative libraries, without replacing all of
`httpx`; just `httpx`'s transport.
## Usage
`AsyncPyCurlTransport` is the focus of this package. It uses the curl
[`multi_socket`](https://everything.curl.dev/transfers/drive/multi-socket.html)
interface to integrate `curl` with the `asyncio` event loop.
```python
import httpx
from httpx_pycurl import AsyncPyCurlTransport
transport = AsyncPyCurlTransport(timeout=10.0)
async with httpx.AsyncClient(transport=transport) as client:
responses = await asyncio.gather(*(client.get(url) for url in urls))
```
`AsyncPyCurlTransport` delegates SSL/TLS handling to `curl` and does not use
Python's builtin `ssl` module. By default it calls
[`certifi.where()`](https://github.com/certifi/python-certifi) to set root
certificates. It is also possible to pass a custom path to the root certificates
with `AsyncPyCurlTransport(cainfo=ca_cert_path)`.
`PyCurlTransport` is not the focus of this project and is less likely to work.
In the future it may delegate to `AsyncPyCurlTransport`.
```python
import httpx
from httpx_pycurl import PyCurlTransport
transport = PyCurlTransport(timeout=10.0)
with httpx.Client(transport=transport) as client:
response = client.get("https://example.com")
print(response.status_code)
print(response.text)
debug_transport = PyCurlTransport(
verbose=True,
debug_callback=lambda info_type, data: print(info_type, data),
)
```
## Installation
```bash
pip install httpx-pycurl
```
Or with conda,
```bash
conda install -n base conda-pypi
conda pypi install httpx-pycurl
```
## Performance
`httpx-pycurl` is in early development but it passes most `httpx` tests and has
good performance. Our `tests/bench.py` uses `asyncio.gather()` to make many
`http/2` requests to `https://httpbingo.org/get` using `httpx`, `niquests`, and
`httpx` with `httpx-pycurl`'s transport. `httpx-pycurl` is the fastest library
tested.
Running `tests/bench.py [N]` shows that the more efficient `http/2` libraries
shine when performing large numbers of parallel requests, and are closer
together when only groups of 128 parallel requests are made.
```
2 groups of 512 requests each...
Time per group:
httpx: 0.679s ± 0.119s
niquests: 0.432s ± 0.140s
httpx_pycurl: 0.299s ± 0.056s
Paired t-test: httpx_pycurl vs niquests
t-stat: -2.249 (approx p < 0.05 if |t| > 2.365)
Speedup: 1.44x
```
```
4 groups of 256 requests each...
Time per group:
httpx: 0.378s ± 0.077s
niquests: 0.259s ± 0.064s
httpx_pycurl: 0.190s ± 0.034s
Paired t-test: httpx_pycurl vs niquests
t-stat: -4.208 (approx p < 0.05 if |t| > 2.365)
Speedup: 1.36x
```
```
8 groups of 128 requests each...
Time per group:
httpx: 0.215s ± 0.073s
niquests: 0.205s ± 0.038s
httpx_pycurl: 0.162s ± 0.033s
Paired t-test: httpx_pycurl vs niquests
t-stat: -8.949 (approx p < 0.05 if |t| > 2.365)
Speedup: 1.27x
```
## Dependencies
`httpx-pycurl` uses curl to support `http/2` instead of the `h2`, `hpack` and
`hyperframe` dependencies used by `httpx`.
```
$ pip install --dry-run httpx-pycurl
...
Would install anyio-4.13.0 certifi-2026.4.22 h11-0.16.0 httpcore-1.0.9 httpx-0.28.1 httpx-pycurl-0.0.4 idna-3.13 pycurl-7.45.7
$ pip install --dry-run httpx[http2]
...
Would install anyio-4.13.0 certifi-2026.4.22 h11-0.16.0 h2-4.3.0 hpack-4.1.0 httpcore-1.0.9 httpx-0.28.1 hyperframe-6.1.0 idna-3.13
```