https://github.com/matteocontrini/plainhttp
Easy HTTP client with support for serialization, proxies, testing, and more
https://github.com/matteocontrini/plainhttp
dotnet dotnet-core http httpclient
Last synced: 3 months ago
JSON representation
Easy HTTP client with support for serialization, proxies, testing, and more
- Host: GitHub
- URL: https://github.com/matteocontrini/plainhttp
- Owner: matteocontrini
- License: mit
- Created: 2019-08-15T18:30:04.000Z (about 6 years ago)
- Default Branch: main
- Last Pushed: 2024-01-24T17:14:39.000Z (almost 2 years ago)
- Last Synced: 2024-12-01T12:52:41.692Z (11 months ago)
- Topics: dotnet, dotnet-core, http, httpclient
- Language: C#
- Homepage:
- Size: 82 KB
- Stars: 20
- Watchers: 3
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# PlainHttp [](https://www.nuget.org/packages/PlainHttp) [](https://github.com/matteocontrini/PlainHttp/blob/master/LICENSE)
An **easy HTTP client** for .NET 6+ with support for **serialization, deserialization, proxies, testing**, and more.
## Features
- Wraps `HttpClient` and provides a cleaner and easier interface
- Supports any HTTP method
- Per-request timeout with an actual `HttpRequestTimeoutException`
- Per-request proxy with transparent pooling
- Built-in serialization of objects to `JSON`/`XML`/URL-encoded, extensible to any other format
- Built-in deserialization of `JSON`/`XML` responses
- Download files to disk
- Read responses with specific response encodings
- Automatically enabled decompression of responses (all algorithms supported by .NET, i.e. gzip, DEFLATE, and Brotli)
- Proper pooling and connection lifetime defaults to avoid DNS and socket exhaustion issues
- Allows to mock requests for unit testing
- Heavily used in production by [@trackbotpro](https://github.com/trackbotpro/) to send millions of requests per day
## Supported frameworks
This library targets .NET 6 (LTS) because it requires the `PooledConnectionLifetime` property on `SocketsHttpHandler`, introduced in .NET Core 2.2.
This makes sure that reusing the same `HttpClient` for a long time doesn't have [unintended consequences](https://github.com/dotnet/corefx/issues/11224) affecting DNS resolution. This library in fact keeps a pool of `HttpClient` instances that are never disposed.
In particular, the library keeps:
- One `HttpClient` per request host
- One `HttpClient` per proxy URI (including credentials)
There is currently no mechanism that disposes `HttpClient` instances that are unused, so if you use a lot of random proxies or many different hostnames, you might get into trouble. See [Custom `HttpClientFactory`](#custom-httpclientfactory) for instructions on how to override the default behavior.
## Installation
Install the [PlainHttp](https://www.nuget.org/packages/PlainHttp) NuGet package:
```
dotnet add package PlainHttp
```
### Upgrading from 1.x to 2.x
See the release notes for [v2.0.0](https://github.com/matteocontrini/PlainHttp/releases/tag/v2.0.0).
## Usage
- [Basic usage](#basic-usage)
- [Error handling](#error-handling)
- [Request customization](#request-customization)
- [Request serialization](#request-serialization)
- [Response deserialization](#response-deserialization)
- [Efficiently reading the response body](#efficiently-reading-the-response-body)
- [Downloading files](#downloading-files)
- [Proxies](#proxies)
- [URL building](#url-building)
- [Testing mode](#testing-mode)
- [Custom serialization](#custom-serialization)
- [Customizing `HttpClient` defaults](#customizing-httpclient-defaults)
### Basic usage
Basic `GET` request:
```c#
string url = "http://random.org";
IHttpRequest request = new HttpRequest(url);
IHttpResponse response = await request.SendAsync();
string body = await response.ReadString();
```
Also with `Uri`:
```c#
Uri uri = new Uri("http://random.org");
IHttpRequest request = new HttpRequest(uri);
```
### Error handling
Checking if the HTTP status code is in the `2xx` range:
```c#
IHttpResponse response = await request.SendAsync();
if (!response.Succeeded)
{
Console.WriteLine($"Response status code is {response.StatusCode}");
}
else
{
Console.WriteLine($"Successful response in {response.ElapsedMilliseconds} ms");
}
```
Asserting that the HTTP status code is in the `2xx` range:
```c#
IHttpResponse response = await request.SendAsync();
response.EnsureSuccessStatusCode(); // may throw HttpRequestException
```
Every exception is wrapped in an `HttpRequestException`, from which `HttpRequestTimeoutException` is derived:
```c#
try
{
IHttpResponse response = await request.SendAsync();
}
catch (HttpRequestException ex)
{
if (ex is HttpRequestTimeoutException)
{
Console.WriteLine("Request timed out");
}
else
{
Console.WriteLine("Something bad happened: {0}", ex);
// PlainHttp.HttpRequestException: Failed request: [GET https://yyyy.org/] [No such host is known] ---> System.Net.Http.HttpRequestException: No such host is known ---> System.Net.Sockets.SocketException: No such host is known
// etc.
}
}
```
### Request customization
Setting **custom headers**:
```c#
IHttpRequest request = new HttpRequest(url)
{
Headers = new Dictionary
{
// No user agent is set by default
{ "User-Agent", "PlainHttp/1.0" }
}
};
```
Request a **specific HTTP version** to be used. If it's not supported, the default [`HttpVersionPolicy`](https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpversionpolicy) applies (downgrade to a lower version).
```csharp
IHttpRequest request = new HttpRequest(url)
{
Version = new Version(2, 0) // HTTP/2
};
```
**Custom timeout** (by default no timeout is set):
```c#
IHttpRequest request = new HttpRequest(url)
{
Timeout = TimeSpan.FromSeconds(10)
};
```
### Request serialization
`POST` request with **URL-encoded payload**:
```c#
IHttpRequest request = new HttpRequest(url)
{
Method = HttpMethod.Post,
Payload = new FormUrlEncodedPayload(new
{
hello = "world",
buuu = true
})
};
```
`POST` request with **JSON payload** (powered by `System.Text.Json`):
```c#
IHttpRequest request = new HttpRequest(url)
{
Method = HttpMethod.Post,
Payload = new JsonPayload(new
{
hello = "world"
})
};
```
You can pass `JsonSerializerOptions` with the second argument of `JsonPayload`.
If you already have a JSON-serialized string, just pass it directly:
```c#
IHttpRequest request = new HttpRequest(url)
{
Method = HttpMethod.Post,
Payload = new JsonPayload("{ \"key\": true }")
};
```
`POST` request with **XML payload** (powered by `System.Xml.Serialization.XmlSerializer`):
```c#
IHttpRequest request = new HttpRequest(url)
{
Method = HttpMethod.Post,
Payload = new XmlPayload(new
{
something = "web"
})
};
```
You can pass XML serialization options with the second argument of `XmlPayload` (`XmlWriterSettings`). If you already have an XML-serialized string, just pass it directly.
Note that the `XmlPayload` implementation will use the `UTF-8` encoding, which is normally not the default in .NET. If you have different requirements you should pass an already-serialized string or implement a [custom payload type](#custom-serialization).
`POST` request with **plain text payload**:
```c#
IHttpRequest request = new HttpRequest(url)
{
Method = HttpMethod.Post,
Payload = new PlainTextPayload("plain text")
};
```
### Response deserialization
To read the response body **as a string**:
```c#
string body = await response.ReadString();
```
Optionally, you can specify the **encoding** to use:
```c#
string body = await response.ReadString(Encoding.GetEncoding("ISO-8859-1"));
```
To read the body **as a stream**:
```c#
Stream stream = await response.ReadStream();
```
Note that when using `ReadStream` the response message is not automatically disposed, so you must take care of disposing it manually when you're done with it.
To deserialize the response **as JSON**:
```c#
ResponseDTO content = await response.ReadJson();
```
To deserialize the response **as XML**:
```c#
ResponseDTO content = await response.ReadXml();
```
To read the body **as a byte array**:
```c#
byte[] bytes = await response.ReadBytes();
```
### Efficiently reading the response body
By default, the full response body is loaded in memory during the `SendAsync` call. This means that when calling the various `Read*` methods, the response body is already fully downloaded and is thereefore read from a memory stream.
To change this, you can set the `HttpCompletionOption` request option to `HttpCompletionOption.ResponseHeadersRead` (from `System.Net.Http`):
```c#
IHttpRequest request = new HttpRequest(url)
{
Method = HttpMethod.Get,
HttpCompletionOption = HttpCompletionOption.ResponseHeadersRead
};
```
Now, when you call methods such as `ReadString` or `ReadJson`, the response body will be streamed from the socket as it arrives.
The library will also take care of **respecting the timeout you specified in the request**, calculating how much time is left to read the response.
```c#
IHttpRequest request = new HttpRequest(url)
{
Method = HttpMethod.Get,
HttpCompletionOption = HttpCompletionOption.ResponseHeadersRead,
Timeout = TimeSpan.FromSeconds(10)
};
// This call returns immediately after reading the response headers
IHttpResponse response = await request.SendAsync();
Console.WriteLine($"Reading the headers took {response.ElapsedMilliseconds} ms");
// This call will proceed with reading the HTTP response body from the socket
// and will throw HttpRequestTimeoutException if the response body is not
// fully read within 10 total seconds
string body = await response.ReadString();
Console.WriteLine($"Reading the headers+body took {response.ElapsedMilliseconds} ms in total");
```
The exception is if you use the `ReadStream` method: in that case PlainHttp cannot enforce a timeout when reading from that stream outside the library.
You also must take care of disposing the response manually when using `ReadStream` or if you don't read the response body at all:
```csharp
IHttpRequest request = new HttpRequest(url)
{
Method = HttpMethod.Get,
HttpCompletionOption = HttpCompletionOption.ResponseHeadersRead,
Timeout = TimeSpan.FromSeconds(10)
};
// You MUST dispose the response manually
// when using HttpCompletionOption.ResponseHeadersRead and ReadStream(),
// or if you don't read the response body at all
using IHttpResponse response = await request.SendAsync();
Stream stream = await response.ReadStream();
// The timeout is not enforced if you read from `stream` here
```
In all other cases (any other `Read*` method), responses are **always disposed automatically** after reading the response body, also in case of errors.
A note on XML deserialization: the `ReadXml` method uses `XmlSerializer`, which is not asynchronous. Therefore, the response body is unfortunately always fully read in memory (asynchronously) before deserializing it, no matter the `HttpCompletionOption` setting.
### Downloading files
You can use the `HttpCompletionOption.ResponseHeadersRead` option to efficiently download files to disk:
```c#
IHttpRequest request = new HttpRequest(url)
{
Method = HttpMethod.Get,
HttpCompletionOption = HttpCompletionOption.ResponseHeadersRead
};
IHttpResponse response = await request.SendAsync();
await response.DownloadFileAsync("video.mp4");
```
### Proxies
You can set a **custom proxy per request**:
```c#
IHttpRequest request = new HttpRequest(url)
{
Proxy = new Uri("http://example.org:3128")
};
```
**Proxy credentials** are supported and are automatically parsed from the URI:
```c#
IHttpRequest request = new HttpRequest(url)
{
Proxy = new Uri("http://user:pass@example.com:3128")
};
```
Note that due to the implementation of proxies in .NET, proxy credentials are only sent from the second request onwards and only if the proxy responded with *407 Proxy Authentication Required*. See [this issue](https://github.com/dotnet/runtime/issues/66244) for more details.
### URL building
This library includes the `Flurl` URL builder as a dependency. Some `Flurl`-provided utilities are used internally but you can also use it to [build URLs](https://flurl.dev/docs/fluent-url/) in an easier way (thanks Todd Menier!):
```c#
string url = "http://random.org"
.SetQueryParam("locale", "it")
.SetQueryParam("timestamp", DateTimeOffset.UtcNow.ToUnixTimeMilliseconds());
```
### Testing mode
**Unit testing HTTP requests** is easy with PlainHttp. You can enqueue HTTP responses that will be dequeued in sequence.
This mechanism is "async safe": the `TestingMode` property is static but wrapped in to an `AsyncLocal` instance, so that you can run your tests in parallel.
```c#
// Run this once
TestingMode http = new TestingMode();
HttpRequest.SetTestingMode(http);
// Then enqueue HTTP responses
HttpResponseMessage msg = new HttpResponseMessage()
{
StatusCode = (HttpStatusCode)200,
Content = new StringContent("oh hello")
};
http.RequestsQueue.Enqueue(msg);
// Then send your requests normally, in the same async context
```
### Custom serialization
You can implement your own custom serializer by implementing the `IPayload` interface.
For example, here's how you can use `Newtonsoft.Json` instead of `System.Text.Json`:
```c#
public class NewtonsoftJsonPayload : IPayload
{
private readonly object payload;
private readonly JsonSerializerSettings? settings;
public NewtonsoftJsonPayload(object payload)
{
this.payload = payload;
}
public NewtonsoftJsonPayload(object payload, JsonSerializerSettings settings) : this(payload)
{
this.settings = settings;
}
public HttpContent Serialize()
{
return new StringContent(
content: JsonConvert.SerializeObject(payload, settings),
encoding: Encoding.UTF8,
mediaType: "application/json"
);
}
}
```
Then use it like this:
```c#
IHttpRequest request = new HttpRequest(url)
{
Method = HttpMethod.Post,
Payload = new NewtonsoftJsonPayload(new
{
something = "hello"
})
};
```
### Customizing `HttpClient` defaults
You can customize how `HttpClient`s and the underlying `SocketsHttpHandler` are created by changing the static `HttpClientFactory` property.
The default factory provides some level of customization, which you can pass to the constructor. For example:
```c#
HttpRequest.HttpClientFactory = new HttpClientFactory(new HttpClientFactory.HttpHandlerOptions
{
IgnoreCertificateValidationErrors = true
});
```
These options will apply to both proxied and non-proxied `HttpClient`s. You can however choose different settings for proxied and non-proxied clients:
```c#
HttpRequest.HttpClientFactory = new HttpClientFactory(
// Normal requests
new HttpClientFactory.HttpHandlerOptions
{
IgnoreCertificateValidationErrors = true
},
// Proxied requests
new HttpClientFactory.HttpHandlerOptions
{
IgnoreCertificateValidationErrors = false
}
);
```
These are all the available options with their defaults:
```c#
public record HttpHandlerOptions
{
public TimeSpan PooledConnectionLifetime { get; init; } = TimeSpan.FromMinutes(10);
public TimeSpan PooledConnectionIdleTimeout { get; init; } = TimeSpan.FromMinutes(1);
public TimeSpan ConnectTimeout { get; init; } = Timeout.InfiniteTimeSpan;
public DecompressionMethods AutomaticDecompression { get; init; } = DecompressionMethods.All;
public SslProtocols EnabledSslProtocols { get; init; } = SslProtocols.None;
public bool IgnoreCertificateValidationErrors { get; init; }
public bool AllowAutoRedirect { get; set; } = true;
}
```
The meanings of these options (which usually map to `SocketsHttpHandler` properties) are the following:
- `PooledConnectionLifetime`: the maximum lifetime of a connection in the pool.
- `PooledConnectionIdleTimeout`: the maximum idle time of a connection in the pool. If a connection is idle for more than this time, it will be closed.
- `ConnectTimeout`: the timeout for establishing a connection to the server.
- `AutomaticDecompression`: the decompression methods to use for the response body. By default, all methods (gzip, DEFLATE and Brotli) are enabled.
- `EnabledSslProtocols`: the SSL/TLS protocols to use. By default, the system default is used.
- `IgnoreCertificateValidationErrors`: whether to ignore certificate validation errors.
- `AllowAutoRedirect`: whether redirect responses should be automatically followed. By default, redirects are followed.
Note that when applied to proxied clients these options will apply to the connection to the proxy server itself.
### Custom `HttpClientFactory`
If the above options aren't enough or you want more control, you can create your own factory implementation and set it to the static `HttpClientFactory` property:
```c#
HttpRequest.HttpClientFactory = new MyHttpClientFactory();
```
The custom factory must implement the [`IHttpClientFactory`](https://github.com/matteocontrini/PlainHttp/blob/main/PlainHttp/IHttpClientFactory.cs) interface. The default factory implementation can be found [here](https://github.com/matteocontrini/PlainHttp/blob/main/PlainHttp/HttpClientFactory.cs).