Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/ppittle/httpwebrequestwrapper

Test/mock (3rd party) code that relies on HttpClient, WebClient, HttpWebRequest or WebRequest.Create()
https://github.com/ppittle/httpwebrequestwrapper

csharp http-client httpwebrequest httpwebrequestwrapper mock testing-tools

Last synced: 2 months ago
JSON representation

Test/mock (3rd party) code that relies on HttpClient, WebClient, HttpWebRequest or WebRequest.Create()

Awesome Lists containing this project

README 

        
[![Build status](https://ci.appveyor.com/api/projects/status/458hoqaj6dr94jrj?svg=true)](https://ci.appveyor.com/project/ppittle/httpwebrequestwrapper)

[![AppVeyor tests](https://img.shields.io/appveyor/tests/ppittle/httpwebrequestwrapper.svg?logo=appveyor)](https://ci.appveyor.com/project/ppittle/httpwebrequestwrapper/build/tests)

[![NuGet](https://img.shields.io/nuget/v/HttpWebRequestWrapper.svg)](https://www.nuget.org/packages/HttpWebRequestWrapper)

[![NuGet](https://img.shields.io/nuget/dt/HttpWebRequestWrapper.svg)](https://www.nuget.org/packages/HttpWebRequestWrapper)



# HttpWebRequestWrapper



**HttpWebRequestWrapper** is a testing layer for Microsoft's `HttpClient`, `HttpWebRequest` and `WebClient` classes. It overcomes restrictions that would normally prevent mocking a `HttpWebRequest` and allows testing your application with faked HTTP requests in Unit and BDD tests.



`HttpWebRequestWrapper` is built with some serious secret sauce allowing you to intercept nearlly all http traffic, including calls made in 3rd party code and code that doesn't support dependency injection!  It's the ideal testing tool for testing application code that relies on http api calls either directly or through 3rd party libraries!



## NuGet



    PM> Install-Package HttpWebRequestWrapper



HttpWebRequestWrapper has no 3rd Party dependencies!



## Usage



```csharp

// Asserts we can use a HttpWebRequestWrapperSession to 

// intercept and replace the Response to 

// new HttpClient().GetStringAsync("https://www.github.com")

[Test]

public async Task InterceptAndReplaceTrafficToGitHub()

{

    var fakeResponse = "Testing";



    var interceptor = 

        new HttpWebRequestWrapperInterceptorCreator(x => 

            x.HttpWebResponseCreator.Create(fakeResponse));



    using (new HttpWebRequestWrapperSession(interceptor))

    {

        var responseBody = await new HttpClient().GetStringAsync("https://www.github.com");



        Assert.Equal(fakeResponse, responseBody);

    }

}

```



### Testing Application Code

Let's say you have some simple but very hard to test code that uses `WebRequest.Create` to make a live http call:



```csharp

public static class Example

{

    public static int CountCharactersOnAWebPage(string url)

    {

       // makes live network call

       var response = (HttpWebResponse)WebRequest.Create(url).GetResponse();



       if (response.StatusCode != HttpStatusCode.Ok)

          throw new Exception();



       using (var sr = new StreamReader(response.GetResponseStream())

            return sr.ReadToEnd().Length;

    }

}

```

  

It would be ideal if this code used seperation of concerns and dependency injection to be more testable.  But perhaps it's in a 3rd party library, or it'll be too expensive to refacotr.

Fortunatly, it can b easily unit tested with the **HttpWebRequestWrapper** Library!  The test below intercepts the http call made by `Example.CountCharactersOnAWebPage` and returns a fake html response:



```csharp

// ARRANGE

var fakeResponseBody = "Test";



using (new HttpWebRequestWrapperSession(

    new HttpWebRequestWrapperInterceptorCreator(req => 

        req.HttpWebResponseCreator.Create(fakeResponseBody))))

{

    // ACT 

    var charactersOnGitHub = Example.CountCharactersOnAWebPage("http://www.github.com");



    // ASSERT

    Assert.Equal(fakeResponseBody.Length, charactersOnGitHub);

}

```



### HttpClient, WebClient, and WebRequest.Create() Support



 **HttpWebRequestWrapper** fully supports the .net `HttpClient` and `WebClient` classes as well as the static `WebRequest.Create()` method:



```csharp

var fakeResponse = "Testing";



using (new HttpWebRequestWrapperSession(

    new HttpWebRequestWrapperInterceptorCreator(

        x => x.HttpWebResponseCreator.Create(fakeResponse))))

{

    var responseBody1 = await new HttpClient().GetStringAsync("https://www.github.com");

    var responseBody2 = new WebClient().DownloadString("https://www.github.com");



    var response = WebRequest.Create("https://www.github.com").GetResponse();



    Assert.Equal(fakeResponse, responseBody1);

    Assert.Equal(fakeResponse, responseBody2);

    

    using (var sr = new StreamReader(response.GetResponseStream())

        Assert.Equal(fakeResponse, sr.ReadToEnd());

}

```



### Advanced Record and Playback



Use the `HttpWebRequestWrapperRecorder` to capture all http requests and response into a serializable `RecordingSession` for later playback in reliable and consistent tests!



```csharp

// run the application using the recorder



var recordingSession = new RecordingSession();

using (new HttpWebRequestWrapperSession(new HttpWebRequestWrapperRecorderCreator(recordingSession)))

{

   Example.CountCharactersOnAWebPage("http://www.github.com");

}



// serialize the recordingSession to disk (and preferrably embed in test assembly)

File.WriteAllText(@"c:\recordingSession.json", JsonConvert.Serialize(recordingSession));

```

Next, deserialize the recording session json and use the `RecordingSessionInterceptorRequestBuilder`  to feed the `RecordingSession` into the `HttpWebRequestWrapperInterceptor`



```csharp

var recordingSession = JsonConvert.DeserializeObject(json);



using (new HttpWebRequestWrapperSession(new HttpWebRequestWrapperInterceptorCreator(

    new RecordingSessionInterceptorRequestBuilder(recordingSession))))

{

    // now no http calls!

    var count = Example.CountCharactersOnAWebPage("http://www.github.com");

}



Assert.AreEqual(4321, count);

```

#### Playback Kung Fu



`RecordingSessionInterceptorRequestBuilder` exposes multiple customization points to override how matching is performed, how responses are built, and what to do if a match can't be found:



```csharp

var recordingSession = JsonConvert.DeserializeObject(json);



new RecordingSessionInterceptorRequestBuilder(recordingSession)

{

   MatchingAlgorithm = (interceptedRequest, recordedRequet) =>

       // match only on url

       interceptedReq.HttpWebRequest.RequestUri == recordedRequet.Url,



   RecordedResultResponseBuilder = (recordedReq, interceptedReq) =>

       // manipulate the response body

       interceptedReq.HttpWebResponseCreator.Create(recordedRequest.Response.ToLower()),



    RequestNotFoundResponseBuilder = interceptedReq =>

       // return a 500 when page isn't found

       interceptedReq.HttpWebResponseCreator.Create("Server Error", HttpStatusCode.InternalServerError),



    OnMatch = (recordedReq, interceptedReq, httpWebResponse, exception) =>

       // keep a count of requests made or do additional manipulation of the web response

       Log.Write("Application made another request");



    AllowReplayingRecordedRequestsMultipleTimes = 

        // act like a playback script - each recorded request in the recording 

        // session will only be used one - useful for testing error handling / retry

        // behavior

        false

};

```



But if that's not enough, you can easily implement your own `IInterceptorRequestBuilder` for full control!



#### WebException Support



Any Exception thown during `HttpWebRequest.GetResponse()` is captured by `HttpWebRequestWrapperRecorder` and is rethrown by `RecordingSessionInterceptorRequestBuilder` during playback!  This includes a fully populated `WebException` with a `WebException.Response`.



Additionally, `RequestNotFoundResponseBuilder` also supports throwing exceptions, you're free to overload and throw a NotFound WebException.



#### Dynamic Playback



Want to change up playback mid test run?  Not a problem!  You can easily manipulate `RecordingSessionInterceptorRequestBuilder.RecordedRequests` at any time:



```csharp

var builder = new RecordingSessionInterceptorRequestBuilder();

builder.RecordedRequests.Add(new RecordedRequest

{

    Url = "http://www.github.com",

    Method = "GET",

    Response = "Hello World"

};



using (new HttpWebRequestWrapperSession(new HttpWebRequestWrapperInterceptorCreator(

        builder)))

{

    var count1 = Example.CountCharactersOnAWebPage("http://www.github.com");



    builder.RecordedRequests.Clear();

    builder.RecordedRequests.Add(new RecordedRequest

    {

        Url = "http://www.github.com",

        Method = "GET",

        Response = "!!!"

    });



    var count2 = Example.CountCharactersOnAWebPage("http://www.github.com");

}



Assert.AreEqual(count1, count2);



```



#### Playback Multiple Sessions



Have a lot of network requests recorded?  Split them up over multiple recording sessions!



```csharp

var recordingSession1 = JsonConvert.DeserializeObject(json1);

var recordingSession2 = JsonConvert.DeserializeObject(json2);



using (new HttpWebRequestWrapperSession(new HttpWebRequestWrapperInterceptorCreator(

    new RecordingSessionInterceptorRequestBuilder(recordingSession1, recordingSession2))))

```



### Custom HttpWebRequest Implementations



Inspired by `HttpWebRequestWrapperInterceptor` and `HttpWebRequestWrapperRecorder` and want to build your own custom http request wrapper?  Not a problem, add an `IWebRequestCreate` to go with it and you'll be good to use `HttpWebRequestWrapperSession` to take care of the plumbing for you:



```csharp

public class CustomWrapper : HttpWebRequestWrapper

{

   public CustomWrapper(Uri uri) : base Uri{}

}



public class CustomWrapperCreator : IWebRequestCreate

{

   public WebRequest Create(Uri uri)

   {

      return new HttpWebRequestWrapperInterceptor(uri);

   }

}



using(new HttpWebRequestWrapperSession(new CustomWrapperCreator()))

{

   var request = WebRequest.Create("http://www.github.com");



   Assert.IsType(request);

}

```



### Multiple WebRequestCreate



Have an advanced scenario where you need to use multiple `IWebRequestCreate` objects?  You can use the `HttpWebRequestWrapperDelegateCreator` to decide just-in-time which `IWebRequestCreate` to use for a specific Uri:



```csharp

var creatorSelector = new Func(url =>

    url.Contains("api1")

        ? api1InterceptorCreator

        : commonInterceptorCreator);



using (new HttpWebRequestWrapperSession(new HttpWebRequestWrapperDelegateCreator(creatorSelector)))

{

    // handled by api1Interceptor

    WebRequest.Create("http://3rdParty.com/api1/request");

    // handled by commonInterceptor

    WebRequest.Create("http://someother.site");

}

```



### Full Mocking



Go crazy with full mocking support!  The ` HttpWebRequestWrapperInterceptor` provides very powerful faking, but you can easily build your own mock `HttpWebRequestWrapper` to provide custom behavior or expectations.



```csharp

// ARRANGE

var fakeResponseBody = "Fake Response";

var mockWebRequest = new Mock(new Uri("http://www.github.com"));

mockWebRequest

    .Setup(x => x.GetResponse())

    .Returns(HttpWebResponseCreator.Create(

        new Uri("http://www.github.com"), 

        "GET",

         HttpStatusCode.OK, 

         "Fake Response"));



var mockCreator = new Mock();

mockCreator

    .Setup(x => x.Create(It.IsAny()))

    .Returns(mockWebRequest.Object);



// ACT

string responseBody;

using (new HttpWebRequestWrapperSession(mockCreator.Object))

{    

    request = (HttpWebRequest)WebRequest.Create("http://www.github.com");



    using (var sr = new StreamReader(request.GetResponse().GetResponseStream()))

        responseBody = sr.ReadToEnd();

}



// ASSERT

Assert.Equal(fakeResponseBody, responseBody);

mockWebRequest.Verify(x => x.GetResponse());



```



## Secret Sauce



**HttpWebRequestWrapper** works by inheriting from `HttpWebRequest`.  This doesn't seem revolutionary, except these are the `HttpWebRequest` constructors:



```csharp

[Obsolete("This API supports the .NET Framework infrastructure and is not intended to be used directly from your code.", 

    true)]

public HttpWebRequest(){}

internal HttpWebRequest(Uri uri, ServicePoint servicePoint)

{

    // bunch of initialization code

}

```



Niether constructor will allow compilation of code that tries to inherit from `HttpWebRequest`.



### ildasm



Instead `HttpWebRequestWrapper` is compiled to inherit from `WebRequest`. During build, the compiled assembly is deassembled, has it's IL manipulated so that it inherits from `HttpWebRequest` and is then reasembled.  `HttpWebRequestWrapper` also does a bunch of reflection inside its constructor so the end result is a public class with a public constructor that inherits from `HttpWebRequest` and is fully functional!



### WebRequest.PrefixList



The second piece of magic is hooking into `WebRequest.PrefixList`.  `WebRequest` works as a factory for factories.  The `PrefixList` contains the registration of which factory is matched to which protocol.  `HttpWebRequestWrapperSession` works by overriding the default `PrefixList`, replacing the `IWebRequestCreate` for `http` and `https`.  



Disposing of the Session restores the original `Prefix` list.



### HttpClient



`HttpClient` uses an internal `HttpWebRequest` constructor to directly create requests and by-passes `WebRequest.PrefixList`.  So instead,

**HttpWebRequestWrapper** uses a custom `TaskScheduler` to hook into `HttpClientHandler` and hijack the `HttpClientHandler.StartRequeset` method to  replace the underlying `HttpWebRequest` it will use with one created via `WebRequest.Create`.  



### Limitations



**HttpWebRequestWrapper** can *not* support concurrent test execution.  Becuase `HttpWebRequestWrapperSession` works by setting a global static variable it's not possible to have two Sessions in use at one time.  Code inside the Session's using block is free to execute concurrently, you just can't try and use two or more Sessions at once.



`HttpClient` performs IO via the abstract `HttpMessageHandler`.  By default, this is a `HttpClinetHandler`.  **HttpWebRequestWrapper** fully supports `HttpClientHandler` and any handler that inherits from `HttpClientHandler`.  Custom handlers that instead inheirt directly from `HttpMessageHandler` are not supported and will not be intercepted.



## Platform Support



The actual wrapper `HttpWebRequestWrapper` is compiled for .NET Framework 2.0 and supports up to versions 4.7 of the .NET Framework (newest version tested at the time).



The remainder of **HttpWebRequestWrapper** requires .NET Framework 3.5+



Support for `HttpClient` requires .NET Framework 4.5+



## Build



Clone the repository and build `/src/HttpWebRequestWrapper.sln` using Visual Studio. NuGet package restore must be enabled.



To generate a nuget package run:



    nuget pack .\src\HttpWebRequestWrapper\HttpWebRequestWrapper.csproj



NOTE: I recommend unloading the `HttpWebRequestWrapper.LowLevel` project after opening the solution in Visual Studio and building for the first time.  The IDE (and tools like ReSharper) get confused by `HttpWebRequestWrapper` inheriting from `WebRequest` in source code and `HttpWebRequest` in the built dll.  Consequence is a lot of warnings and false errors.