{"id":17759276,"url":"https://github.com/ppittle/httpwebrequestwrapper","last_synced_at":"2025-03-15T09:31:41.455Z","repository":{"id":37978380,"uuid":"120291860","full_name":"ppittle/HttpWebRequestWrapper","owner":"ppittle","description":"Test/mock (3rd party) code that relies on HttpClient, WebClient, HttpWebRequest or WebRequest.Create()","archived":false,"fork":false,"pushed_at":"2022-06-22T19:55:31.000Z","size":562,"stargazers_count":9,"open_issues_count":2,"forks_count":1,"subscribers_count":4,"default_branch":"develop","last_synced_at":"2025-02-22T03:33:32.782Z","etag":null,"topics":["csharp","http-client","httpwebrequest","httpwebrequestwrapper","mock","testing-tools"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ppittle.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-02-05T10:47:40.000Z","updated_at":"2022-08-17T17:02:37.000Z","dependencies_parsed_at":"2022-08-24T16:31:08.521Z","dependency_job_id":null,"html_url":"https://github.com/ppittle/HttpWebRequestWrapper","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ppittle%2FHttpWebRequestWrapper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ppittle%2FHttpWebRequestWrapper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ppittle%2FHttpWebRequestWrapper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ppittle%2FHttpWebRequestWrapper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ppittle","download_url":"https://codeload.github.com/ppittle/HttpWebRequestWrapper/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243713357,"owners_count":20335564,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csharp","http-client","httpwebrequest","httpwebrequestwrapper","mock","testing-tools"],"created_at":"2024-10-26T19:00:01.673Z","updated_at":"2025-03-15T09:31:41.044Z","avatar_url":"https://github.com/ppittle.png","language":"C#","readme":"[![Build status](https://ci.appveyor.com/api/projects/status/458hoqaj6dr94jrj?svg=true)](https://ci.appveyor.com/project/ppittle/httpwebrequestwrapper)\n[![AppVeyor tests](https://img.shields.io/appveyor/tests/ppittle/httpwebrequestwrapper.svg?logo=appveyor)](https://ci.appveyor.com/project/ppittle/httpwebrequestwrapper/build/tests)\n[![NuGet](https://img.shields.io/nuget/v/HttpWebRequestWrapper.svg)](https://www.nuget.org/packages/HttpWebRequestWrapper)\n[![NuGet](https://img.shields.io/nuget/dt/HttpWebRequestWrapper.svg)](https://www.nuget.org/packages/HttpWebRequestWrapper)\n\n# HttpWebRequestWrapper\n\n**HttpWebRequestWrapper** is a testing layer for Microsoft's `HttpClient`, `HttpWebRequest` and `WebClient` classes. It overcomes restrictions that would normally prevent mocking a `HttpWebRequest` and allows testing your application with faked HTTP requests in Unit and BDD tests.\n\n`HttpWebRequestWrapper` is built with some serious secret sauce allowing you to intercept nearlly all http traffic, including calls made in 3rd party code and code that doesn't support dependency injection!  It's the ideal testing tool for testing application code that relies on http api calls either directly or through 3rd party libraries!\n\n## NuGet\n\n    PM\u003e Install-Package HttpWebRequestWrapper\n\nHttpWebRequestWrapper has no 3rd Party dependencies!\n\n## Usage\n\n```csharp\n// Asserts we can use a HttpWebRequestWrapperSession to \n// intercept and replace the Response to \n// new HttpClient().GetStringAsync(\"https://www.github.com\")\n[Test]\npublic async Task InterceptAndReplaceTrafficToGitHub()\n{\n    var fakeResponse = \"Testing\";\n\n    var interceptor = \n        new HttpWebRequestWrapperInterceptorCreator(x =\u003e \n            x.HttpWebResponseCreator.Create(fakeResponse));\n\n    using (new HttpWebRequestWrapperSession(interceptor))\n    {\n        var responseBody = await new HttpClient().GetStringAsync(\"https://www.github.com\");\n\n        Assert.Equal(fakeResponse, responseBody);\n    }\n}\n```\n\n### Testing Application Code\nLet's say you have some simple but very hard to test code that uses `WebRequest.Create` to make a live http call:\n\n```csharp\npublic static class Example\n{\n    public static int CountCharactersOnAWebPage(string url)\n    {\n       // makes live network call\n       var response = (HttpWebResponse)WebRequest.Create(url).GetResponse();\n\n       if (response.StatusCode != HttpStatusCode.Ok)\n          throw new Exception();\n\n       using (var sr = new StreamReader(response.GetResponseStream())\n            return sr.ReadToEnd().Length;\n    }\n}\n```\n  \nIt would be ideal if this code used seperation of concerns and dependency injection to be more testable.  But perhaps it's in a 3rd party library, or it'll be too expensive to refacotr.\nFortunatly, it can b easily unit tested with the **HttpWebRequestWrapper** Library!  The test below intercepts the http call made by `Example.CountCharactersOnAWebPage` and returns a fake html response:\n\n```csharp\n// ARRANGE\nvar fakeResponseBody = \"\u003chtml\u003eTest\u003c/html\u003e\";\n\nusing (new HttpWebRequestWrapperSession(\n    new HttpWebRequestWrapperInterceptorCreator(req =\u003e \n        req.HttpWebResponseCreator.Create(fakeResponseBody))))\n{\n    // ACT \n    var charactersOnGitHub = Example.CountCharactersOnAWebPage(\"http://www.github.com\");\n\n    // ASSERT\n    Assert.Equal(fakeResponseBody.Length, charactersOnGitHub);\n}\n```\n\n### HttpClient, WebClient, and WebRequest.Create() Support\n\n **HttpWebRequestWrapper** fully supports the .net `HttpClient` and `WebClient` classes as well as the static `WebRequest.Create()` method:\n\n```csharp\nvar fakeResponse = \"Testing\";\n\nusing (new HttpWebRequestWrapperSession(\n    new HttpWebRequestWrapperInterceptorCreator(\n        x =\u003e x.HttpWebResponseCreator.Create(fakeResponse))))\n{\n    var responseBody1 = await new HttpClient().GetStringAsync(\"https://www.github.com\");\n    var responseBody2 = new WebClient().DownloadString(\"https://www.github.com\");\n\n    var response = WebRequest.Create(\"https://www.github.com\").GetResponse();\n\n    Assert.Equal(fakeResponse, responseBody1);\n    Assert.Equal(fakeResponse, responseBody2);\n    \n    using (var sr = new StreamReader(response.GetResponseStream())\n        Assert.Equal(fakeResponse, sr.ReadToEnd());\n}\n```\n\n### Advanced Record and Playback\n\nUse the `HttpWebRequestWrapperRecorder` to capture all http requests and response into a serializable `RecordingSession` for later playback in reliable and consistent tests!\n\n```csharp\n// run the application using the recorder\n\nvar recordingSession = new RecordingSession();\nusing (new HttpWebRequestWrapperSession(new HttpWebRequestWrapperRecorderCreator(recordingSession)))\n{\n   Example.CountCharactersOnAWebPage(\"http://www.github.com\");\n}\n\n// serialize the recordingSession to disk (and preferrably embed in test assembly)\nFile.WriteAllText(@\"c:\\recordingSession.json\", JsonConvert.Serialize(recordingSession));\n```\nNext, deserialize the recording session json and use the `RecordingSessionInterceptorRequestBuilder`  to feed the `RecordingSession` into the `HttpWebRequestWrapperInterceptor`\n\n```csharp\nvar recordingSession = JsonConvert.DeserializeObject\u003cRecordingSession\u003e(json);\n\nusing (new HttpWebRequestWrapperSession(new HttpWebRequestWrapperInterceptorCreator(\n    new RecordingSessionInterceptorRequestBuilder(recordingSession))))\n{\n    // now no http calls!\n    var count = Example.CountCharactersOnAWebPage(\"http://www.github.com\");\n}\n\nAssert.AreEqual(4321, count);\n```\n#### Playback Kung Fu\n\n`RecordingSessionInterceptorRequestBuilder` exposes multiple customization points to override how matching is performed, how responses are built, and what to do if a match can't be found:\n\n```csharp\nvar recordingSession = JsonConvert.DeserializeObject\u003cRecordingSession\u003e(json);\n\nnew RecordingSessionInterceptorRequestBuilder(recordingSession)\n{\n   MatchingAlgorithm = (interceptedRequest, recordedRequet) =\u003e\n       // match only on url\n       interceptedReq.HttpWebRequest.RequestUri == recordedRequet.Url,\n\n   RecordedResultResponseBuilder = (recordedReq, interceptedReq) =\u003e\n       // manipulate the response body\n       interceptedReq.HttpWebResponseCreator.Create(recordedRequest.Response.ToLower()),\n\n    RequestNotFoundResponseBuilder = interceptedReq =\u003e\n       // return a 500 when page isn't found\n       interceptedReq.HttpWebResponseCreator.Create(\"Server Error\", HttpStatusCode.InternalServerError),\n\n    OnMatch = (recordedReq, interceptedReq, httpWebResponse, exception) =\u003e\n       // keep a count of requests made or do additional manipulation of the web response\n       Log.Write(\"Application made another request\");\n\n    AllowReplayingRecordedRequestsMultipleTimes = \n        // act like a playback script - each recorded request in the recording \n        // session will only be used one - useful for testing error handling / retry\n        // behavior\n        false\n};\n```\n\nBut if that's not enough, you can easily implement your own `IInterceptorRequestBuilder` for full control!\n\n#### WebException Support\n\nAny Exception thown during `HttpWebRequest.GetResponse()` is captured by `HttpWebRequestWrapperRecorder` and is rethrown by `RecordingSessionInterceptorRequestBuilder` during playback!  This includes a fully populated `WebException` with a `WebException.Response`.\n\nAdditionally, `RequestNotFoundResponseBuilder` also supports throwing exceptions, you're free to overload and throw a NotFound WebException.\n\n#### Dynamic Playback\n\nWant to change up playback mid test run?  Not a problem!  You can easily manipulate `RecordingSessionInterceptorRequestBuilder.RecordedRequests` at any time:\n\n```csharp\nvar builder = new RecordingSessionInterceptorRequestBuilder();\nbuilder.RecordedRequests.Add(new RecordedRequest\n{\n    Url = \"http://www.github.com\",\n    Method = \"GET\",\n    Response = \"Hello World\"\n};\n\nusing (new HttpWebRequestWrapperSession(new HttpWebRequestWrapperInterceptorCreator(\n        builder)))\n{\n    var count1 = Example.CountCharactersOnAWebPage(\"http://www.github.com\");\n\n    builder.RecordedRequests.Clear();\n    builder.RecordedRequests.Add(new RecordedRequest\n    {\n        Url = \"http://www.github.com\",\n        Method = \"GET\",\n        Response = \"!!!\"\n    });\n\n    var count2 = Example.CountCharactersOnAWebPage(\"http://www.github.com\");\n}\n\nAssert.AreEqual(count1, count2);\n\n```\n\n#### Playback Multiple Sessions\n\nHave a lot of network requests recorded?  Split them up over multiple recording sessions!\n\n```csharp\nvar recordingSession1 = JsonConvert.DeserializeObject\u003cRecordingSession\u003e(json1);\nvar recordingSession2 = JsonConvert.DeserializeObject\u003cRecordingSession\u003e(json2);\n\nusing (new HttpWebRequestWrapperSession(new HttpWebRequestWrapperInterceptorCreator(\n    new RecordingSessionInterceptorRequestBuilder(recordingSession1, recordingSession2))))\n```\n\n### Custom HttpWebRequest Implementations\n\nInspired by `HttpWebRequestWrapperInterceptor` and `HttpWebRequestWrapperRecorder` and want to build your own custom http request wrapper?  Not a problem, add an `IWebRequestCreate` to go with it and you'll be good to use `HttpWebRequestWrapperSession` to take care of the plumbing for you:\n\n```csharp\npublic class CustomWrapper : HttpWebRequestWrapper\n{\n   public CustomWrapper(Uri uri) : base Uri{}\n}\n\npublic class CustomWrapperCreator : IWebRequestCreate\n{\n   public WebRequest Create(Uri uri)\n   {\n      return new HttpWebRequestWrapperInterceptor(uri);\n   }\n}\n\nusing(new HttpWebRequestWrapperSession(new CustomWrapperCreator()))\n{\n   var request = WebRequest.Create(\"http://www.github.com\");\n\n   Assert.IsType\u003cCustomWrapper\u003e(request);\n}\n```\n\n### Multiple WebRequestCreate\n\nHave an advanced scenario where you need to use multiple `IWebRequestCreate` objects?  You can use the `HttpWebRequestWrapperDelegateCreator` to decide just-in-time which `IWebRequestCreate` to use for a specific Uri:\n\n```csharp\nvar creatorSelector = new Func\u003cUri, IWebRequestCreate\u003e(url =\u003e\n    url.Contains(\"api1\")\n        ? api1InterceptorCreator\n        : commonInterceptorCreator);\n\nusing (new HttpWebRequestWrapperSession(new HttpWebRequestWrapperDelegateCreator(creatorSelector)))\n{\n    // handled by api1Interceptor\n    WebRequest.Create(\"http://3rdParty.com/api1/request\");\n    // handled by commonInterceptor\n    WebRequest.Create(\"http://someother.site\");\n}\n```\n\n### Full Mocking\n\nGo crazy with full mocking support!  The ` HttpWebRequestWrapperInterceptor` provides very powerful faking, but you can easily build your own mock `HttpWebRequestWrapper` to provide custom behavior or expectations.\n\n```csharp\n// ARRANGE\nvar fakeResponseBody = \"Fake Response\";\nvar mockWebRequest = new Mock\u003cHttpWebRequestWrapper\u003e(new Uri(\"http://www.github.com\"));\nmockWebRequest\n    .Setup(x =\u003e x.GetResponse())\n    .Returns(HttpWebResponseCreator.Create(\n        new Uri(\"http://www.github.com\"), \n        \"GET\",\n         HttpStatusCode.OK, \n         \"Fake Response\"));\n\nvar mockCreator = new Mock\u003cIWebRequestCreate\u003e();\nmockCreator\n    .Setup(x =\u003e x.Create(It.IsAny\u003cUri\u003e()))\n    .Returns(mockWebRequest.Object);\n\n// ACT\nstring responseBody;\nusing (new HttpWebRequestWrapperSession(mockCreator.Object))\n{    \n    request = (HttpWebRequest)WebRequest.Create(\"http://www.github.com\");\n\n    using (var sr = new StreamReader(request.GetResponse().GetResponseStream()))\n        responseBody = sr.ReadToEnd();\n}\n\n// ASSERT\nAssert.Equal(fakeResponseBody, responseBody);\nmockWebRequest.Verify(x =\u003e x.GetResponse());\n\n```\n\n## Secret Sauce\n\n**HttpWebRequestWrapper** works by inheriting from `HttpWebRequest`.  This doesn't seem revolutionary, except these are the `HttpWebRequest` constructors:\n\n```csharp\n[Obsolete(\"This API supports the .NET Framework infrastructure and is not intended to be used directly from your code.\", \n    true)]\npublic HttpWebRequest(){}\ninternal HttpWebRequest(Uri uri, ServicePoint servicePoint)\n{\n    // bunch of initialization code\n}\n```\n\nNiether constructor will allow compilation of code that tries to inherit from `HttpWebRequest`.\n\n### ildasm\n\nInstead `HttpWebRequestWrapper` is compiled to inherit from `WebRequest`. During build, the compiled assembly is deassembled, has it's IL manipulated so that it inherits from `HttpWebRequest` and is then reasembled.  `HttpWebRequestWrapper` also does a bunch of reflection inside its constructor so the end result is a public class with a public constructor that inherits from `HttpWebRequest` and is fully functional!\n\n### WebRequest.PrefixList\n\nThe second piece of magic is hooking into `WebRequest.PrefixList`.  `WebRequest` works as a factory for factories.  The `PrefixList` contains the registration of which factory is matched to which protocol.  `HttpWebRequestWrapperSession` works by overriding the default `PrefixList`, replacing the `IWebRequestCreate` for `http` and `https`.  \n\nDisposing of the Session restores the original `Prefix` list.\n\n### HttpClient\n\n`HttpClient` uses an internal `HttpWebRequest` constructor to directly create requests and by-passes `WebRequest.PrefixList`.  So instead,\n**HttpWebRequestWrapper** uses a custom `TaskScheduler` to hook into `HttpClientHandler` and hijack the `HttpClientHandler.StartRequeset` method to  replace the underlying `HttpWebRequest` it will use with one created via `WebRequest.Create`.  \n\n### Limitations\n\n**HttpWebRequestWrapper** can *not* support concurrent test execution.  Becuase `HttpWebRequestWrapperSession` works by setting a global static variable it's not possible to have two Sessions in use at one time.  Code inside the Session's using block is free to execute concurrently, you just can't try and use two or more Sessions at once.\n\n`HttpClient` performs IO via the abstract `HttpMessageHandler`.  By default, this is a `HttpClinetHandler`.  **HttpWebRequestWrapper** fully supports `HttpClientHandler` and any handler that inherits from `HttpClientHandler`.  Custom handlers that instead inheirt directly from `HttpMessageHandler` are not supported and will not be intercepted.\n\n## Platform Support\n\nThe actual wrapper `HttpWebRequestWrapper` is compiled for .NET Framework 2.0 and supports up to versions 4.7 of the .NET Framework (newest version tested at the time).\n\nThe remainder of **HttpWebRequestWrapper** requires .NET Framework 3.5+\n\nSupport for `HttpClient` requires .NET Framework 4.5+\n\n## Build\n\nClone the repository and build `/src/HttpWebRequestWrapper.sln` using Visual Studio. NuGet package restore must be enabled.\n\nTo generate a nuget package run:\n\n    nuget pack .\\src\\HttpWebRequestWrapper\\HttpWebRequestWrapper.csproj\n\nNOTE: I recommend unloading the `HttpWebRequestWrapper.LowLevel` project after opening the solution in Visual Studio and building for the first time.  The IDE (and tools like ReSharper) get confused by `HttpWebRequestWrapper` inheriting from `WebRequest` in source code and `HttpWebRequest` in the built dll.  Consequence is a lot of warnings and false errors.","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fppittle%2Fhttpwebrequestwrapper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fppittle%2Fhttpwebrequestwrapper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fppittle%2Fhttpwebrequestwrapper/lists"}