{"id":13804284,"url":"https://github.com/ikod/dlang-requests","last_synced_at":"2026-01-15T00:51:40.719Z","repository":{"id":7084052,"uuid":"56168307","full_name":"ikod/dlang-requests","owner":"ikod","description":"dlang http client library inspired by python-requests","archived":false,"fork":false,"pushed_at":"2025-09-16T09:59:54.000Z","size":1163,"stargazers_count":159,"open_issues_count":9,"forks_count":31,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-11-30T03:47:34.366Z","etag":null,"topics":["d","dlang","dlang-requests","http-client"],"latest_commit_sha":null,"homepage":"","language":"D","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsl-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ikod.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2016-04-13T16:28:58.000Z","updated_at":"2025-09-20T04:30:07.000Z","dependencies_parsed_at":"2022-08-08T17:31:20.002Z","dependency_job_id":"4696f879-7501-4b0f-a9f2-dd7b94fbf205","html_url":"https://github.com/ikod/dlang-requests","commit_stats":{"total_commits":361,"total_committers":25,"mean_commits":14.44,"dds":"0.16066481994459836","last_synced_commit":"5888a372e57e7f64dfc4fe4b6fe58f10cef5298e"},"previous_names":[],"tags_count":84,"template":false,"template_full_name":null,"purl":"pkg:github/ikod/dlang-requests","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ikod%2Fdlang-requests","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ikod%2Fdlang-requests/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ikod%2Fdlang-requests/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ikod%2Fdlang-requests/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ikod","download_url":"https://codeload.github.com/ikod/dlang-requests/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ikod%2Fdlang-requests/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28440578,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-15T00:34:46.850Z","status":"ssl_error","status_checked_at":"2026-01-15T00:34:46.551Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["d","dlang","dlang-requests","http-client"],"created_at":"2024-08-04T01:00:45.047Z","updated_at":"2026-01-15T00:51:40.708Z","avatar_url":"https://github.com/ikod.png","language":"D","funding_links":[],"categories":["Web Frameworks"],"sub_categories":["Bare metal / kernel development"],"readme":"# dlang-requests\n[![Run all D Tests](https://github.com/ikod/dlang-requests/actions/workflows/blank.yml/badge.svg)](https://github.com/ikod/dlang-requests/actions/workflows/blank.yml)\n![DUB](https://img.shields.io/dub/dm/requests?color=blue)\n\n![Alt](ua.png?raw=true)\n\n\nHTTP client library, inspired by python-requests with goals:\n\n* small memory footprint\n* performance\n* simple, high level API\n* native D implementation\n\nAPI docs: [Wiki](https://ikod.github.io/dlang-requests/)\n\n## Table of contents\n\n- [Library configurations (std.socket and vibe sockets)](#library-configurations)\n- [Levels of API](#two-levels-of-api)\n- [Quick start](#make-a-simple-request)\n- [Requests with parameters](#request-with-parameters)\n- [Posting data](#posting-data-to-server)\n   - [Posting url-encoded](#form-urlencode)\n   - [Posting multipart](#multipart-form)\n   - [Posting raw data](#posting-raw-data-without-forms)\n- [Properties of Request structure](#request-structure)\n- [Streaming response](#streaming-server-response)\n- [Modifying request(headers, etc.)](#addingreplacing-request-headers)\n- [Interceptors](#interceptors)\n- [SocketFactory](#socketfactory)\n- [SSL](#ssl-settings)\n- [FTP](#ftp-requests)\n- [Request pool](#requests-pool)\n- [Static build](#static-build)\n\n\n### Library configurations ###\n\nThis library doesn't use vibe-d but it can work with vibe-d sockets for network IO, so you can use requests in your vibe-d applications. To build `vibe-d` variant, **with `requests` ver 2.x** use \n\n```json\n    \"dependencies\": {\n        \"requests:vibed\": \"~\u003e2\"\n    }\n```\nFor requests 1.x use subConfiguration:\n\n```json\n\"dependencies\": {\n    \"requests\": \"~\u003e1\"\n},\n\"subConfigurations\": {\n    \"requests\": \"vibed\"\n}\n```\n\n### Two levels of API ###\n* At the highest API level you interested only in retrieving or posting document content.\nUse it when you don't need to add headers, set timeouts, or change any other defaults, \nif you don't interested in result codes or any details of request and/or\nresponse. This level propose next calls: `getContent`, `postContent`, `putContent`\nand `patchContent`. What you receive is a Buffer, which you can use as range, but you can easily \nconvert it to `ubyte[]` using `.data` property. These calls also have `ByLine` counterparts which\nwill lazily receive response from server, split it on `\\n` and convert it into InputRange of ubyte[] (so that something like\n`getContentByLine(\"https://httpbin.org/stream/50\").map!\"cast(string)a\".filter!(a =\u003e a.canFind(\"\\\"id\\\": 28\"))` should work.\n\n* At the next level we have `Request` structure, which encapsulate all details and settings \nrequired for http(s)/ftp transfer. Operating on `Request` instance you can \nchange many aspects of interaction with http/ftp server. Most important API \ncalls are `Request.get()`, `Reuest.post` or `Request.exec!\"method\"` and so \non (you will find examples below). You will receive `Response` with all available\ndetails -document body, status code, headers, timings, etc.\n\n\n#### Windows ssl notes ####\nIn case `requests` can't find opsn ssl library on Windows, here is several steps that can help:\n1. From the [slproweb](https://slproweb.com/products/Win32OpenSSL.html) download latest Win32OpenSSL_Light installer binaries for Windows.\n1. Install it. **Important**: allow installer to install libraries in system folders.\n\nSee step-by-step instructions [here](https://github.com/ikod/dlang-requests/issues/77#issuecomment-405911012).\n\n### Make a simple request ###\n\nMaking HTTP/HTTPS/FTP requests with `dlang-requests` is simple. First of all, install and import `requests` module:\n\n```d\nimport requests;\n```\n\nIf you only need content of some webpage, you can use `getContent()`:\n\n```d\nauto content = getContent(\"http://httpbin.org/\");\n```\n\n`getContent()` will fetch complete document to buffer and return this buffer to the caller(see Straming for lazy content loading). `content` can be converted to string, or can be used as range. For example, if you need to count lines in `content`, you can directly apply `splitter()` and `count`:\n\n```d\nwriteln(content.splitter('\\n').count);\n```\n\nCount non-empty lines:\n\n```d\nwriteln(content.splitter('\\n').filter!\"a!=``\".count);\n```\n\nActually, the buffer is a `ForwardRange` with `length` and random access, so you can apply many algorithms directly to it. Or you can extract data in form of `ubyte[]`, using `data` property:\n\n```d\nubyte[] data = content.data;\n```\n\n\n### Request with parameters ###\n\n`dlang-requests` proposes simple way to make a request with parameters. For example, you have to simulate a search query for person: **name** - person name, **age** - person age, and so on... You can pass all parameters to get using `queryParams()` helper:\n\n```d\nauto content = getContent(\"http://httpbin.org/get\", queryParams(\"name\", \"any name\", \"age\", 42));\n```\n\nIf you check httpbin response, you will see that server recognized all parameters:\n\n```json\n{\n  \"args\": {\n    \"age\": \"42\",\n    \"name\": \"any name\"\n  },\n  \"headers\": {\n    \"Accept-Encoding\": \"gzip, deflate\",\n    \"Host\": \"httpbin.org\",\n    \"User-Agent\": \"dlang-requests\"\n  },\n  \"origin\": \"xxx.xxx.xxx.xxx\",\n  \"url\": \"http://httpbin.org/get?name=any name\u0026age=42\"\n}\n```\n\nOr, you can pass dictionary:\n\n```d\nauto content = getContent(\"http://httpbin.org/get\", [\"name\": \"any name\", \"age\": \"42\"]);\n```\n\nWhich gives you the same response.\n\n\n### If `getContent()` fails ###\n\n `getContent()` (and any other API call) can throw the following exceptions:\n\n * `ConnectError` when it can't connect to document origin for some reason (can't resolve name, connection refused, ...)   \n * `TimeoutException` when any single operation *(connect, receive, send)* timed out.  \n * `ErrnoException` when received `ErrnoException` from any underlying call.\n * `RequestException` in some other cases.\n\n\n### Posting data to server ###\n\nThe easy way to post with `dlang-requests` is `postContent()`. There are several ways to post data to server:\n\n 1. Post to web-form using `application/x-www-form-urlencoded` - for posting short data.\n 1. Post to web-form using `multipart/form-data` - for large data and file uploads.\n 1. Post data to server without forms.\n\n#### Form-urlencode ####\n\nCall `postContent()` in the same way as `getContent()` with parameters:\n\n```d\nimport std.stdio;\nimport requests;\n\nvoid main() {\n    auto content = postContent(\"http://httpbin.org/post\", queryParams(\"name\", \"any name\", \"age\", 42));\n    writeln(content);\n}\n```\n\nOutput:\n\n```json\n{\n  \"args\": {},\n  \"data\": \"\",\n  \"files\": {},\n  \"form\": {\n    \"age\": \"42\",\n    \"name\": \"any name\"\n  },\n  \"headers\": {\n    \"Accept-Encoding\": \"gzip, deflate\",\n    \"Content-Length\": \"22\",\n    \"Content-Type\": \"application/x-www-form-urlencoded\",\n    \"Host\": \"httpbin.org\",\n    \"User-Agent\": \"dlang-requests\"\n  },\n  \"json\": null,\n  \"origin\": \"xxx.xxx.xxx.xxx\",\n  \"url\": \"http://httpbin.org/post\"\n}\n```\n\n#### Multipart form ####\n\nPosting multipart forms requires `MultipartForm` structure to be prepared:\n\n```d\nimport std.stdio;\nimport std.conv;\nimport std.string;\nimport requests;\n\nvoid main() {\n    MultipartForm form;\n    form.add(formData(\"name\", \"any name\"));\n    form.add(formData(\"age\", to!string(42)));\n    form.add(formData(\"raw data\", \"some bytes\".dup.representation));\n    auto content = postContent(\"http://httpbin.org/post\", form);\n    writeln(\"Output:\");\n    writeln(content);\n}\n```\n\nOutput:\n\n```json\n{\n  \"args\": {},\n  \"data\": \"\",\n  \"files\": {},\n  \"form\": {\n    \"age\": \"42\",\n    \"name\": \"any name\",\n    \"raw data\": \"some bytes\"\n  },\n  \"headers\": {\n    \"Accept-Encoding\": \"gzip, deflate\",\n    \"Content-Length\": \"332\",\n    \"Content-Type\": \"multipart/form-data; boundary=e3beab0d-d240-4ec1-91bb-d47b08af5999\",\n    \"Host\": \"httpbin.org\",\n    \"User-Agent\": \"dlang-requests\"\n  },\n  \"json\": null,\n  \"origin\": \"xxx.xxx.xxx.xxx\",\n  \"url\": \"http://httpbin.org/post\"\n}\n```\n\nHere is an example of posting a file:\n\n```d\nimport std.stdio;\nimport std.conv;\nimport std.string;\nimport requests;\n\nvoid main() {\n    MultipartForm form;\n    form.add(formData(\"file\", File(\"test.txt\", \"rb\"), [\"filename\":\"test.txt\", \"Content-Type\": \"text/plain\"]));\n    form.add(formData(\"age\", \"42\"));\n    auto content = postContent(\"http://httpbin.org/post\", form);\n\n    writeln(\"Output:\");\n    writeln(content);\n}\n```\n\nOutput:\n\n```json\n{\n  \"args\": {},\n  \"data\": \"\",\n  \"files\": {\n    \"file\": \"this is test file\\n\"\n  },\n  \"form\": {\n    \"age\": \"42\"\n  },\n  \"headers\": {\n    \"Accept-Encoding\": \"gzip, deflate\",\n    \"Content-Length\": \"282\",\n    \"Content-Type\": \"multipart/form-data; boundary=3fd7317f-7082-4d63-82e2-16cfeaa416b4\",\n    \"Host\": \"httpbin.org\",\n    \"User-Agent\": \"dlang-requests\"\n  },\n  \"json\": null,\n  \"origin\": \"xxx.xxx.xxx.xxx\",\n  \"url\": \"http://httpbin.org/post\"\n}\n```\n\n#### Posting raw data without forms ####\n\n`postContent()` can post from `InputRange`s. For example, to post file content:\n\n```d\nimport std.stdio;\nimport requests;\n\nvoid main() {\n    auto f = File(\"test.txt\", \"rb\");\n    auto content = postContent(\"http://httpbin.org/post\", f.byChunk(5), \"application/binary\");\n    writeln(\"Output:\");\n    writeln(content);\n}\n```\n\nOutput:\n\n```json\n{\n  \"args\": {},\n  \"data\": \"this is test file\\n\",\n  \"files\": {},\n  \"form\": {},\n  \"headers\": {\n    \"Accept-Encoding\": \"gzip, deflate\",\n    \"Content-Length\": \"18\",\n    \"Content-Type\": \"application/binary\",\n    \"Host\": \"httpbin.org\",\n    \"User-Agent\": \"dlang-requests\"\n  },\n  \"json\": null,\n  \"origin\": \"xxx.xxx.xxx.xxx\",\n  \"url\": \"http://httpbin.org/post\"\n}\n```\n\nOr, if you keep your data in memory, you can use something like this:\n\n```d\nauto content = postContent(\"http://httpbin.org/post\", \"ABCDEFGH\", \"application/binary\");\n```\n\nThose are all details about simple API with default request parameters. The next section will describe a lower-level interface through `Request` structure.\n\n\n### `Request` structure ###\n\nWhen you need to configure request details (like timeouts and other limits, keep-alive, ssl properties), or response details (code, headers), you have to use `Request` and `Response` structures:\n\n```d\nRequest rq = Request();\nResponse rs = rq.get(\"https://httpbin.org/\");\nassert(rs.code==200);\n```\n\n\nBy default Keep-Alive requests are used, so you can reuse the connection:\n\n```d\nimport std.stdio;\nimport requests;\n\nvoid main()\n{\n    auto rq = Request();\n    rq.verbosity = 2;\n    auto rs = rq.get(\"http://httpbin.org/image/jpeg\");\n    writeln(rs.responseBody.length);\n    rs = rq.get(\"http://httpbin.org/image/png\");\n    writeln(rs.responseBody.length);\n}\n```\n\nIn the latter case `rq.get()` will reuse previous connection to server.\n`Request` will automatically reopen connection when host, protocol or port change (so it is safe\nto send different requests through single instance of `Request`).\nIt also recovers when server prematurely closes keep-alive connection.\nYou can turn `keepAlive` off when needed:\n\n```d\nrq.keepAlive = false;\n```\n\nFor anything other than default, you can configure `Request` structure for keep-alive, redirects handling, to add/remove headers, set IO buffer size and maximum size of response headers and body.\n\nFor example, to authorize with basic authentication, use the following code (works both for HTTP and FTP URLs):\n\n```d\nrq = Request();\nrq.authenticator = new BasicAuthentication(\"user\", \"passwd\");\nrs = rq.get(\"http://httpbin.org/basic-auth/user/passwd\");\n```\n\nHere is a short description of some `Request` options you can set:\n\n| name                | type             | meaning                                 | default    |\n|---------------------|------------------|-----------------------------------------|------------|\n| keepAlive           | `bool`           | request keepalive connection            | true       |\n| maxRedirects *)     | `uint`           | maximum redirect depth (0 to disable)   | 10         |\n| maxHeadersLength *) | `size_t`         | max. acceptable response headers length | 32KB       |\n| maxContentLength *) | `size_t`         | max. acceptable content length          | 5MB        |\n| timeout *)          | `Duration`       | timeout on connect or data transfer     | 30.seconds |\n| bufferSize          | `size_t`         | socket io buffer size                   | 16KB       |\n| verbosity           | `uint`           | verbosity level (0, 1, 2 or 3)          | 0          |\n| proxy               | `string`         | url of the HTTP proxy                   | null       |\n| addHeaders          | `string[string]` | additional headers                      | null       |\n| useStreaming        | `bool`           | receive data as lazy `InputRange`       | false      |\n| cookie              | `Cookie[]`       | cookies you will send to server         | null       |\n| authenticator       | `Auth`           | authenticatior                          | null       |\n| bind                | `string`         | use local address whan connect          | null       |\n| socketFactory       | [NetworkStream](#socketfactory)      | user-provided connection factory        | null       |\n\n*) Throws exception when limit is reached.\n\n`Request` properties that are read-only:\n\n| name             | type             | meaning                                             |\n|------------------|------------------|-----------------------------------------------------|\n| cookie           | `Cookie[]`       | cookie the server sent to us                        |\n| contentLength    | `long`           | current document's content length or -1 if unknown  |\n| contentReceived  | `long`           | content received                                    |\n\n\n##### Redirect and connection optimisations #####\n\n`Request` keep results of Permanent redirections in small cache. It also keep map\n`(schema,host,port) -\u003e connection` of opened connections, for subsequent usage.\n\n### Streaming server response ###\nWhen you plan to receive something really large in response (file download) you don't want to receive\ngigabytes of content into the response buffer. With `useStreaming`, you can receive response from server as input range.\nElements of the range are chunks of data (of type ubyte[]).\n`contentLength` and `contentReceived` can be used to monitor progress:\n\n```d\nimport std.stdio;\nimport requests;\n\nvoid main()\n{\n    auto rq = Request();\n    rq.useStreaming = true;\n    rq.verbosity = 2;\n    auto rs = rq.get(\"http://httpbin.org/image/jpeg\");\n    auto stream = rs.receiveAsRange();\n    while(!stream.empty) {\n        writefln(\"Received %d bytes, total received %d from document length %d\", stream.front.length, rs.contentReceived, rs.contentLength);\n        stream.popFront;\n    }\n}\n```\n\nOutput:\n\n```\n\u003e GET /image/jpeg HTTP/1.1\n\u003e Connection: Keep-Alive\n\u003e User-Agent: dlang-requests\n\u003e Accept-Encoding: gzip, deflate\n\u003e Host: httpbin.org\n\u003e\n\u003c HTTP/1.1 200 OK\n\u003c server: nginx\n\u003c date: Thu, 09 Jun 2016 16:25:57 GMT\n\u003c content-type: image/jpeg\n\u003c content-length: 35588\n\u003c connection: keep-alive\n\u003c access-control-allow-origin: *\n\u003c access-control-allow-credentials: true\n\u003c 1232 bytes of body received\n\u003c 1448 bytes of body received\nReceived 2680 bytes, total received 2680 from document length 35588\nReceived 2896 bytes, total received 5576 from document length 35588\nReceived 2896 bytes, total received 8472 from document length 35588\nReceived 2896 bytes, total received 11368 from document length 35588\nReceived 1448 bytes, total received 12816 from document length 35588\nReceived 1448 bytes, total received 14264 from document length 35588\nReceived 1448 bytes, total received 15712 from document length 35588\nReceived 2896 bytes, total received 18608 from document length 35588\nReceived 2896 bytes, total received 21504 from document length 35588\nReceived 2896 bytes, total received 24400 from document length 35588\nReceived 1448 bytes, total received 25848 from document length 35588\nReceived 2896 bytes, total received 28744 from document length 35588\nReceived 2896 bytes, total received 31640 from document length 35588\nReceived 2896 bytes, total received 34536 from document length 35588\nReceived 1052 bytes, total received 35588 from document length 35588\n```\n\nWith `verbosity \u003e= 3`, you will also receive a dump of each data portion received from server:\n\n```\n00000  48 54 54 50 2F 31 2E 31  20 32 30 30 20 4F 4B 0D  |HTTP/1.1 200 OK.|\n00010  0A 53 65 72 76 65 72 3A  20 6E 67 69 6E 78 0D 0A  |.Server: nginx..|\n00020  44 61 74 65 3A 20 53 75  6E 2C 20 32 36 20 4A 75  |Date: Sun, 26 Ju|\n00030  6E 20 32 30 31 36 20 31  36 3A 31 36 3A 30 30 20  |n 2016 16:16:00 |\n00040  47 4D 54 0D 0A 43 6F 6E  74 65 6E 74 2D 54 79 70  |GMT..Content-Typ|\n00050  65 3A 20 61 70 70 6C 69  63 61 74 69 6F 6E 2F 6A  |e: application/j|\n00060  73 6F 6E 0D 0A 54 72 61  6E 73 66 65 72 2D 45 6E  |son..Transfer-En|\n00070  63 6F 64 69 6E 67 3A 20  63 68 75 6E 6B 65 64 0D  |coding: chunked.|\n00080  0A 43 6F 6E 6E 65 63 74  69 6F 6E 3A 20 6B 65 65  |.Connection: kee|\n...\n\n```\n\nJust for fun: with streaming you can forward content between servers in just two code lines. `postContent` will automatically receive next data portion from source and send it to destination:\n\n```d\nimport requests;                                                                                                            \nimport std.stdio;                                                                                                           \n                                                                                                                            \nvoid main()                                                                                                                 \n{                                                                                                                           \n    auto rq = Request();                                                                                                    \n    rq.useStreaming = true;                                                                                                 \n    auto stream = rq.get(\"http://httpbin.org/get\").receiveAsRange();                                                        \n    auto content = postContent(\"http://httpbin.org/post\", stream);                                                          \n    writeln(content);                                                                                                       \n}                                                                                                                           \n```\n\nYou can use `dlang-requests` in parallel tasks (but you can't share the same `Request` structure between threads):\n\n```d\nimport std.stdio;\nimport std.parallelism;\nimport std.algorithm;\nimport std.string;\nimport core.atomic;\nimport requests;\n\nimmutable auto urls = [\n    \"http://httpbin.org/stream/10\",\n    \"http://httpbin.org/stream/20\",\n    \"http://httpbin.org/stream/30\",\n    \"http://httpbin.org/stream/40\",\n    \"http://httpbin.org/stream/50\",\n    \"http://httpbin.org/stream/60\",\n    \"http://httpbin.org/stream/70\",\n];\n\nvoid main() {\n    defaultPoolThreads(5);\n\n    shared short lines;\n\n    foreach(url; parallel(urls)) {\n        atomicOp!\"+=\"(lines, getContent(url).splitter(\"\\n\").count);\n    }\n    assert(lines == 287);\n}\n```\n\n##### File download example #####\n\nNote: use \"wb\" and `rawWrite` with file.\n\n```d\nimport requests;\nimport std.stdio;\n\nvoid main() {\n    Request rq = Request();\n    Response rs = rq.get(\"http://geoserver.readthedocs.io/en/latest/_images/imagemosaiccreate1.png\");\n    File f = File(\"123.png\", \"wb\"); // do not forget to use both \"w\" and \"b\" modes when open file.\n    f.rawWrite(rs.responseBody.data);\n    f.close();\n}\n```\nLoading whole document to memory and then save it might be impractical or impossible.\nUse streams in this case:\n```d\nimport requests;\nimport std.stdio;\n\nvoid main() {\n    Request rq = Request();\n\n    rq.useStreaming = true;\n    auto rs = rq.get(\"http://geoserver.readthedocs.io/en/latest/_images/imagemosaiccreate1.png\");\n    auto stream = rs.receiveAsRange();\n    File file = File(\"123.png\", \"wb\");\n\n    while(!stream.empty)  {\n        file.rawWrite(stream.front);\n        stream.popFront;\n    }\n    file.close();\n}\n```\n\n\n#### vibe.d ####\n\nYou can safely use `dlang-requests` with `vibe.d`. When `dlang-requests` is compiled with support for `vibe.d` sockets (`--config=vibed`), each call to `dlang-requests` API can block only the current fiber, not the thread:\n\n```d\nimport requests, vibe.d;\n\nshared static this()\n{\n    void taskMain()\n    {\n        logInfo(\"Task created\");\n        auto r1 = getContent(\"http://httpbin.org/delay/3\");\n        logInfo(\"Delay request finished\");\n        auto r2 = getContent(\"http://google.com\");\n        logInfo(\"Google request finished\");\n    }\n\n    setLogFormat(FileLogger.Format.threadTime, FileLogger.Format.threadTime);\n    for(size_t i = 0; i \u003c 3; i++)\n        runTask(\u0026taskMain);\n}\n```\n\nOutput:\n\n```\n[F7EC2FAB:F7ECD7AB 2016.07.05 16:55:54.115 INF] Task created\n[F7EC2FAB:F7ECD3AB 2016.07.05 16:55:54.116 INF] Task created\n[F7EC2FAB:F7ED6FAB 2016.07.05 16:55:54.116 INF] Task created\n[F7EC2FAB:F7ECD7AB 2016.07.05 16:55:57.451 INF] Delay request finished\n[F7EC2FAB:F7ECD3AB 2016.07.05 16:55:57.464 INF] Delay request finished\n[F7EC2FAB:F7ED6FAB 2016.07.05 16:55:57.474 INF] Delay request finished\n[F7EC2FAB:F7ECD7AB 2016.07.05 16:55:57.827 INF] Google request finished\n[F7EC2FAB:F7ECD3AB 2016.07.05 16:55:57.836 INF] Google request finished\n[F7EC2FAB:F7ED6FAB 2016.07.05 16:55:57.856 INF] Google request finished\n```\n\n### Adding/replacing request headers ###\n\nUse `string[string]` and `addHeaders()` method to add or replace some request headers.\n\nUser-supplied headers override headers, created by library code,\nso you have to be careful adding common headers, like Content-Type, Content-Length, etc..\n\n\n```d\nimport requests;\n\nvoid main() {\n    auto rq = Request();\n    rq.verbosity = 2;\n    rq.addHeaders([\"User-Agent\": \"test-123\", \"X-Header\": \"x-value\"]);\n    auto rs = rq.post(\"http://httpbin.org/post\", `{\"a\":\"b\"}`, \"application/x-www-form-urlencoded\");\n}\n```\n\nOutput:\n\n```\n\u003e POST /post HTTP/1.1\n\u003e Content-Length: 9\n\u003e Connection: Keep-Alive\n\u003e User-Agent: test-123\n\u003e Accept-Encoding: gzip, deflate\n\u003e Host: httpbin.org\n\u003e X-Header: x-value\n\u003e Content-Type: application/x-www-form-urlencoded\n\u003e\n\u003c HTTP/1.1 200 OK\n\u003c server: nginx\n...\n```\n\n### SSL settings ###\n\nHTTP requests can be configured for SSL options: you can enable or disable remote server certificate verification, set key and certificate to use for authorizing to remote server:\n\n* sslSetVerifyPeer(bool) - turn ssl peer verification **on** or **off** (**on** by default since v0.8.0)\n* sslSetKeyFile(string) - load client key from file\n* sslSetCertFile(string) - load client cert from file\n* sslSetCaCert(string) - load server CA cert for private or self-signed server certificates\n* sslUseKeyLogFile(bool) - enable or disable SSLKEYLOGFILE (you have to set file name in env variable SSLKEYLOGFILE)\n\n```d\nimport std.stdio;\nimport requests;\nimport std.experimental.logger;\n\nvoid main() {\n    globalLogLevel(LogLevel.trace);\n    auto rq = Request();\n    rq.sslSetKeyFile(\"client01.key\"); // set key file\n    rq.sslSetCertFile(\"client01.crt\"); // set cert file\n    auto rs = rq.get(\"https://dlang.org/\");\n    writeln(rs.code);\n    writeln(rs.responseBody);\n}\n```\n\nPlease note that with `vibe.d` you have to add the following call\n\n```d\nrq.sslSetCaCert(\"/opt/local/etc/openssl/cert.pem\");\n```\n\nwith path to CA cert file (location may differ for different OS or openssl packaging).\n\nBy default ssl peer verification turned ON. This can lead to problems in case you use server-side self-signed certificates.\nTo fix, you have either add server ca.crt to trusted store on local side(see https://unix.stackexchange.com/questions/90450/adding-a-self-signed-certificate-to-the-trusted-list for example), or use sslSetCaCert to add it for single requests call(`rq.sslSetCaCert(\"ca.crt\");`), or just disable peer verification with\n`rq.sslSetVerifyPeer(false);`\n\n### FTP requests ###\n\nYou can use the same structure to make ftp requests, both get and post.\n\nHTTP-specific methods do not work if request uses `ftp` scheme.\n\nHere is an example:\n\n```d\nimport std.stdio;\nimport requests;\n\nvoid main() {\n    auto rq = Request();\n    rq.verbosity = 3;\n    rq.authenticator = new BasicAuthentication(\"login\", \"password\");\n    auto f = File(\"test.txt\", \"rb\");\n    auto rs = rq.post(\"ftp://example.com/test.txt\", f.byChunk(1024));\n    writeln(rs.code);\n    rs = rq.get(\"ftp://@example.com/test.txt\");\n    writeln(rs.code);\n}\n```\n\nSecond argument for FTP posts can be anything that can be casted to `ubyte[]` or any `InputRange` with element type like `ubyte[]`.\nIf the path in the post request doesn't exist, it will try to create all the required directories.\nAs with HTTP, you can call several FTP requests using the same `Request` structure - it will reuse established connection (and authorization as well).\n\n### FTP LIST command ###\n\nTo retrieve single file properties or directory listing use `rq.execute` method:\n```d\nimport std.stdio;\nimport requests;\n\nvoid main()\n{\n    auto rq = Request();\n    auto rs = rq.execute(\"LIST\", \"ftp://ftp.iij.ad.jp/pub/FreeBSD/\");\n    writeln(rs.responseBody);\n\n}\n\noutput:\n\n-rw-rw-r--    1 ftp      ftp            35 May 12 09:00 TIMESTAMP\ndrwxrwxr-x    9 ftp      ftp           169 Oct 05  2015 development\n-rw-r--r--    1 ftp      ftp          2871 May 11 10:00 dir.sizes\ndrwxrwxr-x   22 ftp      ftp          8192 May 09 23:00 doc\ndrwxrwxr-x    6 ftp      ftp            92 Jan 10 21:38 ports\ndrwxrwxr-x   12 ftp      ftp           237 Feb 06  2021 releases\ndrwxrwxr-x   12 ftp      ftp           237 May 05 18:00 snapshots\n\n``` \n\n\n### Interceptors ###\n\nInterceptors provide a way to modify, or log, or cache request. They can form a chain attached to Request structure so that\neach request will pass through whole chain.\n\nEach interceptor receive request as input, do whatever it need and pass request to the handler, which finally serve request and\nreturn `Response` back.\n\n\nHere is small example how interceptors can be used. Consider situation where you have  main app and some module. Main code:\n\n```d\nimport std.stdio;\nimport mymodule;\n\nvoid main()\n{\n    auto r = mymodule.doSomething();\n    writeln(r.length);\n}\n```\nmodule:\n```d\nmodule mymodule;\n\nimport requests;\n\nauto doSomething()\n{\n    return getContent(\"http://google.com\");                                                                              \n}\n```\nOne day you decide that you need to log every http request to external services.\n\nOne solution is to add logging code to each \nfunction of `mymodule` where external http calls executed. This can require lot of work and code changes, and sometimes\neven not really possible.\n\nAnother, and more effective solution is to use interceptors. First we have to create `logger` class:\n\n```d\nclass LoggerInterceptor : Interceptor {\n    Response opCall(Request r, RequestHandler next)\n    {\n        writefln(\"Request  %s\", r);\n        auto rs = next.handle(r);\n        writefln(\"Response %s\", rs);\n        return rs;\n    }\n}\n```\nThen we can instrument every call to `request` with this call:\n\n```d\nimport std.stdio;\nimport requests;\nimport mymodule;\n\nclass LoggerInterceptor : Interceptor {\n    Response opCall(Request r, RequestHandler next)\n    {\n        writefln(\"Request  %s\", r);\n        auto rs = next.handle(r);\n        writefln(\"Response %s\", rs);\n        return rs;\n    }\n}\n\nvoid main()\n{\n    requests.addInterceptor(new LoggerInterceptor());\n    auto r = mymodule.doSomething();\n    writeln(r.length);\n}\n```\n\nThe only change required is call addInterceptor().\n\nYou may intercept single Request structure (instead of whole `requests` module) attaching interceptors directly to this structure:\n```d\nRequest rq;\nrq.addInterceptor(new LoggerInterceptor());\n\n```\n\nInterceptor can change Request `r`, using Request() getters/setters before pass it to next handler.\nFor example, authentication methods can be added using interceptors and headers injection.\nYou can implement some kind of cache and return cached response immediately.\n\nTo change POST data in the interceptors you can use makeAdapter (since v1.1.2).\nExample:\n\n```\nimport std.stdio;\nimport std.string;\nimport std.algorithm;\nimport requests;\n\nclass I : Interceptor\n{\n    Response opCall(Request rq, RequestHandler next)\n    {\n        rq.postData = makeAdapter(rq.postData.map!\"toUpper(cast(string)a)\");\n        auto rs = next.handle(rq);\n        return rs;\n    }\n}\n\nvoid main()\n{\n    auto f = File(\"text.txt\", \"rb\");\n    Request rq;\n    rq.verbosity = 1;\n    rq.addInterceptor(new I);\n    auto rs = rq.post(\"https://httpbin.org/post\", f.byChunk(5));\n    writeln(rs.responseBody);\n}\n```\n\n### SocketFactory ###\n\nIf configured - each time when `Request` need new connection it will call factory to create instance of NetworkStream.\nThis way you can implement (outside of this library) lot of useful things: various proxies, unix-socket connections, etc.\n\n\n### `Response` structure ###\n\nThis structure provides details about received response.\n\nMost frequently needed parts of `Response` are:\n\n* `code` - HTTP or FTP response code as received from server.\n* `responseBody` - contain complete document body when no streaming is in use. You can't use it when in streaming mode.\n* `responseHeaders` - response headers in form of `string[string]` (not available for FTP requests)\n* `receiveAsRange` - if you set `useStreaming` in the `Request`, then `receiveAsRange` will provide elements (type `ubyte[]`) of `InputRange` while receiving data from the server.\n\n\n### Requests Pool ###\n\nWhen you have a large number of requests to execute, you can use a request pool to speed things up.\n\nA `pool` is a fixed set of worker threads, which receives requests in form of `Job`s and returns `Result`s.\n\nEach `Job` can be configured for an URL, method, data (for POST requests) and some other parameters.\n\n`pool` acts as a parallel `map` from `Job` to `Result` - it consumes `InputRange` of `Job`s, and produces `InputRange` of `Result`s as fast as it can.\n\nIt is important to note that `pool` does not preserve result order. If you need to tie jobs and results somehow, you can use the `opaque` field of `Job`.\n\nHere is an example usage:\n\n```d\nimport std.algorithm;\nimport std.datetime;\nimport std.string;\nimport std.range;\nimport requests;\n\nvoid main() {\n    Job[] jobs = [\n        Job(\"http://httpbin.org/get\").addHeaders([\n                            \"X-Header\": \"X-Value\",\n                            \"Y-Header\": \"Y-Value\"\n                        ]),\n        Job(\"http://httpbin.org/gzip\"),\n        Job(\"http://httpbin.org/deflate\"),\n        Job(\"http://httpbin.org/absolute-redirect/3\")\n                .maxRedirects(2),\n        Job(\"http://httpbin.org/range/1024\"),\n        Job(\"http://httpbin.org/post\")                                                                               \n                .method(\"POST\")                     // change default GET to POST\n                .data(\"test\".representation())      // attach data for POST\n                .opaque(\"id\".representation),       // opaque data - you will receive the same in Result\n        Job(\"http://httpbin.org/delay/3\")\n                .timeout(1.seconds),                // set timeout to 1.seconds - this request will throw exception and fails\n        Job(\"http://httpbin.org/stream/1024\"),\n    ];\n\n    auto count = jobs.\n        pool(6).\n        filter!(r =\u003e r.code==200).\n        count();\n\n    assert(count == jobs.length - 2, \"pool test failed\");\n    iota(20)\n        .map!(n =\u003e Job(\"http://httpbin.org/post\")\n                        .data(\"%d\".format(n).representation))\n        .pool(10)\n        .each!(r =\u003e assert(r.code==200));\n}\n```\n\nOne more example, with more features combined:\n\n```d\nimport requests;\nimport std.stdio;\nimport std.string;\n\nvoid main() {\n    Job[] jobs_array = [\n        Job(\"http://0.0.0.0:9998/3\"),\n        Job(\"http://httpbin.org/post\").method(\"POST\").data(\"test\".representation()).addHeaders([\"a\":\"b\"]),\n        Job(\"http://httpbin.org/post\", Job.Method.POST, \"test\".representation()).opaque([1,2,3]),\n        Job(\"http://httpbin.org/absolute-redirect/4\").maxRedirects(2),\n    ];\n    auto p = pool(jobs_array, 10);\n    while(!p.empty) {\n        auto r = p.front;\n        p.popFront;\n        switch(r.flags) {\n        case Result.OK:\n            writeln(r.code);\n            writeln(cast(string)r.data);\n            writeln(r.opaque);\n            break;\n        case Result.EXCEPTION:\n            writefln(\"Exception: %s\", cast(string)r.data);\n            break;\n        default:\n            continue;\n        }\n        writeln(\"---\");\n    }\n}\n```\n\nOutput:\n\n```\n2016-12-29T10:22:00.861:streams.d:connect:973 Failed to connect to 0.0.0.0:9998(0.0.0.0:9998): Unable to connect socket: Connection refused\n2016-12-29T10:22:00.861:streams.d:connect:973 Failed to connect to 0.0.0.0:9998(0.0.0.0:9998): Unable to connect socket: Connection refused\nException: Can't connect to 0.0.0.0:9998\n---\n200\n{\n  \"args\": {},\n  \"data\": \"test\",\n  \"files\": {},\n  \"form\": {},\n  \"headers\": {\n    \"A\": \"b\",\n    \"Accept-Encoding\": \"gzip, deflate\",\n    \"Content-Length\": \"4\",\n    \"Content-Type\": \"application/octet-stream\",\n    \"Host\": \"httpbin.org\",\n    \"User-Agent\": \"dlang-requests\"\n  },\n  \"json\": null,\n  \"origin\": \"xxx.xxx.xxx.xxx\",\n  \"url\": \"http://httpbin.org/post\"\n}\n\n[]\n---\n200\n{\n  \"args\": {},\n  \"data\": \"test\",\n  \"files\": {},\n  \"form\": {},\n  \"headers\": {\n    \"Accept-Encoding\": \"gzip, deflate\",\n    \"Content-Length\": \"4\",\n    \"Content-Type\": \"application/octet-stream\",\n    \"Host\": \"httpbin.org\",\n    \"User-Agent\": \"dlang-requests\"\n  },\n  \"json\": null,\n  \"origin\": \"xxx.xxx.xxx.xxx\",\n  \"url\": \"http://httpbin.org/post\"\n}\n\n[1, 2, 3]\n---\nException: 2 redirects reached maxRedirects 2.\n---\n```\n\n`Job` methods\n\n| name        | parameter type             | description               |\n|-------------|----------------------------|---------------------------|\n| method      | `string` \"GET\" or \"POST\"   | request method            |\n| data        | `immutable(ubyte)[]`       | data for POST request     |\n| timeout     | `Duration`                 | timeout for network IO    |\n| maxRedirects| `uint`                     | max. no. of redirects     |\n| opaque      | `immutable(ubyte)[]`       | opaque data               |\n| addHeaders  | `string[string]`           | headers to add to request |\n\n`Result` fields\n\n| name   | type                 | description          |\n|--------|----------------------|----------------------|\n| flags  | `uint`               | flags (OK,EXCEPTION) |\n| code   | `ushort`             | response code        |\n| data   | `immutable(ubyte)[]` | response body        |\n| opaque | `immutable(ubyte)[]` | opaque data from job |\n\n\n### Pool limitations ###\n\n1. Currently it doesn't work under `vibe.d` - use `vibe.d` parallelisation.\n1. It limits you in tuning request (e.g. you can add authorization only through `addHeaders()`, you can't tune SSL parameters, etc).\n1. `Job`'s' and `Result`'s' `data` are immutable byte arrays (as it uses send/receive for data exchange).\n\n#### International Domain names ####\n\ndlang-requests supports IDNA through `idna` package.\nIt provide correct conversion between unicode domain names and punycode, but have limited ability to check names for standard compliance.\n\n### Static Build ###\n\nBy default dlang-requests links to the openssl and crypto libraries dynamically. There is a `staticssl` dub configuration (linux only!) in order to link statically to these libraries.\n\nThis is often used to generate a \"distro-less\" binaries, typically done on `alpine` using musl's libc.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fikod%2Fdlang-requests","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fikod%2Fdlang-requests","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fikod%2Fdlang-requests/lists"}