{"id":13636315,"url":"https://github.com/timebug/lua-resty-httpipe","last_synced_at":"2025-04-19T08:31:50.404Z","repository":{"id":16196186,"uuid":"18942888","full_name":"timebug/lua-resty-httpipe","owner":"timebug","description":"Lua HTTP client cosocket driver for OpenResty / ngx_lua, interfaces are more flexible","archived":false,"fork":false,"pushed_at":"2020-07-22T14:36:55.000Z","size":95,"stargazers_count":71,"open_issues_count":1,"forks_count":12,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-02-14T17:32:14.906Z","etag":null,"topics":["flexible","http-client","lua-resty"],"latest_commit_sha":null,"homepage":"","language":"Perl","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"addyosmani/polymer-boilerplate","license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/timebug.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-04-19T14:32:00.000Z","updated_at":"2023-10-31T11:37:15.000Z","dependencies_parsed_at":"2022-09-11T15:23:16.748Z","dependency_job_id":null,"html_url":"https://github.com/timebug/lua-resty-httpipe","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timebug%2Flua-resty-httpipe","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timebug%2Flua-resty-httpipe/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timebug%2Flua-resty-httpipe/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timebug%2Flua-resty-httpipe/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/timebug","download_url":"https://codeload.github.com/timebug/lua-resty-httpipe/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249650155,"owners_count":21305976,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["flexible","http-client","lua-resty"],"created_at":"2024-08-02T00:00:59.849Z","updated_at":"2025-04-19T08:31:50.123Z","avatar_url":"https://github.com/timebug.png","language":"Perl","funding_links":[],"categories":["Libraries"],"sub_categories":[],"readme":"# Name\n\n[![Build Status](https://travis-ci.org/timebug/lua-resty-httpipe.svg)](https://travis-ci.org/timebug/lua-resty-httpipe)\n\nlua-resty-httpipe - Lua HTTP client cosocket driver for [OpenResty](http://openresty.org/) / [ngx_lua](https://github.com/chaoslawful/lua-nginx-module), interfaces are more flexible.\n\n# Table of Contents\n\n* [Status](#status)\n* [Features](#features)\n* [Synopsis](#synopsis)\n* [Methods](#methods)\n  * [Connection](#connection)\n    * [new](#new)\n    * [connect](#connect)\n    * [set_timeout](#set_timeout)\n    * [ssl_handshake](#ssl_handshake)\n    * [set_keepalive](#set_keepalive)\n    * [get_reused_times](#get_reused_times)\n    * [close](#close)\n  * [Requesting](#requesting)\n    * [request](#request)\n    * [request_uri](#request_uri)\n    * [res.body_reader](#res.body_reader)\n    * [send_request](#send_request)\n    * [read_response](#read_response)\n    * [read](#read)\n    * [eof](#eof)\n  * [Utility](#utility)\n    * [parse_uri](#parse_uri)\n    * [get_client_body_reader](#get_client_body_reader)\n* [Author](#author)\n* [Copyright and License](#copyright-and-license)\n* [See Also](#see-also)\n\n\n# Status\n\nReady for testing. Probably production ready in most cases, though not yet proven in the wild. Please check the issues list and let me know if you have any problems / questions.\n\n# Features\n\n* HTTP 1.0/1.1 and HTTPS\n* Flexible interface design\n* Streaming reader and uploads\n* Chunked-encoding request / response body\n* Sets the timeout for read and send operations\n* Limit the maximum response body size\n* Keepalive\n\n# Synopsis\n\n````lua\nlua_package_path \"/path/to/lua-resty-httpipe/lib/?.lua;;\";\n\nserver {\n\n  listen 9090;\n\n  location /echo {\n    content_by_lua_block {\n      local raw_header = ngx.req.raw_header()\n\n      if ngx.req.get_method() == \"GET\" then\n          ngx.header[\"Content-Length\"] = #raw_header\n      end\n\n      ngx.req.read_body()\n      local body, err = ngx.req.get_body_data()\n\n      ngx.print(raw_header)\n      ngx.print(body)\n    }\n  }\n\n  location /simple {\n    content_by_lua_block {\n      local httpipe = require \"resty.httpipe\"\n\n      local hp, err = httpipe:new()\n      if not hp then\n          ngx.log(ngx.ERR, \"failed to new httpipe: \", err)\n          return ngx.exit(503)\n      end\n\n      hp:set_timeout(5 * 1000) -- 5 sec\n\n      local res, err = hp:request(\"127.0.0.1\", 9090, {\n                                     method = \"GET\", path = \"/echo\" })\n      if not res then\n          ngx.log(ngx.ERR, \"failed to request: \", err)\n          return ngx.exit(503)\n      end\n\n      ngx.status = res.status\n\n      for k, v in pairs(res.headers) do\n          ngx.header[k] = v\n      end\n\n      ngx.say(res.body)\n    }\n  }\n\n  location /generic {\n    content_by_lua_block {\n      local cjson = require \"cjson\"\n      local httpipe = require \"resty.httpipe\"\n\n      local hp, err = httpipe:new(10) -- chunk_size = 10\n      if not hp then\n          ngx.log(ngx.ERR, \"failed to new httpipe: \", err)\n          return ngx.exit(503)\n      end\n\n      hp:set_timeout(5 * 1000) -- 5 sec\n\n      local ok, err = hp:connect(\"127.0.0.1\", 9090)\n      if not ok then\n          ngx.log(ngx.ERR, \"failed to connect: \", err)\n          return ngx.exit(503)\n      end\n\n      local ok, err = hp:send_request{ method = \"GET\", path = \"/echo\" }\n      if not ok then\n          ngx.log(ngx.ERR, \"failed to send request: \", err)\n          return ngx.exit(503)\n      end\n\n      -- full streaming parser\n\n      while true do\n          local typ, res, err = hp:read()\n          if not typ then\n              ngx.say(\"failed to read: \", err)\n              return\n          end\n\n          ngx.say(\"read: \", cjson.encode({typ, res}))\n\n          if typ == 'eof' then\n              break\n          end\n      end\n    }\n  }\n\n  location /advanced {\n    content_by_lua_block {\n      local httpipe = require \"resty.httpipe\"\n\n      local hp, err = httpipe:new()\n\n      hp:set_timeout(5 * 1000) -- 5 sec\n\n      local r0, err = hp:request(\"127.0.0.1\", 9090, {\n                                     method = \"GET\", path = \"/echo\",\n                                     stream = true })\n\n      -- from one http stream to another, just like a unix pipe\n\n      local pipe = r0.pipe\n\n      pipe:set_timeout(5 * 1000) -- 5 sec\n\n      --[[\n          local headers = {[\"Content-Length\"] = r0.headers[\"Content-Length\"]}\n          local r1, err = pipe:request(\"127.0.0.1\", 9090, {\n                                           method = \"POST\", path = \"/echo\",\n                                           headers = headers,\n                                           body = r0.body_reader })\n      --]]\n      local r1, err = pipe:request(\"127.0.0.1\", 9090, {\n                                       method = \"POST\", path = \"/echo\" })\n\n      ngx.status = r1.status\n\n      for k, v in pairs(r1.headers) do\n          ngx.header[k] = v\n      end\n\n      ngx.say(r1.body)\n    }\n  }\n\n}\n````\n\nA typical output of the `/simple` location defined above is:\n\n```\nGET /echo HTTP/1.1\nHost: 127.0.0.1\nUser-Agent: Resty/HTTPipe-1.00\nAccept: */*\n\n```\n\nA typical output of the `/generic` location defined above is:\n\n```\nread: [\"statusline\",\"200\"]\nread: [\"header\",[\"Server\",\"openresty\\/1.5.12.1\",\"Server: openresty\\/1.5.12.1\"]]\nread: [\"header\",[\"Date\",\"Tue, 10 Jun 2014 07:29:57 GMT\",\"Date: Tue, 10 Jun 2014 07:29:57 GMT\"]]\nread: [\"header\",[\"Content-Type\",\"text\\/plain\",\"Content-Type: text\\/plain\"]]\nread: [\"header\",[\"Connection\",\"keep-alive\",\"Connection: keep-alive\"]]\nread: [\"header\",[\"Content-Length\",\"84\",\"Content-Length: 84\"]]\nread: [\"header_end\"]\nread: [\"body\",\"GET \\/echo \"]\nread: [\"body\",\"HTTP\\/1.1\\r\\n\"]\nread: [\"body\",\"Host: 127.\"]\nread: [\"body\",\"0.0.1\\r\\nUse\"]\nread: [\"body\",\"r-Agent: R\"]\nread: [\"body\",\"esty\\/HTTPi\"]\nread: [\"body\",\"pe-1.00\\r\\nA\"]\nread: [\"body\",\"ccept: *\\/*\"]\nread: [\"body\",\"\\r\\n\\r\\n\"]\nread: [\"body_end\"]\nread: [\"eof\"]\n```\n\nA typical output of the `/advanced` location defined above is:\n\n```\nPOST /echo HTTP/1.1\nContent-Length: 84\nUser-Agent: Resty/HTTPipe-1.00\nAccept: */*\nHost: 127.0.0.1\n\nGET /echo HTTP/1.1\nHost: 127.0.0.1\nUser-Agent: Resty/HTTPipe-1.00\nAccept: */*\n\n\n```\n\n# Methods\n\n[Back to TOC](#table-of-contents)\n\n## Connection\n\n### new\n\n**syntax:** `hp, err = httpipe:new(chunk_size?, sock?)`\n\nCreates the httpipe object. In case of failures, returns `nil` and a string describing the error.\n\nThe argument, `chunk_size`, specifies the buffer size used by cosocket reading operations. Defaults to `8192`.\n\n[Back to TOC](#table-of-contents)\n\n### connect\n\n`syntax: ok, err = hp:connect(host, port, options_table?)`\n\n`syntax: ok, err = hp:connect(\"unix:/path/to/unix.sock\", options_table?)`\n\nAttempts to connect to the web server.\n\nBefore actually resolving the host name and connecting to the remote backend, this method will always look up the connection pool for matched idle connections created by previous calls of this method.\n\nAn optional Lua table can be specified as the last argument to this method to specify various connect options:\n\n* `pool`\n: Specifies a custom name for the connection pool being used. If omitted, then the connection pool name will be generated from the string template `\u003chost\u003e:\u003cport\u003e` or `\u003cunix-socket-path\u003e`.\n\n[Back to TOC](#table-of-contents)\n\n### set_timeout\n\n**syntax:** `hp:set_timeout(time)`\n\nSets the timeout (in ms) protection for subsequent operations, including the `connect` method.\n\n[Back to TOC](#table-of-contents)\n\n### ssl_handshake\n\n**syntax:** `hp:ssl_handshake(reused_session?, server_name?, ssl_verify?)`\n\nDoes SSL/TLS handshake on the currently established connection.\n\nSee more: \u003chttp://wiki.nginx.org/HttpLuaModule#tcpsock:sslhandshake\u003e\n\n[Back to TOC](#table-of-contents)\n\n### set_keepalive\n\n**syntax:** `ok, err = hp:set_keepalive(max_idle_timeout, pool_size)`\n\nAttempts to puts the current connection into the ngx_lua cosocket connection pool.\n\n**Note** Normally, it will be called automatically after processing the request. In other words, we cannot release the connection back to the pool unless you consume all the data.\n\nYou can specify the max idle timeout (in ms) when the connection is in the pool and the maximal size of the pool every nginx worker process.\n\nIn case of success, returns 1. In case of errors, returns nil with a string describing the error.\n\n[Back to TOC](#table-of-contents)\n\n### get_reused_times\n\n**syntax:** `times, err = hp:get_reused_times()`\n\nThis method returns the (successfully) reused times for the current connection. In case of error, it returns `nil` and a string describing the error.\n\nIf the current connection does not come from the built-in connection pool, then this method always returns `0`, that is, the connection has never been reused (yet). If the connection comes from the connection pool, then the return value is always non-zero. So this method can also be used to determine if the current connection comes from the pool.\n\n[Back to TOC](#table-of-contents)\n\n### close\n\n**syntax:** `ok, err = hp:close()`\n\nCloses the current connection and returns the status.\n\nIn case of success, returns `1`. In case of errors, returns `nil` with a string describing the error.\n\n[Back to TOC](#table-of-contents)\n\n\n## Requesting\n\n### request\n\n**syntax:** `res, err = hp:request(opts?)`\n\n**syntax:** `res, err = hp:request(host, port, opts?)`\n\n**syntax:** `res, err = hp:request(\"unix:/path/to/unix-domain.socket\", opts?)`\n\nThe `opts` table accepts the following fields:\n\n* `version`: Sets the HTTP version. Use `10` for HTTP/1.0 and `11` for HTTP/1.1. Defaults to `11`.\n* `method`: The HTTP method string. Defaults to `GET`.\n* `path`: The path string. Default to `/`.\n* `query`: Specifies query parameters. Accepts either a string or a Lua table.\n* `headers`: A table of request headers. Accepts a Lua table.\n* `body`: The request body as a string, or an iterator function.\n* `read_timeout`: Sets the timeout in milliseconds for network read operations specially.\n* `send_timeout`: Sets the timeout in milliseconds for network send operations specially.\n* `stream`: If set to `true`, return an iterable `res.body_reader` object instead of `res.body`.\n* `maxsize`: Sets the maximum size in bytes to fetch. A response body larger than this will cause the fucntion to return a `exceeds maxsize` error. Defaults to nil which means no limit.\n* `ssl_verify`: A Lua boolean value to control whether to perform SSL verification.\n\nWhen the request is successful, `res` will contain the following fields:\n\n* `res.status` (number): The resonse status, e.g. 200\n* `res.headers` (table): A Lua table with response headers.\n* `res.body` (string): The plain response body.\n* `res.body_reader` (function): An iterator function for reading the body in a streaming fashion.\n* `res.pipe` (httpipe): A new http pipe which use the current `body_reader` as input body by default.\n\n**Note** All headers (request and response) are noramlized for capitalization - e.g., Accept-Encoding, ETag, Foo-Bar, Baz - in the normal HTTP \"standard.\"\n\nIn case of errors, returns nil with a string describing the error.\n\n[Back to TOC](#table-of-contents)\n\n### request_uri\n\n**syntax:** `res, err = hp:request_uri(uri, opts?)`\n\nThe simple interface. Options supplied in the `opts` table are the same as in the generic interface, and will override components found in the uri itself.\n\nReturns a res object as same as `hp:request` method.\n\nIn case of errors, returns nil with a string describing the error.\n\n[Back to TOC](#table-of-contents)\n\n### res.body_reader\n\nThe `body_reader` iterator can be used to stream the response body in chunk sizes of your choosing, as follows:\n\n````lua\nlocal reader = res.body_reader\n\nrepeat\n  local chunk, err = reader(8192)\n  if err then\n    ngx.log(ngx.ERR, err)\n    break\n  end\n\n  if chunk then\n    -- process\n  end\nuntil not chunk\n````\n\n[Back to TOC](#table-of-contents)\n\n### send_request\n\n**syntax:** `ok, err = hp:send_request(opts?)`\n\nIn case of errors, returns nil with a string describing the error.\n\n[Back to TOC](#table-of-contents)\n\n### read_response\n\n**syntax:** `local res, err = hp:read_response(callback?)`\n\nThe `callback` table accepts the following fields:\n\n* `header_filter`: A callback function for response headers filter\n\n````lua\nlocal res, err = hp:read_response{\n    header_filter = function (status, headers)\n        if status == 200 then\n        \treturn 1\n        end\nend }\n````\n\n* `body_filter`: A callback function for response body filter\n\n````lua\nlocal res, err = hp:read_response{\n    body_filter = function (chunk)\n        ngx.print(chunk)\n    end\n}\n````\n\nAdditionally there is no ability to stream the response body in this method. If the response is successful, res will contain the following fields: `res.status`, `res.headers`, `res.body`.\n\n**Note** When return true in callback function，filter process will be interrupted.\n\nIn case of errors, returns nil with a string describing the error.\n\n[Back to TOC](#table-of-contents)\n\n### read\n\n**syntax:** `local typ, res, err = hp:read()`\n\nStreaming parser for the full response.\n\nThe user just needs to call the read method repeatedly until a nil token type is returned. For each token returned from the read method, just check the first return value for the current token type. The token type can be `statusline`, `header`, `header_end`, `body`, `body_end` and `eof`. About the format of `res` value, please refer to the above example. For example, several body tokens holding each body data chunk, so `res` value is equal to the body data chunk.\n\nIn case of errors, returns nil with a string describing the error.\n\n[Back to TOC](#table-of-contents)\n\n### eof\n\n**syntax:** `local eof = hp:eof()`\n\nIf return `true` indicating already consume all the data; Otherwise, the request there is still no end, you need call `hp:close` to close the connection forcibly.\n\n[Back to TOC](#table-of-contents)\n\n## Utility\n\n### parse_uri\n\n**syntax:** `local scheme, host, port, path, args = unpack(hp:parse_uri(uri))`\n\nThis is a convenience function allowing one to more easily use the generic interface, when the input data is a URI.\n\n[Back to TOC](#table-of-contents)\n\n### get_client_body_reader\n\n**syntax:** `reader, err = hp:get_client_body_reader(chunk_size?)`\n\nReturns an iterator function which can be used to read the downstream client request body in a streaming fashion. For example:\n\n```lua\nlocal req_reader = hp:get_client_body_reader()\n\nrepeat\n  local chunk, err = req_reader(8192)\n  if err then\n    ngx.log(ngx.ERR, err)\n    break\n  end\n\n  if chunk then\n    -- process\n  end\nuntil not chunk\n```\n\nThis iterator can also be used as the value for the body field in request params, allowing one to stream the request body into a proxied upstream request.\n\n```lua\nlocal client_body_reader, err = hp:get_client_body_reader()\n\nlocal res, err = hp:request{\n   path = \"/helloworld\",\n   body = client_body_reader,\n}\n```\n\n[Back to TOC](#table-of-contents)\n\n# Author\n\nMonkey Zhang \u003ctimebug.info@gmail.com\u003e, UPYUN Inc.\n\nOriginally started life based on \u003chttps://github.com/bakins/lua-resty-http-simple\u003e.\n\nThe part of the interface design inspired from \u003chttps://github.com/pintsized/lua-resty-http\u003e.\n\nCosocket docs and implementation borrowed from the other lua-resty-* cosocket modules.\n\n[Back to TOC](#table-of-contents)\n\n# Copyright and License\n\nThis module is licensed under the 2-clause BSD license.\n\nCopyright (c) 2015 - 2017, Monkey Zhang \u003ctimebug.info@gmail.com\u003e, UPYUN Inc.\n\nAll rights reserved.\n\nRedistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:\n\n* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.\n\n* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.\n\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n\n[Back to TOC](#table-of-contents)\n\n# See Also\n\n* the ngx_lua module: https://github.com/openresty/lua-nginx-module\n* OpenResty: https://openresty.org/\n\n[Back to TOC](#table-of-contents)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimebug%2Flua-resty-httpipe","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftimebug%2Flua-resty-httpipe","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimebug%2Flua-resty-httpipe/lists"}