{"id":25676842,"url":"https://github.com/digital-fabric/h1p","last_synced_at":"2025-04-23T14:05:39.465Z","repository":{"id":56875641,"uuid":"398000073","full_name":"digital-fabric/h1p","owner":"digital-fabric","description":"HTTP 1.1 parser for Ruby","archived":false,"fork":false,"pushed_at":"2023-08-25T05:54:37.000Z","size":101,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-04-26T08:04:25.027Z","etag":null,"topics":["http","http1","parser","ruby"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/digital-fabric.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-08-19T16:02:57.000Z","updated_at":"2022-11-17T00:36:48.000Z","dependencies_parsed_at":"2023-02-03T20:00:24.077Z","dependency_job_id":null,"html_url":"https://github.com/digital-fabric/h1p","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/digital-fabric%2Fh1p","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/digital-fabric%2Fh1p/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/digital-fabric%2Fh1p/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/digital-fabric%2Fh1p/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/digital-fabric","download_url":"https://codeload.github.com/digital-fabric/h1p/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240496491,"owners_count":19810859,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["http","http1","parser","ruby"],"created_at":"2025-02-24T14:38:11.816Z","updated_at":"2025-02-24T14:38:12.437Z","avatar_url":"https://github.com/digital-fabric.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# H1P - HTTP/1 tools for Ruby\n\n[![Gem Version](https://badge.fury.io/rb/h1p.svg)](http://rubygems.org/gems/h1p)\n[![H1P Test](https://github.com/digital-fabric/h1p/workflows/Tests/badge.svg)](https://github.com/digital-fabric/h1p/actions?query=workflow%3ATests)\n[![MIT licensed](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/digital-fabric/h1p/blob/master/LICENSE)\n\nH1P is a blocking/synchronous HTTP/1 parser for Ruby with a simple and intuitive\nAPI. Its design lends itself to writing HTTP servers in a sequential style. As\nsuch, it might prove useful in conjunction with the new fiber scheduler\nintroduced in Ruby 3.0, but is also useful with a normal thread-based server\n(see\n[example](https://github.com/digital-fabric/h1p/blob/main/examples/http_server.rb).)\nThe H1P was originally written as part of\n[Tipi](https://github.com/digital-fabric/tipi), a web server running on top of\n[Polyphony](https://github.com/digital-fabric/polyphony).\n\nIn addition to parsing, H1P offers APIs for formatting and writing HTTP/1\nrequests and responses.\n\n## Features\n\n- Simple, blocking/synchronous API\n- Zero dependencies\n- Transport-agnostic\n- Parses both HTTP request and HTTP response\n- Support for chunked encoding\n- Support for both `LF` and `CRLF` line breaks\n- Support for **splicing** request/response bodies (when used with\n  [Polyphony](https://github.com/digital-fabric/polyphony))\n- Track total incoming traffic\n- Write HTTP requests and responses to any IO instance, with support for chunked\n  transfer encoding.\n\n## Installing\n\nIf you're using bundler just add it to your `Gemfile`:\n\n```ruby\nsource 'https://rubygems.org'\n\ngem 'h1p'\n```\n\nYou can then run `bundle install` to install it. Otherwise, just run `gem install h1p`.\n\n## Usage\n\nStart by creating an instance of `H1P::Parser`, passing a connection instance and the parsing mode:\n\n```ruby\nrequire 'h1p'\n\nparser = H1P::Parser.new(conn, :server)\n```\n\nIn order to parse HTTP responses, change the mode to `:client`:\n\n```ruby\nparser = H1P::Parser.new(conn, :client)\n```\n\nTo read the next message from the connection, call `#parse_headers`:\n\n```ruby\nloop do\n  headers = parser.parse_headers\n  break unless headers\n\n  handle_request(headers)\nend\n```\n\nThe `#parse_headers` method returns a single hash containing the different HTTP\nheaders. In case the client has closed the connection, `#parse_headers` will\nreturn `nil` (see the guard clause above).\n\nIn addition to the header keys and values, the resulting hash also contains the\nfollowing \"pseudo-headers\" (in server mode):\n\n- `:method`: the HTTP method (in upper case)\n- `:path`: the request target\n- `:protocol`: the protocol used (either `'http/1.0'` or `'http/1.1'`)\n- `:rx`: the total bytes read by the parser\n\nIn client mode, the following pseudo-headers will be present:\n\n- `:protocol`: the protocol used (either `'http/1.0'` or `'http/1.1'`)\n- `:status': the HTTP status as an integer\n- `:status_message`: the HTTP status message\n- `:rx`: the total bytes read by the parser\n\n\nThe header keys are always lower-cased. Consider the following HTTP request:\n\n```\nGET /foo HTTP/1.1\nHost: example.com\nUser-Agent: curl/7.74.0\nAccept: */*\n\n```\n\nThe request will be parsed into the following Ruby hash:\n\n```ruby\n{\n  \":method\"     =\u003e \"get\",\n  \":path\"       =\u003e \"/foo\",\n  \":protocol\"   =\u003e \"http/1.1\",\n  \"host\"        =\u003e \"example.com\",\n  \"user-agent\"  =\u003e \"curl/7.74.0\",\n  \"accept\"      =\u003e \"*/*\",\n  \":rx\"         =\u003e 78\n}\n```\n\nMultiple headers with the same key will be coalesced into a single key-value\nwhere the value is an array containing the corresponding values. For example,\nmultiple `Cookie` headers will appear in the hash as a single `\"cookie\"` entry,\ne.g. `{ \"cookie\" =\u003e ['a=1', 'b=2'] }`\n\n### Handling of invalid message\n\nWhen an invalid message is encountered, the parser will raise a `H1P::Error`\nexception. An incoming message may be considered invalid if an invalid character\nhas been encountered at any point in parsing the message, or if any of the\ntokens have an invalid length. You can consult the limits used by the parser\n[here](https://github.com/digital-fabric/h1p/blob/main/ext/h1p/limits.rb).\n\n### Reading the message body\n\nTo read the message body use `#read_body`:\n\n```ruby\n# read entire body\nbody = parser.read_body\n```\n\nThe H1P parser knows how to read both message bodies with a specified\n`Content-Length` and request bodies in chunked encoding. The method call will\nreturn when the entire body has been read. If the body is incomplete or has\ninvalid formatting, the parser will raise a `H1P::Error` exception.\n\nYou can also read a single chunk of the body by calling `#read_body_chunk`:\n\n```ruby\n# read a body chunk\nchunk = parser.read_body_chunk(false)\n\n# read chunk only from buffer:\nchunk = parser.read_body_chunk(true)\n```\n\nIf no more chunks are availble, `#read_body_chunk` will return nil. To test\nwhether the request is complete, you can call `#complete?`:\n\n```ruby\nheaders = parser.parse_headers\nunless parser.complete?\n  body = parser.read_body\nend\n```\n\nThe `#read_body` and `#read_body_chunk` methods will return `nil` if no body is\nexpected (based on the received headers).\n\n## Splicing request/response bodies\n\n\u003e Splicing of request/response bodies is available only on Linux, and works only\n\u003e with [Polyphony](https://github.com/digital-fabric/polyphony).\n\nH1P also lets you [splice](https://man7.org/linux/man-pages/man2/splice.2.html)\nrequest or response bodies directly to a pipe. This is particularly useful for\nuploading or downloading large files, as the data does not need to be loaded\ninto Ruby strings. In fact, the data will stay almost entirely in kernel\nbuffers, which means any data copying is reduced to the absolute minimum.\n\nThe following example sends a request, then splices the response body to a file:\n\n```ruby\nrequire 'polyphony'\nrequire 'h1p'\n\nsocket = TCPSocket.new('example.com', 80)\nsocket \u003c\u003c \"GET /bigfile HTTP/1.1\\r\\nHost: example.com\\r\\n\\r\\n\"\n\nparser = H1P::Parser.new(socket, :client)\nheaders = parser.parse_headers\n\npipe = Polyphony.pipe\nFile.open('bigfile', 'w+') do |f|\n  spin { parser.splice_body_to(pipe) }\n  f.splice_from(pipe)\nend\n```\n\n## Parsing from arbitrary transports\n\nThe H1P parser was built to read from any arbitrary transport or source, as long\nas they conform to one of two alternative interfaces:\n\n- An object implementing a `__read_method__` method, which returns any of\n  the following values:\n\n  - `:stock_readpartial` - to be used for instances of `IO`, `Socket`,\n    `TCPSocket`, `SSLSocket` etc.\n  - `:backend_read` - for use in Polyphony-based servers.\n  - `:backend_recv` - for use in Polyphony-based servers.\n  - `:readpartial` - for use in Polyphony-based servers.\n\n- An object implementing a `call` method, such as a `Proc` or any other. The\n  call is given a single argument signifying the maximum number of bytes to\n  read, and is expected to return either a string with the read data, or `nil`\n  if no more data is available. The callable can be passed as an argument or as\n  a block. Here's an example for parsing from a callable:\n\n  ```ruby\n  data = ['GET ', '/foo', \" HTTP/1.1\\r\\n\", \"\\r\\n\"]\n  data = ['GET ', '/foo', \" HTTP/1.1\\r\\n\", \"\\r\\n\"]\n  parser = H1P::Parser.new { data.shift }\n  parser.parse_headers\n  #=\u003e {\":method\"=\u003e\"get\", \":path\"=\u003e\"/foo\", \":protocol\"=\u003e\"http/1.1\", \":rx\"=\u003e21}\n  ```\n\n## Writing HTTP requests and responses\n\nH1P implements optimized methods for writing HTTP requests and responses to\narbitrary IO instances. To write a response with or without a body, use\n`H1P.send_response(io, headers, body = nil)`:\n\n```ruby\nH1P.send_response(socket, { 'Some-Header' =\u003e 'header value'}, 'foobar')\n# HTTP/1.1 200 OK\n# Some-Header: header value\n# \n# foobar\n\n# The :protocol pseudo header sets the protocol in the status line:\nH1P.send_response(socket, { ':protocol' =\u003e 'HTTP/0.9' })\n# HTTP/0.9 200 OK\n#\n#\n\n# The :status pseudo header sets the response status:\nH1P.send_response(socket, { ':status' =\u003e '418 I\\'m a teapot' })\n# HTTP/1.1 418 I'm a teapot\n#\n#\n```\n\nTo send responses using chunked transfer encoding use\n`H1P.send_chunked_response(io, header, body = nil)`:\n\n```ruby\nH1P.send_chunked_response(socket, {}, \"foobar\")\n# HTTP/1.1 200 OK\n# Transfer-Encoding: chunked\n# 6\n# foobar\n# 0\n#\n#\n```\n\nYou can also call `H1P.send_chunked_response` with a block that provides the\nnext chunk to send. The last chunk is signalled by returning `nil` from the\nblock:\n\n```ruby\nIO.open('/path/to/file') do |f|\n  H1P.send_chunked_response(socket, {}) { f.read(CHUNK_SIZE) }\nend\n```\n\nTo send individual chunks use `H1P.send_body_chunk`:\n\n```ruby\nH1P.send_body_chunk(socket, 'foo')\n# 3\n# foo\n#\n\nH1P.send_body_chunk(socket, nil)\n# 0\n#\n#\n```\n\n## Parser Design\n\nThe H1P parser design is based on the following principles:\n\n- Implement a blocking API for use with a sequential programming style.\n- Minimize copying of data between buffers.\n- Parse each piece of data only once.\n- Minimize object and buffer allocations.\n- Minimize the API surface area.\n\nOne of the unique aspects of H1P is that instead of the server needing to feed\ndata to the parser, the parser itself reads data from its source whenever it\nneeds more of it. If no data is yet available, the parser blocks until more data\nis received.\n\nThe different parts of the request are parsed one byte at a time, and once each\ntoken is considered complete, it is copied from the buffer into a new string, to\nbe stored in the headers hash.\n\n## Performance\n\nThe included benchmark (against\n[http_parser.rb](https://github.com/tmm1/http_parser.rb), based on the *old*\n[node.js HTTP parser](https://github.com/nodejs/http-parser)) shows the H1P\nparser to be about 10-20% slower than http_parser.rb.\n\nHowever, in a fiber-based environment such as\n[Polyphony](https://github.com/digital-fabric/polyphony), H1P is slightly\nfaster, as the overhead of dealing with pipelined requests (which will cause\n`http_parser.rb` to emit callbacks multiple times) significantly affects its\nperformance.\n\n## Roadmap\n\nHere are some of the features and enhancements planned for H1P:\n\n- Add conformance and security tests\n- Add ability to splice the message body into an arbitrary fd\n  (Polyphony-specific)\n- Improve performance\n\n## Contributing\n\nIssues and pull requests will be gladly accepted. If you have found this gem\nuseful, please let me know.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdigital-fabric%2Fh1p","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdigital-fabric%2Fh1p","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdigital-fabric%2Fh1p/lists"}