{"id":17228648,"url":"https://github.com/njonsson/structured_io","last_synced_at":"2025-10-12T08:43:00.759Z","repository":{"id":48308402,"uuid":"115075617","full_name":"njonsson/structured_io","owner":"njonsson","description":"An Elixir API for consuming structured input streams","archived":false,"fork":false,"pushed_at":"2021-08-02T11:15:17.000Z","size":176,"stargazers_count":11,"open_issues_count":12,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-14T01:35:59.387Z","etag":null,"topics":["binary","data","elixir","io","parser","stream"],"latest_commit_sha":null,"homepage":"https://njonsson.github.io/structured_io","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/njonsson.png","metadata":{"files":{"readme":"README.md","changelog":"History.md","contributing":null,"funding":null,"license":"License.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-12-22T04:12:20.000Z","updated_at":"2024-04-14T04:46:26.000Z","dependencies_parsed_at":"2022-09-21T07:50:23.718Z","dependency_job_id":null,"html_url":"https://github.com/njonsson/structured_io","commit_stats":null,"previous_names":[],"tags_count":20,"template":false,"template_full_name":null,"purl":"pkg:github/njonsson/structured_io","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/njonsson%2Fstructured_io","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/njonsson%2Fstructured_io/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/njonsson%2Fstructured_io/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/njonsson%2Fstructured_io/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/njonsson","download_url":"https://codeload.github.com/njonsson/structured_io/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/njonsson%2Fstructured_io/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279010788,"owners_count":26084807,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-12T02:00:06.719Z","response_time":53,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["binary","data","elixir","io","parser","stream"],"created_at":"2024-10-15T04:44:41.379Z","updated_at":"2025-10-12T08:43:00.742Z","avatar_url":"https://github.com/njonsson.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# StructuredIO\n\n[\u003cimg alt=\"Travis CI build status\" src=\"https://secure.travis-ci.org/njonsson/structured_io.svg?branch=master\" /\u003e][Travis-CI-build-status]\n[\u003cimg alt=\"Coveralls test coverage status\" src=\"https://coveralls.io/repos/njonsson/structured_io/badge.svg?branch=master\" /\u003e][Coveralls-test-coverage-status]\n[\u003cimg alt=\"Hex release\" src=\"https://img.shields.io/hexpm/v/structured_io.svg\" /\u003e][Hex-release]\n\n*StructuredIO* resembles Elixir’s [*IO*][HexDocs-Elixir-IO] module. The\ndifference is that whereas *IO* exposes a freeform stream of bytes or lines of\ndata, *StructuredIO* guarantees that only complete data elements are returned\nfrom its reader functions. This simplifies your application logic with respect\nto fragmentary input.\n\nThere are two main features of this library:\n\n1. **It provides a stateful process with a writer function for writing binary\n   data.** (No big deal since *IO* gives you that.)\n2. **It provides a variety of reader functions for conditional reading according\n   to a specified data structure.** Virtually any wire format can be specified,\n   including binary encodings, nested and flat markup, and delimited data. If a\n   complete data element has not (yet) been written to the process, nothing is\n   read.\n\n**See what’s changed lately by reading\n[the project history][GitHub-project-history].**\n\n## Usage\n\nHere’s a contrived example that shows how to write and read structured data\nusing the `StructuredIO.write` and `.read_*` functions. This example depicts\nUnicode data, but binary data of any kind can be written and read, too. See\n[the API reference][HexDocs-project-API-reference] for detailed examples.\n\n```elixir\niex\u003e {:ok,\n...\u003e  structured_io} = StructuredIO.start_link(:unicode)\n```\n\nNow we have a running *StructuredIO* process that expects properly encoded\nUnicode data.\n\n```elixir\niex\u003e StructuredIO.write structured_io,\n...\u003e                    \"  \u003cp\u003efoo\u003c/p\"\n:ok\n```\n\nWe’ve written some markup to the process. Note that the `\u003cp\u003e` element is\npreceded by whitespace and is not properly closed.\n\n```elixir\niex\u003e StructuredIO.read_across structured_io,\n...\u003e                          \"\u003cp\u003e\",\n...\u003e                          \"\u003c/p\u003e\"\n\"\"\n```\n\nNo `\u003cp\u003e` element is read because the available data in the process doesn’t begin\nwith a `\u003cp\u003e`.\n\n```elixir\niex\u003e StructuredIO.read_to structured_io,\n...\u003e                      \"\u003cp\u003e\"\n\"  \"\niex\u003e StructuredIO.read_across structured_io,\n...\u003e                          \"\u003cp\u003e\",\n...\u003e                          \"\u003c/p\u003e\"\n\"\"\n```\n\nWe managed to get past the whitespace, but no `\u003cp\u003e` element is read because the\navailable data in the process doesn’t contain a complete element.\n\n```elixir\niex\u003e StructuredIO.write structured_io,\n...\u003e                    \"\u003e\u003chr /\u003e\u003cp\u003ebar\u003c/p\u003e\"\n:ok\n```\n\nNow the first element is properly closed, and a second complete element has been\nwritten to the process.\n\n```elixir\niex\u003e StructuredIO.read_across structured_io,\n...\u003e                          \"\u003cp\u003e\",\n...\u003e                          \"\u003c/p\u003e\"\n\"\u003cp\u003efoo\u003c/p\u003e\"\niex\u003e StructuredIO.read_through structured_io,\n...\u003e                           \"\u003chr /\u003e\"\n\"\u003chr /\u003e\"\niex\u003e StructuredIO.read_between structured_io,\n...\u003e                           \"\u003cp\u003e\",\n...\u003e                           \"\u003c/p\u003e\"\n\"bar\"\n```\n\nWe’ve read one element at a time from the available data in the process. The\nread operations demonstrate both seeking and skipping.\n\n```elixir\niex\u003e StructuredIO.read_across structured_io,\n...\u003e                          \"\u003cp\u003e\",\n...\u003e                          \"\u003c/p\u003e\"\n\"\"\n```\n\nNo more elements can be read unless more data is written to the process.\n\n```elixir\niex\u003e collector = StructuredIO.collect(structured_io)\niex\u003e [\"\u003cp\u003ebaz\u003c/p\u003e\",\n...\u003e  \"\u003cp\u003equx\u003c/p\u003e\",\n...\u003e  \"\u003cp\u003equux\u003c/p\u003e\"]\n...\u003e |\u003e Enum.into(collector)\niex\u003e structured_io\n...\u003e |\u003e StructuredIO.enumerate_with(:read_between,\n...\u003e                                \"\u003cp\u003e\",\n...\u003e                                \"\u003c/p\u003e\")\n...\u003e |\u003e Enum.map(\u0026String.upcase/1)\n[\"BAZ\",\n \"QUX\",\n \"QUUX\"]\n```\n\nThe `StructuredIO.collect` function returns a struct that implements Elixir’s\n[*Collectable*][HexDocs-Elixir-Collectable] protocol, which lets you **pipe data\ninto the process** instead of performing individual write operations. Likewise,\nthe `StructuredIO.enumerate_with` function returns a struct that implements\nElixir’s [*Enumerable*][HexDocs-Elixir-Enumerable] protocol, which lets you\n**pipe data elements out of the process** instead of performing individual read\noperations.\n\n```elixir\niex\u003e StructuredIO.stop structured_io\n:ok\n```\n\nDon’t forget to stop the process when you’re finished with it.\n\nYou’ll find more detailed examples in\n[the documentation][HexDocs-project-API-reference] for the *StructuredIO*\nmodule.\n\n## Installation\n\nInstall [the Hex package][Hex-release] by adding `:structured_io` to the list of\ndependencies in your project’s *mix.exs* file:\n\n```elixir\n# mix.exs\n\n# ...\ndef deps do\n  [\n    {:structured_io, \"~\u003e 1.5.0\"}\n  ]\nend\n# ...\n```\n\n## Contributing\n\nTo submit a patch to the project:\n\n1. [Fork][GitHub-fork-project] the official repository.\n2. Create your feature branch: `git checkout -b my-new-feature`.\n3. Commit your changes: `git commit -am 'Add some feature'`.\n4. Push to the branch: `git push origin my-new-feature`.\n5. [Create][GitHub-compare-project-branches] a new pull request.\n\nAfter cloning the repository, `mix deps.get` to install dependencies. Then\n`mix test` to run the tests. You can also `iex -S mix` to get an interactive\nprompt that will allow you to experiment. To build this package,\n`mix hex.build`.\n\nTo release a new version:\n\n1. Update [the project history in *History.md*][GitHub-project-history], and\n   then commit.\n2. Update the version number in [*mix.exs*][GitHub-mix-dot-exs-file] respecting\n   [Semantic Versioning][Semantic-Versioning], update\n   [the “Installation” section](#installation) of\n   [this readme][GitHub-readme-dot-md-file] to reference the new version, and\n   then commit.\n3. Build and publish [the Hex package][Hex-release] with `mix hex.publish`.\n4. Tag with a name like `vMAJOR.MINOR.PATCH` corresponding to the new version,\n   and then push commits and tags.\n\n## License\n\nReleased under the [MIT License][GitHub-project-MIT-License].\n\n[Travis-CI-build-status]:          https://www.travis-ci.org/njonsson/structured_io                             \"Travis CI build status for ‘StructuredIO’\"\n[Coveralls-test-coverage-status]:  https://coveralls.io/r/njonsson/structured_io?branch=master                  \"Coveralls test coverage status for ‘StructuredIO’\"\n[Hex-release]:                     https://hex.pm/packages/structured_io                                        \"Hex release of ‘StructuredIO’\"\n[HexDocs-Elixir-IO]:               https://hexdocs.pm/elixir/IO.html                                            \"Elixir’s ‘IO’ module at HexDocs\"\n[HexDocs-Elixir-Collectable]:      https://hexdocs.pm/elixir/Collectable.html                                   \"Elixir’s ‘Collectable’ protocol at HexDocs\"\n[HexDocs-Elixir-Enumerable]:       https://hexdocs.pm/elixir/Enumerable.html                                    \"Elixir’s ‘Enumerable’ protocol at HexDocs\"\n[HexDocs-project-API-reference]:   https://hexdocs.pm/structured_io/api-reference.html                          \"‘StructuredIO’ API reference at HexDocs\"\n[GitHub-project-history]:          https://github.com/njonsson/structured_io/blob/master/History.md             \"‘StructuredIO’ project history at GitHub\"\n[GitHub-fork-project]:             https://github.com/njonsson/structured_io/fork                               \"Fork the official repository of ‘StructuredIO’\"\n[GitHub-compare-project-branches]: https://github.com/njonsson/structured_io/compare                            \"Compare branches of ‘StructuredIO’ repositories\"\n[GitHub-mix-dot-exs-file]:         https://github.com/njonsson/structured_io/blob/master/mix.exs                \"‘StructuredIO’ project ‘mix.exs’ file at GitHub\"\n[Semantic-Versioning]:             https://semver.org/\n[GitHub-readme-dot-md-file]:       https://github.com/njonsson/structured_io/blob/master/README.md#installation \"‘StructuredIO’ project ‘README.md’ file at GitHub\"\n[GitHub-project-MIT-License]:      https://github.com/njonsson/structured_io/blob/master/License.md             \"MIT License claim for ‘StructuredIO’ at GitHub\"\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnjonsson%2Fstructured_io","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnjonsson%2Fstructured_io","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnjonsson%2Fstructured_io/lists"}