{"id":24905766,"url":"https://github.com/ckampfe/httpipe","last_synced_at":"2025-03-27T21:45:56.197Z","repository":{"id":275230520,"uuid":"925482004","full_name":"ckampfe/httpipe","owner":"ckampfe","description":"super easy ad-hoc inter-process communication via HTTP","archived":false,"fork":false,"pushed_at":"2025-02-24T00:24:03.000Z","size":149,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-24T01:24:57.151Z","etag":null,"topics":["http","networking","patchbay","proxy","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ckampfe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-01T01:07:58.000Z","updated_at":"2025-02-24T00:24:07.000Z","dependencies_parsed_at":null,"dependency_job_id":"e0cd6ea2-bb26-4354-803e-7efd3e7b6318","html_url":"https://github.com/ckampfe/httpipe","commit_stats":null,"previous_names":["ckampfe/httpipe"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ckampfe%2Fhttpipe","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ckampfe%2Fhttpipe/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ckampfe%2Fhttpipe/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ckampfe%2Fhttpipe/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ckampfe","download_url":"https://codeload.github.com/ckampfe/httpipe/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245931756,"owners_count":20695958,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["http","networking","patchbay","proxy","rust"],"created_at":"2025-02-02T00:24:46.697Z","updated_at":"2025-03-27T21:45:56.169Z","avatar_url":"https://github.com/ckampfe.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# httpipe\n\n[![Rust](https://github.com/ckampfe/httpipe/actions/workflows/rust.yml/badge.svg)](https://github.com/ckampfe/httpipe/actions/workflows/rust.yml)\n\n\n`httpipe` is an HTTP server that allows for HTTP clients to forward data to each other in a queue-like way using named channels.\n\n\n## What is this \n\nIn `httpipe`, channels are named, logical queues for conveying data from one HTTP request to another, so `httpipe` is like its name describes: a way to pipe data from one HTTP client to another, via a central server.\n\nA producer enqueues data to send by sending a `POST` request to a channel.\nA consumer receives that data by sending a `GET` request to the same channel.\n\nFor example:\n\n```sh\n# in one terminal, a producer client enqueues data...\ncurl -XPOST /channels/v1/do_work -d\"hello\"\n```\n\n```sh\n# ...and in another terminal, a consumer client receives it\ncurl -XGET /channels/v1/do_work\nhello%\n```\n\nThe data was piped from the first request (the producer) to the second (the consumer).\n\n\n## installation\n\n```\n$ cargo install --git https://github.com/ckampfe/httpipe.git\n```\n\nThis downloads and builds `httpipe` from source from this repo, assuming you have [Rust installed](https://www.rust-lang.org/tools/install).\n\n\n## Details\n\n### Naming\n\nChannels are uniquely identified by their namespace and name, both of which are required.\n\nThis is the URL scheme: `/channels/{namespace}/{channel_name}`.\n\nFor example, the channel `/channels/v1/do_work` has the namespace `v1` and the name `do_work`.\n\n- There can be arbitrarily many namespaces.\n- Namespaces are globally unique (the namespace `foo` always refers to the same namespace).\n- Namespaces can have many arbitrarily many channels.\n- Channel names are unique per namespace, meaning `/channels/v1/foo` is a different channel than `/channels/v2/foo`, even though both have the same channel name.\n\nBoth namespaces and channel names are automatically created on first use and cannot be manually created out of band.\n\nNamespaces and channels can be destroyed manually by deleting either a specific channel (`DELETE /channels/{namespace}/{channel_name}`), or by deleting the namespace and all of its associated channels (`DELETE /channels/{namespace}`).\n\n\n### Concurrency and order\n\nThe concurrency of a given channel is 1.\n\nChannels are \"rendezvous\" queues, in that both producer and consumer requests to a given channel will block until there is a counterpart on the other side of the channel to either receive or produce data, respectively. That is, a producer will block until there is a consumer, and a consumer will block until there is a producer.\n\nThere can be arbitrarily many producer and consumer requests currently pending against an instance of `httpipe`, but only one producer-consumer pair at a time for a given channel can be exchanging data.\n\nProducers and consumers are matched in the order in which they made their requests.\nYou can think of this logically as there being a \"producer queue\" and a \"consumer queue\".\n\nFor example, if we make 4 consecutive producer requests to `/channels/v1/do_work` (call them `p1`, `p2`, `p3`, and `p4`), they would all block until a consumer connects for each of them, because we have enqueued 4 producer requests into the \"producer queue\".\n\nIf we then make a single consumer request to `/channels/v1/do_work` (call it `c1`), `httpipe` will match `p1` with `c1` by popping `p1` off the \"producer queue\" and popping `c1` off the \"consumer queue\", sending `p1`'s data to `c1`. Producer requests `p2`, `p3`, and `p4` would still remain blocking in the \"producer queue\", waiting for consumers to connect.\n\n(Note that the \"producer queue\" and \"consumer queue\" terms are just metaphors. `httpipe` implements this behavior in a slightly different way, but this \"queue\" logic still applies.)\n\n\n### Use cases\n\nIt turns out this functionality, limited as it is, is enough to do a lot of useful stuff.\nYou can use `httpipe` to send notifications, build a concurrent job queue, share files, chat, and all kinds of other stuff. See the [archived patchbay site](https://web.archive.org/web/20241105063704/https://patchbay.pub/) for other ideas.\n\n\n### Options\n\n```\nUsage: httpipe [OPTIONS]\n\nOptions:\n  -p, --port \u003cPORT\u003e\n          the port to bind the server to [env: PORT=] [default: 3000]\n  -r, --request-timeout \u003cREQUEST_TIMEOUT\u003e\n          the maximum request timeout, in seconds [env: REQUEST_TIMEOUT=]\n  -h, --help\n          Print help\n```\n\n\n### Credit\n\nThis is not my idea! I first discovered this idea a few years ago as [patchbay](https://web.archive.org/web/20241105063704/https://patchbay.pub/) by [Anders Pitman](https://github.com/anderspitman). Recently I remembered patchbay, and found out that it is apparently no longer available, so I decided to try to build my own based on what I could read on the archive of the patchbay site.\n\nNote that this project is an entirely new work and does not draw from or use the original patchbay code in any way, so any flaws in it are mine alone.\n\n### Todo\n\nThis originally had patchbay's additional pubsub functionality but I removed it while I work on the design. I may add it back at some point, or not.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fckampfe%2Fhttpipe","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fckampfe%2Fhttpipe","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fckampfe%2Fhttpipe/lists"}