{"id":24117195,"url":"https://github.com/fluvio-community/http-sink-connector","last_synced_at":"2026-04-26T12:02:36.510Z","repository":{"id":156487444,"uuid":"626577797","full_name":"infinyon/http-sink-connector","owner":"infinyon","description":"Official Infinyon HTTP Sink Connector","archived":false,"fork":false,"pushed_at":"2025-01-06T02:22:11.000Z","size":385,"stargazers_count":9,"open_issues_count":10,"forks_count":2,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-01-06T11:14:03.932Z","etag":null,"topics":["connector","fluvio","http","sink"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/infinyon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-11T18:44:39.000Z","updated_at":"2024-10-30T15:29:39.000Z","dependencies_parsed_at":null,"dependency_job_id":"b4763d22-a1f0-4270-be55-ad0b92540fc3","html_url":"https://github.com/infinyon/http-sink-connector","commit_stats":null,"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/infinyon%2Fhttp-sink-connector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/infinyon%2Fhttp-sink-connector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/infinyon%2Fhttp-sink-connector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/infinyon%2Fhttp-sink-connector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/infinyon","download_url":"https://codeload.github.com/infinyon/http-sink-connector/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":233456087,"owners_count":18678963,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["connector","fluvio","http","sink"],"created_at":"2025-01-11T07:38:31.334Z","updated_at":"2026-04-26T12:02:36.480Z","avatar_url":"https://github.com/infinyon.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Fluvio HTTP Outbound Connector\n\nOfficial Infinyon HTTP Sink connector\n\n## Sink Connector\n\nHTTP sink connector reads records from data streaming and generates an HTTP request.\n\n\u003e Supports HTTP/1.0, HTTP/1.1, HTTP/2.0 protocols.\n\n### Configuration\n\nHTTP Sink is configured using a YAML file:\n\n```yaml\n# config-example.yaml\napiVersion: 0.1.0\nmeta:\n  version: 0.3.0\n  name: my-http-sink\n  type: http-sink\n  topic: http-sink-topic\n  secrets:\n    - name: HTTP_TOKEN\nhttp:\n  endpoint: \"http://127.0.0.1/post\"\n  headers:\n    - \"Authorization: token ${{ secrets.HTTP_TOKEN }}\"\n    - \"Cache-Control: no-cache\"\n```\n\n| Option               | default                    | type            | description                                       |\n| :--------------------| :--------------------------| :-------------- | :-------------------------------------------------|\n| method               | POST                       | String          | POST, PUT                                         |\n| endpoint             | -                          | String          | HTTP URL endpoint                                 |\n| headers              | -                          | Array\\\u003cString\\\u003e | Request header(s) \"Key:Value\" pairs               |\n| user-agent           | `fluvio/http-sink 0.3.0`   | String          | Request user-agent                                |\n| http_request_timeout | 1s                         | String          | HTTP Request Timeout                              |\n| http_connect_timeout | 15s                        | String          | HTTP Connect Timeout                              |\n\n\u003e By default HTTP headers will use `Content-Type: text/html` unless anothed value\n\u003e is provided to the Headers configuration.\n\n### Usage\n\nLogin to your Fluvio Cloud Account via Fluvio CLI\n\n```bash\nfluvio cloud login --use-oauth2\n```\n\nThen configure the HTTP Request to be sent using a YAML file following the\nconnector configuration schema. The following configuration will send a POST\nHTTP request to `http://httpbin.org/post`.\n\n```yaml\n# config.yaml\napiVersion: 0.1.0\nmeta:\n  version: 0.3.0\n  name: httpbin\n  type: http-sink\n  topic: httpbin-send-post\n\nhttp:\n  endpoint: http://httpbin.org/post\n  interval: 3s\n```\n\nFinally create your connector by running:\n\n```bash\nfluvio cloud connector create --config ./config.yaml\n```\n\n\u003e You can see active connectors by running the following command:\n\u003e\n\u003e ```bash\n\u003e fluvio cloud connector list\n\u003e ```\n\nCheck connector logs by running\n\n```bash\nfluvio cloud connector logs httpbin\n```\n\n```log\nINFO connect:connect_with_config:connect: fluvio_socket::versioned: connect to socket add=fluvio-sc-public:9003\nINFO dispatcher_loop{self=MultiplexDisp(10)}: fluvio_socket::multiplexing: multiplexer terminated\n2023-05-02T20:59:50.192104Z  INFO stream_with_config:inner_stream_batches_with_config:request_stream{offset=Offset { inner: FromEnd(0) }}:create_serial_socket:create_serial_socket_from_leader{leader_id=0}:connect_to_leader{leader=0}:connect: fluvio_socket::versioned: connect to socket add=fluvio-spu-main-0.acct-584fd564-1d4a-4308-9061-09acea387bea.svc.cluster.local:9005\nINFO fluvio_connector_common::monitoring: using metric path: /fluvio_metrics/connector.sock\nINFO fluvio_connector_common::monitoring: monitoring started\n```\n\n### Produce Records to send as HTTP POST Requests\n\nYou can produce records using `fluvio produce \u003cTOPIC\u003e`, values produced will\nbe sent as HTTP Body payloads on HTTP Sink Connector.\n\nRunning the following command will attach stdin to the topic stream, any data\nwritten to stdin will be sent as a record through the `httpbin-send-post` topic,\nand as a side effect of the HTTP Sink Connector, these records will also be sent\nas HTTP POST requests to http://httpbin.org/post, based on our configuration.\n\n```bash\nfluvio produce httpbin-send-post\n```\n\nThen send data:\n\n```log\n\u003e {\\\"hello\\\": \\\"world\\\"}\nOk!\n```\n\n### Teardown\n\nTo stop your connector just use `fluvio cloud connector delete \u003cNAME\u003e`\n\n```bash\nfluvio cloud connector delete httpbin\n```\n\n\u003e `httpbin` is our connector instance name from the configuration file shown above\n\n### Transformations\n\nFluvio HTTP Sink Connector supports [Transformations](https://www.fluvio.io/docs/concepts/transformations-chain/). Records can be modified before sending to endpoint.\n\nThe previous example can be extended to add extra transformations to outgoing records:\n```yaml\n# config-example.yaml\napiVersion: 0.1.0\nmeta:\n  version: 0.3.0\n  name: my-http-sink\n  type: http-sink\n  topic: http-sink-topic\n  secrets:\n    - name: AUTHORIZATION_TOKEN\nhttp:\n  endpoint: \"http://127.0.0.1/post\"\n  headers:\n    - \"Authorization: token ${{ secrets.AUTHORIZATION_TOKEN }}\"\n    - \"Content-Type: application/json\"\ntransforms:\n  - uses: infinyon/jolt@0.1.0\n    with:\n      spec:\n        - operation: shift\n          spec:\n            \"result\": \"text\"\n```\n\nIn this case, additional transformation will be performed before records are sent the http endpoint. A json field called `result` will be renamed to `text`.\n\n\nRead more about [JSON to JSON transformations](https://www.fluvio.io/smartmodules/certified/jolt/).\n\n### Offset Management\n\nFluvio Consumer Offset feature allows for a connector to store the offset in the Fluvio cluster and use it on restart.\nTo activate it, you need to provide the `consumer` name and set the `strategy: auto`.\nSee the example below:\n\n```yaml\napiVersion: 0.1.0\nmeta:\n  version: 0.3.0\n  name: my-http-sink\n  type: http-sink\n  topic: http-sink-topic\n  consumer:\n    id: my-http-sink\n    offset:\n      strategy: auto\n      start: beginning\n      flush-period:\n        secs: 10\n        nanos: 0\nhttp:\n  endpoint: \"http://127.0.0.1/post\"\n```\n\nAfter the connector processed any records, you can check the last stored offset value via:\n```bash\n$ fluvio consumer list\n  CONSUMER      TOPIC            PARTITION  OFFSET  LAST SEEN\n  my-http-sink  http-sink-topic  0          0       3s\n```\n\n## Contributing\n\nFollow on the conventional `CONTRIBUTING.md` file to setup your environment and\ncontribute to this project.\n\n[1]: https://www.fluvio.io/connectors/outbound/http/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffluvio-community%2Fhttp-sink-connector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffluvio-community%2Fhttp-sink-connector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffluvio-community%2Fhttp-sink-connector/lists"}