https://github.com/daichitakahashi/cf-eventhub

Event Hub works with Cloudflare Workers and Queues
https://github.com/daichitakahashi/cf-eventhub

cloudflare cloudflare-queues cloudflare-workers eventhub fan-out iac postgresql pulumi pulumi-typescript

Last synced: about 1 month ago
JSON representation

Event Hub works with Cloudflare Workers and Queues

• Host: GitHub
• URL: https://github.com/daichitakahashi/cf-eventhub
• Owner: daichitakahashi
• Created: 2024-09-18T13:05:11.000Z (5 months ago)
• Default Branch: main
• Last Pushed: 2024-12-09T17:09:35.000Z (2 months ago)
• Last Synced: 2024-12-09T17:35:58.123Z (2 months ago)
• Topics: cloudflare, cloudflare-queues, cloudflare-workers, eventhub, fan-out, iac, postgresql, pulumi, pulumi-typescript
• Language: TypeScript
• Homepage: https://www.npmjs.com/package/cf-eventhub
• Size: 383 KB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 5
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# cf-eventhub

Developing message hub that works with Cloudflare Workers and Queues with following functionalities.

- Configure destinations of each message in IaC(currently, we will support Pulumi)

- Record execution info including lost job(**Cloudflare Queues is Beta**)

## Designs

### Sequence

```mermaid

sequenceDiagram

  participant Worker1 as Worker(producer)

  participant eventhub as eventhub

  participant Queue as Queue

  participant DB

  participant executor as executor

  participant Worker2 as Worker(consumer)

  autonumber

  Worker1 ->> eventhub: emit() [RPC]

  activate eventhub

  eventhub ->> eventhub: routing

  eventhub ->> DB: begin

  activate DB

  eventhub ->> DB: save event payload

  opt if routes matched

    eventhub ->> DB: create dispatches of the event
for matched routes

    eventhub ->> Queue: enqueue dispatches

    activate Queue

  end

  eventhub ->> DB: commit

  deactivate DB

  eventhub -->> Worker1: return Promise

  deactivate eventhub

  Queue ->> Queue: wait delaySeconds

  par execute each dispatch

    Queue ->> executor: dequeue dispatch for matched route

    activate executor

    executor ->> DB: begin

    activate DB

    executor ->> DB: load payload
(UPDATE RETURNING)

    opt dispatch is found and not completed

      executor ->> Worker2: handle() [RPC]

      activate Worker2

      Worker2 ->> Worker2: event handling

      Worker2 -->> executor: return
Promise<"complete" | "ignored" | "failed">

      deactivate Worker2

      executor ->> DB: record execution with its status

      opt execution succeeds or max retry exceeded

        executor ->> DB: record dispatch result

      end

    end

    executor ->> DB: commit

    deactivate DB

    executor ->> Queue: ack() on "complete" | "ignored" | "misconfigured" | "notfound"
or
retry() on "failed"

    deactivate executor

    deactivate Queue

  end

```

## To run demo

1. Launch demo workers

    ```shell

    $ pnpm dev

    ```

2. Open `http://localhost:3041` in your browser