https://github.com/pior/fastjob

Fast and robust job queue using GoogleCloud PubSub ☁️
https://github.com/pior/fastjob

google-pubsub job-queue pubsub

Last synced: 1 day ago
JSON representation

Fast and robust job queue using GoogleCloud PubSub ☁️

• Host: GitHub
• URL: https://github.com/pior/fastjob
• Owner: pior
• License: mit
• Created: 2019-04-05T02:55:19.000Z (almost 6 years ago)
• Default Branch: master
• Last Pushed: 2019-04-08T16:02:26.000Z (almost 6 years ago)
• Last Synced: 2024-12-04T18:41:01.399Z (about 2 months ago)
• Topics: google-pubsub, job-queue, pubsub
• Language: Go
• Homepage:
• Size: 32.2 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# fastjob

[![GoDoc](https://godoc.org/github.com/pior/fastjob?status.svg)](https://godoc.org/github.com/pior/fastjob)

[![Go Report Card](https://goreportcard.com/badge/github.com/pior/fastjob)](https://goreportcard.com/report/github.com/pior/fastjob)

[![CircleCI](https://circleci.com/gh/pior/fastjob.svg?style=svg)](https://circleci.com/gh/pior/fastjob)

[![codecov](https://codecov.io/gh/pior/fastjob/branch/master/graph/badge.svg)](https://codecov.io/gh/pior/fastjob)

Fastjob is a fast and robust job queue using GoogleCloud PubSub 🛰

**Work In Progress**

Design objectives:

- Robustness: never lose a job.

- Reliability: never let the main queue be blocked by failing jobs.

Strategies:

- Robustness: only one external dependencies: PubSub.

- Robustness: the durability is garanteed by PubSub.

- Reliability: the core features are mostly only the PubSub semantics and features (but extensible).

- Reliability: route failing jobs to a dead letter queue.

## Usage

#### Define a job:

```golang

type PingHTTP struct{

    Url string

}

func (m *PingHTTP) Name() string {

	return "PingHTTP"

}

func (m *PingHTTP) Perform(ctx context.Context) error {

    _, err := http.Post(m.Url)

	return err

}

```

Note: the job will be JSON encoded, only **public** fields should be used to define the job inputs.

#### Register the job:

```golang

registry := fastjob.NewRegistry().WithJob(&PingHTTP{})

```

#### Run the worker:

```golang

client, err := pubsub.NewClient(ctx, "my-gcp-project-id")

sub := client.Subscription("sub-test")

config := fastjob.NewConfig(registry)

worker := fastjob.NewPubsubWorker(config, sub)

worker.Run(ctx)

```

#### Enqueue a job:

```golang

runner := fastjob.NewPubSubRunner(client, topicName)

job := &PingHTTP{Url: "http://example.org/hello"}

err = runner.Enqueue(ctx, job)

```

#### Use a local runner for testing:

```golang

runner := fastjob.NewLocalRunner()

err = runner.Enqueue(ctx, job)

```

## License

[MIT](LICENSE)