https://github.com/pior/fastjob
Fast and robust job queue using GoogleCloud PubSub ☁️
https://github.com/pior/fastjob
google-pubsub job-queue pubsub
Last synced: about 1 month ago
JSON representation
Fast and robust job queue using GoogleCloud PubSub ☁️
- Host: GitHub
- URL: https://github.com/pior/fastjob
- Owner: pior
- License: mit
- Created: 2019-04-05T02:55:19.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2019-04-08T16:02:26.000Z (about 7 years ago)
- Last Synced: 2025-01-26T08:14:48.756Z (over 1 year ago)
- Topics: google-pubsub, job-queue, pubsub
- Language: Go
- Homepage:
- Size: 32.2 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# fastjob
[](https://godoc.org/github.com/pior/fastjob)
[](https://goreportcard.com/report/github.com/pior/fastjob)
[](https://circleci.com/gh/pior/fastjob)
[](https://codecov.io/gh/pior/fastjob)
Fastjob is a fast and robust job queue using GoogleCloud PubSub 🛰
**Work In Progress**
Design objectives:
- Robustness: never lose a job.
- Reliability: never let the main queue be blocked by failing jobs.
Strategies:
- Robustness: only one external dependencies: PubSub.
- Robustness: the durability is garanteed by PubSub.
- Reliability: the core features are mostly only the PubSub semantics and features (but extensible).
- Reliability: route failing jobs to a dead letter queue.
## Usage
#### Define a job:
```golang
type PingHTTP struct{
Url string
}
func (m *PingHTTP) Name() string {
return "PingHTTP"
}
func (m *PingHTTP) Perform(ctx context.Context) error {
_, err := http.Post(m.Url)
return err
}
```
Note: the job will be JSON encoded, only **public** fields should be used to define the job inputs.
#### Register the job:
```golang
registry := fastjob.NewRegistry().WithJob(&PingHTTP{})
```
#### Run the worker:
```golang
client, err := pubsub.NewClient(ctx, "my-gcp-project-id")
sub := client.Subscription("sub-test")
config := fastjob.NewConfig(registry)
worker := fastjob.NewPubsubWorker(config, sub)
worker.Run(ctx)
```
#### Enqueue a job:
```golang
runner := fastjob.NewPubSubRunner(client, topicName)
job := &PingHTTP{Url: "http://example.org/hello"}
err = runner.Enqueue(ctx, job)
```
#### Use a local runner for testing:
```golang
runner := fastjob.NewLocalRunner()
err = runner.Enqueue(ctx, job)
```
## License
[MIT](LICENSE)