https://github.com/pyk/packagebug-dispatcher

Dispatcher dispatchs a jobs for a bunch of worker once a day
https://github.com/pyk/packagebug-dispatcher

Last synced: 10 months ago
JSON representation

Dispatcher dispatchs a jobs for a bunch of worker once a day

• Host: GitHub
• URL: https://github.com/pyk/packagebug-dispatcher
• Owner: pyk
• License: bsd-3-clause
• Created: 2015-08-29T01:20:41.000Z (almost 11 years ago)
• Default Branch: master
• Last Pushed: 2015-09-01T00:36:49.000Z (almost 11 years ago)
• Last Synced: 2025-02-07T17:23:19.723Z (over 1 year ago)
• Language: Go
• Size: 137 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# Dispatcher of Packagebug

Dispatcher dispatchs a job for a bunch of worker. Dispatcher send the job as

a message to Amazon SQS, which is where the worker receive from. Each message

contain information about the package separated by commas. That information

is used by worker to fetch a bugs from the repository of package.

There are over 20K packages in the database, sending messsage to Amazon SQS

for each package require 20K requests. It's not optimal though. So, we send

message as a batch instead of 1 request per message.

## Setup

Make sure this enviroment variable already set

    DATABASE_URL

    PACKAGEBUG_SQS_ENDPOINT

    PACKAGEBUG_SQS_REGION

    AWS_ACCESS_KEY_ID

    AWS_SECRET_ACCESS_KEY

Run

    go install

    packagebug-dispatcher