{"id":22989150,"url":"https://github.com/cookpad/barbeque","last_synced_at":"2025-05-15T10:02:57.393Z","repository":{"id":10816226,"uuid":"66923637","full_name":"cookpad/barbeque","owner":"cookpad","description":"Job queue system to run job with Docker","archived":false,"fork":false,"pushed_at":"2025-01-27T07:24:19.000Z","size":1372,"stargazers_count":198,"open_issues_count":3,"forks_count":21,"subscribers_count":14,"default_branch":"master","last_synced_at":"2025-04-14T16:53:52.663Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cookpad.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"MIT-LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-08-30T08:51:55.000Z","updated_at":"2025-01-27T07:24:24.000Z","dependencies_parsed_at":"2025-03-17T01:10:43.940Z","dependency_job_id":"1f94423b-1aa9-40e1-a9bf-53dd8c29daa2","html_url":"https://github.com/cookpad/barbeque","commit_stats":{"total_commits":356,"total_committers":9,"mean_commits":39.55555555555556,"dds":0.547752808988764,"last_synced_commit":"8da0c071c2c5e766185bad9d059b69da8ab13e84"},"previous_names":[],"tags_count":51,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cookpad%2Fbarbeque","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cookpad%2Fbarbeque/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cookpad%2Fbarbeque/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cookpad%2Fbarbeque/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cookpad","download_url":"https://codeload.github.com/cookpad/barbeque/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254319716,"owners_count":22051072,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-15T04:16:36.459Z","updated_at":"2025-05-15T10:02:55.315Z","avatar_url":"https://github.com/cookpad.png","language":"Ruby","readme":"# Barbeque [![Build Status](https://github.com/cookpad/barbeque/actions/workflows/ci.yml/badge.svg?branch=master)](https://github.com/cookpad/barbeque/actions/workflows/ci.yml)\n\nJob queue system to run job with Docker\n\n\u003cimg src=\"https://raw.githubusercontent.com/cookpad/barbeque/master/doc/images/job_definitions.png\" height=\"280px\" /\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/cookpad/barbeque/master/doc/images/statistics.png\" height=\"280px\" /\u003e\n\n## Project Status\n\nBarbeque is used on production at Cookpad.\n\n## What's Barbeque?\n\nBarbeque is a job queue system that consists of:\n\n- Web console to manage jobs\n- Web API to queue a job\n- Worker to execute a job\n\nA job for Barbeque is a command you configured on web console.\nA message serialized by JSON and a job name are given to the command when performed.\nIn Barbeque worker, they are done on Docker container.\n\n## Why Barbeque?\n\n- You can achieve job-level auto scaling using tools like [Amazon ECS](https://aws.amazon.com/ecs/) [EC2 Auto Scaling group](https://aws.amazon.com/autoscaling/)\n  - For Amazon ECS, Barbeque has Hako executor\n- You don't have to manage infrastructure for each application like Resque or Sidekiq\n\nFor details, see [Scalable Job Queue System Built with Docker // Speaker Deck](https://speakerdeck.com/k0kubun/scalable-job-queue-system-built-with-docker).\n\n## Deployment\n\n### Web API \u0026 console\n\nInstall barbeque.gem to an empty Rails app and mount `Barbeque::Engine`.\nAnd deploy it as you like.\n\nYou also need to prepare MySQL, Amazon SQS and Amazon S3.\n\n#### For sandbox environment\nBarbeque's enqueue API tries to be independent of MySQL by design.\nAlthough that design policy, verifying the enqueued job is useful in some environment (such as sandboxed environment).\nPassing `BARBEQUE_VERIFY_ENQUEUED_JOBS=1` to the Web API server enables the feature that verifies the enqueued job by accessing MySQL.\n\n### Worker\n\n```bash\n$ rake barbeque:worker BARBEQUE_QUEUE=default\n```\n\nThe rake task launches four worker processes.\n\n- Two runners\n  - receives message from SQS queue, starts job execution and stores its identifier to the database\n- One execution poller\n  - gets execution status and reflect it to the database\n- One retry poller\n  - gets retried execution status and reflect it to the database\n\n## Usage\n\nWeb API documentation is available at [doc/toc.md](./doc/toc.md).\n\n### Ruby\n\n[barbeque\\_client.gem](https://github.com/cookpad/barbeque_client) has API client and ActiveJob integration.\n\n## Executor\nBarbeque executor can be customized in config/barbeque.yml. Executor is responsible for starting executions and getting status of executions.\n\nBarbeque has currently two executors.\n\n### Docker (default)\nBarbeque::Executor::Docker starts execution by `docker run --detach` and gets status by `docker inspect`.\n\n### Hako\nBarbeque::Executor::Hako starts execution by `hako oneshot --no-wait` and gets status from S3 task notification.\n\n#### Requirement\nYou must configure CloudWatch Events for putting S3 task notification.\nSee Hako's documentation for detail.\nhttps://github.com/eagletmt/hako/blob/master/docs/ecs-task-notification.md\n\n## License\nThe gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).\n","funding_links":[],"categories":["Day 1"],"sub_categories":["Scalable job queue system built with Docker"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcookpad%2Fbarbeque","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcookpad%2Fbarbeque","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcookpad%2Fbarbeque/lists"}