{"id":13462960,"url":"https://github.com/xing/beetle","last_synced_at":"2025-05-14T09:20:27.904Z","repository":{"id":780073,"uuid":"472219","full_name":"xing/beetle","owner":"xing","description":"High Availability AMQP Messaging With Redundant Queues","archived":false,"fork":false,"pushed_at":"2025-05-12T12:29:06.000Z","size":3008,"stargazers_count":160,"open_issues_count":3,"forks_count":22,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-05-12T12:45:06.168Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xing.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"MIT-LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2010-01-14T16:54:21.000Z","updated_at":"2025-05-07T06:15:42.000Z","dependencies_parsed_at":"2024-06-19T00:12:19.211Z","dependency_job_id":"640d5f2b-f3fe-4820-a28c-36a2803e7a83","html_url":"https://github.com/xing/beetle","commit_stats":{"total_commits":1697,"total_committers":30,"mean_commits":56.56666666666667,"dds":0.6275780789628757,"last_synced_commit":"a1ec421b092156136d39c7c38a386c52ef946b20"},"previous_names":[],"tags_count":138,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xing%2Fbeetle","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xing%2Fbeetle/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xing%2Fbeetle/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xing%2Fbeetle/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xing","download_url":"https://codeload.github.com/xing/beetle/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253744960,"owners_count":21957313,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T13:00:42.452Z","updated_at":"2025-05-14T09:20:27.840Z","avatar_url":"https://github.com/xing.png","language":"Ruby","readme":"# Beetle\n\nHigh Availability AMQP Messaging with Redundant Queues\n\n## IMPORTANT: Project Status\n\nBeetle has been developed within Xing for an extended period and made accessible to external parties. Due to organizational changes, we must implement rapid modifications to meet internal platform requirements. \nHowever, we lack the resources and expertise to maintain this library for a broader audience. The project's future is uncertain and may be discontinued as we transition away from it internally.\n\nFuture development will be merged into the master branch, with changes indicated by semantic versioning. We cannot guarantee that these changes will be beneficial or applicable to a wider audience.\n\nA [v3.x branch](https://github.com/xing/beetle/tree/v3.x) is available, containing all updates prior to version 4.0.0. This branch will not be maintained long-term but is available for stable reference.\n\nWe will not publish new releases on rubygems.org; please reference this repository for versions beyond 3.x.\n\nIn summary:\n\n* The project's future is uncertain and may be discontinued\n* The master branch will undergo frequent changes\n* New releases will not be published on rubygems.org\n* The [v3.x branch](https://github.com/xing/beetle/tree/v3.x) is available for reference but will not be maintained\n\n## About\n\nBeetle grew out of a project to improve an existing ActiveMQ based messaging\ninfrastructure. It offers the following features:\n\n* High Availability (by using multiple message broker instances)\n* Redundancy (by replicating queues)\n* Simple client API (by encapsulating the publishing/ deduplication logic)\n\n\n## Release notes\n\nSee [RELEASE_NOTES.md](./RELEASE_NOTES.md)\n\n## Usage\n\n### Configuration\n\n    # configure machines\n\n    Beetle.config do |config|\n      config.servers = \"broker1:5672, broker2:5672\"\n      config.redis_server = \"redis1:6379\"\n    end\n\n    # instantiate a beetle client\n\n    b = Beetle::Client.new\n\n    # configure exchanges, queues, bindings, messages and handlers\n\n    b.configure do\n      queue :test\n      message :test\n      handler(:test) { |message| puts message.data }\n    end\n\n### Publishing\n\n    b.publish :test, \"I'm a test message\"\n\n### Subscribing\n\n    b.listen_queues\n\n### Examples\n\nBeetle ships with a number of [example scripts](http://github.com/xing/beetle/tree/master/examples/).\n\nThe top level Rakefile comes with targets to start several RabbitMQ and redis instances\nlocally. Make sure the corresponding binaries are in your search path. Open four new shell\nwindows and execute the following commands:\n\n    rake rabbit:start1\n    rake rabbit:start2\n    rake redis:start:master\n    rake redis:start:slave\n\n## Prerequisites\n\nTo set up a redundant messaging system you will need\n* at least 2 AMQP servers (we use [RabbitMQ](http://www.rabbitmq.com/))\n* at least one [Redis](http://github.com/antirez/redis) server (better are two in a\n  master/slave setup, see [REDIS_AUTO_FAILOVER.md](./REDIS_AUTO_FAILOVER.md))\n\n## Test environment\n\nFor testing purposes, you will need a MySQL database with the database\n`beetle_test` created. This is needed to test special cases in which\nBeetle handles the connection with ActiveRecord:\n\n     mysql -e 'create database beetle_test;'\n\nYou also need a Redis instance running. The default configuration of Redis will work:\n\n    redis-server\n\nIf you want to run the integration tests you need GO installed and you\nwill need to build the beetle binary. We provide a Makefile for this\npurpose, so simply running\n\n    make\n\nshould suffice.\n\n## Gem Dependencies\n\nAt runtime, Beetle will use\n* [bunny](http://github.com/ruby-amqp/bunny)\n* [redis](http://github.com/redis/redis-rb)\n* [amqp](http://github.com/ruby-amqp/amqp)\n  (which is based on [eventmachine](http://github.com/eventmachine/eventmachine))\n* [daemons](http://daemons.rubyforge.org/)\n* [activesupport](https://github.com/rails/rails/tree/master/activesupport)\n\nFor development, you'll need\n* [mocha](http://github.com/floehopper/mocha)\n* [cucumber](http://github.com/aslakhellesoy/cucumber)\n* [daemon_controller](http://github.com/FooBarWidget/daemon_controller)\n* [consul](https://www.consul.io/downloads.html)\n\nFor tests, you'll need\n* [activerecord](https://github.com/rails/rails/tree/master/activerecord)\n* [mysql2](https://github.com/brianmario/mysql2/)\n\nDependencies are managed by `bundler`.\n\nIf you want to use a `redis-rb` version after 5.0.0, you must add the\n`hiredis-client` gem to your application.\n\n\n## Authors\n\n[Stefan Kaes](http://github.com/skaes),\n[Pascal Friederich](http://github.com/paukul),\n[Ali Jelveh](http://github.com/dudemeister),\n[Bjoern Rochel](http://github.com/bjro) and\n[Sebastian Roebke](http://github.com/boosty).\n\nYou can find out more about our work on our [dev blog](http://devblog.xing.com).\n\nCopyright (c) 2010-2019 [XING AG](http://www.xing.com/)\n\nReleased under the MIT license. For full details see MIT-LICENSE included in this\ndistribution.\n\n## Contributing\n\n1. Fork it\n2. Create your feature branch (`git checkout -b my-new-feature`)\n3. Hack along and test your code.\n4. Commit your changes (`git commit -am 'Add some feature'`)\n5. Push to the branch (`git push origin my-new-feature`)\n6. Create new Pull Request\n\nDon't increase the gem version in your pull requests. It will be done after merging the request,\nto allow merging of pull requests in a flexible order.\n\n## Compiling beetle and running tests\n\nIn order to execute the unit tests, you need Ruby, a running rabbitmq server, a running\nredis-server, a running mysql server and a runnning consul server.\n\nIn addition, beetle ships with a cucumber feature to test the automatic redis failover as\nan integration test. For this you need a recent Go installation in order to compile the\nbeetle go binary. Just invoke `make` in the top level directory.\n\nThere are two ways to start the required test dependencies: using `docker compose` or\nstarting the services manually.\n\n### Testing with docker compose\n\nOpen a separate terminal window and run\n\n     docker compose pull\n\nfollowed by\n\n     docker compose up\n\nThis will start mysql, two redis servers, two RabbitMQ instances and a single consul\ndevelopment node.\n\nNote: make sure to wait until all services are properly started.\n\n\n## Testing with  locally installed services\n\nThe top level Rakefile comes with targets to start several RabbitMQ instances locally.\nMake sure the corresponding binaries are in your search path. Open three shell windows and\nexecute the following command:\n\n    rake rabbit:start1\n\nand\n\n   rake redis:start:master\n\nas well as\n\n   rake consul:start\n\nThen you can run the cucumber feature by running:\n\n    cucumber\n\nor\n\n    rake cucumber\n\nNote: Cucumber will automatically run after the unit tests when you run `rake` without\narguments.\n\n\n## How to release a new gem version\n\nUpdate [RELEASE_NOTES.md](./RELEASE_NOTES.md)!\n\nWe use [semantic versioning](http://semver.org/) and create a git tag\nfor each release.\n\nEdit `lib/beetle/version.rb` and\n`go/src/github.com/xing/beetle/version.go` to set the new version\nnumber (`Major.Minor.Patch`).\n\nIn short (see [semver.org](http://semver.org) for details):\n\n* *Major* version MUST be incremented if any backwards incompatible changes\n  are introduced to the public API.\n* *Minor* version MUST be incremented if new, backwards compatible functionality\n  is introduced to the public API. It MUST be incremented if any public API\n  functionality is marked as deprecated.\n* *Patch* version MUST be incremented if only backwards compatible bug fixes\n  are introduced.\n\nThen use `rake release` which will create the git tag and upload the\ngem to github.com:\n\n    bundle exec rake release\n\nThe generated gem is located in the `pkg/` directory.\n\nIn order to build go binaries and upload the docker container with the\nbeetle GO binary to docker hub, run\n\n    make release\n\nThis will upload the go binaries to https://github.com/xing/beetle/\nand push the beetle container to\nhttps://hub.docker.com/r/xingarchitects/gobeetle/.\n\nRun\n\n    make tag push TAG=X.X.X\n\nto tag and push the container with a specific version number.\n","funding_links":[],"categories":["Background Processing","Ruby"],"sub_categories":["Background Jobs"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxing%2Fbeetle","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxing%2Fbeetle","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxing%2Fbeetle/lists"}