{"id":32211834,"url":"https://github.com/jmico/beekeeper","last_synced_at":"2025-10-22T06:56:01.643Z","repository":{"id":56834329,"uuid":"42574021","full_name":"jmico/beekeeper","owner":"jmico","description":"Perl framework for building applications with a microservices architecture","archived":false,"fork":false,"pushed_at":"2023-08-02T22:33:37.000Z","size":800,"stargazers_count":9,"open_issues_count":1,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-10-22T06:55:58.438Z","etag":null,"topics":["anyevent","framework","json-rpc","linux","microservices","mqtt","perl"],"latest_commit_sha":null,"homepage":"","language":"Perl","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jmico.png","metadata":{"files":{"readme":"README.md","changelog":"Changes","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-09-16T08:13:07.000Z","updated_at":"2023-07-27T13:35:39.000Z","dependencies_parsed_at":"2024-12-01T18:37:22.270Z","dependency_job_id":"fdca172e-853c-45d2-9a59-3c656192f886","html_url":"https://github.com/jmico/beekeeper","commit_stats":{"total_commits":333,"total_committers":1,"mean_commits":333.0,"dds":0.0,"last_synced_commit":"eca0949e892df8f6e23a2fc8a69f89545ea1b7b4"},"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"purl":"pkg:github/jmico/beekeeper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmico%2Fbeekeeper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmico%2Fbeekeeper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmico%2Fbeekeeper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmico%2Fbeekeeper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jmico","download_url":"https://codeload.github.com/jmico/beekeeper/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmico%2Fbeekeeper/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":280395516,"owners_count":26323517,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-22T02:00:06.515Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anyevent","framework","json-rpc","linux","microservices","mqtt","perl"],"created_at":"2025-10-22T06:55:54.446Z","updated_at":"2025-10-22T06:56:01.632Z","avatar_url":"https://github.com/jmico.png","language":"Perl","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Beekeeper\n\nBeekeeper is a framework for building applications with a microservices architecture.\n\n![](./doc/images/beekeeper.svg)\n\nA pool of worker processes handle requests and communicate with each other through a common message bus.\n\nClients send requests through a different set of message buses, which are isolated for security reasons.\n\nRequests and responses are shoveled between buses by a few router processes.\n\n\n**Benefits of this architecture:**\n\n- Scales horizontally very well. It is easy to add or remove workers, routers or brokers.\n\n- High availability. The system remains responsive even when several components fail.\n\n- Easy integration of browsers via WebSockets or clients written in other languages.\n\n\n**Key characteristics:**\n\n- The broker is an MQTT messaging server, like Mosquitto, HiveMQ or EMQ X.\n\n- The messaging protocol is MQTT 5 (see the [specification](https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html)).\n\n- The RPC protocol is JSON-RPC 2.0 (see the [specification](https://www.jsonrpc.org/specification)).\n\n- There is no message persistence in the broker, it just passes on messages.\n\n- There is no routing logic defined in the broker.\n\n- Synchronous and asynchronous workers or clients can be integrated seamlessly.\n\n- Efficient multicast and unicast push notifications.\n\n- Inherent load balancing.\n\n\n**What does this framework provides:**\n\n- [`Beekeeper::Worker`](https://metacpan.org/pod/Beekeeper::Worker), to create service workers.\n\n- [`Beekeeper::Client`](https://metacpan.org/pod/Beekeeper::Client), to create service clients.\n\n- [`bkpr`](https://metacpan.org/dist/Beekeeper/view/bin/bkpr) command which spawns and controls\n  worker processes.\n\n- Command line tools for monitoring and controlling worker pools.\n\n- An internal broker suitable for development or running tests. \n\n- Automatic message routing between frontend and backend buses.\n\n- Centralized logging, which can be shoveled to an external monitoring application.\n\n- Performance metrics gathering, which can be shoveled to an external monitoring application.\n\n- A nice HTML [dashboard](./examples/dashboard), which can be used in any project. \n([live demo](https://beekeeper.net.ar/dashboard/))\n\n\n## Getting Started\n\n### Creating workers\n\nWorkers provide a service accepting certain RPC calls from clients. The base class `Beekeeper::Worker` \nprovides all the glue needed to accept requests and communicate trough the message bus with clients \nor another workers.\n\nA worker class just declares on startup which methods it will accept, then implements them:\n\n```\npackage MyApp::Worker;\n\nuse base 'Beekeeper::Worker';\n\nsub on_startup {\n    my $self = shift;\n\n    $self-\u003eaccept_remote_calls(\n        'myapp.str.uc' =\u003e 'uppercase',\n    );\n}\n\nsub uppercase {\n    my ($self, $params) = @_;\n\n    return uc $params-\u003e{'string'};\n}\n```\n\n### Creating clients\n\nClients of the service need an interface to use it without knowledge of the underlying RPC mechanisms.\nThe class `Beekeeper::Client` provides methods to connect to the broker and make RPC calls.\n\nThis is the interface of the above service:\n\n```\npackage MyApp::Client;\n\nuse Beekeeper::Client;\n\nsub uppercase {\n    my ($class, $str) = @_;\n\n    my $client = Beekeeper::Client-\u003einstance;\n\n    my $resp = $client-\u003ecall_remote(\n        method =\u003e 'myapp.str.uc',\n        params =\u003e { string =\u003e $str },\n    );\n\n    return $resp-\u003eresult;\n}\n```\nThen other workers or clients can just:\n```\nuse MyApp::Client;\n\nprint MyApp::Client-\u003euppercase(\"hello!\");\n```\n\n### Configuring\n\nBeekeeper applications use two config files to define how clients, workers and brokers connect to \neach other. These files are looked for in ENV `BEEKEEPER_CONFIG_DIR`, `~/.config/beekeeper` and \nthen `/etc/beekeeper`. File format is relaxed JSON, which allows comments and trailing commas.\n\nThe file `pool.config.json` defines all worker pools running on a host, specifying which logical bus\nshould be used and which services it will run. For example:\n\n```\n[{\n    \"pool_id\" : \"myapp\",\n    \"bus_id\"  : \"backend\",\n    \"workers\" : {\n        \"MyApp::Worker\" : { \"worker_count\" : 4 },\n    },\n}]\n```\nThe file `bus.config.json` defines all logical buses used by the application, specifying the connection\nparameters to the brokers that will service them. For example:\n\n```\n[{\n    \"bus_id\"   : \"backend\",\n    \"host\"     : \"localhost\",\n    \"username\" : \"backend\",\n    \"password\" : \"def456\",\n}]\n```\nNeither the worker code nor the client code have hardcoded references to the logical message bus or\nthe broker connection parameters, these communicate to each other using the definitions in these two files.\n\n\n### Running\n\nTo start or stop a pool of workers you use the `bkpr` command. Given the above example config, this \nwill start 4 processes running `MyApp::Worker` code:\n```\nbkpr --pool \"myapp\" start\n```\nWhen started it daemonizes itself and forks all worker processes, then continues monitoring those forked\nprocesses and immediately respawns defunct ones.\n\nThe framework includes these command line tools to manage worker pools:\n\n- [`bkpr-top`](https://metacpan.org/dist/Beekeeper/view/bin/bkpr-top) allows to monitor in real\n  time the performance of workers.\n\n- [`bkpr-log`](https://metacpan.org/dist/Beekeeper/view/bin/bkpr-log) allows to monitor in real\n  time the log output of workers.\n\n- [`bkpr-restart`](https://metacpan.org/dist/Beekeeper/view/bin/bkpr-restart) gracefully restarts \n  worker pools.\n\n\n## Performance\n\nBeekeeper is pretty lightweight for being pure Perl, but the performance depends mostly on the broker\nperformance, particularly on the broker introduced latency. The following are conservative performance\nestimations:\n\n- A `call_remote` synchronous call to a remote method involves 4 MQTT messages and takes 0.7 ms.\n  This limits a client to make a maximum of 1400 synchronous calls per second. The CPU load will be\n  very low, as the client spends most of the time just waiting for the response.\n\n- A `call_remote_async` asynchronous call to a remote method also involves 4 MQTT messages, but it\n  can sustain a rate of 8000 calls per second because it does not block waiting for responses.\n\n- Launching a remote task with `fire_remote` involves 1 MQTT message and takes 0.1 ms. This implies\n  a maximum of 10000 calls per second.\n\n- Sending a notification with `send_notification` involves 1 MQTT message and takes 0.1 ms. A worker\n  can emit more than 10000 notifications per second, up to 15000 if these are smaller than 1 KiB.\n\n- A worker processing remote calls can handle a maximum of 4000 requests per second. It will be I/O\n  bound, the CPU load will be low for simple tasks, as the worker will spend a significant chunk of\n  time waiting for messages.\n\n- A worker can receive a maximum of 15000 notifications per second. It will be CPU bound.\n\n- An empty worker uses 10 MiB of resident memory for the perl interpreter and the few required modules.\n  After adding actual code to do useful work the memory usage will of course increase.\n\n- A single router can handle around 5000 messages per second.\n\n- Routers add 2 ms to frontend requests roundtrip.\n\n\n## Examples\n\nThis distribution includes some examples that can be run out of the box using an internal `ToyBroker`\n(so no install of a proper broker is needed):\n\n[examples/basic](./examples/basic) is a barebones example of the usage of Beekeper.\n\n[examples/flood](./examples/flood) allows to estimate the performance of a Beekeper setup.\n\n[examples/scraper](./examples/scraper) demonstrates asynchronous workers and clients.\n\n[examples/websocket](./examples/websocket) uses a service from a browser using WebSockets. \n([live demo](https://beekeeper.net.ar/examples/calculator.html))\n\n[examples/chat](./examples/chat) implements a real world setup with isolated buses and redundancy. \n([live demo](https://beekeeper.net.ar/examples/chat.html))\n\n[examples/dashboard](./examples/dashboard) is an HTML dashboard for Beekeeper projects. \n([live demo](https://beekeeper.net.ar/dashboard/))\n\n\n## See also\n\n- [How to run Beekeeper pools as systemd services](./doc/Install.md).\n\n- [Notes about supported MQTT brokers](./doc/Brokers.md) configuration.\n\n- [Diagram of message routing](https://raw.githubusercontent.com/jmico/beekeeper/master/doc/images/routing.svg)\n  between clients, workers and buses.\n\n- https://metacpan.org/release/Beekeeper\n\n\n## Dependencies\n\nThis framework requires `Anyevent`, `JSON::XS`, `Net::SSLeay`, `Term::ReadKey` and `ps`.\n\nTo install these dependencies on a Debian system run:\n```\napt install libanyevent-perl libjson-xs-perl libnet-ssleay-perl libterm-readkey-perl procps\n```\n\n## License\n\nCopyright 2015-2023 José Micó.\n\nThis is free software; you can redistribute it and/or modify it under the same\nterms as the Perl 5 programming language itself.\n\n[![CI - master](https://github.com/jmico/beekeeper/actions/workflows/CI.yml/badge.svg)](https://github.com/jmico/beekeeper/actions/workflows/CI.yml)\n[![CI - devel](https://github.com/jmico/beekeeper/actions/workflows/CI.yml/badge.svg?branch=devel)](https://github.com/jmico/beekeeper/actions/workflows/CI.yml)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjmico%2Fbeekeeper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjmico%2Fbeekeeper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjmico%2Fbeekeeper/lists"}