{"id":24732654,"url":"https://github.com/gonzalo123/playing_with_nameko","last_synced_at":"2025-03-22T16:16:28.212Z","repository":{"id":66582115,"uuid":"163684018","full_name":"gonzalo123/playing_with_nameko","owner":"gonzalo123","description":"Communications between nameko microservices and non-python microservices asynchronously.","archived":false,"fork":false,"pushed_at":"2019-01-01T19:31:51.000Z","size":6,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-01-27T17:59:45.665Z","etag":null,"topics":["microservices","nameko","nameko-services","python","queues","rabbitmq"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gonzalo123.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-12-31T16:45:13.000Z","updated_at":"2019-01-01T19:31:52.000Z","dependencies_parsed_at":null,"dependency_job_id":"f2109ce4-1207-432a-bc61-b068cb0f1fe6","html_url":"https://github.com/gonzalo123/playing_with_nameko","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gonzalo123%2Fplaying_with_nameko","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gonzalo123%2Fplaying_with_nameko/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gonzalo123%2Fplaying_with_nameko/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gonzalo123%2Fplaying_with_nameko/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gonzalo123","download_url":"https://codeload.github.com/gonzalo123/playing_with_nameko/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244982060,"owners_count":20542301,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["microservices","nameko","nameko-services","python","queues","rabbitmq"],"created_at":"2025-01-27T17:53:32.267Z","updated_at":"2025-03-22T16:16:28.206Z","avatar_url":"https://github.com/gonzalo123.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Communications between nameko microservices and non-python microservices asynchronously.\n\nIn my last projects I've playing with microservices. Mostly written in Python. Months ago I discovered nameko. It's great. I've written a couple of posts about it. Nameko relies on RabbitMQ and basically it does the common operations that I normally do when I work with microservices in a very clean way. That's cool but I sometimes need to work with other services (micro or macro) that they aren't written in Python. With nameko I can create easily http gateways. And also we can send http requests from our nameko microservices to the rest of the world. It works but it's synchronous. If synchronous way fits to your needs it's perfect, but sometimes we need asynchronous way. What can we do?\n\nSince nameko relies on RabbitMQ we can use nameko's queues and exchanges outside nameko. It should be possible but afaik nameko encapsulates its messages with one kind of encoder that I don't understand. Years ago I've tried to do one kind of reverse engineering to figure out how to encode those messages. But now I'm getting older and I don't want to spend so much time trying to discover it. Let me show you how I've done it\n\n### From nameko to another services\nHere we only need to send one RabbitMQ message. To do it, I've created a simple dependency Provider in Nameko\n\n```python\nfrom nameko.extensions import DependencyProvider\nfrom pika.adapters.blocking_connection import BlockingConnection\nimport pika\n\n\nclass Rabbit:\n    def __init__(self, conn: BlockingConnection):\n        self.conn = conn\n\n    def get_connection(self) -\u003e BlockingConnection:\n        return self.conn\n\n    def close(self):\n        self.conn.close()\n\n    def publish_to_queue(self, queue_name, data, headers=None):\n        headers = headers if headers is None else {}\n        channel = self.conn.channel()\n        channel.queue_declare(queue=queue_name)\n\n        channel.basic_publish(exchange='',\n                              routing_key=queue_name,\n                              body=data,\n                              properties=pika.BasicProperties(\n                                  headers=headers\n                              ))\n        return self\n\n\nclass RabbitService(DependencyProvider):\n\n    def get_dependency(self, worker_ctx):\n        uri = self.container.config['AMQP_URI']\n        return Rabbit(pika.BlockingConnection(pika.URLParameters(uri)))\n\n```\n\nHere a simple example of nameko microservice that sends a meesage to one queue each 3 seconds\n\n```python\nfrom nameko.timer import timer\nfrom ext.rabbit import RabbitService, Rabbit\nfrom dotenv import load_dotenv\nimport os\n\ncurrent_dir = os.path.dirname(os.path.abspath(__file__))\nload_dotenv(dotenv_path=\"{}/.env\".format(current_dir))\n\n\nclass Timer:\n    name = 'timer_service'\n\n    rabbit: Rabbit = RabbitService()\n\n    @timer(interval=3)\n    def ping(self):\n        self.rabbit.publish_to_queue(queue_name='queue3',\n                                     data='ping').close()\n```\n\n### From another services to nameko\n\nIn the opposite way I've found two possible solutions. \n\nThe first one is to create a simple RabbitMQ consumer listening to one queue (acting as a proxy) and redirecting messages to nameko.\n \n```python\nimport pika\nimport logging\nimport yaml\nfrom nameko.standalone.rpc import ClusterRpcProxy\n\nlogging.basicConfig(level=logging.WARNING)\nqueue_name = 'queue3'\n\nwith open(\"config.yaml\", 'r') as stream:\n    amqp_uri = yaml.load(stream)['AMQP_URI']\n\nparameters = pika.URLParameters(amqp_uri)\n\nconfig = {'AMQP_URI': amqp_uri}\n\ndef callback(ch, method, properties, body):\n    logging.warning(\"rabbit receive: {}\".format(body))\n    with ClusterRpcProxy(config) as rpc:\n        rpc.listener_example.remote_method(body.decode('utf-8'))\n\n\nbroker_connection = pika.BlockingConnection(parameters=parameters)\nchannel = broker_connection.channel()\nchannel.queue_declare(queue=queue_name)\n\nchannel.basic_consume(consumer_callback=callback,\n                      queue=queue_name,\n                      no_ack=True)\nchannel.basic_qos(prefetch_count=1)\nchannel.start_consuming()\n```\n\nThe other way is using a Nameko Entrypoint\n\n```python\nfrom nameko.extensions import Entrypoint\nimport pika\nfrom functools import partial\nimport os\n\n\nclass QueueListener(Entrypoint):\n    channel = None\n\n    def __init__(self, queue, **kwargs):\n        self.queue = queue\n        super(QueueListener, self).__init__(**kwargs)\n\n    def setup(self):\n        pass\n\n    def start(self):\n        self.container.spawn_managed_thread(self.run, identifier=\"QueueListener.run\")\n\n    def run(self):\n        broker_connection = pika.BlockingConnection(pika.URLParameters(os.getenv('AMQP_URI')))\n\n        channel = broker_connection.channel()\n        channel.queue_declare(queue=self.queue)\n\n        channel.basic_consume(consumer_callback=self.handle_message,\n                              queue=self.queue,\n                              no_ack=False)\n        channel.basic_qos(prefetch_count=1)\n        channel.start_consuming()\n\n    def handle_message(self, ch, method, properties, body):\n        handle_result = partial(self.handle_result)\n        ch.basic_ack(delivery_tag=method.delivery_tag)\n        args = (body,)\n        kwargs = {}\n\n        self.container.spawn_worker(\n            self, args, kwargs, handle_result=handle_result\n        )\n\n    def handle_result(self, worker_ctx, result, exc_info):\n        return result, exc_info\n\n\nlistener = QueueListener.decorator\n```\n\nNow we can use a nameko service listening to the queue via our nameko entrypoint\n\n```python\nfrom nameko.events import EventDispatcher\nfrom ext.entrypoint import listener\nimport logging\nfrom dotenv import load_dotenv\nimport os\n\ncurrent_dir = os.path.dirname(os.path.abspath(__file__))\nload_dotenv(dotenv_path=\"{}/.env\".format(current_dir))\n\n\nclass Listener:\n    name = 'listener_service'\n    dispatcher = EventDispatcher()\n\n    @listener(queue='queue3')\n    def handle(self, body):\n        logging.info(\"listener_service listen via entrypoint: {}\".format(body))\n        self.dispatcher(\"event_type\", body.decode('utf-8'))\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgonzalo123%2Fplaying_with_nameko","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgonzalo123%2Fplaying_with_nameko","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgonzalo123%2Fplaying_with_nameko/lists"}