{"id":13495142,"url":"https://github.com/Fatal1ty/amqpipe","last_synced_at":"2025-03-28T16:31:37.583Z","repository":{"id":5829603,"uuid":"44819116","full_name":"Fatal1ty/amqpipe","owner":"Fatal1ty","description":"Twisted based pipeline framework for AMQP","archived":true,"fork":false,"pushed_at":"2022-04-06T17:32:20.000Z","size":38,"stargazers_count":10,"open_issues_count":1,"forks_count":2,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-05T05:46:24.310Z","etag":null,"topics":["amqp","pika","pipeline-framework","python","queue","rabbitmq","twisted"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Fatal1ty.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-10-23T14:45:14.000Z","updated_at":"2023-01-28T01:44:53.000Z","dependencies_parsed_at":"2022-08-06T19:00:54.835Z","dependency_job_id":null,"html_url":"https://github.com/Fatal1ty/amqpipe","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Fatal1ty%2Famqpipe","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Fatal1ty%2Famqpipe/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Fatal1ty%2Famqpipe/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Fatal1ty%2Famqpipe/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Fatal1ty","download_url":"https://codeload.github.com/Fatal1ty/amqpipe/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246062830,"owners_count":20717696,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["amqp","pika","pipeline-framework","python","queue","rabbitmq","twisted"],"created_at":"2024-07-31T19:01:31.547Z","updated_at":"2025-03-28T16:31:37.272Z","avatar_url":"https://github.com/Fatal1ty.png","language":"Python","readme":"AMQPipe\n=======\n\n.. image:: https://travis-ci.org/Fatal1ty/amqpipe.svg?branch=master\n    :target: https://travis-ci.org/Fatal1ty/amqpipe\n\n.. image:: https://requires.io/github/Fatal1ty/amqpipe/requirements.svg?branch=master\n    :target: https://requires.io/github/Fatal1ty/amqpipe/requirements/?branch=master\n    :alt: Requirements Status\n\n.. image:: https://img.shields.io/pypi/v/amqpipe.svg\n    :target: https://pypi.python.org/pypi/amqpipe\n\n.. image:: https://img.shields.io/pypi/pyversions/amqpipe.svg\n    :target: https://pypi.python.org/pypi/amqpipe/\n\n.. image:: https://img.shields.io/badge/license-MIT-blue.svg\n    :target: https://raw.githubusercontent.com/Fatal1ty/amqpipe/master/LICENSE\n\nTwisted based pipeline framework for AMQP. It allow you to create fast\nasynchronous services which follow ideology:\n\n-  get message from queue\n-  doing something with message\n-  publish some result\n\nInstallation\n------------\n\nInstall via pip:\n\n::\n\n        pip install amqpipe\n\nBasic usage\n-----------\n\nThe minimal module based on AMQPipe is:\n\n.. code:: python\n\n    from amqpipe import AMQPipe\n\n    pipe = AMQPipe()\n    pipe.run()\n\nIt will simply get all messages from one RabbitMQ queue and publish them\nto other RabbitMQ exchange.\n\nNow we define some action on messages:\n\n.. code:: python\n\n    import hashlib\n    from amqpipe import AMQPipe\n\n    def action(message):\n        return hashlib.md5(message).hexdigest()\n\n    pipe = AMQPipe(action=action)\n    pipe.run()\n\nIt will publish md5 checksum for every message as result.\n\nIf messages in input queue are in predefined format then you can define\nconverter-function:\n\n.. code:: python\n\n    import hashlib\n    from amqpipe import AMQPipe\n\n    def converter(message):\n        return message['text']\n\n    def action(text):\n        return hashlib.md5(text).hexdigest()\n\n    pipe = AMQPipe(converter=converter, action=action)\n    pipe.run()\n\nYou can define service-specific arguments:\n\n.. code:: python\n\n    import hashlib\n    from amqpipe import AMQPipe\n\n    class Processor:\n        def set_field(self, field):\n            self.field = field\n\n    processor = Processor()\n\n    def init(args):\n        processor.set_field(args.field)\n\n    def converter(message):\n        return message.get(processor.field)\n\n    def action(text):\n        return hashlib.md5(text).hexdigest()\n\n    pipe = AMQPipe(converter, action, init)\n    pipe.parser.add_argument('--field', default='text', help='Field name for retrieving message value')\n    pipe.run()\n\nYou can connect to database in ``init`` function or do some other things\nfor initialization.\n\nIf your action returns Deferred then result would be published to\nRabbitMQ when this Deferred will be resolved:\n\n.. code:: python\n\n    import logging\n    from twisted.internet import defer\n    from amqpipe import AMQPipe\n\n    logger = logging.getLogger(__name__)\n\n    class Processor:\n        def set_field(self, field):\n            self.field = field\n\n    processor = Processor()\n\n    def init(args):\n        connect_to_db()\n        ...\n\n    def converter(message):\n        return message.get(processor.field)\n\n    @defer.inlineCallbacks\n    def action(text):\n        result = yield db_query(text)\n        logger.info('Get from db: %s', result)\n        defer.returnValue(result)\n\n    pipe = AMQPipe(converter, action, init)\n    pipe.parser.add_argument('--field', default='text', help='Field name for retrieving message value')\n    pipe.run()\n\nInit function may return Deferred too.","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FFatal1ty%2Famqpipe","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FFatal1ty%2Famqpipe","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FFatal1ty%2Famqpipe/lists"}