{"id":27603532,"url":"https://github.com/taskiq-python/taskiq-memphis","last_synced_at":"2025-04-22T19:17:20.840Z","repository":{"id":168004106,"uuid":"628385271","full_name":"taskiq-python/taskiq-memphis","owner":"taskiq-python","description":"Memphis integration for taskiq","archived":false,"fork":false,"pushed_at":"2023-06-11T13:04:46.000Z","size":158,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-10T07:06:27.450Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/taskiq-python.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-04-15T19:30:04.000Z","updated_at":"2023-05-21T16:22:38.000Z","dependencies_parsed_at":"2023-09-21T15:32:48.195Z","dependency_job_id":null,"html_url":"https://github.com/taskiq-python/taskiq-memphis","commit_stats":{"total_commits":22,"total_committers":3,"mean_commits":7.333333333333333,"dds":"0.18181818181818177","last_synced_commit":"0c89aeb16467a2c2c27775e75e6d49aedbe3cb10"},"previous_names":["taskiq-python/taskiq-memphis"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taskiq-python%2Ftaskiq-memphis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taskiq-python%2Ftaskiq-memphis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taskiq-python%2Ftaskiq-memphis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taskiq-python%2Ftaskiq-memphis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/taskiq-python","download_url":"https://codeload.github.com/taskiq-python/taskiq-memphis/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249981844,"owners_count":21355579,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-22T19:17:20.247Z","updated_at":"2025-04-22T19:17:20.807Z","avatar_url":"https://github.com/taskiq-python.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# taskiq-memphis\n\nThis library provides you with memphis broker for taskiq.\n**You need python version \u003e=3.8**\n\n### Usage:\n```python\nfrom taskiq_memphis import MemphisBroker\n\nbroker = MemphisBroker(\n    memphis_host=\"localhost\",\n    username=\"root\",\n    password=\"memphis\",\n)\n\n\n@broker.task\nasync def the_best_task_ever() -\u003e None:\n    print(\"The best task ever\")\n```\n\n## Configuration\n\nMemphisBroker parameters:\n* `memphis_host` - host to memphis.\n* `port` - memphis server port.\n* `username` - username.\n* `connection_token` - connection token.\n* `password` - password for username.\n* `result_backend` - custom result backend.\n* `task_id_generator` - custom task_id genertaor.\n* `reconnect` - turn on/off reconnection while connection is lost.\n* `max_reconnect` - maximum reconnection attempts.\n* `reconnect_interval_ms` - interval in milliseconds between reconnect attempts.\n* `timeout_ms` - connection timeout in milliseconds.\n* `cert_file` - path to tls cert file.\n* `key_file` - path to tls key file.\n* `ca_file` - path to tls ca file.\n* `consumer_batch_size` - batch size for the consumer.\n* `destroy_station_on_shutdown` - close station on shutdown.\n* `destroy_producer_on_shutdown` - close producer on shutdown.\n* `destroy_consumer_on_shutdown` - close consumer on shutdown.\n\n\n## Non-obvious things\n\nYou can configure memphis topic, consumer, producer and `produce` method with:\n```python\n# Create broker\nbroker = MemphisBroker(\n    memphis_host=\"localhost\",\n    username=\"root\",\n    password=\"memphis\",\n    destroy_station_on_shutdown=True,\n)\n\n# Configure station\nbroker.configure_station(...)\n\n# Configure producer\nbroker.configure_producer(...)\n\n# Configure produce method\nbroker.configure_produce_method(...)\n\n# Configure consumer\nbroker.configure_consumer(...)\n```\n\nMemphis `station` parameters you can configure:\n* `name` - name of the station. Required.\n* `retention_type` - type of message retention.\n* `retention_value` - how long it takes to keep message, based on retention_type.\n* `storage_type` - type of the storage, DISK or MEMORY.\n* `replicas` - number of replicas.\n* `idempotency_window_ms` - time frame in which idempotent messages will be tracked.\n* `schema_name` - name of the schema. (You can create it only via memphis UI now)\n* `send_poison_msg_to_dls` - send poisoned message to dead letter station or not.\n* `send_schema_failed_msg_to_dls` - send schema failed message to dead letter station or not.\n* `tiered_storage_enabled` - tiered storage enabled or not.\n\nMemphis `producer` parameters you can configure:\n* `producer_name` - producer name. Required.\n* `generate_random_suffix` - add suffix to producer name. Default - `True`.\n**DON'T SET THIS VARIABLE TO `FALSE` IF YOU WANT TO USE MORE THAN ONE PRODUCER.**\n\nMemphis `produce` method parameters you can configure:\n* `ack_wait_sec` - wait ack time in seconds.\n* `headers` - `Headers` instance from memphis.\n* `async_produce` - produce message in async way or not.\n\nMemphis `consumer` parameters you can configure:\n\n* `consumer_name` - name of the consumer. Required.\n* `consumer_group` - name of the consumer group.\n* `pull_interval_ms` - interval in milliseconds between pulls.\n* `batch_size` - pull batch size.\n* `batch_max_time_to_wait_ms` - max time in milliseconds to wait between pulls.\n* `max_ack_time_ms` - max time for ack a message in milliseconds.\n* `max_msg_deliveries` - max number of message deliveries.\n* `generate_random_suffix` - concatenate a random suffix to consumer's name.\n**DON'T SET THIS VARIABLE TO `FALSE` IF YOU WANT TO USE MORE THAN ONE CONSUMER.**\n* `start_consume_from_sequence` - start consuming from a specific sequence.\n* `last_messages` - consume the last N messages.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftaskiq-python%2Ftaskiq-memphis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftaskiq-python%2Ftaskiq-memphis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftaskiq-python%2Ftaskiq-memphis/lists"}