https://github.com/50-course/dj-celery-results-multiple-worker-failure

This repository aims to reproduce and perform root cause analysis of the production bug on @celery:django-celery-results
https://github.com/50-course/dj-celery-results-multiple-worker-failure

Last synced: 24 days ago
JSON representation

This repository aims to reproduce and perform root cause analysis of the production bug on @celery:django-celery-results

• Host: GitHub
• URL: https://github.com/50-course/dj-celery-results-multiple-worker-failure
• Owner: 50-Course
• Created: 2024-01-19T16:59:00.000Z (10 months ago)
• Default Branch: main
• Last Pushed: 2024-01-23T22:42:05.000Z (10 months ago)
• Last Synced: 2024-05-01T13:17:21.361Z (6 months ago)
• Language: Python
• Size: 11.7 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
This repository aims to perform post-mortem of the production bug in @celery/django-celery-results #417:

## Post-mortem

Title: Unable to retrieve results across workers

Ticket No:

 - #417

Reference: https://github.com/celery/django-celery-results/issues/417

Associated Patch: 

### Bug description:

> I have two queues although only one is actively processing tasks, call them A and B. Each is deployed in an individual docker deployment consisting of an app container and a celery worker. Both queues share one rabbit MQ running in its own container.

> I can successfully start tasks in B.celery from A.app. I can watch the task in B's log and see it completes. In B.app I am able to retrieve the task's state and result.

> If, however, I try to retrieve the task's result/ state in A.app, those are always empty and PENDING, and get() / wait() just hang.

> This is problematic because A is where, for the most part, all tasks are started, and A needs to send the same task to each of B, C, D, ... then wait for all tasks to complete before moving on. I can't use chained tasks or similar because the tasks all need to run in parallel.

## Installation

## Bug tracker

Please report error regarding this issue to official `django-celery-results` repository. Or continue the discussion at: celery/django-celery-results#417