{"id":13424698,"url":"https://github.com/snok/asgi-correlation-id","last_synced_at":"2025-05-14T08:08:47.555Z","repository":{"id":38409464,"uuid":"421007644","full_name":"snok/asgi-correlation-id","owner":"snok","description":"Request ID propagation for ASGI apps","archived":false,"fork":false,"pushed_at":"2024-10-17T11:41:24.000Z","size":320,"stargazers_count":474,"open_issues_count":2,"forks_count":36,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-04-09T03:08:55.367Z","etag":null,"topics":["celery","correlation-id","logging","python","request-id","sentry","tracing"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/snok.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-10-25T12:07:15.000Z","updated_at":"2025-04-05T05:14:59.000Z","dependencies_parsed_at":"2024-01-03T02:29:55.460Z","dependency_job_id":"2a8cb479-c483-46b4-aab4-b2171a26ce4e","html_url":"https://github.com/snok/asgi-correlation-id","commit_stats":{"total_commits":109,"total_committers":12,"mean_commits":9.083333333333334,"dds":0.2844036697247706,"last_synced_commit":"bf3b500290567d84b0a724703dadd1906996573e"},"previous_names":[],"tags_count":24,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snok%2Fasgi-correlation-id","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snok%2Fasgi-correlation-id/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snok%2Fasgi-correlation-id/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snok%2Fasgi-correlation-id/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/snok","download_url":"https://codeload.github.com/snok/asgi-correlation-id/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254101558,"owners_count":22014908,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["celery","correlation-id","logging","python","request-id","sentry","tracing"],"created_at":"2024-07-31T00:00:58.033Z","updated_at":"2025-05-14T08:08:42.544Z","avatar_url":"https://github.com/snok.png","language":"Python","funding_links":[],"categories":["Third-Party Extensions","Monitoring"],"sub_categories":["Utils"],"readme":"[![pypi](https://img.shields.io/pypi/v/asgi-correlation-id)](https://pypi.org/project/asgi-correlation-id/)\n[![test](https://github.com/snok/asgi-correlation-id/actions/workflows/test.yml/badge.svg)](https://github.com/snok/asgi-correlation-id/actions/workflows/test.yml)\n[![codecov](https://codecov.io/gh/snok/asgi-correlation-id/branch/main/graph/badge.svg?token=1aXlWPm2gb)](https://codecov.io/gh/snok/asgi-correlation-id)\n\n# ASGI Correlation ID middleware\n\nMiddleware for reading or generating correlation IDs for each incoming request. Correlation IDs can then be added to your\nlogs, making it simple to retrieve all logs generated from a single HTTP request.\n\nWhen the middleware detects a correlation ID HTTP header in an incoming request, the ID is stored. If no header is\nfound, a correlation ID is generated for the request instead.\n\nThe middleware checks for the `X-Request-ID` header by default, but can be set to any key.\n`X-Correlation-ID` is also pretty commonly used.\n\n## Example\n\nOnce logging is configured, your output will go from this:\n\n```\nINFO    ... project.views  This is an info log\nWARNING ... project.models This is a warning log\nINFO    ... project.views  This is an info log\nINFO    ... project.views  This is an info log\nWARNING ... project.models This is a warning log\nWARNING ... project.models This is a warning log\n```\n\nto this:\n\n```docker\nINFO    ... [773fa6885] project.views  This is an info log\nWARNING ... [773fa6885] project.models This is a warning log\nINFO    ... [0d1c3919e] project.views  This is an info log\nINFO    ... [99d44111e] project.views  This is an info log\nWARNING ... [0d1c3919e] project.models This is a warning log\nWARNING ... [99d44111e] project.models This is a warning log\n```\n\nNow we're actually able to see which logs are related.\n\n# Installation\n\n```\npip install asgi-correlation-id\n```\n\n# Setup\n\nTo set up the package, you need to add the middleware and configure logging.\n\n## Adding the middleware\n\nThe middleware can be added like this:\n\n```python\nfrom fastapi import FastAPI\n\nfrom asgi_correlation_id import CorrelationIdMiddleware\n\napp = FastAPI()\napp.add_middleware(CorrelationIdMiddleware)\n```\n\nor any other way your framework allows.\n\nFor [Starlette](https://github.com/encode/starlette) apps, just substitute `FastAPI` with `Starlette` in all examples.\n\n## Configure logging\n\nThis section assumes you have already started configuring logging in your project. If this is not the case, check out\nthe section on [setting up logging from scratch](#setting-up-logging-from-scratch) instead.\n\nTo set up logging of the correlation ID, you simply have to add the log-filter the package provides.\n\nIf your current log-config looked like this:\n\n```python\nLOGGING = {\n    'version': 1,\n    'disable_existing_loggers': False,\n    'formatters': {\n        'web': {\n            'class': 'logging.Formatter',\n            'datefmt': '%H:%M:%S',\n            'format': '%(levelname)s ... %(name)s %(message)s',\n        },\n    },\n    'handlers': {\n        'web': {\n            'class': 'logging.StreamHandler',\n            'formatter': 'web',\n        },\n    },\n    'loggers': {\n        'my_project': {\n            'handlers': ['web'],\n            'level': 'DEBUG',\n            'propagate': True,\n        },\n    },\n}\n```\n\nYou simply have to add the filter, like this:\n\n```diff\nLOGGING = {\n    'version': 1,\n    'disable_existing_loggers': False,\n+   'filters': {\n+       'correlation_id': {\n+           '()': 'asgi_correlation_id.CorrelationIdFilter',\n+           'uuid_length': 32,\n+           'default_value': '-',\n+       },\n+   },\n    'formatters': {\n        'web': {\n            'class': 'logging.Formatter',\n            'datefmt': '%H:%M:%S',\n+           'format': '%(levelname)s ... [%(correlation_id)s] %(name)s %(message)s',\n        },\n    },\n    'handlers': {\n        'web': {\n            'class': 'logging.StreamHandler',\n+           'filters': ['correlation_id'],\n            'formatter': 'web',\n        },\n    },\n    'loggers': {\n        'my_project': {\n            'handlers': ['web'],\n            'level': 'DEBUG',\n            'propagate': True,\n        },\n    },\n}\n```\n\nIf you're using a json log-formatter, just add `correlation-id: %(correlation_id)s` to your list of properties.\n\n## Middleware configuration\n\nThe middleware can be configured in a few ways, but there are no required arguments.\n\n```python\napp.add_middleware(\n    CorrelationIdMiddleware,\n    header_name='X-Request-ID',\n    update_request_header=True,\n    generator=lambda: uuid4().hex,\n    validator=is_valid_uuid4,\n    transformer=lambda a: a,\n)\n```\n\nConfigurable middleware arguments include:\n\n**header_name**\n\n- Type: `str`\n- Default: `X-Request-ID`\n- Description: The header name decides which HTTP header value to read correlation IDs from. `X-Request-ID` and\n  `X-Correlation-ID` are common choices.\n\n**update_request_header**\n\n- Type: `bool`\n- Default: `True`\n- Description: Whether to update incoming request's header value with the generated correlation ID. This is to support\n  use cases where it's relied on the presence of the request header (like various tracing middlewares).\n\n**generator**\n\n- Type: `Callable[[], str]`\n- Default: `lambda: uuid4().hex`\n- Description: The generator function is responsible for generating new correlation IDs when no ID is received from an\n  incoming request's headers. We use UUIDs by default, but if you prefer, you could use libraries\n  like [nanoid](https://github.com/puyuan/py-nanoid) or your own custom function.\n\n**validator**\n\n- Type: `Callable[[str], bool]`\n- Default: `is_valid_uuid4` (\n  found [here](https://github.com/snok/asgi-correlation-id/blob/main/asgi_correlation_id/middleware.py#L17))\n- Description: The validator function is used when reading incoming HTTP header values. By default, we discard non-UUID\n  formatted header values, to enforce correlation ID uniqueness. If you prefer to allow any header value, you can set\n  this setting to `None`, or pass your own validator.\n\n**transformer**\n\n- Type: `Callable[[str], str]`\n- Default: `lambda a: a`\n- Description: Most users won't need a transformer, and by default we do nothing.\n  The argument was added for cases where users might want to alter incoming or generated ID values in some way. It\n  provides a mechanism for transforming an incoming ID in a way you see fit. See the middleware code for more context.\n\n## CORS\n\nIf you are using cross-origin resource sharing ([CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS)), e.g.\nyou are making requests to an API from a frontend JavaScript code served from a different origin, you have to ensure\ntwo things:\n\n- permit correlation ID header in the incoming requests' HTTP headers so the value can be reused by the middleware,\n- add the correlation ID header to the allowlist in responses' HTTP headers so it can be accessed by the browser.\n\nThis can be best accomplished by using a dedicated middleware for your framework of choice. Here are some examples.\n\n### Starlette\n\nDocs: https://www.starlette.io/middleware/#corsmiddleware\n\n```python\nfrom starlette.applications import Starlette\nfrom starlette.middleware import Middleware\nfrom starlette.middleware.cors import CORSMiddleware\n\n\nmiddleware = [\n    Middleware(\n        CORSMiddleware,\n        allow_origins=['*'],\n        allow_methods=['*'],\n        allow_headers=['X-Requested-With', 'X-Request-ID'],\n        expose_headers=['X-Request-ID']\n    )\n]\n\napp = Starlette(..., middleware=middleware)\n```\n\n### FastAPI\n\nDocs: https://fastapi.tiangolo.com/tutorial/cors/\n\n```python\nfrom app.main import app\nfrom fastapi.middleware.cors import CORSMiddleware\n\n\napp.add_middleware(\n    CORSMiddleware,\n    allow_origins=['*'],\n    allow_methods=['*'],\n    allow_headers=['X-Requested-With', 'X-Request-ID'],\n    expose_headers=['X-Request-ID']\n)\n```\n\nFor more details on the topic, refer to the [CORS protocol](https://fetch.spec.whatwg.org/#http-cors-protocol).\n\n## Exception handling\n\nBy default, the `X-Request-ID` response header will be included in all responses from the server, *except* in the case\nof unhandled server errors. If you wish to include request IDs in the case of a `500` error you can add a custom\nexception handler.\n\nHere are some simple examples to help you get started. See each framework's documentation for more info.\n\n### Starlette\n\nDocs: https://www.starlette.io/exceptions/\n\n```python\nfrom starlette.requests import Request\nfrom starlette.responses import PlainTextResponse\nfrom starlette.applications import Starlette\n\nfrom asgi_correlation_id import correlation_id\n\n\nasync def custom_exception_handler(request: Request, exc: Exception) -\u003e PlainTextResponse:\n    return PlainTextResponse(\n        \"Internal Server Error\",\n        status_code=500,\n        headers={'X-Request-ID': correlation_id.get() or \"\"}\n    )\n\n\napp = Starlette(\n    ...,\n    exception_handlers={500: custom_exception_handler}\n)\n```\n\n### FastAPI\n\nDocs: https://fastapi.tiangolo.com/tutorial/handling-errors/\n\n```python\nfrom app.main import app\nfrom fastapi import HTTPException, Request\nfrom fastapi.exception_handlers import http_exception_handler\nfrom fastapi.responses import JSONResponse\n\nfrom asgi_correlation_id import correlation_id\n\n\n@app.exception_handler(Exception)\nasync def unhandled_exception_handler(request: Request, exc: Exception) -\u003e JSONResponse:\n    return await http_exception_handler(\n        request,\n        HTTPException(\n            500,\n            'Internal server error',\n            headers={'X-Request-ID': correlation_id.get() or \"\"}\n        ))\n```\n\nIf you are using CORS, you also have to include the `Access-Control-Allow-Origin` and `Access-Control-Expose-Headers`\nheaders in the error response. For more details, see the [CORS section](#cors) above.\n\n# Setting up logging from scratch\n\nIf your project does not have logging configured, this section will explain how to get started. If you want even more\ndetails, take a look\nat [this blogpost](https://medium.com/@sondrelg_12432/setting-up-request-id-logging-for-your-fastapi-application-4dc190aac0ea)\n.\n\nThe Python [docs](https://docs.python.org/3/library/logging.config.html) explain there are a few configuration functions\nyou may use for simpler setup. For this example we will use `dictConfig`, because that's what, e.g., Django users should\nfind most familiar, but the different configuration methods are interchangable, so if you want to use another method,\njust browse the python docs and change the configuration method as you please.\n\nThe benefit of `dictConfig` is that it lets you specify your entire logging configuration in a single data structure,\nand it lets you add conditional logic to it. The following example shows how to set up both console and JSON logging:\n\n```python\nfrom logging.config import dictConfig\n\nfrom app.core.config import settings\n\n\ndef configure_logging() -\u003e None:\n    dictConfig(\n        {\n            'version': 1,\n            'disable_existing_loggers': False,\n            'filters': {  # correlation ID filter must be added here to make the %(correlation_id)s formatter work\n                'correlation_id': {\n                    '()': 'asgi_correlation_id.CorrelationIdFilter',\n                    'uuid_length': 8 if not settings.ENVIRONMENT == 'local' else 32,\n                    'default_value': '-',\n                },\n            },\n            'formatters': {\n                'console': {\n                    'class': 'logging.Formatter',\n                    'datefmt': '%H:%M:%S',\n                    # formatter decides how our console logs look, and what info is included.\n                    # adding %(correlation_id)s to this format is what make correlation IDs appear in our logs\n                    'format': '%(levelname)s:\\t\\b%(asctime)s %(name)s:%(lineno)d [%(correlation_id)s] %(message)s',\n                },\n            },\n            'handlers': {\n                'console': {\n                    'class': 'logging.StreamHandler',\n                    # Filter must be declared in the handler, otherwise it won't be included\n                    'filters': ['correlation_id'],\n                    'formatter': 'console',\n                },\n            },\n            # Loggers can be specified to set the log-level to log, and which handlers to use\n            'loggers': {\n                # project logger\n                'app': {'handlers': ['console'], 'level': 'DEBUG', 'propagate': True},\n                # third-party package loggers\n                'databases': {'handlers': ['console'], 'level': 'WARNING'},\n                'httpx': {'handlers': ['console'], 'level': 'INFO'},\n                'asgi_correlation_id': {'handlers': ['console'], 'level': 'WARNING'},\n            },\n        }\n    )\n```\n\nWith the logging configuration defined within a function like this, all you have to do is make sure to run the function\non startup somehow, and logging should work for you. You can do this any way you'd like, but passing it to\nthe `FastAPI.on_startup` list of callables is a good starting point.\n\n# Integration with structlog\n\n[structlog](https://www.structlog.org/) is a Python library that enables structured logging.\n\nIt is trivial to configure with `asgi_correlation_id`:\n\n```python\nimport logging\nfrom typing import Any\n\nimport structlog\nfrom asgi_correlation_id import correlation_id\n\n\ndef add_correlation(\n    logger: logging.Logger, method_name: str, event_dict: dict[str, Any]\n) -\u003e dict[str, Any]:\n    \"\"\"Add request id to log message.\"\"\"\n    if request_id := correlation_id.get():\n        event_dict[\"request_id\"] = request_id\n    return event_dict\n\n\nstructlog.configure(\n    processors=[\n        add_correlation,\n        structlog.stdlib.filter_by_level,\n        structlog.stdlib.add_logger_name,\n        structlog.stdlib.add_log_level,\n        structlog.processors.TimeStamper(fmt=\"%Y-%m-%d %H:%M.%S\"),\n        structlog.processors.StackInfoRenderer(),\n        structlog.processors.format_exc_info,\n        structlog.processors.JSONRenderer(),\n    ],\n    wrapper_class=structlog.stdlib.BoundLogger,\n    logger_factory=structlog.stdlib.LoggerFactory(),\n    cache_logger_on_first_use=True,\n)\n```\n\n# Integration with [SAQ](https://github.com/tobymao/saq)\n\nIf you're using [saq](https://github.com/tobymao/saq/), you\ncan easily transfer request IDs from the web server to your\nworkers by using the event hooks provided by the library:\n\n```python\nfrom uuid import uuid4\n\nfrom asgi_correlation_id import correlation_id\nfrom saq import Job, Queue\n\n\nCID_TRANSFER_KEY = 'correlation_id'\n\n\nasync def before_enqueue(job: Job) -\u003e None:\n    \"\"\"\n    Transfer the correlation ID from the current context to the worker.\n\n    This might be called from a web server or a worker process.\n    \"\"\"\n    job.meta[CID_TRANSFER_KEY] = correlation_id.get() or uuid4()\n\n\nasync def before_process(ctx: dict) -\u003e None:\n    \"\"\"\n    Load correlation ID from the enqueueing process to this one.\n    \"\"\"\n    correlation_id.set(ctx['job'].meta.get(CID_TRANSFER_KEY, uuid4()))\n\n\nasync def after_process(ctx: dict) -\u003e None:\n    \"\"\"\n    Reset correlation ID for this process.\n    \"\"\"\n    correlation_id.set(None)\n\nqueue = Queue(...)\nqueue.register_before_enqueue(before_enqueue)\n\npriority_settings = {\n    ...,\n    'queue': queue,\n    'before_process': before_process,\n    'after_process': after_process,\n}\n```\n\n# Integration with [hypercorn](https://github.com/pgjones/hypercorn)\nTo add a correlation ID to your [hypercorn](https://github.com/pgjones/hypercorn) logs, you'll need to add a log filter and change the log formatting. Here's an example of how to configure hypercorn, if you're running a [FastAPI](https://fastapi.tiangolo.com/deployment/manually/) app:\n\n```python\nimport logging\nimport os\n\nfrom fastapi import APIRouter, FastAPI\nfrom hypercorn.config import Config\nfrom hypercorn.asyncio import serve\nimport asgi_correlation_id\nimport asyncio\nimport hypercorn\n\n\ndef configure_logging():\n    console_handler = logging.StreamHandler()\n    console_handler.addFilter(asgi_correlation_id.CorrelationIdFilter())\n    logging.basicConfig(\n        handlers=[console_handler],\n        level=\"INFO\",\n\tformat=\"%(levelname)s log [%(correlation_id)s] %(name)s %(message)s\")\n\n\napp = FastAPI(on_startup=[configure_logging])\napp.add_middleware(asgi_correlation_id.CorrelationIdMiddleware)\nrouter = APIRouter()\n\n\n@router.get(\"/test\")\nasync def test_get():\n    print(\"toto\")\n    logger = logging.getLogger()\n    logger.info(\"test_get\")\n\n\napp.include_router(router)\n\n\nif __name__ == \"__main__\":\n    logConfig = {\n        \"handlers\": {\n            \"hypercorn.access\": {\n                \"formatter\": \"hypercorn.access\",\n                \"level\": \"INFO\",\n                \"class\": \"logging.StreamHandler\",\n                \"stream\": \"ext://sys.stdout\",\n                \"filters\": [\n                    asgi_correlation_id.CorrelationIdFilter()\n                ],\n        }},\n        \"formatters\": {\n            \"hypercorn.access\": {\n                \"format\": \"%(message)s %(correlation_id)s\",\n            }\n        },\n        \"loggers\": {\n            \"hypercorn.access\": {\n                \"handlers\": [\n                    \"hypercorn.access\"\n                ],\n                \"level\": \"INFO\",\n            },\n        },\n        \"version\": 1\n    }\n\n    config = Config()\n    # write access log to stdout\n    config.accesslog = \"-\"\n\n    config.logconfig_dict = logConfig\n    asyncio.run(serve(app, config))\n```\n\n```\n# run it\n$ python3 test.py\n\n# test it:\n$ curl http://localhost:8080/test\n\n# log on stdout:\nINFO log [7e7ccfff352a428991920d1da2502674] root test_get\n127.0.0.1:34754 - - [14/Dec/2023:10:34:08 +0100] \"GET /test 1.1\" 200 4 \"-\" \"curl/7.76.1\" 7e7ccfff352a428991920d1da2502674\n```\n\n# Integration with [Uvicorn](https://github.com/encode/uvicorn)\nTo add a correlation ID to your [uvicorn](https://github.com/encode/uvicorn) logs, you'll need to add a log filter and change the log formatting. Here's an example of how to configure uvicorn, if you're running a [FastAPI](https://fastapi.tiangolo.com/deployment/manually/) app:\n\n```python\nimport logging\nimport os\n\nimport asgi_correlation_id\nimport uvicorn\nfrom fastapi import APIRouter, FastAPI\nfrom uvicorn.config import LOGGING_CONFIG\n\n\ndef configure_logging():\n    console_handler = logging.StreamHandler()\n    console_handler.addFilter(asgi_correlation_id.CorrelationIdFilter())\n    logging.basicConfig(\n        handlers=[console_handler],\n        level=\"INFO\",\n        format=\"%(levelname)s log [%(correlation_id)s] %(name)s %(message)s\")\n\n\napp = FastAPI(on_startup=[configure_logging])\napp.add_middleware(asgi_correlation_id.CorrelationIdMiddleware)\nrouter = APIRouter()\n\n\n@router.get(\"/test\")\nasync def test_get():\n    logger = logging.getLogger()\n    logger.info(\"test_get\")\n\n\napp.include_router(router)\n\n\nif __name__ == \"__main__\":\n    LOGGING_CONFIG[\"handlers\"][\"access\"][\"filters\"] = [asgi_correlation_id.CorrelationIdFilter()]\n    LOGGING_CONFIG[\"formatters\"][\"access\"][\"fmt\"] = \"%(levelname)s access [%(correlation_id)s] %(name)s %(message)s\"\n    uvicorn.run(\"test:app\", port=8080, log_level=os.environ.get(\"LOGLEVEL\", \"DEBUG\").lower())\n```\n\n```\n# run it\npython test.py\n\n# test it\ncurl http://localhost:8080/test\n\n# log on stdout\nINFO log [16b61d57f9ff4a85ac80f5cd406e0aa2] root test_get\nINFO access [16b61d57f9ff4a85ac80f5cd406e0aa2] uvicorn.access 127.0.0.1:24810 - \"GET /test HTTP/1.1\" 200\n```\n\n# Extensions\n\nIn addition to the middleware, we've added a couple of extensions for third-party packages.\n\n## Sentry\n\nIf your project has [sentry-sdk](https://pypi.org/project/sentry-sdk/)\ninstalled, correlation IDs will automatically be added to Sentry events as a `transaction_id`.\n\nSee\nthis [blogpost](https://blog.sentry.io/2019/04/04/trace-errors-through-stack-using-unique-identifiers-in-sentry#1-generate-a-unique-identifier-and-set-as-a-sentry-tag-on-issuing-service)\nfor a little bit of detail. The transaction ID is displayed in the event detail view in Sentry and is just an easy way\nto connect logs to a Sentry event.\n\n## Celery\n\n\u003e Note: If you're using the celery integration, install the package with `pip install asgi-correlation-id[celery]`\n\nFor Celery user's there's one primary issue: workers run as completely separate processes, so correlation IDs are lost\nwhen spawning background tasks from requests.\n\nHowever, with some Celery signal magic, we can actually transfer correlation IDs to worker processes, like this:\n\n```python\n@before_task_publish.connect()\ndef transfer_correlation_id(headers) -\u003e None:\n    # This is called before task.delay() finishes\n    # Here we're able to transfer the correlation ID via the headers kept in our backend\n    headers[header_key] = correlation_id.get()\n\n\n@task_prerun.connect()\ndef load_correlation_id(task) -\u003e None:\n    # This is called when the worker picks up the task\n    # Here we're able to load the correlation ID from the headers\n    id_value = task.request.get(header_key)\n    correlation_id.set(id_value)\n```\n\nTo configure correlation ID transfer, simply import and run the setup function the package provides:\n\n```python\nfrom asgi_correlation_id.extensions.celery import load_correlation_ids\n\nload_correlation_ids()\n```\n\n### Taking it one step further - Adding Celery tracing IDs\n\nIn addition to transferring request IDs to Celery workers, we've added one more log filter for improving tracing in\ncelery processes. This is completely separate from correlation ID functionality, but is something we use ourselves, so\nkeep in the package with the rest of the signals.\n\nThe log filter adds an ID, `celery_current_id` for each worker process, and an ID, `celery_parent_id` for the process\nthat spawned it.\n\nHere's a quick summary of outputs from different scenarios:\n\n| Scenario                                           | Correlation ID     | Celery Current ID | Celery Parent ID |\n|------------------------------------------          |--------------------|-------------------|------------------|\n| Request                                            | ✅                |                   |                  |\n| Request -\u003e Worker                                  | ✅                | ✅               |                  |\n| Request -\u003e Worker -\u003e Another worker                | ✅                | ✅               | ✅              |\n| Beat -\u003e Worker     | ✅*               | ✅                |                   |                  |\n| Beat -\u003e Worker -\u003e Worker     | ✅*     | ✅                | ✅               | ✅              |\n\n*When we're in a process spawned separately from an HTTP request, a correlation ID is still spawned for the first\nprocess in the chain, and passed down. You can think of the correlation ID as an origin ID, while the combination of\ncurrent and parent-ids as a way of linking the chain.\n\nTo add the current and parent IDs, just alter your `celery.py` to this:\n\n```diff\n+ from asgi_correlation_id.extensions.celery import load_correlation_ids, load_celery_current_and_parent_ids\n\nload_correlation_ids()\n+ load_celery_current_and_parent_ids()\n```\n\nIf you wish to correlate celery task IDs through the IDs found in your broker (i.e., the celery `task_id`), use the `use_internal_celery_task_id` argument on `load_celery_current_and_parent_ids`\n```diff\nfrom asgi_correlation_id.extensions.celery import load_correlation_ids, load_celery_current_and_parent_ids\n\nload_correlation_ids()\n+ load_celery_current_and_parent_ids(use_internal_celery_task_id=True)\n```\nNote: `load_celery_current_and_parent_ids` will ignore the `generator` argument when `use_internal_celery_task_id` is set to `True`\n\nTo set up the additional log filters, update your log config like this:\n\n```diff\nLOGGING = {\n    'version': 1,\n    'disable_existing_loggers': False,\n    'filters': {\n        'correlation_id': {\n+           '()': 'asgi_correlation_id.CorrelationIdFilter',\n+           'uuid_length': 32,\n+           'default_value': '-',\n+       },\n+       'celery_tracing': {\n+            '()': 'asgi_correlation_id.CeleryTracingIdsFilter',\n+            'uuid_length': 32,\n+            'default_value': '-',\n+       },\n    },\n    'formatters': {\n        'web': {\n            'class': 'logging.Formatter',\n            'datefmt': '%H:%M:%S',\n            'format': '%(levelname)s ... [%(correlation_id)s] %(name)s %(message)s',\n        },\n+       'celery': {\n+           'class': 'logging.Formatter',\n+           'datefmt': '%H:%M:%S',\n+           'format': '%(levelname)s ... [%(correlation_id)s] [%(celery_parent_id)s-%(celery_current_id)s] %(name)s %(message)s',\n+       },\n    },\n    'handlers': {\n        'web': {\n            'class': 'logging.StreamHandler',\n            'filters': ['correlation_id'],\n            'formatter': 'web',\n        },\n+       'celery': {\n+           'class': 'logging.StreamHandler',\n+           'filters': ['correlation_id', 'celery_tracing'],\n+           'formatter': 'celery',\n+       },\n    },\n    'loggers': {\n        'my_project': {\n+           'handlers': ['celery' if any('celery' in i for i in sys.argv) else 'web'],\n            'level': 'DEBUG',\n            'propagate': True,\n        },\n    },\n}\n```\n\nWith these IDs configured you should be able to:\n\n1. correlate all logs from a single origin, and\n2. piece together the order each log was run, and which process spawned which\n\n#### Example\n\nWith everything configured, assuming you have a set of tasks like this:\n\n```python\n@celery.task()\ndef debug_task() -\u003e None:\n    logger.info('Debug task 1')\n    second_debug_task.delay()\n    second_debug_task.delay()\n\n\n@celery.task()\ndef second_debug_task() -\u003e None:\n    logger.info('Debug task 2')\n    third_debug_task.delay()\n    fourth_debug_task.delay()\n\n\n@celery.task()\ndef third_debug_task() -\u003e None:\n    logger.info('Debug task 3')\n    fourth_debug_task.delay()\n    fourth_debug_task.delay()\n\n\n@celery.task()\ndef fourth_debug_task() -\u003e None:\n    logger.info('Debug task 4')\n```\n\nyour logs could look something like this:\n\n```\n   correlation-id               current-id\n          |        parent-id        |\n          |            |            |\nINFO [3b162382e1] [    -     ] [93ddf3639c] project.tasks - Debug task 1\nINFO [3b162382e1] [93ddf3639c] [24046ab022] project.tasks - Debug task 2\nINFO [3b162382e1] [93ddf3639c] [cb5595a417] project.tasks - Debug task 2\nINFO [3b162382e1] [24046ab022] [08f5428a66] project.tasks - Debug task 3\nINFO [3b162382e1] [24046ab022] [32f40041c6] project.tasks - Debug task 4\nINFO [3b162382e1] [cb5595a417] [1c75a4ed2c] project.tasks - Debug task 3\nINFO [3b162382e1] [08f5428a66] [578ad2d141] project.tasks - Debug task 4\nINFO [3b162382e1] [cb5595a417] [21b2ef77ae] project.tasks - Debug task 4\nINFO [3b162382e1] [08f5428a66] [8cad7fc4d7] project.tasks - Debug task 4\nINFO [3b162382e1] [1c75a4ed2c] [72a43319f0] project.tasks - Debug task 4\nINFO [3b162382e1] [1c75a4ed2c] [ec3cf4113e] project.tasks - Debug task 4\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnok%2Fasgi-correlation-id","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsnok%2Fasgi-correlation-id","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnok%2Fasgi-correlation-id/lists"}