{"id":18799426,"url":"https://github.com/messa/aiohttp-request-id-logging","last_synced_at":"2025-04-13T17:34:01.163Z","repository":{"id":136842946,"uuid":"221445688","full_name":"messa/aiohttp-request-id-logging","owner":"messa","description":"Add [request id] to Python logging records for the Aiohttp server.","archived":false,"fork":false,"pushed_at":"2024-12-04T15:37:57.000Z","size":42,"stargazers_count":12,"open_issues_count":5,"forks_count":1,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-12-23T22:21:34.755Z","etag":null,"topics":["aiohttp","aiohttp-server","correlation-id","logging","python-logging","python3","request-id","sentry"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/messa.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-11-13T11:40:42.000Z","updated_at":"2024-12-19T19:06:33.000Z","dependencies_parsed_at":null,"dependency_job_id":"8f9e713b-4b90-424e-91e8-bdfd574cd65f","html_url":"https://github.com/messa/aiohttp-request-id-logging","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/messa%2Faiohttp-request-id-logging","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/messa%2Faiohttp-request-id-logging/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/messa%2Faiohttp-request-id-logging/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/messa%2Faiohttp-request-id-logging/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/messa","download_url":"https://codeload.github.com/messa/aiohttp-request-id-logging/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":231487791,"owners_count":18384253,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aiohttp","aiohttp-server","correlation-id","logging","python-logging","python3","request-id","sentry"],"created_at":"2024-11-07T22:15:16.197Z","updated_at":"2025-04-13T17:34:01.148Z","avatar_url":"https://github.com/messa.png","language":"Python","readme":"aiohttp-request-id-logging\n==========================\n\nMotivation\n----------\n\nWhen logging from an async application (e.g. aiohttp web application),\nlog messages from different async tasks, http requests etc. are intertwined and you cannot surely tell which line was generated by what request.\nFor example:\n\n```\n2020-01-15 15:35:37,501  INFO: Processing transfer id 1234\n2020-01-15 15:35:37,976  INFO: Processing transfer id 5678\n2020-01-15 15:35:38,201 ERROR: Oh no, something bad has happened! Cannot finish the transfer.\n2020-01-15 15:35:38,504  INFO: 127.0.0.1 [15/Jan/2020:14:35:36 +0000] \"POST / HTTP/1.1\" 200 165 \"-\" \"curl/7.68.0\"\n2020-01-15 15:35:38,982  INFO: 127.0.0.1 [15/Jan/2020:14:35:36 +0000] \"POST / HTTP/1.1\" 500 165 \"-\" \"curl/7.68.0\"\n```\n\nSo, which transfer has failed? The one with id 1234, or the one with id 5678?\n\nOf course, ideally the error message should contain also the transfer id.\nBut it is not always this case, or you may want to inspect also other log messages that were logged by that specific task/request.\n\nWhen you start to use this library in your aiohttp web application, this is how your log messages will look like:\n\n```\n2020-01-15 15:58:47,238  INFO: [req:O5bvIlU] Processing GET / (__main__:hello)\n2020-01-15 15:58:47,950  INFO: [req:xtMacpA] Processing GET / (__main__:hello)\n2020-01-15 15:58:48,240  INFO: [req:O5bvIlU] Processing transfer id 1234\n2020-01-15 15:58:48,953  INFO: [req:xtMacpA] Processing transfer id 5678\n2020-01-15 15:58:49,182 ERROR: [req:xtMacpA] Oh no, something bad has happened! Cannot finish the transfer.\n2020-01-15 15:58:49,242  INFO: [req:O5bvIlU] 127.0.0.1 \"GET / HTTP/1.1\" 200 165 \"-\" \"curl/7.68.0\"\n2020-01-15 15:58:49,959  INFO: [req:xtMacpA] 127.0.0.1 \"GET / HTTP/1.1\" 500 165 \"-\" \"curl/7.68.0\"\n```\n\n\nInstallation\n------------\n\n```shell\n$ python3 -m pip install https://github.com/messa/aiohttp-request-id-logging/archive/v0.0.6.zip\n```\n\nOr add this line to `requirements.txt`:\n\n```\naiohttp-request-id-logging @ https://github.com/messa/aiohttp-request-id-logging/archive/v0.0.6.zip\n```\n\n\nUsage\n-----\n\n```python\nfrom aiohttp.web import Application, Response, RouteTableDef, run_app\nfrom aiohttp.web_log import AccessLogger\nfrom aiohttp_request_id_logging import (\n    setup_logging_request_id_prefix,\n    request_id_middleware,\n    RequestIdContextAccessLogger)\n\nroutes = RouteTableDef()\n\n@routes.get('/')\nasync def hello(request):\n    return Response(text=\"Hello, world!\\n\")\n\nlogging.basicConfig(\n    level=logging.DEBUG,\n    format='%(asctime)s [%(threadName)s] %(name)-26s %(levelname)5s: %(requestIdPrefix)s%(message)s')\n\nsetup_logging_request_id_prefix()\n\napp = Application(middlewares=[request_id_middleware()])\napp.router.add_routes(routes)\n\nrun_app(app, access_log_class=RequestIdContextAccessLogger)\n```\n\nFor more complete example see [demo.py](demo.py).\n\n\nHow it works\n------------\n\nThis library helps you to add request (correlation) id to the log messages in a few steps:\n\n1. **`request_id_middleware()`:** Generate random `request_id` for each aiohttp request and\n\n   - store it in a ContextVar `aiohttp_request_id_logging.request_id`\n   - store it also in `request['request_id']`\n\n2. **`setup_logging_request_id_prefix()`:** Modify logging record factory so that the request_id is attached to every logging record created\n\n   - so you should modify your log format, for example `logging.basicConfig(format=... %(levelname)5s: %(requestIdPrefix)s%(message)s')`\n\n3. Because the aiohttp access logging happens out of the middleware scope, the request id ContextVar would be already resetted. So **`RequestIdContextAccessLogger`** is provided that adds the request_id to the access log message.\n\n4. If you use **[Sentry](https://docs.sentry.io/platforms/python/aiohttp/)**, a `request_id` [tag](https://docs.sentry.io/enriching-error-data/context/?platform=python#tagging-events) is added when the request is processed.\n\nSentry integration will be active only if you have `sentry_sdk` installed.\n\n\nAlternatives\n------------\n\n- TODO: find other solutions to this problem, perhaps also from different frameworks or platforms\n\n\nLinks\n-----\n\nIn general, you should be familiar with Aiohttp documentation related to production deployment:\n\n- https://docs.aiohttp.org/en/stable/web_advanced.html#deploying-behind-a-proxy\n- https://docs.aiohttp.org/en/stable/deployment.html\n\nThis is where this started: https://stackoverflow.com/a/58801740/196206\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmessa%2Faiohttp-request-id-logging","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmessa%2Faiohttp-request-id-logging","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmessa%2Faiohttp-request-id-logging/lists"}