{"id":15587990,"url":"https://github.com/david-lor/logging-requests-poc","last_synced_at":"2026-04-27T23:36:07.107Z","repository":{"id":104269076,"uuid":"236299546","full_name":"David-Lor/Logging-Requests-POC","owner":"David-Lor","description":"POC of an API with per-request contextualized logs","archived":false,"fork":false,"pushed_at":"2020-02-28T19:02:34.000Z","size":337,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-29T09:18:07.403Z","etag":null,"topics":["api","context","context-vars","contextlib","fastapi","logging","logging-example","logs","loguru","python","python3"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/David-Lor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-01-26T10:56:09.000Z","updated_at":"2020-02-28T19:02:17.000Z","dependencies_parsed_at":null,"dependency_job_id":"95971175-cf5c-427d-82f8-4393fdb9afa4","html_url":"https://github.com/David-Lor/Logging-Requests-POC","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/David-Lor/Logging-Requests-POC","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/David-Lor%2FLogging-Requests-POC","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/David-Lor%2FLogging-Requests-POC/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/David-Lor%2FLogging-Requests-POC/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/David-Lor%2FLogging-Requests-POC/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/David-Lor","download_url":"https://codeload.github.com/David-Lor/Logging-Requests-POC/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/David-Lor%2FLogging-Requests-POC/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32360114,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-27T20:07:02.737Z","status":"ssl_error","status_checked_at":"2026-04-27T20:07:00.910Z","response_time":128,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","context","context-vars","contextlib","fastapi","logging","logging-example","logs","loguru","python","python3"],"created_at":"2024-10-02T22:20:43.455Z","updated_at":"2026-04-27T23:36:07.089Z","avatar_url":"https://github.com/David-Lor.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Logging Requests POC\n\n![Test](https://github.com/David-Lor/Logging-Requests-POC/workflows/Test/badge.svg)\n\n## Introduction\n\n### TL;DR\n\n- Having an API\n- I want to add logging to all the functions involved in resolving the requests\n- I need a way to identify to which individual request each log record corresponds\n- Python natively features [context variables](https://docs.python.org/3/library/contextvars.html)\n- But I will be using [loguru](https://github.com/Delgan/loguru), that already implements them on its logging system\n- Additionally, I want to persist all the log records for failed requests on a database\n\n### Full description\n\nHaving a service based on request handling (like a web service, a Telegram bot or an API), we want to group\nlog records identified by the unique request they were caused by.\n\nThe common workflow on these type of services usually are based on a request handler (function that receives the \nrequest and returns a response), and multiple misc functions (like services or helpers) called from the main\nrequest handler to fulfill the request (like accessing a database, an external API/service, and so on).\n\nAll these functions (the main request handler and all the services and helpers involved) have their own\nlog records. But, as the service is usually async (so it can process multiple requests \"at the same time\"), we can end\nhaving mixed log records for multiple requests, without knowing that record correspond to which individual request.\n\nWhat we need is to track the context of each request, by appending a unique request identifier to each individual\nrequest. While in Python we can use the built-in [context variables](https://docs.python.org/3/library/contextvars.html)\nto do so, we will use the [loguru](https://github.com/Delgan/loguru) logging library, which already have a built-in\ncontext system to transparently append context variables to the log records.\n\nThe main advantage of context variables is that they are created on the main request handler, and all the functions\ncalled from the request handler can acquire their value. They do not required to be passed as function arguments.\n\n![Context Variables Schema](docs/ContextVariables.png)\n\n## Technical parts\n\n### API\n\nAs a demonstration, we have a simple API built with FastAPI, which serves endpoints to perform CRUD operations on an\nentity called User. Users are stored on a MongoDB database (where the log records will be stored as well).\n\n#### Endpoints\n\n- GET /users\n- GET /users/{user_id}\n- DELETE /users/{user_id}\n- POST /users - body:\n  ```json\n    {\n      \"username\": \"foo\"\n    }\n  ```\n\n_On [tools](tools) you can find scripts to POST successful and failing requests to create users_\n\n#### Forcing an exception\n\n- The API will throw an exception (resulting on a 500 status code response, thus a failed request) when trying to POST\n  a User where the \"username\" field contains the word \"fail\".\n- The API will record a WARNING on the logger when trying to POST a User where the \"username\" field contains the word \"warn\".\n\n### Logging\n\nWe work with two different logs:\n- A System Logger, used during initialization of the API and events related with it. Records are sent to sys.stderr.\n- A Request Logger, for functions called while handling a request (the main handler and all the services and helpers)\n\n_Since Loguru works with a single logger, we create \"virtual\" loggers by binding a context variable \"logger_name\"\nto records (more information on the [logger.py](logging_requests_poc/logger.py) file)._\n\n#### Request Log Handler (persisting failed requests log records)\n\nNow that we have our request log records identified by each individual request, we want something else.\nWhen a request fails, we want to persist ALL the log records of that request on a MongoDB collection.\n\nFor this, we create a custom log handling function, for each request log record, with the following logic inside:\n\n![Log Record Persistence Schema](docs/LogRecordPersistenceSchema.png)\n\nAdditionally, we can customize at which level the request record list will be stored (maybe not at ERROR, but at WARNING).\n\n### Libraries used\n\n- **fastapi** \u0026 **uvicorn** for the API\n- **loguru** for the logging\n- **pymongo** as the MongoDB connector (we should be using an async connector)\n- **ward**, a new testing library (just for trying it out)\n- **pydantic** for dataclasses (used by fastapi)\n- **python-dotenv** \u0026 **dotenv-settings-handler** to load and handle settings through environment variables or .env file\n\n## Running\n\n```bash\ngit clone https://github.com/David-Lor/Logging-Requests-POC\ncd Logging-Requests-POC\n\npip install -r requirements.txt\npython .\n\n# running tests\nward\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavid-lor%2Flogging-requests-poc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdavid-lor%2Flogging-requests-poc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavid-lor%2Flogging-requests-poc/lists"}