{"id":27947908,"url":"https://github.com/iloveitaly/structlog-config","last_synced_at":"2026-02-05T01:01:17.642Z","repository":{"id":282969351,"uuid":"950246852","full_name":"iloveitaly/structlog-config","owner":"iloveitaly","description":"A comprehensive structlog configuration with sensible defaults for development and production environments, featuring context management, exception formatting, and path prettification.","archived":false,"fork":false,"pushed_at":"2026-01-28T18:24:22.000Z","size":463,"stargazers_count":9,"open_issues_count":6,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-01-29T08:53:04.846Z","etag":null,"topics":["json-logging","logging","structlog","structured-logging"],"latest_commit_sha":null,"homepage":"https://github.com/iloveitaly/structlog-config","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/iloveitaly.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":["iloveitaly"]}},"created_at":"2025-03-17T21:33:06.000Z","updated_at":"2026-01-28T18:24:31.000Z","dependencies_parsed_at":"2025-05-07T14:44:54.514Z","dependency_job_id":"b7861ac6-b2aa-47f5-b382-732d70e5fe66","html_url":"https://github.com/iloveitaly/structlog-config","commit_stats":null,"previous_names":["iloveitaly/structlog_config","iloveitaly/structlog-config"],"tags_count":10,"template":false,"template_full_name":null,"purl":"pkg:github/iloveitaly/structlog-config","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iloveitaly%2Fstructlog-config","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iloveitaly%2Fstructlog-config/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iloveitaly%2Fstructlog-config/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iloveitaly%2Fstructlog-config/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/iloveitaly","download_url":"https://codeload.github.com/iloveitaly/structlog-config/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iloveitaly%2Fstructlog-config/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29105268,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-05T00:52:08.035Z","status":"ssl_error","status_checked_at":"2026-02-05T00:52:07.703Z","response_time":62,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["json-logging","logging","structlog","structured-logging"],"created_at":"2025-05-07T14:39:03.952Z","updated_at":"2026-02-05T01:01:17.625Z","avatar_url":"https://github.com/iloveitaly.png","language":"Python","readme":"# Opinionated Defaults for Structlog\n\nLogging is really important. Getting logging to work well in python feels like black magic: there's a ton of configuration\nacross structlog, warnings, std loggers, fastapi + celery context, JSON logging in production, etc that requires lots of\nfiddling and testing to get working. I finally got this working for me in my [project template](https://github.com/iloveitaly/python-starter-template) and extracted this out into a nice package.\n\nHere are the main goals:\n\n* High performance JSON logging in production\n* All loggers, even plugin or system loggers, should route through the same formatter\n* Structured logging everywhere\n* Pytest plugin to easily capture logs and dump to a directory on failure. This is really important for LLMs so they can\n  easily consume logs and context for each test and handle them sequentially.\n* Ability to easily set thread-local log context\n* Nice log formatters for stack traces, ORM ([ActiveModel/SQLModel](https://github.com/iloveitaly/activemodel)), etc\n* Ability to log level and output (i.e. file path) *by logger* for easy development debugging\n* If you are using fastapi, structured logging for access logs\n* [Improved exception logging with beautiful-traceback](https://github.com/iloveitaly/beautiful-traceback)\n\n## Installation\n\n```bash\nuv add structlog-config\n```\n\n## Usage\n\n```python\nfrom structlog_config import configure_logger\n\nlog = configure_logger()\n\nlog.info(\"the log\", key=\"value\")\n\n# named logger just like stdlib, but with a different syntax\ncustom_named_logger = structlog.get_logger(logger_name=\"test\")\n```\n\n## JSON Logging in Production\n\nJSON logging is automatically enabled in production and staging environments (`PYTHON_ENV=production` or `PYTHON_ENV=staging`):\n\n```python\nfrom structlog_config import configure_logger\n\n# Automatic JSON logging in production\nlog = configure_logger()\nlog.info(\"User login\", user_id=\"123\", action=\"login\")\n# Output: {\"action\":\"login\",\"event\":\"User login\",\"level\":\"info\",\"timestamp\":\"2025-09-24T18:03:00Z\",\"user_id\":\"123\"}\n\n# Force JSON logging regardless of environment\nlog = configure_logger(json_logger=True)\n\n# Force console logging regardless of environment\nlog = configure_logger(json_logger=False)\n```\n\nJSON logs use [orjson](https://github.com/ijl/orjson) for performance, include sorted keys and ISO timestamps, and serialize exceptions cleanly.\n\nNote that `PYTHON_LOG_PATH` is ignored with JSON logging (stdout only).\n\n## TRACE Logging Level\n\nThis package adds support for a custom `TRACE` logging level (level 5) that's even more verbose than `DEBUG`.\n\nThe `TRACE` level is automatically set up when you call `configure_logger()`. You can use it like any other logging level:\n\n```python\nimport logging\nfrom structlog_config import configure_logger\n\nlog = configure_logger()\n\n# Using structlog\nlog.info(\"This is info\")\nlog.debug(\"This is debug\")\nlog.trace(\"This is trace\")  # Most verbose\n\n# Using stdlib logging\nlogging.trace(\"Module-level trace message\")\nlogger = logging.getLogger(__name__)\nlogger.trace(\"Instance trace message\")\n```\n\nSet the log level to TRACE using the environment variable:\n\n```bash\nLOG_LEVEL=TRACE\n```\n\n## Stdlib Log Management\n\nBy default, all stdlib loggers are:\n\n1. Given the same global logging level, with some default adjustments for noisy loggers (looking at you, `httpx`)\n2. Use a structlog formatter (you get structured logging, context, etc in any stdlib logger calls)\n3. The root processor is overwritten so any child loggers created after initialization will use the same formatter\n\nYou can customize loggers by name (i.e. the name used in `logging.getLogger(__name__)`) using ENV variables.\n\nFor example, if you wanted to [mimic `OPENAI_LOG` functionality](https://github.com/openai/openai-python/blob/de7c0e2d9375d042a42e3db6c17e5af9a5701a99/src/openai/_utils/_logs.py#L16):\n\n* `LOG_LEVEL_OPENAI=DEBUG`\n* `LOG_PATH_OPENAI=tmp/openai.log`\n* `LOG_LEVEL_HTTPX=DEBUG`\n* `LOG_PATH_HTTPX=tmp/openai.log`\n\n## Custom Formatters\n\nThis package includes several custom formatters that automatically clean up log output:\n\n### Path Prettifier\n\nAutomatically formats `pathlib.Path` and `PosixPath` objects to show relative paths when possible, removing the wrapper class names:\n\n```python\nfrom pathlib import Path\nlog.info(\"Processing file\", file_path=Path.cwd() / \"data\" / \"users.csv\")\n# Output: file_path=data/users.csv (instead of PosixPath('/home/user/data/users.csv'))\n```\n\n### Whenever Datetime Formatter\n\nFormats [whenever](https://github.com/ariebovenberg/whenever) datetime objects without their class wrappers for cleaner output:\n\n```python\nfrom whenever import ZonedDateTime\n\nlog.info(\"Event scheduled\", event_time=ZonedDateTime(2025, 11, 2, 0, 0, 0, tz=\"UTC\"))\n# Output: event_time=2025-11-02T00:00:00+00:00[UTC]\n# Instead of: event_time=ZonedDateTime(\"2025-11-02T00:00:00+00:00[UTC]\")\n```\n\nSupports all whenever datetime types: `ZonedDateTime`, `Instant`, `LocalDateTime`, `PlainDateTime`, etc.\n\n### ActiveModel Object Formatter\n\nAutomatically converts [ActiveModel](https://github.com/iloveitaly/activemodel) BaseModel instances to their ID representation and TypeID objects to strings:\n\n```python\nfrom activemodel import BaseModel\n\nuser = User(id=\"user_123\", name=\"Alice\")\nlog.info(\"User action\", user=user)\n# Output: user_id=user_123 (instead of full object representation)\n```\n\n### FastAPI Context\n\nAutomatically includes all context data from [starlette-context](https://github.com/tomwojcik/starlette-context) in your logs, useful for request tracing:\n\n```python\n# Context data (request_id, correlation_id, etc.) automatically included in all logs\nlog.info(\"Processing request\")\n# Output includes: request_id=abc-123 correlation_id=xyz-789 ...\n```\n\nAll formatters are optional and automatically enabled when their respective dependencies are installed. They work seamlessly in both development (console) and production (JSON) logging modes.\n\n## FastAPI Access Logger\n\n**Note:** Requires `pip install structlog-config[fastapi]` for FastAPI dependencies.\n\nStructured, simple access log with request timing to replace the default fastapi access log. Why?\n\n1. It's less verbose\n2. Uses structured logging params instead of string interpolation\n3. debug level logs any static assets\n\nHere's how to use it:\n\n1. [Disable fastapi's default logging.](https://github.com/iloveitaly/python-starter-template/blob/f54cb47d8d104987f2e4a668f9045a62e0d6818a/main.py#L55-L56)\n2. [Add the middleware to your FastAPI app.](https://github.com/iloveitaly/python-starter-template/blob/f54cb47d8d104987f2e4a668f9045a62e0d6818a/app/routes/middleware/__init__.py#L63-L65)\n\n## Pytest Plugin: Capture Output on Failure\n\nA pytest plugin that captures stdout, stderr, and exceptions from failing tests and writes them to organized output files. This is useful for debugging test failures, especially in CI/CD environments where you need to inspect output after the fact.\n\n### Features\n\n- Captures stdout, stderr, and exception tracebacks for failing tests\n- Only creates output for failing tests (keeps directories clean)\n- Separate files for each output type (stdout.txt, stderr.txt, exception.txt)\n- Captures all test phases (setup, call, teardown)\n- Optional fd-level capture for subprocess output\n\n### Usage\n\nEnable the plugin with the `--structlog-output` flag and `-s` (to disable pytest's built-in capture):\n\n```bash\npytest --structlog-output=./test-output -s\n```\n\nThe `--structlog-output` flag both enables the plugin and specifies where output files should be written.\n\n**Recommended:** Also disable pytest's logging plugin with `-p no:logging` to avoid duplicate/interfering capture:\n\n```bash\npytest --structlog-output=./test-output -s -p no:logging\n```\n\nWhile the plugin works without this flag, disabling pytest's logging capture ensures cleaner output and avoids any potential conflicts between the two capture mechanisms.\n\n### Output Structure\n\nEach failing test gets its own directory with separate files:\n\n```\ntest-output/\n    test_module__test_name/\n        stdout.txt      # stdout from test (includes setup, call, and teardown phases)\n        stderr.txt      # stderr from test (includes setup, call, and teardown phases)\n        exception.txt   # exception traceback\n```\n\n### Advanced: fd-level Capture\n\nFor tests that spawn subprocesses or write directly to file descriptors, you can enable fd-level capture. This is useful for integration tests that run external processes (such a server which replicates a production environment).\n\n#### Add fixture to function signature\n\nGreat for a single single test:\n\n```python\ndef test_with_subprocess(file_descriptor_output_capture):\n    # subprocess.run() output will be captured\n    subprocess.run([\"echo\", \"hello from subprocess\"])\n\n    # multiprocessing.Process output will be captured\n    from multiprocessing import Process\n    proc = Process(target=lambda: print(\"hello from process\"))\n    proc.start()\n    proc.join()\n\n    assert False  # Trigger failure to write output files\n```\n\nAlternatively, you can use `@pytest.mark.usefixtures(\"file_descriptor_output_capture\")`\n\n\n#### All tests in directory\n\nAdd to `conftest.py`:\n\n```python\nimport pytest\n\npytestmark = pytest.mark.usefixtures(\"file_descriptor_output_capture\")\n```\n\n### Example\n\nWhen a test fails:\n\n```python\ndef test_user_login():\n    print(\"Starting login process\")\n    print(\"ERROR: Connection failed\", file=sys.stderr)\n    assert False, \"Login failed\"\n```\n\nYou'll get:\n\n```\ntest-output/test_user__test_user_login/\n    stdout.txt: \"Starting login process\"\n    stderr.txt: \"ERROR: Connection failed\"\n    exception.txt: Full traceback with \"AssertionError: Login failed\"\n```\n\n## Beautiful Traceback Support\n\nOptional support for [beautiful-traceback](https://github.com/iloveitaly/beautiful-traceback) provides enhanced exception formatting with improved readability, smart coloring, path aliasing (e.g., `\u003cpwd\u003e`, `\u003csite\u003e`), and better alignment. Automatically activates when installed:\n\n```bash\nuv add beautiful-traceback --group dev\n```\n\nNo configuration needed - just install and `configure_logger()` will use it automatically.\n\n## iPython\n\nOften it's helpful to update logging level within an iPython session. You can do this and make sure all loggers pick up on it.\n\n```\n%env LOG_LEVEL=DEBUG\nfrom structlog_config import configure_logger\nconfigure_logger()\n```\n\n## Related Projects\n\n* https://github.com/underyx/structlog-pretty\n* https://pypi.org/project/httpx-structlog/\n\n## References\n\nGeneral logging:\n\n- https://github.com/replicate/cog/blob/2e57549e18e044982bd100e286a1929f50880383/python/cog/logging.py#L20\n- https://github.com/apache/airflow/blob/4280b83977cd5a53c2b24143f3c9a6a63e298acc/task_sdk/src/airflow/sdk/log.py#L187\n- https://github.com/kiwicom/structlog-sentry\n- https://github.com/jeremyh/datacube-explorer/blob/b289b0cde0973a38a9d50233fe0fff00e8eb2c8e/cubedash/logs.py#L40C21-L40C42\n- https://stackoverflow.com/questions/76256249/logging-in-the-open-ai-python-library/78214464#78214464\n- https://github.com/openai/openai-python/blob/de7c0e2d9375d042a42e3db6c17e5af9a5701a99/src/openai/_utils/_logs.py#L16\n- https://www.python-httpx.org/logging/\n\nFastAPI access logger:\n\n- https://github.com/iloveitaly/fastapi-logger/blob/main/fastapi_structlog/middleware/access_log.py#L70\n- https://github.com/fastapiutils/fastapi-utils/blob/master/fastapi_utils/timing.py\n- https://pypi.org/project/fastapi-structlog/\n- https://pypi.org/project/asgi-correlation-id/\n- https://gist.github.com/nymous/f138c7f06062b7c43c060bf03759c29e\n- https://github.com/sharu1204/fastapi-structlog/blob/master/app/main.py\n","funding_links":["https://github.com/sponsors/iloveitaly"],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Filoveitaly%2Fstructlog-config","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Filoveitaly%2Fstructlog-config","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Filoveitaly%2Fstructlog-config/lists"}