{"id":15297481,"url":"https://github.com/jteppinette/python-logfmter","last_synced_at":"2025-04-03T02:11:04.256Z","repository":{"id":62576671,"uuid":"455972874","full_name":"jteppinette/python-logfmter","owner":"jteppinette","description":"Using the stdlib logging module and without changing a single logging call, logfmter supports global (first and third party) logfmt structured logging.","archived":false,"fork":false,"pushed_at":"2024-03-01T13:51:11.000Z","size":56,"stargazers_count":43,"open_issues_count":4,"forks_count":3,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-05-02T22:40:05.170Z","etag":null,"topics":["log","logfmt","logging","python","python-3","structured","structured-logging"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/logfmter/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jteppinette.png","metadata":{"files":{"readme":"README.md","changelog":"HISTORY.md","contributing":null,"funding":"FUNDING.yml","license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"jteppinette"}},"created_at":"2022-02-05T20:02:03.000Z","updated_at":"2024-06-19T12:12:45.483Z","dependencies_parsed_at":"2024-06-19T12:12:43.744Z","dependency_job_id":"7c80f0d4-c798-4cde-8d02-c7d41474b5bc","html_url":"https://github.com/jteppinette/python-logfmter","commit_stats":{"total_commits":42,"total_committers":2,"mean_commits":21.0,"dds":0.09523809523809523,"last_synced_commit":"89b4d16d0545d84a7f8aa6a6ca0d038d1b4fc998"},"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jteppinette%2Fpython-logfmter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jteppinette%2Fpython-logfmter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jteppinette%2Fpython-logfmter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jteppinette%2Fpython-logfmter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jteppinette","download_url":"https://codeload.github.com/jteppinette/python-logfmter/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246922247,"owners_count":20855345,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["log","logfmt","logging","python","python-3","structured","structured-logging"],"created_at":"2024-09-30T19:17:47.483Z","updated_at":"2025-04-03T02:11:04.233Z","avatar_url":"https://github.com/jteppinette.png","language":"Python","readme":"![python logfmter](./banner.png)\n\n\u003cdiv align=\"center\"\u003e\n\n[![pre-commit](https://github.com/jteppinette/python-logfmter/actions/workflows/pre-commit.yml/badge.svg)](https://github.com/jteppinette/python-logfmter/actions/workflows/pre-commit.yml)\n[![test](https://github.com/jteppinette/python-logfmter/actions/workflows/test.yml/badge.svg)](https://github.com/jteppinette/python-logfmter/actions/workflows/test.yml)\n[![python-3.9-3.10-3.11-3.12-3.13](https://img.shields.io/badge/python-3.9%20|%203.10%20|%203.11|%203.12|%203.13-blue.svg)](.github/workflows/test.yml)\n\nAdd [logfmt](https://www.brandur.org/logfmt) structured logging using the stdlib logging module and without changing a single log call.\n\n\u003c/div\u003e\n\n```python\n\u003e logging.warn(\"user created\", extra=user)\n\nat=WARNING msg=\"user created\" first_name=John last_name=Doe age=25\n```\n\n# Table of Contents\n\n1. [Why](#why)\n2. [Install](#install)\n3. [Usage](#usage)\n   1. [Integration](#integration)\n   2. [Configuration](#configuration)\n   3. [Extension](#extension)\n   4. [Guides](#guides)\n4. [Development](#development)\n   1. [Required Software](#required-software)\n   2. [Getting Started](#getting-started)\n   3. [Publishing](#publishing)\n\n# Why\n\n- enables both human and computer readable logs, [recommended as a \"best practice\" by Splunk](https://dev.splunk.com/enterprise/docs/developapps/addsupport/logging/loggingbestpractices/)\n- formats all first and third party logs, you never have to worry about a library using a different logging format\n- simple to integrate into any existing application, requires no changes to existing log statements i.e. [structlog](https://github.com/hynek/structlog)\n\n# Install\n\n```sh\n$ pip install logfmter\n```\n\n# Usage\n\nThis package exposes a single `Logfmter` class that can be integrated into\nthe [standard library logging system](https://docs.python.org/3/howto/logging.html) like any [`logging.Formatter`](https://docs.python.org/3/howto/logging.html#formatters).\n\n## Integration\n\n**[basicConfig](https://docs.python.org/3/library/logging.html#logging.basicConfig)**\n\n```python\nimport logging\nfrom logfmter import Logfmter\n\nhandler = logging.StreamHandler()\nhandler.setFormatter(Logfmter())\n\nlogging.basicConfig(handlers=[handler])\n\nlogging.error(\"hello\", extra={\"alpha\": 1}) # at=ERROR msg=hello alpha=1\nlogging.error({\"token\": \"Hello, World!\"}) # at=ERROR token=\"Hello, World!\"\n```\n\n**[dictConfig](https://docs.python.org/3/library/logging.config.html#logging.config.dictConfig)**\n\n_If you are using `dictConfig`, you need to consider your setting\nof `disable_existing_loggers`. It is enabled by default, and causes\nany third party module loggers to be disabled._\n\n```python\nimport logging.config\n\nlogging.config.dictConfig(\n    {\n        \"version\": 1,\n        \"formatters\": {\n            \"logfmt\": {\n                \"()\": \"logfmter.Logfmter\",\n            }\n        },\n        \"handlers\": {\n            \"console\": {\"class\": \"logging.StreamHandler\", \"formatter\": \"logfmt\"}\n        },\n        \"loggers\": {\"\": {\"handlers\": [\"console\"], \"level\": \"INFO\"}},\n    }\n)\n\nlogging.info(\"hello\", extra={\"alpha\": 1}) # at=INFO msg=hello alpha=1\n```\n\n_Notice, you can configure the `Logfmter` by providing keyword arguments as dictionary\nitems after `\"()\"`:_\n\n```python\n...\n\n    \"logfmt\": {\n        \"()\": \"logfmter.Logfmter\",\n        \"keys\": [...],\n        \"mapping\": {...}\n    }\n\n...\n```\n\n**[fileConfig](https://docs.python.org/3/library/logging.config.html#logging.config.fileConfig)**\n\nUsing logfmter via fileConfig is not supported, because fileConfig does not support custom formatter initialization. There may be some hacks to make this work in the future. Let me know if you have ideas or really need this.\n\n## Configuration\n\n**keys**\n\nBy default, the `at=\u003clevelname\u003e` key/value will be included in all log messages. These\ndefault keys can be overridden using the `keys` parameter. If the key you want to include\nin your output is represented by a different attribute on the log record, then you can\nuse the `mapping` parameter to provide that key/attribute mapping.\n\nReference the Python [`logging.LogRecord` Documentation](https://docs.python.org/3/library/logging.html?highlight=logrecord#logging.LogRecord)\nfor a list of available attributes.\n\n```python\nimport logging\nfrom logfmter import Logfmter\n\nformatter = Logfmter(keys=[\"at\", \"processName\"])\n\nhandler = logging.StreamHandler()\nhandler.setFormatter(formatter)\n\nlogging.basicConfig(handlers=[handler])\n\nlogging.error(\"hello\") # at=ERROR processName=MainProceess msg=hello\n```\n\n**mapping**\n\nBy default, a mapping of `{\"at\": \"levelname\"}` is used to allow the `at` key to reference\nthe log record's `levelname` attribute. You can override this parameter to provide your\nown mappings.\n\n```python\nimport logging\nfrom logfmter import Logfmter\n\nformatter = Logfmter(\n    keys=[\"at\", \"process\"],\n    mapping={\"at\": \"levelname\", \"process\": \"processName\"}\n)\n\nhandler = logging.StreamHandler()\nhandler.setFormatter(formatter)\n\nlogging.basicConfig(handlers=[handler])\n\nlogging.error(\"hello\") # at=ERROR process=MainProceess msg=hello\n```\n\n**datefmt**\n\nIf you request the `asctime` attribute (directly or through a mapping), then the date format\ncan be overridden through the `datefmt` parameter.\n\n```python\nimport logging\nfrom logfmter import Logfmter\n\nformatter = Logfmter(\n    keys=[\"at\", \"when\"],\n    mapping={\"at\": \"levelname\", \"when\": \"asctime\"},\n    datefmt=\"%Y-%m-%d\"\n)\n\nhandler = logging.StreamHandler()\nhandler.setFormatter(formatter)\n\nlogging.basicConfig(handlers=[handler])\n\nlogging.error(\"hello\") # at=ERROR when=2022-04-20 msg=hello\n```\n\n## Extension\n\nYou can subclass the formatter to change its behavior.\n\n```python\nimport logging\nfrom logfmter import Logfmter\n\n\nclass CustomLogfmter(Logfmter):\n    \"\"\"\n    Provide a custom logfmt formatter which formats\n    booleans as \"yes\" or \"no\" strings.\n    \"\"\"\n\n    @classmethod\n    def format_value(cls, value):\n        if isinstance(value, bool):\n            return \"yes\" if value else \"no\"\n\n\treturn super().format_value(value)\n\nhandler = logging.StreamHandler()\nhandler.setFormatter(CustomLogfmter())\n\nlogging.basicConfig(handlers=[handler])\n\nlogging.error({\"example\": True}) # at=ERROR example=yes\n```\n\n## Guides\n\n**Default Key/Value Pairs**\n\nInstead of providing key/value pairs at each log call, you can override\nthe log record factory to provide defaults:\n\n```py\n_record_factory = logging.getLogRecordFactory()\n\ndef record_factory(*args, **kwargs):\n    record = _record_factory(*args, **kwargs)\n    record.trace_id = 123\n    return record\n\nlogging.setLogRecordFactory(record_factory)\n```\n\nThis will cause all logs to have the `trace_id=123` pair regardless of including\n`trace_id` in keys or manually adding `trace_id` to the `extra` parameter or the `msg` object.\n\n# Development\n\n## Required Software\n\nIf you are using [nix](https://zero-to-nix.com/start/install/) \u0026 [direnv](https://direnv.net/docs/installation.html), then your dev environment will be managed automatically. Otherwise, you will need to manually install the following software:\n\n- [direnv](https://direnv.net)\n- [git](https://git-scm.com/)\n- [pyenv](https://github.com/pyenv/pyenv#installation)\n\n## Getting Started\n\n**Setup**\n\n\u003e If you are using pyenv, you will need to install the correct versions of python using `\u003cruntimes.txt xargs -n 1 pyenv install -s`.\n\n```sh\n$ direnv allow\n$ pip install -r requirements/dev.txt\n$ pre-commit install\n$ pip install -e .\n```\n\n**Tests**\n\n_Run the test suite against the active python environment._\n\n```sh\n$ pytest\n```\n\n_Run the test suite against the active python environment and\nwatch the codebase for any changes._\n\n```sh\n$ ptw\n```\n\n_Run the test suite against all supported python versions._\n\n```sh\n$ tox\n```\n\n## Publishing\n\n**Create**\n\n1. Update the version number in `logfmter/__init__.py`.\n\n2. Add an entry in `HISTORY.md`.\n\n3. Commit the changes, tag the commit, and push the tags:\n\n   ```sh\n   $ git commit -am \"v\u003cmajor\u003e.\u003cminor\u003e.\u003cpatch\u003e\"\n   $ git tag v\u003cmajor\u003e.\u003cminor\u003e.\u003cpatch\u003e\n   $ git push origin main --tags\n   ```\n\n4. Convert the tag to a release in GitHub with the history\n   entry as the description.\n\n**Build**\n\n```sh\n$ python -m build\n```\n\n**Upload**\n\n```\n$ twine upload dist/*\n```\n","funding_links":["https://github.com/sponsors/jteppinette"],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjteppinette%2Fpython-logfmter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjteppinette%2Fpython-logfmter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjteppinette%2Fpython-logfmter/lists"}