{"id":20333939,"url":"https://github.com/braedon/python-jog","last_synced_at":"2025-08-09T09:27:06.700Z","repository":{"id":144031174,"uuid":"89355109","full_name":"braedon/python-jog","owner":"braedon","description":"JSON Structured Logging for Python","archived":false,"fork":false,"pushed_at":"2017-06-30T02:49:12.000Z","size":9,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-06-06T22:04:32.172Z","etag":null,"topics":["elk","json-logging","logging","logstash","python","python2","python3","structured-logging"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/braedon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-04-25T11:55:55.000Z","updated_at":"2024-12-19T10:54:50.000Z","dependencies_parsed_at":null,"dependency_job_id":"3afba457-7d1d-4ebb-9379-36969b7de601","html_url":"https://github.com/braedon/python-jog","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/braedon/python-jog","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/braedon%2Fpython-jog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/braedon%2Fpython-jog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/braedon%2Fpython-jog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/braedon%2Fpython-jog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/braedon","download_url":"https://codeload.github.com/braedon/python-jog/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/braedon%2Fpython-jog/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261495472,"owners_count":23167321,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elk","json-logging","logging","logstash","python","python2","python3","structured-logging"],"created_at":"2024-11-14T20:35:22.145Z","updated_at":"2025-06-23T14:36:51.155Z","avatar_url":"https://github.com/braedon.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Jog: Python Json Structured Logging\n====\nFormat your python logs as JSON objects, perfect for easy ingestion into centralised logging systems.\n\n# Installation\nInstallation is pretty simple with pip:\n```\n\u003e pip install jog\n```\n\nDepending on your system, you might need to use pip3 to install for Python 3 (ditto for any other pip commands):\n```\n\u003e pip3 install jog\n```\n\n# Usage\nOnce installed, import the `JogFormatter` and configure the logging system to use it. e.g.:\n```\nimport logging\nfrom jog import JogFormatter\n\nlog_handler = logging.StreamHandler()\nlog_format = '[%(asctime)s] %(name)s.%(levelname)s %(threadName)s %(module)s.%(funcName)s %(filename)s:%(lineno)s %(message)s'\nlog_handler.setFormatter(JogFormatter(log_format))\n\nroot_logger = logging.getLogger()\nroot_logger.setLevel(logging.INFO)\nroot_logger.addHandler(log_handler)\n```\n\nThen log as normal:\n```\nlogging.info('foo')\nlogging.info('foo %s', 'bar')\nlogging.info('foo %(baz)s', {'baz': 'bar'})\nlogging.info('foo %(baz)s', {'baz': 'bar'}, extra={'ekki': 'ptang'})\n```\n\nOr log an arbitrary dict:\n```\nlogging.info({'foo': 'bar'})\n```\n\n# Output Format\nHere's an example of a simple log and its output:\n```\nlogging.info('foo')\n```\n```\n{\n    \"@timestamp\": \"2017-05-30T11:25:51.168769+00:00\",\n    \"@version\": 1,\n    \"filename\": \"test.py\",\n    \"funcName\": \"\u003cmodule\u003e\",\n    \"levelname\": \"INFO\",\n    \"levelno\": 20,\n    \"lineno\": 56,\n    \"log\": \"[2017-05-30 23:25:51,168] __main__.INFO MainThread test.\u003cmodule\u003e test.py:56 foo\",\n    \"message\": \"foo\",\n    \"message_format\": \"foo\",\n    \"module\": \"test\",\n    \"name\": \"__main__\",\n    \"pathname\": \"test.py\",\n    \"process\": 7476,\n    \"processName\": \"MainProcess\",\n    \"relativeCreated\": 7.751226425170898,\n    \"thread\": 139963963160320,\n    \"threadName\": \"MainThread\"\n}\n```\nNote that the output has been pretty-printed for ease of examination - each log Json object is actually output on a single line.\n\nThe Json objects produced are primarily a conversion of LogRecords into dicts, which are then rendered as Json. See the [official docs](https://docs.python.org/3/library/logging.html#logrecord-attributes) for details on fields included in a LogRecord. Note that the logging system will merge a `extra` dict into the LogRecord before it is passed to JogFormatter, so `extra` fields are treated as part of the LogRecord.\n\nAfter the conversion of LogRecord to dict a number of transformations are applied.\n\n## String Messages\nA `msg` string is formatted using the `args` (if provided), and the result is put in the `message` field. The unformatted `msg` is put in `message_format`, and the `args` in `message_args`. An unstructured \"traditional\" log is also created using the `fmt` used to create the JogFormatter, and put in the `log` field:\n```\nlogging.info('foo %s', 'bar')\n```\n```\n{\n    ...\n    \"message\": \"foo bar\",\n    \"message_format\": \"foo %s\",\n    \"message_args\": [\"bar\"],\n    \"log\": \"[2017-05-30 22:03:48,217] __main__.INFO MainThread test.\u003cmodule\u003e test.py:45 foo bar\",\n    ...\n}\n```\n\nIf keyword formatting is used, the arguments dict is put in `message_kwargs` instead:\n```\nlogging.info('foo %(baz)s', {'baz': 'bar'})\n```\n```\n{\n    ...\n    \"message\": \"foo bar\",\n    \"message_format\": \"foo %(baz)s\",\n    \"message_kwargs\": {\"baz\": \"bar\"},\n    \"log\": \"[2017-05-30 22:03:48,217] __main__.INFO MainThread test.\u003cmodule\u003e test.py:45 foo bar\",\n    ...\n}\n```\n\n## Dict Messages\nA `msg` dict is merged straight into the log dict. Note that this will override any existing fields from the LogRecord (including fields from `extra`) with the same name.\n```\nlogging.info({'foo': 'bar'})\n```\n```\n{\n    ...\n    \"foo\": \"bar\",\n    ...\n}\n```\n\nNo formatting is done, so attempting to provide `args` along with a dict `msg` will result in a logging error.\n```\nlogging.info({'foo': 'bar'}, 'baz')  # results in a logging error\n```\n\n## Exceptions\nIf a LogRecord contains a `exc_info`, the exception is rendered and put into the `exc_text` field.\n```\ntry:\n    raise Exception('Foo!')\nexcept Exception:\n    logging.exception('bar')\n```\n```\n{\n    ...\n    \"exc_text\": \"Traceback (most recent call last):\\n  File \\\"test.py\\\", line 51, in \u003cmodule\u003e\\n    raise Exception('Foo!')\\nException: Foo!\",\n    ...\n}\n```\n\n## Post Processing Functions\nJogFormatter takes a keyword argument `fn`, which is a function take takes a log dict (produced as described above) and returns the final dict to be rendered as Json, allowing arbitrary changes to be made before the log is written out.\n\nBy default, the provided `jog.elk.format_log` function is used. This function makes a number of changes to make the final Json suitable for ingestion into the ELK centralised logging system.\n\nThe `@version` field is set to `1`, and the `created` timestamp field is removed, converted to ISO format, and put in `@timestamp`. `asctime` and `msecs` are removed as they are not needed with `@timestamp` set. `exc_info` is also removed, as no special formatting is done with it, and `exc_text` is available.\n\nFinally, the dict is cleaned to avoid (statelessly detectable) mapping conflicts in Elasticsearch. Specifically, if a list or tuple's contents are not homogenously a basic Json type (None, str, int|float, bool), the (non None) contents are converted to strings.\n```\nlog.info('foo %(baz)s', {'baz': ['bar', 0, None]})\n```\n```\n{\n    ...\n    \"@version\": 1,\n    \"@timestamp\": \"2017-05-30T11:34:31.901264+00:00\",\n    \"message\": \"foo ['bar', 0, None]\",\n    \"message_kwargs\": {\"baz\": [\"bar\", \"0\", null]},\n    ...\n}\n```\n\n## Json Encoding\nThe final log dict is rendered as Json using the standard `json` library. Unencodable objects are automatically converted to strings to avoid losing logs when they are logged by mistake.\n\n# Development\nTo install directly from the git repo, run the following in the root project directory:\n```\n\u003e pip install .\n```\n\nThe library can be installed in \"editable\" mode, using pip's `-e` flag. This allows you to test out changes without having to re-install.\n```\n\u003e pip install -e .\n```\n\nSend me a PR if you have a change you want to contribute!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbraedon%2Fpython-jog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbraedon%2Fpython-jog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbraedon%2Fpython-jog/lists"}