{"id":20687655,"url":"https://github.com/klauer/python-logging-notes","last_synced_at":"2026-06-06T00:31:15.207Z","repository":{"id":72187210,"uuid":"373321914","full_name":"klauer/python-logging-notes","owner":"klauer","description":"Python logging notes, for when you're looking for additional confusion beyond the standard library docs","archived":false,"fork":false,"pushed_at":"2021-06-04T16:23:10.000Z","size":21,"stargazers_count":1,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-12-13T22:25:43.698Z","etag":null,"topics":["adapters","confusion","filters","handlers","logging","python","python3"],"latest_commit_sha":null,"homepage":"https://klauer.github.io/python-logging-notes/","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/klauer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-06-02T22:48:54.000Z","updated_at":"2024-11-19T14:04:44.000Z","dependencies_parsed_at":"2023-02-23T01:15:21.866Z","dependency_job_id":null,"html_url":"https://github.com/klauer/python-logging-notes","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/klauer/python-logging-notes","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/klauer%2Fpython-logging-notes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/klauer%2Fpython-logging-notes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/klauer%2Fpython-logging-notes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/klauer%2Fpython-logging-notes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/klauer","download_url":"https://codeload.github.com/klauer/python-logging-notes/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/klauer%2Fpython-logging-notes/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33965591,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-05T02:00:06.157Z","response_time":120,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adapters","confusion","filters","handlers","logging","python","python3"],"created_at":"2024-11-16T22:57:54.558Z","updated_at":"2026-06-06T00:31:15.192Z","avatar_url":"https://github.com/klauer.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"## Python Logging Notes\n\nPython logging is confusing.\n\n### Levels\n\n```\nCRITICAL = 50\nFATAL = CRITICAL\nERROR = 40\nWARNING = 30\nWARN = WARNING\nINFO = 20\nDEBUG = 10\nNOTSET = 0\n```\n\nTo pass a message through, a log message must have the following relationship:\n``record.level \u003e= logger.level``\n\nSo:\n1. A logger set to ``DEBUG`` would see ``DEBUG, INFO, WARNING, ...``.\n2. LOWER levels are relatively less important for the user to see.\n3. HIGHER levels are relatively more important for the user to see.\n4. Consider the levels like a \"high water marker\", indicating the nearby lake\n   is nearing overflowing at WARNING (30) and a flood is imminent at CRITICAL\n   (50).\n\n### ``isEnabledFor``\n\n``isEnabledFor`` performs this check. \n\n``isEnabledFor`` works like the following:\n\n1. if ``logger.disabled``, return immediately\n2. Otherwise,\n    1. Check ``manager.disable \u003e= level`` (usually not the case)\n    2. Check ``level \u003e= logger.getEffectiveLevel()``\n\n3. ``logger.getEffectiveLevel()``\n    1. Find the first logger ``[self, self.parent, self.parent.parent, ...]``\n       that has a ``level`` set.\n    2. Return the level found in (a) or fall back to ``NOTSET``.\n\n For performance reasons, the per-level ``isEnabledFor`` information is\n **cached** in a dictionary ``logger._cache[level]``.\n\nThis cache is locked using a **module-level** ``threading.RLock``.\n\n### Level conversion\n\n```python\n\u003e\u003e\u003e logging.getLevelName(10)\n'DEBUG'\n```\n\nBut the same function **reverses** it, too!\n\n```python\n\u003e\u003e\u003e logging.getLevelName(\"DEBUG\")\n10\n```\n\nSo using something like the following would make it consistent:\n\n\n```python\nfrom typing import Union\n\ndef validate_log_level(level: Union[str, int]) -\u003e int:\n    \"\"\"\n    Return a logging level integer for level comparison.\n\n    Parameters\n    ----------\n    level : str or int\n        The logging level string or integer value.\n\n    Returns\n    -------\n    log_level : int\n        The integral log level.\n\n    Raises\n    ------\n    ValueError\n        If the logging level is invalid.\n    \"\"\"\n    if isinstance(level, int):\n        return level\n    if isinstance(level, str):\n        levelno = logging.getLevelName(level)\n\n    if not isinstance(levelno, int):\n        raise ValueError(\"Invalid logging level (use e.g., DEBUG or 6)\")\n\n    return levelno\n```\n\nConsistent numerical values:\n```\n\u003e\u003e\u003e validate_log_level(10)\n10\n\n\u003e\u003e\u003e validate_log_level(\"DEBUG\")\n10\n\n\u003e\u003e\u003e logging.getLevelName(validate_log_level(10))\n'DEBUG'\n```\n\n## Manager\n\nNever hear of the ``logging.Manager``? Me either.\n\nNormally, it appears there is just one ``Manager`` instance.\n\nIt contains the following information:\n\n1. The root logger node (``Logger.root``)\n2. A method to disable logging **globally** below a certain level.\n3. If a no-handler-warning has been emitted (no handler found and no fallback)\n4. ``loggerDict`` - a mapping of ``{logger_name: Logger}``\n5. ``loggerClass`` the default ``Logger`` class to create on ``getLogger``.\n6. ``logRecordFactory`` the factory used to create records\n\n### Global disable logging\n\nWhile the manager takes care of this, the public API is\n\n``logging.disable(level)``\n\nThis also forces a cache clearing, so that loggers have to find their\neffective levels again.\n\n### getLogger\n\n``logging.getLogger`` is a shortcut for ``Logger.manager.getLogger``!\n\n... with the exception of the root logger, that is returned directly\nfrom ``logging.root``.\n\nSome things to note:\n1. You can override the default Logger class by doing ``logging.``\n2. Custom ``Logger`` classes **must** be subclasses of ``Logger``.\n3. ``manager.setLoggerClass`` takes precedence over ``logging.setLoggerClass``\n    1. They are two separate state variables!\n\n### Placeholders?\n\nInternal API for managing parent/child logger relationships. Quoting the\nsource code:\n\n\u003e Get a logger with the specified name (channel name), creating it\n\u003e if it doesn't yet exist. This name is a dot-separated hierarchical\n\u003e name, such as \"a\", \"a.b\", \"a.b.c\" or similar.\n\u003e\n\u003e If a PlaceHolder existed for the specified name [i.e. the logger\n\u003e didn't exist but a child of it did], replace it with the created\n\u003e logger and fix up the parent/child references which pointed to the\n\u003e placeholder to now point to the logger.\n\n\n## Records\n\n### extras\n\n``LogRecord`` ignores unknown kwargs, so where do ``extra``s get added as\nattributes?\n\nExtras get patched in by way of ``record.__dict__[key] = value``.\n\n\n## Adapters\n\n``LoggerAdapter`` is a simple way to add contextual information onto a logger.\nThe adapter itself, however, is not 100% API-compatible with the ``Logger``\nclass.\n\nIt should be used when you have multiple contexts (objects, etc.) utilizing the\nsame logger instance, but you want the generated logger records to be easily\ndifferentiable.\n\n## Filters\n\nOnly the following classes implement the ``Filterer`` interface:\n\n* Handler\n* Logger\n\n### Interface of a filter\n\nFilters should either be callable or have a callable attribute ``.filter()``.\nEither should take **one** positional argument: the ``LogRecord``.\n\n```\nclass MyFilter:\n    def filter(self, record):\n        return True\n\ndef filter(record):\n    return True\n```\n\n``MyFilter()`` and ``filter()`` are both acceptable (and equivalent in this\nsimple example).\n\n\n### Application of filters\n\nFilters are applied uniformly for either handlers or loggers:\n\n```python\nshould_handle = all(filter() for filter in logger.filters)\nshould_handler_0_emit_record = all(filter() for filter in logger.handlers[0].filters)\n```\n\n### Handler filter\n\nA **filter applied to a handler** takes in a record.\nIf **any** filter returns ``False``, the final message will not be emitted.\n\n## Log Messages\n\n### Hierarchy\n\nThe parent logger of ``logging.getLogger(\"a.b\")`` is ``logging.getLogger(\"a\")``.\n\n### Propagation (during callHandlers)\n\nBasic rule is that if:\n\n```python\nlogger.propagate = False\n```\n\nThat the log message will not propagate to `logger.parent`.\nThere's more to it than this, though.\n\nA ``False``-returning filter in ``logger.filters`` or setting ``logger.disabled`` \nwill stop records from reaching ``callHandlers`` and thus avoid propagation entirely.\n\nHandler-level filters **have no effect** on propagation.\n\n### Effective Level\n\nUnset log levels are configured as ``logging.NOTSET = 0``\n\n### Flow [Logger version]\n\n1. ``logger.debug(\"Message\")``\n    1. Ensure ``logger.isEnabledFor(DEBUG)``\n2. ``logger._log()`` \n    1. Create a ``LogRecord`` by way of ``logger.makeRecord()``\n    2. This is where exception information and stack information is determined.\n3. ``logger.handle(record)``\n    1. Stop if ``logger.disabled``\n    2. Stop if ``not logger.filter(record)``\n4. ``logger.callHandlers(record)``\n    1. For each handler in ``logger.handlers``\n        1. If the **handler** level is ``\u003c=`` the ``record.levelno``, pass the\n            message to the handler: ``handler.handle(record)``\n    2. Repeat (a) for each parent/ancestor logger **if** ``.propagate``\n    3. If **zero** handlers were found in this step, use the \"last resort\" logger.\n       The log level number requirement still applies for the last resort logger.\n\nFor each handler that passed step (4)(a)(1):\n\n1. Stop if ``not handler.filter(record)`` \n    1. All filters must return ``True``\n2. Emit the record by way of:\n    1. Acquire ``threading.RLock`` dedicated to the handler (``handler.lock``).\n    2. ``handler.emit(record)``\n    3. Release the lock.\n\nFor each emitted record above in ``handler.emit(record)``:\n1. Call ``handler.format(record)`` to get the message\n2. Send message out to target (this is custom depending on the handler\n   implementation itself)\n\n### Flow [Adapter version]\n\nThe adapter is a thin wrapper around its public ``logger`` attribute.\n\nAssuming:\n\n```python\nextra = {\"a\": \"b\"}\nadapter = logging.LoggerAdapter(logging.getLogger(\"logger\"), extra=extra)\n```\n\n1. ``adapter.debug(\"Test\")``\n    1. ``adapter.log(DEBUG, \"Test\")``\n2. ``adapter.log(level, msg, *args, **kwargs)``\n    1. Ensure ``adapter.logger.isEnabledFor(DEBUG)``\n    2. Call ``adapter.process(DEBUG, \"Test\")``\n3. ``adapter.process(DEBUG, \"Test\", *args, **kwargs)``\n    1. Attach ``kwargs[\"extra\"] = self.extra``\n4. Pass on the message and args to the original logger instance.\n    1. ``self.logger.log(level, msg, *args, **kwargs)``\n\n### Last Resort\n\nIf no other handlers are found in a log message chain, a \"last resort\" logger\nis used.  \n\nThis is by default configured at the ``WARNING`` level.\n\nThe log level number requirement still applies for the last resort logger.\n\n### Creating a log record\n\n``logger.makeRecord`` takes care of creating a record.\n\nUnder the hood, log records are created through a factory\n``logging.getLogRecordFactory()``, which can be set with\n``logging.setLogRecordFactory(factory)``.\n\nThis factory (as of the time of writing) should have arguments:\n```python\n_logRecordFactory(\n    name,\n    level,\n    filename,\n    line_number,\n    msg,\n    args,\n    exception_info,\n    function,\n    stack_info,\n)\n```\n\n## Shutdown\n\nThere's an ``atexit`` hook which will look through every handler ever defined\n(well, weakrefs of handlers, stashed in ``logging._handlerList``) and call the\nfollowing:\n\n1. If the weakref is invalid, stop\n2. Call the following, ignoring OSError/ValueError:\n    1. ``handler.acquire()``\n    2. ``handler.flush()``\n    3. ``handler.close()``\n4. And finally, ``handler.release()``\n\n## Root Logger\n\nHelper functions for using the root logger are there, but you really shouldn't\nbe using them (in my opinion).\n\nThere is at least one upside to using them, an application without logging\nsetup may still see the messages due to a check for handlers on the root logger\nin these helpers.\n\n``logging.info(...)`` -\u003e ``basicConfig`` + ``root.info(...)``\n\n``logging.debug(...)`` -\u003e ``basicConfig`` + ``root.debug(...)``\n\n``logging.warning(...)`` -\u003e ``basicConfig`` + ``root.warning(...)``\n\n``logging.error(...)`` -\u003e ``basicConfig`` + ``root.error(...)``\n\n``logging.exception(...)`` -\u003e ``basicConfig`` + ``root.exception(...)``\n\n``logging.critical(...)`` -\u003e ``basicConfig`` + ``root.critical(...)``\n\n## Capturing warnings\n\nWarnings can be captured in the logging system by way of:\n\n```python\nlogging.captureWarnings(True)\n```\n\nThis (surprisingly, to me) monkey-patches ``warnings.showwarning`` to\nredirect it to ``logging._showwarning``.  This formats the warning\nand routes it to ``logging.getLogger(\"py.warnings\").warning()``.\n\n\n## basicConfig\n\nThis is used pretty much everywhere for simple configuration of logging.\n\nAs someone clearly interested in the Python logging system, it's worth reading\nthrough the docstring once to see what it's actually doing.\n\n\u003e   Do basic configuration for the logging system.\n\u003e\n\u003e   This function does nothing if the root logger already has handlers\n\u003e   configured, unless the keyword argument *force* is set to ``True``.\n\u003e   It is a convenience method intended for use by simple scripts\n\u003e   to do one-shot configuration of the logging package.\n\u003e\n\u003e   The default behaviour is to create a StreamHandler which writes to\n\u003e   sys.stderr, set a formatter using the BASIC_FORMAT format string, and\n\u003e   add the handler to the root logger.\n\u003e\n\u003e   A number of optional keyword arguments may be specified, which can alter\n\u003e   the default behaviour.\n\nAlso, you can \n\n1. Specify ``filename``/``filemode`` to quickly create a ``FileHandler``\n2. Configure formatting by way of ``format``/``datefmt``/``style`` and \n   ``encoding``\n3. Configure the root logger level (``level``)\n4. Redirect the default ``stderr`` stream (``stream``)\n5. Pass in already-created handlers (``handlers = [...]``)\n\n\n## Formatting\n\nTODO\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fklauer%2Fpython-logging-notes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fklauer%2Fpython-logging-notes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fklauer%2Fpython-logging-notes/lists"}