{"id":15070062,"url":"https://github.com/metwork-framework/mflog","last_synced_at":"2025-04-10T17:10:51.254Z","repository":{"id":56281337,"uuid":"166773307","full_name":"metwork-framework/mflog","owner":"metwork-framework","description":"opinionated python (structured) logging library built on structlog","archived":false,"fork":false,"pushed_at":"2023-12-22T16:49:30.000Z","size":339,"stargazers_count":9,"open_issues_count":5,"forks_count":1,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-04-24T04:31:53.808Z","etag":null,"topics":["autoreadme","github-actions","integration-level-3","library","logging","python2","python3","structlog"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/metwork-framework.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-01-21T08:07:50.000Z","updated_at":"2023-12-04T09:10:35.000Z","dependencies_parsed_at":"2023-12-22T17:29:04.293Z","dependency_job_id":null,"html_url":"https://github.com/metwork-framework/mflog","commit_stats":{"total_commits":145,"total_committers":7,"mean_commits":"20.714285714285715","dds":0.6620689655172414,"last_synced_commit":"74bb0d570695976d3c8496d992be39d79077e974"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metwork-framework%2Fmflog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metwork-framework%2Fmflog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metwork-framework%2Fmflog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metwork-framework%2Fmflog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/metwork-framework","download_url":"https://codeload.github.com/metwork-framework/mflog/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247294483,"owners_count":20915340,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autoreadme","github-actions","integration-level-3","library","logging","python2","python3","structlog"],"created_at":"2024-09-25T01:46:48.308Z","updated_at":"2025-04-10T17:10:51.210Z","avatar_url":"https://github.com/metwork-framework.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# mflog\n\n[//]: # (automatically generated from https://github.com/metwork-framework/github_organization_management/blob/master/common_files/README.md)\n\n**Status (master branch)**\n\n[![GitHub CI](https://github.com/metwork-framework/mflog/actions/workflows/ci.yml/badge.svg?branch=master)](https://github.com/metwork-framework/mflog/actions?query=workflow%3ACI+branch%3Amaster)\n[![Maintenance](https://raw.githubusercontent.com/metwork-framework/resources/master/badges/maintained.svg)](https://github.com/metwork-framework/resources/blob/master/badges/maintained.svg)\n\n\n\n\n## What is it ?\n\nIt is an opinionated python (structured) logging library built on [structlog](https://www.structlog.org/)\nfor the [MetWork Framework](http://metwork-framework.org) (but it can be used in any context).\n\n\u003e Structured logging means that you don’t write hard-to-parse and hard-to-keep-consistent prose in your logs but that you log events that happen in a context instead.\n\u003e - https://www.structlog.org/en/stable/why.html\n\nExample:\n\n```python\n\nfrom mflog import get_logger\n\n# Get a logger\nlog = get_logger(\"foo.bar\")\n\n# Bind some attributes to the logger depending on the context\nlog = log.bind(user=\"john\")\nlog = log.bind(user_id=123)\n\n# [...]\n\n# Log something\nlog.warning(\"user logged in\", happy=True, another_key=42)\n```\n\nOn `stderr`, you will get:\n\n```\n2019-01-28T07:52:42.903067Z  [WARNING] (foo.bar#7343) user logged in {another_key=42 happy=True user=john user_id=123}\n```\n\nOn `json output file`, you will get:\n\n```json\n{\n    \"timestamp\": \"2019-01-28T08:16:40.047710Z\",\n    \"level\": \"warning\",\n    \"name\": \"foo.bar\",\n    \"pid\": 29317,\n    \"event\": \"user logged in\",\n    \"another_key\": 42,\n    \"happy\": true,\n    \"user\": \"john\",\n    \"user_id\": 123\n}\n```\n\nIf the [python/rich library](https://github.com/willmcgugan/rich) is installed (this is not a\nmandatory requirement) and if the output is a real terminal (and not a redirection or a pipe),\nthe library will automatically configure a fancy color output (of course you can disable it if\nyou don't like):\n\nWith following demo python program:\n```python\nimport mflog\n\n# Get a logger\nlogger = mflog.get_logger(\"foobar\")\n\n# Bind two context variables to this logger\nlogger = logger.bind(user_id=1234, is_logged=True)\n\n# Log something\nlogger.info(\"This is an info message\", special_value=\"foo\")\nlogger.critical(\"This is a very interesting critical message\")\n\n# Let's play with exception\ntry:\n    # Just set a variable to get a demo of locals variable dump\n    var = {\"key1\": [1, 2, 3], \"key2\": \"foobar\"}\n    1/0\nexcept Exception:\n    logger.exception(\"exception raised (a variables dump should follow)\")\n\n```\n\nYou will get this color ouput:\n\n![color output](./demo/demo.png)\n\n## (opinionated) Choices and Features\n\n- we use main ideas from `structlog` library\n- we log `[DEBUG]` and `[INFO]` messages on `stdout` (in a human friendly way)\n- we log `[WARNING]`, `[ERROR]` and `[CRITICAL]` on `stderr` (in a human friendly way)\n- (and optionally) we log all messages (worse than a minimal configurable level) in a configurable file in `JSON` (for easy automatic parsing)\n- (and optionally) we send all messages (worse than a minimal configurable level) to an UDP syslog server (in JSON or in plain text)\n- we can configure a global minimal level to ignore all messages below\n- we reconfigure automatically python standard logging library to use `mflog`\n- Unicode and Bytes messages are supported (in Python2 and Python3)\n- good support for exceptions (with backtraces)\n- override easily minimal levels (for patterns of logger names) programmatically or with plain text configuration files\n- if the [python/rich library](https://github.com/willmcgugan/rich) is installed (this is not a mandatory requirement) and if the output is a real terminal (and not a redirection), the library will automatically configure a fancy color output (can be really useful but of course you can disable this feature if you don't like it)\n\n## How to use ?\n\nA `mflog` logger can be used as a standard `logging` logger.\n\nFor example:\n\n```python\n# Import\nfrom mflog import get_logger\n\n# Get a logger\nx = get_logger(\"foo.bar\")\n\n# Usage\nx.warning(\"basic message\")\nx.critical(\"message with templates: %i, %s\", 2, \"foo\")\nx.debug(\"message with key/values\", foo=True, bar=\"string\")\n\ntry:\n    1/0\nexcept Exception:\n    x.exception(\"we catched an exception with automatic traceback\")\n\nx = x.bind(context1=\"foo\")\nx = x.bind(context2=\"bar\")\nx.info(\"this is a contexted message\", extra_var=123)\n```\n\n## How to configure ?\n\n### In python\n\n```python\nimport mflog\n\n# Configure\nmflog.set_config(minimal_level=\"DEBUG\", json_minimal_level=\"WARNING\",\n                 json_file=\"/foo/bar/my_output.json\")\n\n# Get a logger\nx = mflog.get_logger(\"foo.bar\")\n\n# [...]\n```\n\n### With environment variables\n\n```bash\n\n$ export MFLOG_MINIMAL_LEVEL=\"DEBUG\"\n$ export MFLOG_JSON_MINIMAL_LEVEL=\"WARNING\"\n$ export MFLOG_JSON_FILE=\"/foo/bar/my_output.json\"\n\n$ python\n\n\u003e\u003e\u003e import mflog\n\u003e\u003e\u003e\n\u003e\u003e\u003e # Get a logger\n\u003e\u003e\u003e x = mflog.get_logger(\"foo.bar\")\n\u003e\u003e\u003e\n\u003e\u003e\u003e # [...]\n```\n\n### Note\n\nWhen you get a `mflog` logger, if default configuration is applied automatically\nif not set manually before.\n\n## How to override minimal level for a specific logger\n\nIf you have a \"noisy\" specific logger, you can override its minimal log level.\n\nThe idea is to configure this in a file like this:\n\n```\n# lines beginning with # are comments\n\n# this line say 'foo.bar' logger will have a minimal level of WARNING\nfoo.bar =\u003e WARNING\n\n# this line say 'foo.*' loggers will have a minimal level of DEBUG\n# (see python fnmatch for accepted wildcards)\nfoo.* =\u003e DEBUG\n\n# The first match wins\n```\n\nThen, you can use\n\n```python\n\n# yes we use a list here because you can use several files\n# (the first match wins)\nmflog.set_config([...], override_files=[\"/full/path/to/your/override.conf\"])\n```\n\nor\n\n```\n# if you want to provide multiple files, use ';' as a separator\nexport MFLOG_MINIMAL_LEVEL_OVERRIDE_FILES=/full/path/to/your/override.conf\n```\n\n## Link with standard python logging library\n\nWhen you get a `mflog` logger or when you call `set_config()` function,\nthe standard python `logging` library is reconfigured to use `mflog`.\n\nExample:\n\n```python\nimport logging\nimport mflog\n\n# standard use of logging library\nx = logging.getLogger(\"standard.logger\")\nprint(\"\u003coutput of the standard logging library\u003e\")\nx.warning(\"foo bar\")\nprint(\"\u003c/output of the standard logging library\u003e\")\n\n# we set the mflog configuration\nmflog.set_config()\n\n# now logging library use mflog\nprint()\nprint(\"\u003coutput of the standard logging library through mflog\u003e\")\nx.warning(\"foo bar\")\nprint(\"\u003c/output of the standard logging library through mflog\u003e\")\n```\n\nOutput:\n\n```\n\u003coutput of the standard logging library\u003e\nfoo bar\n\u003c/output of the standard logging library\u003e\n\n\u003coutput of the standard logging library through mflog\u003e\n2019-01-29T09:32:37.093240Z  [WARNING] (standard.logger#15809) foo bar\n\u003c/output of the standard logging library through mflog\u003e\n```\n\n## mflog loggers API\n\n### `.debug(message, *args, **kwargs)`\n\nLog the given message as `[DEBUG]`.\n\n- `*args` can be used for placeholders (to format the given message)\n- `**kwargs` can be used for key/values (log context).\n\nExamples:\n\n```python\nfrom mflog import get_logger\n\nx = get_logger('my.logger')\nx.debug(\"my debug message with placeholders: %s and %i\", \"foo\", 123,\n        key1=\"value1, key2=True, key5=123)\n```\n\n### `.info(message, *args, **kwargs)`\n\nSame as `.debug` but with `[INFO]` severity level.\n\n### `.warning(message, *args, **kwargs)`\n\nSame as `.debug` but with `[WARNING]` severity level.\n\n### `.error(message, *args, **kwargs)`\n\nSame as `.debug` but with `[ERROR]` severity level.\n\n### `.critical(message, *args, **kwargs)`\n\nSame as `.debug` but with `[CRITICAL]` severity level.\n\n### `.exception(message, *args, **kwargs)`\n\nSame as `.error` (so with `[ERROR]` severity level) but we automatically add\nthe current stacktrace in the message through special key/values.\n\n### `.bind(**new_values)`\n\nReturn a new logger with `**new_values` added to the existing ones\n(see examples at the beginning).\n\n### `.unbind(*keys)`\n\nReturn a new logger with `*keys` removed from the context.\nIt raises `KeyError` if the key is not part of the context.\n\n### `.try_unbind(*keys)`\n\nLike `.unbind` but best effort:  missing keys are ignored.\n\n### `.die(optional_message, *args, **kwargs)`\n\nSame as `.exception()` but also do a `.dump_locals()` call and exit the program\nwith `sys.exit(1)`.\n\n### `.dump_locals()`\n\nDump locals variables on `stderr` (for debugging).\n\n### `mflog.*`\n\nAll previous loggers method are also available in `mflog` module.\n\nExample:\n\n```python\n\nimport mflog\n\nmflog.warning(\"this is a warning message\", context1=\"foobar\", user_id=123)\n```\n\n## FAQ\n\n## If I want to use mflog inside my library ?\n\nIf you write a library and if you want to use `mflog`, use `mflog` normally.\nYou just should avoid to call `set_config()` inside your library.\n\n\n## Do you have \"thread local context mode\" ?\n\nThis mode is explained [here](https://www.structlog.org/en/stable/thread-local.html).\n\nYou have to understand what you are doing.\n\nIf you want to use it, just add `thread_local_context=True` to your `set_config()`\ncall. And you can use `.new(**new_values)` on mflog loggers to clear context\nand binds some initial values.\n\n\n## Can I globally add an extra context to each log line ?\n\nIf you add `extra_context_func=your_python_func` to your `set_config()` call,\nand if `your_python_func` returns a dict of key/values as strings when called\nwith no argument, these key/values will be added to your log context.\n\nAnother way to do that without even calling `set_config()` is to define\nan environment variable called `MFLOG_EXTRA_CONTEXT_FUNC` containing the\nfull path to your python func.\n\nFull example:\n\n```bash\n# in shell\nexport MFLOG_EXTRA_CONTEXT_FUNC=\"mflog.unittests.extra_context\"\n```\n\nthen, in your python interpreter:\n\n```python\n\u003e\u003e\u003e from mflog import get_logger\n\u003e\u003e\u003e get_logger(\"foo\").info(\"bar\")\n2019-04-11T07:32:53.517260Z     [INFO] (foo#15379) bar {extra_context_key1=extra_context_value1 extra_context_key2=extra_context_value2}\n```\n\nHere is the code of `mflog.unittests.extra_context`:\n\n```python\ndef extra_context():\n    return {\"extra_context_key1\": \"extra_context_value1\",\n            \"extra_context_key2\": \"extra_context_value2\"}\n```\n\n## Can I filter some context keys in stdout/stderr output (but keep them in json output) ?\n\nYes, add `json_only_keys=[\"key1\", \"key2\"]` to your `set_config()` call or use\n`MFLOG_JSON_ONLY_KEYS=key1,key2` environment variable.\n\n## What about if I don't want to redirect standard python `logging` to `mflog` ?\n\nYou can add `standard_logging_redirect=False` in your `set_config()` call\nof set `MFLOG_STANDARD_LOGGING_REDIRECT=0` environment variable.\n\n## Can I silent a specific noisy logger?\n\nYou can use `override_files` feature to do that or you can also use the\n`mflog.add_override` function.\n\nFor example:\n\n```python\nimport mflog\n\n# for all mylogger.* loggers (fnmatch pattern), the minimal level is CRITICAL\nmflog.add_override(\"mylogger.*\", CRITICAL)\n\n# Not very interesting but this call will be ignored\nmflog.get_logger(\"mylogger.foo\").warning(\"foo\")\n```\n\n## How can I use syslog logging?\n\nYou can configure it with these keyword arguments during `set_config()` call:\n\n- `syslog_minimal_level`: `WARNING`, `CRITICAL`...\n- `syslog_address`: `null` (no syslog (defaut)), `127.0.0.1:514` (send packets to 127.0.0.1:514), `/dev/log` (unix socket)...\n- `syslog_format`: `msg_only` (default) or `json`\n\nor with corresponding env vars:\n\n- `MFLOG_SYSLOG_MINIMAL_LEVEL`\n- `MFLOG_SYSLOG_ADDRESS`\n- `MFLOG_SYSLOG_FORMAT`\n\n## How to disable the fancy color output?\n\nThis feature is automatically enabled when:\n\n- [python/rich](https://github.com/willmcgugan/rich) library is installed\n- the corresponding output (stdout, stderr) is a real terminal (and not a redirection to a file)\n\nBut you can manually disable it by adding `fancy_output=False` to your `set_config()`.\n\n## Coverage\n\nSee [Coverage report](https://metwork-framework.org/pub/misc/mflog/coverage/)\n\n\n\n\n\n\n## Contributing guide\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) file.\n\n\n\n## Code of Conduct\n\nSee [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) file.\n\n\n\n## Sponsors\n\n*(If you are officially paid to work on MetWork Framework, please contact us to add your company logo here!)*\n\n[![logo](https://raw.githubusercontent.com/metwork-framework/resources/master/sponsors/meteofrance-small.jpeg)](http://www.meteofrance.com)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmetwork-framework%2Fmflog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmetwork-framework%2Fmflog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmetwork-framework%2Fmflog/lists"}