{"id":17483415,"url":"https://github.com/cogini/logger_formatter_json","last_synced_at":"2025-04-10T19:31:57.854Z","repository":{"id":85035599,"uuid":"538249404","full_name":"cogini/logger_formatter_json","owner":"cogini","description":"Erlang OTP logger formatter that outputs JSON","archived":false,"fork":false,"pushed_at":"2024-10-03T19:06:41.000Z","size":197,"stargazers_count":6,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-07T18:08:08.695Z","etag":null,"topics":["erlang","erlang-library","erlang-otp","formatter","json","logger"],"latest_commit_sha":null,"homepage":"","language":"Erlang","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cogini.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-09-18T21:45:55.000Z","updated_at":"2024-10-21T14:11:33.000Z","dependencies_parsed_at":null,"dependency_job_id":"ed9b16fd-ecff-4092-9d49-266a462028ca","html_url":"https://github.com/cogini/logger_formatter_json","commit_stats":{"total_commits":180,"total_committers":3,"mean_commits":60.0,"dds":0.07222222222222219,"last_synced_commit":"cfc84c0cbdf810b52250be17d57e88b5d437798f"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cogini%2Flogger_formatter_json","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cogini%2Flogger_formatter_json/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cogini%2Flogger_formatter_json/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cogini%2Flogger_formatter_json/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cogini","download_url":"https://codeload.github.com/cogini/logger_formatter_json/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248281415,"owners_count":21077423,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["erlang","erlang-library","erlang-otp","formatter","json","logger"],"created_at":"2024-10-19T00:04:54.149Z","updated_at":"2025-04-10T19:31:56.669Z","avatar_url":"https://github.com/cogini.png","language":"Erlang","funding_links":[],"categories":[],"sub_categories":[],"readme":"![test workflow](https://github.com/cogini/logger_formatter_json/actions/workflows/test.yml/badge.svg)\n[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg)](CODE_OF_CONDUCT.md)\n\n# logger_formatter_json\n\nThis is a formatter for the Erlang logger application that outputs JSON.\n\nWhen logging text messages, it outputs the text message under the `msg` key\nalong with logger metadata such as OpenTelemetry trace ids.\n\nIt also supports \"structured\" logging of maps, e.g. `logger:info(#{foo =\u003e bar})`.\nMetadata values can also be maps.\n\nYou can control the order of output keys to put the message first, avoiding\nhaving to read a big mess of JSON to find the important parts.\n\nYou can configure the names of output keys to support naming conventions from services\nsuch as [Datadog](https://www.erlang.org/doc/man/logger_formatter.html) and\n[Google Cloud](https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#LogSeverity)\n\nIt implements the [formatter](https://www.erlang.org/doc/apps/kernel/logger_chapter.html#formatters)\nAPI for the high-performance `logger` application introduced in OTP 21.\n\nIt is written in Erlang with no dependencies except for the Erlang JSON library\n[thoas](https://github.com/lpil/thoas). It can be used by pure Erlang projects\nas well as other BEAM languages such as Elixir.\n\n## Installation\n\nErlang:\n\nAdd `logger_formatter_json` to the list of dependencies in `rebar.config`:\n\n```erlang\n{deps, [logger_formatter_json]}.\n```\n\nElixir:\n\nAdd `logger_formatter_json` to the list of dependencies in `mix.exs`:\n\n```elixir\ndef deps do\n  [\n    {:logger_formatter_json, \"~\u003e 0.8\"},\n  ]\nend\n```\n\n## Configuration\n\nJSON output is useful for production, as it allows you parse and query\nlog records to, e.g., match on a particular user. It can be excessively verbose\nfor development, however, so we stick with the normal Elixir logger for development.\nFor Erlang, [flatlog](https://github.com/ferd/flatlog) is a similar user-friendly library.\n\nIn order to make all log output in consistent JSON format, including system\nmessages, we configure the formatter as the default for all applications\nrunning on the VM.\n\nErlang:\n\nConfigure the kernel default handler in the `sys.config` file for the release:\n\n```erlang\n[\n {kernel, [\n    {logger, [\n        {handler, default, logger_std_h,\n         #{formatter =\u003e {logger_formatter_json, #{}}}\n        }\n    ]},\n    {logger_level, info}\n ]}\n].\n```\n\nElixir:\n\nAs of Elixir 15, we can override the formatter for the default log handler\neasily. In `config/prod.exs` configure `:default_handler`:\n\n```elixir\nconfig :logger, :default_handler,\n  formatter: {:logger_formatter_json, %{}}\n```\n\nOr, with options (see below):\n\n```elixir\nconfig :logger, :default_handler,\n  formatter: {\n    :logger_formatter_json,\n    %{\n      template: [\n        :msg,\n        :time,\n        :level,\n        :file,\n        :line,\n        # :mfa,\n        :pid,\n        :request_id,\n        :otel_trace_id,\n        :otel_span_id,\n        :otel_trace_flags\n      ]\n    }\n  }\n```\n\nIn older Elixir versions, it was tricky to configure the default handler from\nElixir. Instead, we would reconfigure the default handler in the app startup.\n\nIn `config/prod.exs` or `config/runtime.exs`, define the formatter config:\n\n```elixir\nconfig :foo, :logger_formatter_config, {:logger_formatter_json, %{}}\n```\n\nOr, with options (see below):\n\n```elixir\nconfig :foo, :logger_formatter_config, {:logger_formatter_json,\n %{\n   template: [\n     :msg,\n     :time,\n     :level,\n     :file,\n     :line,\n     # :mfa,\n     :pid,\n     :request_id,\n     :otel_trace_id,\n     :otel_span_id,\n     :otel_trace_flags\n   ]\n }}\n```\n\nNext, in in your application startup file, e.g., `lib/foo/application.ex`, add a\ncall to reconfigure the logger:\n\n```elixir\ndef start(_type, _args) do\n  logger_formatter_config = Application.get_env(:foo, :logger_formatter_config)\n\n  if logger_formatter_config do\n    :logger.update_handler_config(:default, :formatter, logger_formatter_config)\n  end\n```\n\nIf you want all the messages from the initial startup in JSON as well, you have to\nconfigure the logger as a VM arg for the release.\n\nIn `rel/vm.args.eex`, set up the logger:\n\n```erlang\n-kernel logger '[{handler, default, logger_std_h, #{formatter =\u003e {logger_formatter_json, #{}}}}]'\n```\nor, with options:\n\n```erlang\n-kernel logger '[{handler, default, logger_std_h, #{formatter =\u003e {logger_formatter_json, #{template =\u003e [msg, time, level, file, line, mfa, pid, otel_trace_id, otel_span_id, otel_trace_flags]}}}}]'\n```\n\nThere used to be a way of doing this in Elixir, but it seems to have stopped\nworking. In `config/prod.exs` or `config/runtime.exs`, define the formatter\nconfig:\n\n```elixir\nif System.get_env(\"RELEASE_MODE\") do\n  config :kernel, :logger, [\n    {:handler, :default, :logger_std_h, %{\n       formatter: {:logger_formatter_json, %{}}\n    }}\n  ]\nend\n```\n\nThe check for the `RELEASE_MODE` environment variable makes the code only run\nwhen building a release.\n\n\n## Usage\n\nThe formatter accepts a map of options, e.g.:\n\n```elixir\nconfig :foo, :logger, [\n  {:handler, :default, :logger_std_h, %{\n     formatter: {:logger_formatter_json, %{\n          names: %{\n            time: \"date\",\n            level: \"status\",\n            msg: \"message\"\n          }\n     }}\n  }}\n]\n```\n\n`names` is a map of keys in the metadata map to string keys in the JSON output.\n\nThe module has predefined sets of keys for `datadog` and `gcp`.\n\n```elixir\nconfig :foo, :logger, [\n  {:handler, :default, :logger_std_h, %{formatter: {:logger_formatter_json, %{names: :datadog}}}}\n]\n```\n\nYou can also specify a list to add your own tags to the predefined ones, e.g.\nof options, e.g. `names: [datadog, %{foo: \"bar\"}]`.\n\n`types` is a map which identifies keys with a special format that the module understands\n(`level`, `system_time`, `mfa`).\n\n`template` is a list of metadata to format. This lets you put keys in specific order to\nmake them easier to read in the output.\n\nFor example:\n\n```elixir\ntemplate: [\n  :msg,\n  :time,\n  :level,\n  :file,\n  :line,\n  :mfa,\n  :pid,\n  :otel_trace_id,\n  :otel_span_id,\n  :otel_trace_flags\n]\n```\n\nList elements are metadata key names, with a few special keys:\n\n* `msg` represents the text message, if any.\n\nIf you call `logger:info(\"the message\")`, then it would be rendered in the JSON\nas `{\"msg\": \"the message\", ...}`. You can map the key `msg` to e.g. `message`\nvia the `names` config option.\n\n* `all` represents all the metadata keys.\n* `rest` represents all the metadata keys which have not been handled explicitly.\n\nYou can specify a group of keys as a tuple like\n`{group, \u003cname\u003e, [\u003clist of metadata keys\u003e]}`, and they will be collected into a\nmap in the output.\n\nFor example:\n\n```elixir\n{group, source_location, [file, line, mfa]},\n{group, tags, [rest]}\n```\n\nThis would result in a log message like:\n\n```json\n{\n    ...\n    \"source_location\": {\"file\": \"mymodule.ex\", \"line\": 17, \"mfa\": \"mymodule:thefunction/1\"},\n    \"tags\": {\"foo\": \"bar\", \"biz\": \"baz\"}\n}\n```\n\nThe default template is `[msg, rest]`.\n\nYou can also use a tuple to specify a standard set of keys to be used:\n\n`{keys, basic}`: `[time, level, msg]`\n\n`{keys, trace}`: `[trace_id, span_id]`\n\n`{keys, otel}`: `[otel_trace_id, otel_span_id, otel_trace_flags]`\n\n`{keys, gcp}`:\n\n```erlang\n[\n    msg,\n    time,\n    level,\n    otel_trace_id,\n    otel_span_id,\n    {group, source_location, [file, line, mfa]},\n    {group, tags, [rest]}\n]\n```\n\nYou can specify multiple templates, so you can add your own metadata keys to one\nof the standard templates, e.g., `[{keys, basic}, request_id, trace_id, span_id]`.\n\nWhen logging maps, default (`embed`) puts the map under the `msg` key.\nIf you set the config option `map_msg` to `merge`, then the keys of the map\nwill be merged into the metadata.\n\nSo `logger:info(#{hi =\u003e there}}` is logged as `{\"level\":\"info\",\"hi\":\"there\"}`.\nThis lets you control how the fields appear in the output with the template, e.g.,\na template of `[level, biz]` would result in output of `{\"level\":\"info\",\"biz\":\"baz\"}`\nfor the message `logger:info(#{foo =\u003e bar, biz =\u003e baz}}`.\n\n## Acknowledgements\n\nMuch thanks to Fred Hebert, as always. His [flatlog](https://github.com/ferd/flatlog)\nlibrary was a significant inspiration.\n\n## Links\n\n* [Erlang/OTP 21's new logger](https://ferd.ca/erlang-otp-21-s-new-logger.html)\n* [Canonical log lines](https://brandur.org/canonical-log-lines)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcogini%2Flogger_formatter_json","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcogini%2Flogger_formatter_json","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcogini%2Flogger_formatter_json/lists"}