{"id":20816390,"url":"https://github.com/nebo15/logger_json","last_synced_at":"2025-04-14T22:06:04.981Z","repository":{"id":37547394,"uuid":"84937151","full_name":"Nebo15/logger_json","owner":"Nebo15","description":"JSON logger formatter with support for Google Cloud, DataDog and other for Elixir.","archived":false,"fork":false,"pushed_at":"2025-04-06T20:58:53.000Z","size":485,"stargazers_count":256,"open_issues_count":1,"forks_count":100,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-04-14T22:05:51.823Z","etag":null,"topics":["datadog","dockerized","ecto","elixir","elixir-lang","elk-stack","google-cloud-logging","json","log","log-collector","logging","package","phoenix-framework","plug"],"latest_commit_sha":null,"homepage":"https://nebo15.github.io/logger_json/","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Nebo15.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-03-14T10:38:43.000Z","updated_at":"2025-04-14T13:44:36.000Z","dependencies_parsed_at":"2024-04-29T17:30:09.539Z","dependency_job_id":"5953cc76-610e-47c8-bf87-2b71173547f4","html_url":"https://github.com/Nebo15/logger_json","commit_stats":{"total_commits":175,"total_committers":33,"mean_commits":5.303030303030303,"dds":"0.24571428571428566","last_synced_commit":"987aabc835c6b37555aa62a6a76b6cac65ceed93"},"previous_names":[],"tags_count":36,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Nebo15%2Flogger_json","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Nebo15%2Flogger_json/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Nebo15%2Flogger_json/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Nebo15%2Flogger_json/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Nebo15","download_url":"https://codeload.github.com/Nebo15/logger_json/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248968834,"owners_count":21191160,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["datadog","dockerized","ecto","elixir","elixir-lang","elk-stack","google-cloud-logging","json","log","log-collector","logging","package","phoenix-framework","plug"],"created_at":"2024-11-17T21:33:34.068Z","updated_at":"2025-04-14T22:06:04.963Z","avatar_url":"https://github.com/Nebo15.png","language":"Elixir","readme":"# LoggerJSON\n\n[![Build Status](https://github.com/Nebo15/logger_json/actions/workflows/ci.yml/badge.svg)](https://github.com/Nebo15/logger_json/actions/workflows/ci.yml)\n[![Coverage Status](https://coveralls.io/repos/github/Nebo15/logger_json/badge.svg?branch=master)](https://coveralls.io/github/Nebo15/logger_json?branch=master)\n[![Module Version](https://img.shields.io/hexpm/v/logger_json.svg)](https://hex.pm/packages/logger_json)\n[![Hex Docs](https://img.shields.io/badge/hex-docs-lightgreen.svg)](https://hexdocs.pm/logger_json/)\n[![Hex Download Total](https://img.shields.io/hexpm/dt/logger_json.svg)](https://hex.pm/packages/logger_json)\n[![License](https://img.shields.io/hexpm/l/logger_json.svg)](https://github.com/Nebo15/logger_json/blob/master/LICENSE.md)\n\nA collection of formatters and utilities for JSON-based logging for various cloud tools and platforms.\n\n## Supported formatters\n\n- [`LoggerJSON.Formatters.Basic`](https://hexdocs.pm/logger_json/LoggerJSON.Formatters.Basic.html) - a basic JSON formatter that logs messages in a structured, but generic format, can be used with any JSON-based logging system.\n\n- [`LoggerJSON.Formatters.GoogleCloud`](https://hexdocs.pm/logger_json/LoggerJSON.Formatters.GoogleCloud.html) - a formatter that logs messages in a structured format that can be consumed by Google Cloud Logger and Google Cloud Error Reporter.\n\n- [`LoggerJSON.Formatters.Datadog`](https://hexdocs.pm/logger_json/LoggerJSON.Formatters.Datadog.html) - a formatter that logs messages in a structured format that can be consumed by Datadog.\n\n- [`LoggerJSON.Formatters.Elastic`](https://hexdocs.pm/logger_json/LoggerJSON.Formatters.Elastic.html) - a formatter that logs messages in a structured format that conforms to the [Elastic Common Schema (ECS)](https://www.elastic.co/guide/en/ecs/8.11/ecs-reference.html), so it can be consumed by ElasticSearch, LogStash, FileBeat and Kibana.\n\n## Installation\n\nAdd `logger_json` to your list of dependencies in `mix.exs`:\n\n```elixir\ndef deps do\n  [\n    # ...\n    {:logger_json, \"~\u003e 7.0\"}\n    # ...\n  ]\nend\n```\n\nand install it running `mix deps.get`.\n\nThen, enable the formatter in your `config.exs`:\n\n```elixir\nconfig :logger, :default_handler,\n  formatter: LoggerJSON.Formatters.Basic.new(metadata: [:request_id])\n```\n\nor during runtime (eg. in your `application.ex`):\n\n```elixir\nformatter = LoggerJSON.Formatters.Basic.new(metadata: :all)\n:logger.update_handler_config(:default, :formatter, formatter)\n```\n\nYou might also want to format the log messages when migrations are running:\n\n```elixir\nconfig :domain, MyApp.Repo,\n  # ...\n  start_apps_before_migration: [:logger_json]\n```\n\nAdditionally, you may also be try [redirecting otp reports to Logger](https://hexdocs.pm/logger/Logger.html#module-configuration) (see \"Configuration\" section).\n\n## Configuration\n\nConfiguration can be set using 2nd element of the tuple of the `:formatter` option in `Logger` configuration.\nFor example in `config.exs`:\n\n```elixir\nconfig :logger, :default_handler,\n  formatter: LoggerJSON.Formatters.GoogleCloud.new(metadata: :all, project_id: \"logger-101\")\n```\n\nor during runtime:\n\n```elixir\nformatter = LoggerJSON.Formatters.Basic.new(%{metadata: {:all_except, [:conn]}})\n:logger.update_handler_config(:default, :formatter, formatter)\n```\n\nBy default, `LoggerJSON` is using `Jason` as the JSON encoder. If you use Elixir 1.18 or later, you can\nuse the built-in `JSON` module as the encoder. To do this, you need to set the `:encoder` option in your\n`config.exs` file. This setting is only available at compile-time:\n\n    config :logger_json, encoder: JSON\n\n## Docs\n\nThe docs can be found at [https://hexdocs.pm/logger_json](https://hexdocs.pm/logger_json).\n\n## Examples\n\n### Basic\n\n```json\n{\n  \"message\": \"Hello\",\n  \"metadata\": {\n    \"domain\": [\"elixir\"]\n  },\n  \"severity\": \"notice\",\n  \"time\": \"2024-04-11T21:31:01.403Z\"\n}\n```\n\n### Google Cloud Logger\n\nFollows the [Google Cloud Logger LogEntry](https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry) format,\nfor more details see [special fields in structured payloads](https://cloud.google.com/logging/docs/agent/configuration#special_fields_in_structured_payloads).\n\n```json\n{\n  \"logging.googleapis.com/trace\": \"projects/my-projectid/traces/0679686673a\",\n  \"logging.googleapis.com/spanId\": \"000000000000004a\",\n  \"logging.googleapis.com/operation\": {\n    \"pid\": \"#PID\u003c0.29081.0\u003e\"\n  },\n  \"logging.googleapis.com/sourceLocation\": {\n    \"file\": \"/Users/andrew/Projects/os/logger_json/test/formatters/google_cloud_test.exs\",\n    \"function\": \"Elixir.LoggerJSON.Formatters.GoogleCloudTest.test logs an LogEntry of a given level/1\",\n    \"line\": 44\n  },\n  \"message\": {\n    \"domain\": [\"elixir\"],\n    \"message\": \"Hello\"\n  },\n  \"severity\": \"NOTICE\",\n  \"time\": \"2024-04-12T15:07:55.020Z\"\n}\n```\n\nand this is how it looks in Google Cloud Logger:\n\n```json\n{\n  \"insertId\": \"1d4hmnafsj7vy1\",\n  \"jsonPayload\": {\n    \"message\": \"Hello\",\n    \"logging.googleapis.com/spanId\": \"000000000000004a\",\n    \"domain\": [\"elixir\"],\n    \"time\": \"2024-04-12T15:07:55.020Z\"\n  },\n  \"resource\": {\n    \"type\": \"gce_instance\",\n    \"labels\": {\n      \"zone\": \"us-east1-d\",\n      \"project_id\": \"firezone-staging\",\n      \"instance_id\": \"3168853301020468373\"\n    }\n  },\n  \"timestamp\": \"2024-04-12T15:07:55.023307594Z\",\n  \"severity\": \"NOTICE\",\n  \"logName\": \"projects/firezone-staging/logs/cos_containers\",\n  \"operation\": {\n    \"id\": \"F8WQ1FsdFAm5ZY0AC1PB\",\n    \"producer\": \"#PID\u003c0.29081.0\u003e\"\n  },\n  \"trace\": \"projects/firezone-staging/traces/bc007e40a2e9edffa23785d8badc43b8\",\n  \"sourceLocation\": {\n    \"file\": \"lib/phoenix/logger.ex\",\n    \"line\": \"231\",\n    \"function\": \"Elixir.Phoenix.Logger.phoenix_endpoint_stop/4\"\n  },\n  \"receiveTimestamp\": \"2024-04-12T15:07:55.678986520Z\"\n}\n```\n\nException that can be sent to Google Cloud Error Reporter:\n\n```json\n{\n  \"httpRequest\": {\n    \"protocol\": \"HTTP/1.1\",\n    \"referer\": \"http://www.example.com/\",\n    \"remoteIp\": \"\",\n    \"requestMethod\": \"PATCH\",\n    \"requestUrl\": \"http://www.example.com/\",\n    \"status\": 503,\n    \"userAgent\": \"Mozilla/5.0\"\n  },\n  \"logging.googleapis.com/operation\": {\n    \"pid\": \"#PID\u003c0.250.0\u003e\"\n  },\n  \"logging.googleapis.com/sourceLocation\": {\n    \"file\": \"/Users/andrew/Projects/os/logger_json/test/formatters/google_cloud_test.exs\",\n    \"function\": \"Elixir.LoggerJSON.Formatters.GoogleCloudTest.test logs exception http context/1\",\n    \"line\": 301\n  },\n  \"@type\": \"type.googleapis.com/google.devtools.clouderrorreporting.v1beta1.ReportedErrorEvent\",\n  \"context\": {\n    \"httpRequest\": {\n      \"protocol\": \"HTTP/1.1\",\n      \"referer\": \"http://www.example.com/\",\n      \"remoteIp\": \"\",\n      \"requestMethod\": \"PATCH\",\n      \"requestUrl\": \"http://www.example.com/\",\n      \"status\": 503,\n      \"userAgent\": \"Mozilla/5.0\"\n    },\n    \"reportLocation\": {\n      \"filePath\": \"/Users/andrew/Projects/os/logger_json/test/formatters/google_cloud_test.exs\",\n      \"functionName\": \"Elixir.LoggerJSON.Formatters.GoogleCloudTest.test logs exception http context/1\",\n      \"lineNumber\": 301\n    }\n  },\n  \"domain\": [\"elixir\"],\n  \"message\": \"Hello\",\n  \"serviceContext\": {\n    \"service\": \"nonode@nohost\"\n  },\n  \"stack_trace\": \"** (EXIT from #PID\u003c0.250.0\u003e) :foo\",\n  \"severity\": \"DEBUG\",\n  \"time\": \"2024-04-11T21:34:53.503Z\"\n}\n```\n\n## Datadog\n\nAdheres to the [default standard attribute list](https://docs.datadoghq.com/logs/processing/attributes_naming_convention/#default-standard-attribute-list)\nas much as possible.\n\n```json\n{\n  \"domain\": [\"elixir\"],\n  \"http\": {\n    \"method\": \"GET\",\n    \"referer\": \"http://www.example2.com/\",\n    \"request_id\": null,\n    \"status_code\": 200,\n    \"url\": \"http://www.example.com/\",\n    \"url_details\": {\n      \"host\": \"www.example.com\",\n      \"path\": \"/\",\n      \"port\": 80,\n      \"queryString\": \"\",\n      \"scheme\": \"http\"\n    },\n    \"useragent\": \"Mozilla/5.0\"\n  },\n  \"logger\": {\n    \"file_name\": \"/Users/andrew/Projects/os/logger_json/test/formatters/datadog_test.exs\",\n    \"line\": 239,\n    \"method_name\": \"Elixir.LoggerJSON.Formatters.DatadogTest.test logs http context/1\",\n    \"thread_name\": \"#PID\u003c0.225.0\u003e\"\n  },\n  \"message\": \"Hello\",\n  \"network\": {\n    \"client\": {\n      \"ip\": \"127.0.0.1\"\n    }\n  },\n  \"syslog\": {\n    \"hostname\": \"MacBook-Pro\",\n    \"severity\": \"debug\",\n    \"timestamp\": \"2024-04-11T23:10:47.967Z\"\n  }\n}\n```\n\n## Elastic\n\nFollows the [Elastic Common Schema (ECS)](https://www.elastic.co/guide/en/ecs/8.11/ecs-reference.html) format.\n\n```json\n{\n  \"@timestamp\": \"2024-05-21T15:17:35.374Z\",\n  \"ecs.version\": \"8.11.0\",\n  \"log.level\": \"info\",\n  \"log.logger\": \"Elixir.LoggerJSON.Formatters.ElasticTest\",\n  \"log.origin\": {\n    \"file.line\": 18,\n    \"file.name\": \"/app/logger_json/test/logger_json/formatters/elastic_test.exs\",\n    \"function\": \"test logs message of every level/1\"\n  },\n  \"message\": \"Hello\"\n}\n```\n\nWhen an error is thrown, the message field is populated with the error message and the `error.` fields will be set:\n\n\u003e Note: when throwing a custom exception type that defines the fields `id` and/or `code`, then the `error.id` and/or `error.code` fields will be set respectively.\n\n```json\n{\n  \"@timestamp\": \"2024-05-21T15:20:11.623Z\",\n  \"ecs.version\": \"8.11.0\",\n  \"error.message\": \"runtime error\",\n  \"error.stack_trace\": \"** (RuntimeError) runtime error\\n    test/logger_json/formatters/elastic_test.exs:191: anonymous fn/0 in LoggerJSON.Formatters.ElasticTest.\\\"test logs exceptions\\\"/1\\n\",\n  \"error.type\": \"Elixir.RuntimeError\",\n  \"log.level\": \"error\",\n  \"message\": \"runtime error\"\n}\n```\n\nAny custom metadata fields will be added to the root of the message, so that your application can fill any other ECS fields that you require:\n\n\u003e Note that this also allows you to produce messages that do not strictly adhere to the ECS specification.\n\n```json\n// Logger.info(\"Hello\") with Logger.metadata(:\"device.model.name\": \"My Awesome Device\")\n// or Logger.info(\"Hello\", \"device.model.name\": \"My Awesome Device\")\n{\n  \"@timestamp\": \"2024-05-21T15:17:35.374Z\",\n  \"ecs.version\": \"8.11.0\",\n  \"log.level\": \"info\",\n  \"log.logger\": \"Elixir.LoggerJSON.Formatters.ElasticTest\",\n  \"log.origin\": {\n    \"file.line\": 18,\n    \"file.name\": \"/app/logger_json/test/logger_json/formatters/elastic_test.exs\",\n    \"function\": \"test logs message of every level/1\"\n  },\n  \"message\": \"Hello\",\n  \"device.model.name\": \"My Awesome Device\"\n}\n```\n\n## Copyright and License\n\nCopyright (c) 2016 Andrew Dryga\n\nReleased under the MIT License, which can be found in [LICENSE.md](./LICENSE.md).\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnebo15%2Flogger_json","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnebo15%2Flogger_json","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnebo15%2Flogger_json/lists"}