{"id":16701478,"url":"https://github.com/ferd/flatlog","last_synced_at":"2025-09-12T04:33:40.873Z","repository":{"id":62429617,"uuid":"157755444","full_name":"ferd/flatlog","owner":"ferd","description":"A custom formatter for the Erlang logger application that turns maps into single line text logs","archived":false,"fork":false,"pushed_at":"2022-08-17T14:38:26.000Z","size":23,"stargazers_count":53,"open_issues_count":0,"forks_count":9,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-08-16T08:44:05.382Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Erlang","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ferd.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-11-15T18:33:23.000Z","updated_at":"2025-02-21T10:41:44.000Z","dependencies_parsed_at":"2022-11-01T19:47:09.814Z","dependency_job_id":null,"html_url":"https://github.com/ferd/flatlog","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/ferd/flatlog","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ferd%2Fflatlog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ferd%2Fflatlog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ferd%2Fflatlog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ferd%2Fflatlog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ferd","download_url":"https://codeload.github.com/ferd/flatlog/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ferd%2Fflatlog/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274753617,"owners_count":25342825,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-12T02:00:09.324Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-12T18:44:14.476Z","updated_at":"2025-09-12T04:33:40.856Z","avatar_url":"https://github.com/ferd.png","language":"Erlang","funding_links":[],"categories":[],"sub_categories":[],"readme":"flatlog\n=====\n\nA custom formatter for the logger application that turns maps into single line\ntext logs.\n\nWhy?\n----\n\nStructured logging is a better approach to logging in general. Fields are more\nclearly defined, tool-assisted parsing and consuming of logs is simpler, and\nthe structure is amenable to good filtering in global or handler filters for\nthe Erlang `logger` library.\n\nYou could, for example, emit all your logs as structured logs (just maps), and\nset up multiple handlers for them:\n\n- an audit handler, for all critical issues and configuration changes, which\n  get stored on disk and remotely for long periods of time\n- an info log for users, which gets a shorter term durability\n- an error log for support tickets, which may instead have a targeted retention\n  for a few weeks\n- a special handler that parses the structured logs and forwards them to a\n  distributed tracing framework such as opencensus\n- extract or hide metrics from logs if you integrate with such a system, and do\n  it cheaply by just nesting (or removing) a metrics map in the overall report.\n\nThis can be done transparently and after the fact, without major structural\nimpact to the call site. It lets you far more easily decouple log generation\nfrom its consumption at no heavy cost.\n\nThis formatter focuses on providing a text-based single-line format for\nstructured logs, which can be human-readable, while being useful to people who\nuse grep or awk to process logs, or want to forward them to a consumer like\nsyslogd.\n\nUsage\n-----\n\nIt is recommended that if you are providing a library, you do not add this\nproject as a dependency. A code formatter of this kind should be added to a\nproject in its release repository as a top-level final presentational concern.\n\nOnce the project is added, replace the formatter of the default handler (or add\na custom handler) for structured logging to your `sys.config` file:\n\n```erlang\n[\n {kernel, [\n    {logger, [\n        {handler, default, logger_std_h,\n         #{formatter =\u003e {flatlog, #{\n            map_depth =\u003e 3,\n            term_depth =\u003e 50\n          }}}\n        }\n    ]},\n    {logger_level, info}\n ]}\n].\n```\n\nThe logging output will then be supported. Calling the logger like:\n\n```erlang\n?LOG_ERROR(\n    #{type =\u003e event, in =\u003e config, user =\u003e #{name =\u003e \u003c\u003c\"bobby\"\u003e\u003e, id =\u003e 12345},\n      action =\u003e change_password, result =\u003e error, details =\u003e {entropy, too_low},\n      txt =\u003e \u003c\u003c\"user password not strong enough\"\u003e\u003e}\n)\n```\n\nWill produce a single log line like:\n\n```text\nwhen=2018-11-15T18:16:03.411822+00:00 level=error pid=\u003c0.134.0\u003e\nat=config:update/3:450 user_name=bobby user_id=12345 type=event\ntxt=\"user password not strong enough\" result=error in=config\ndetails={entropy,too_low} action=change_password\n```\n\nDo note that the `user` map gets flattened such that `#{user =\u003e #{name =\u003e\nbobby}}` gets turned into `user_name=bobby`, ensuring that various subfields in\ndistinct maps will not clash.\n\nThe default template supplied with the library also includes optional fields for\nidentifiers as used in distributed tracing framework which can be set in the metadata\nfor the logger framework, either explicitly or as a process state. The fields are:\n\n- `id` for individual request identifiers\n- `parent_id` for the event or command that initially caused the current\n  logging event to happen\n- `correlation_id` for groupings of related events\n\nLogs that are not reports (maps) are going to be formatted and handled such\nthat they can be put inside a structured log. For example:\n\n```erlang\n?LOG_INFO(\"hello ~s\", [\"world\"])\n```\n\nWill result in:\n\n```text\nwhen=2018-11-15T18:16:03.411822+00:00 level=info pid=\u003c0.134.0\u003e\nat=some:code/0:15 unstructured_log=\"hello world\"\n```\n\nDo note that if you are building a release, you will need to manually add\nthe `flatlog` dependency to your `relx` configuration, since it is\ntechnically not a direct dependency of any application in your system.\n\nTest\n----\n\n```sh\nrebar3 check\n```\n\nFeatures\n--------\n\n- Printing rules similar to the default Erlang logger formatter, but extended\n  for binary values that can be represented as text. I.e. rather than\n  `\u003c\u003c\"hello\"\u003e\u003e`, the value `hello` will be output. A non-representable value\n  will revert to `\u003c\u003c...\u003e\u003e`\n- Linebreaks are escaped to ensure all logs are always on one line, and strings\n  that contain spaces or equal signs (` ` and `=`) are quoted such that\n  `\"key=name\"=\"hello world\"` to be clear.\n- Term depth applies on a per-term basis before a data structure is elided with `...`\n- Map depth is controllable independently to deal with recursion vs. complexity\n  of terms\n- Colored output can be enabled with `colored =\u003e true`. One can color certain\n  parts of the output using `colored_start` and `colored_end` in `template`.\n  Per-level colors can be configured with `colored_{log level}`.\n\nCaveats\n-------\n\n- No max line length is enforced at the formatter level, since the ordering of\n  terms in maps is not defined and it could be risky to cut logs early. If a max\n  line length is to be enforced, you should wrap this formatter into your own.\n- Escaping of keys does not carry well to nested maps. I.e. the map\n  `#{a_b =\u003e #{\"c d\" =\u003e x}}` is not well supported: `a_b_\"c d\"=x` will be\n  returned, which is nonsensical. For nested maps, you have the\n  responsibility of ensuring composability.\n- The transformations to the log line format is not lossless; it is not\n  serialization.\n\n  Information is lost regarding whether the initial term was a binary, a\n  string, or an atom. Similarly, naming a key `user_password` may make it seem\n  like the `user` map leaks a `password` field, but it is an unrelated field\n  that looks similar due to flattening.\n\n  If this is unacceptable, you might want to choose another structured log\n  format such as JSON.\n\nRoadmap\n-------\n\n- integration tests\n- add example basic usage\n- add example usage with optional tracing for IDs\n- clean up test suites\n- incorporating lager's safer truncating logic (might be a breaking change\n  prior to 1.0.0)\n\nChangelog\n---------\n\n- 0.1.2: added a check for old `error_logger` calls (thanks @hommeabeil)\n- 0.1.1: added optionally colored logs (thanks @pfenoll)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fferd%2Fflatlog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fferd%2Fflatlog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fferd%2Fflatlog/lists"}