{"id":15022551,"url":"https://github.com/puppetlabs/structured-logging","last_synced_at":"2025-04-05T16:08:05.106Z","repository":{"id":1622825,"uuid":"43309909","full_name":"puppetlabs/structured-logging","owner":"puppetlabs","description":"Write data structures to your logs from clojure","archived":false,"fork":false,"pushed_at":"2025-02-04T02:23:19.000Z","size":30,"stargazers_count":29,"open_issues_count":2,"forks_count":8,"subscribers_count":142,"default_branch":"main","last_synced_at":"2025-03-29T15:07:41.794Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Clojure","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/puppetlabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null}},"created_at":"2015-09-28T15:38:30.000Z","updated_at":"2024-08-14T14:24:48.000Z","dependencies_parsed_at":"2022-09-11T08:21:24.826Z","dependency_job_id":null,"html_url":"https://github.com/puppetlabs/structured-logging","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/puppetlabs%2Fstructured-logging","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/puppetlabs%2Fstructured-logging/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/puppetlabs%2Fstructured-logging/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/puppetlabs%2Fstructured-logging/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/puppetlabs","download_url":"https://codeload.github.com/puppetlabs/structured-logging/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247361689,"owners_count":20926643,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-24T19:58:06.434Z","updated_at":"2025-04-05T16:08:05.081Z","avatar_url":"https://github.com/puppetlabs.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"# puppetlabs/structured-logging [![Clojars Project](https://img.shields.io/clojars/v/puppetlabs/structured-logging.svg)](https://clojars.org/puppetlabs/structured-logging) [![Build Status](https://travis-ci.org/puppetlabs/structured-logging.svg?branch=master)](https://travis-ci.org/puppetlabs/structured-logging)\n\n`structured-logging` is a library that helps you to:\n - write arbitrary JSON to your logs, making it easier to interface with log analysis tools.\n - write messages to arbitrarily named loggers, instead of just the one named\n   after the namespace in which the log statement appears.\n\nIt is built on `clojure.tools.logging`, but it only works with logback. \n\n## Usage\n\n### maplog\n\nStructured log entries are created by calling `maplog` like this:\n\n    (maplog :warn {:user-service \"https://...\", :status 503, :elapsed 27}\n            #(format \"Failed to query user-service %s. Response: status %d\"\n                     (:user-service %) (:status %)))\n\nThe second parameter is the structured data you want to log, and any\nclojure map is ok, as long as\n[Cheshire](https://github.com/dakrone/cheshire) can handle it.\n\nThe log message itself is generated by passing the structured data to\nthe function provided as the final argument.  This provides a general\nway to support arbitrary formatters, including some like those in\n[clj-i18n](https://github.com/puppetlabs/clj-i18n), which require\nthat their invocations appear literally, at each translation call\nsite, i.e.:\n\n    (maplog :info {:temperature t :stamp (now)}\n            #(i18n/trs \"Temperature has exceeded {0} at {1}\"\n                       (:temperature %) (:stamp %)))\n\n### Named loggers\n\nWhen you would pass a log-level parameter, you may instead supply a vector of\n`[:custom-logger :level]`. This will log a message to the logger named\n\"custom-logger\". Then you can specifically address that logger in your logback\nconfiguration (to redirect it to a different log file, for example).\n\n## Common fields\n\nIf you use the recommended logger configuration, as described below, you will\nsee the following fields in each JSON log message:\n\n* @timestamp\n* message\n* logger_name\n* thread_name\n* level\n* level_value (numeric, suitable for sorting)\n* stack_trace\n\nAdditional relevant fields may be added in any given message, but this base set\nwill always be present.\n\n## Configuration\n### JSON text\n\nIf you want to log JSON data where you would otherwise log regular text, replace the `encoder` element in\nyour `logback.xml` with this one:\n\n    \u003cencoder class=\"net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder\"\u003e\n      \u003cproviders\u003e\n        \u003ctimestamp/\u003e\n        \u003cmessage/\u003e\n        \u003cloggerName/\u003e\n        \u003cthreadName/\u003e\n        \u003clogLevel/\u003e\n        \u003clogLevelValue/\u003e\n        \u003cstackTrace/\u003e\n        \u003clogstashMarkers/\u003e\n      \u003c/providers\u003e\n    \u003c/encoder\u003e\n\nEven though this says 'logstash' on it, it works completely independently from\nany log aggregation system. The final `\u003clogstashMarkers/\u003e` element is\nimportant - that inserts our custom properties into each json message.\n\n## Logstash integration\n\nYou can also log directly to logstash with an appender configured like this:\n\n    \u003cappender name=\"stash\" class=\"net.logstash.logback.appender.LogstashTcpSocketAppender\"\u003e\n      \u003cremoteHost\u003emy.great.logstash.server.com\u003c/remoteHost\u003e\n      \u003cport\u003e4560\u003c/port\u003e\n\n      \u003cencoder class=\"net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder\"\u003e\n        \u003cproviders\u003e\n          \u003ctimestamp/\u003e\n          \u003cmessage/\u003e\n          \u003cloggerName/\u003e\n          \u003cthreadName/\u003e\n          \u003clogLevel/\u003e\n          \u003clogLevelValue/\u003e\n          \u003cstackTrace/\u003e\n          \u003clogstashMarkers/\u003e\n        \u003c/providers\u003e\n      \u003c/encoder\u003e\n    \u003c/appender\u003e\n\nYou will also need to add a reference to the appender from the `\u003croot\u003e` element:\n\n    \u003croot\u003e\n      ... \n      \u003cappender-ref ref=\"stash\" /\u003e\n    \u003c/root\u003e\n\n\n\n## Log Output\n\nWe are using a json encoder that comes with the `logstash-logback` integration\nlibrary. (https://github.com/logstash/logstash-logback-encoder) This can be used\nin conjunction with the logstash tcp appender, or in conjunction with a typical\nfile appender. In either case, the json for the above log message looks\nsomething like this:\n\n    {\n      \"@timestamp\": \"...\",\n      \"level\": \"WARN\",\n      \"user-service\": \"https://...\",\n      \"status\": 503,\n      \"elapsed\": 27\n    }\n\nThe exact fields that show up depend on how you've configured the encoder;\n'@timestamp' is a logstash convention.\n\n## Answers to expected questions\n\n*Can I embed arbitrary structures in there?*\n\nYou can log anything that cheshire can serialize, but you may wish to stick\nto simple key-value formats to keep your logs easy to analyze.\n\n*What happens when I log a map with :level as the key?*\n\nDon't do that. It's not yet clear how this case should be handled. For now,\navoid such keys. Namespacing your map keys may be prudent if you're worried\nabout collisions.\n\n*Does this library require logstash?*\n\nNo. It does depend on the json encoder that comes with the logback-logstash\nintegration library, but you can point the json it spits out at any old logback\nappender. Critically, it has a feature that allows us to edit the json before it\ngets written, which is where we add our custom information. We *could* write\nsuch an encoder ourselves, but they already wrote it.\n\n*Why didn't you use one of the other logback-json encoders?*\n\nBecause those just encode the normal log event fields as json; the useful\nfeature here is the ability to include arbitrary data as part of the json.\n\n## Tips\n\nStructured logging is a slippery slope and should be applied where it has\nclear benefits. While we've streamlined it a lot, this is heavier than regular\nlogging, and it's still non-application code that you have to scatter throughout\nyour program.\n\n## License\n\nCopyright © 2015 Puppet Labs\n\nDistributed under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0.html)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpuppetlabs%2Fstructured-logging","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpuppetlabs%2Fstructured-logging","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpuppetlabs%2Fstructured-logging/lists"}