https://github.com/logfellow/logstash-logback-encoder

Logback JSON encoder and appenders
https://github.com/logfellow/logstash-logback-encoder

json logback logback-appender logstash

Last synced: 6 months ago
JSON representation

Logback JSON encoder and appenders

Host: GitHub
URL: https://github.com/logfellow/logstash-logback-encoder
Owner: logfellow
License: apache-2.0
Created: 2013-03-04T16:19:08.000Z (over 13 years ago)
Default Branch: main
Last Pushed: 2025-11-10T00:03:27.000Z (7 months ago)
Last Synced: 2025-12-08T19:21:36.579Z (6 months ago)
Topics: json, logback, logback-appender, logstash
Language: Java
Homepage:
Size: 3.64 MB
Stars: 2,511
Watchers: 62
Forks: 415
Open Issues: 35
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md

Awesome Lists containing this project

awesome-java - Logstash Logback Encoder

README

> [!IMPORTANT]
> This document applies to the next version under development.
>
> See [here for documentation on the latest released version](https://github.com/logfellow/logstash-logback-encoder/tree/logstash-logback-encoder-9.0).

# Logstash Logback Encoder

[![Build](https://github.com/logfellow/logstash-logback-encoder/actions/workflows/build.yml/badge.svg?branch=main)](https://github.com/logfellow/logstash-logback-encoder/actions/workflows/build.yml)
[![Javadocs](http://www.javadoc.io/badge/net.logstash.logback/logstash-logback-encoder.svg)](http://www.javadoc.io/doc/net.logstash.logback/logstash-logback-encoder)
[![Maven Central](https://img.shields.io/maven-central/v/net.logstash.logback/logstash-logback-encoder)](https://search.maven.org/artifact/net.logstash.logback/logstash-logback-encoder)
[![Release Notes](https://img.shields.io/github/v/release/logfellow/logstash-logback-encoder?label=release%20notes)](https://github.com/logfellow/logstash-logback-encoder/releases/latest)

Provides [logback](http://logback.qos.ch/) encoders, layouts, and appenders to log in JSON and [other formats supported by Jackson](#data-format).

Supports both regular _LoggingEvents_ (logged through a `Logger`) and _AccessEvents_ (logged via [logback-access](http://logback.qos.ch/access.html)).

Originally written to support output in [logstash](https://www.elastic.co/guide/en/logstash/current)'s JSON format, but has evolved into a highly-configurable, general-purpose, structured logging mechanism for JSON and other Jackson dataformats.
The structure of the output, and the data it contains, is fully configurable.

#### Contents:

* [Including it in your project](#including-it-in-your-project)
* [Java Version Requirements](#java-version-requirements)
* [Usage](#usage)
* [UDP Appenders](#udp-appenders)
* [TCP Appenders](#tcp-appenders)
* [Keep-alive](#keep-alive)
* [Multiple Destinations](#multiple-destinations)
* [Reconnection Delay](#reconnection-delay)
* [Connection Timeout](#connection-timeout)
* [Write Buffer Size](#write-buffer-size)
* [Write Timeout](#write-timeout)
* [Initial Send Delay](#initial-send-delay)
* [SSL](#ssl)
* [Async Appenders](#async-appenders)
* [RingBuffer Size](#ringbuffer-size)
* [RingBuffer Full](#ringbuffer-full)
* [Graceful Shutdown](#graceful-shutdown)
* [Wait Strategy](#wait-strategy)
* [Appender Listeners](#appender-listeners)
* [Encoders / Layouts](#encoders--layouts)
* [Threads and ThreadLocals](#threads-and-threadlocals)
* [LoggingEvent Fields](#loggingevent-fields)
* [Standard Fields](#standard-fields)
* [MDC fields](#mdc-fields)
* [Key Value Pair fields](#key-value-pair-fields)
* [Context fields](#context-fields)
* [Caller Info Fields](#caller-info-fields)
* [Custom Fields](#custom-fields)
* [Global Custom Fields](#global-custom-fields)
* [Event-specific Custom Fields](#event-specific-custom-fields)
* [AccessEvent Fields](#accessevent-fields)
* [Standard Fields](#standard-fields-1)
* [Header Fields](#header-fields)
* [Customizing Jackson](#customizing-jackson)
* [Data Format](#data-format)
* [Customizing TokenStreamFactory, ObjectMapper, and JsonGenerator](#customizing-tokenstreamfactory-objectmapper-and-jsongenerator)
* [Registering Jackson Modules](#registering-jackson-modules)
* [Customizing Character Escapes](#customizing-character-escapes)
* [Masking](#masking)
* [Customizing Standard Field Names](#customizing-standard-field-names)
* [Customizing Version](#customizing-version)
* [Customizing Timestamp](#customizing-timestamp)
* [Customizing LoggingEvent Message](#customizing-loggingevent-message)
* [Customizing AccessEvent Message](#customizing-accessevent-message)
* [Customizing Logger Name Length](#customizing-logger-name-length)
* [Customizing Stack Traces](#customizing-stack-traces)
* [Omit Common Frames](#omit-common-frames)
* [Truncate after Regex](#truncate-after-regex)
* [Exclude Frames per Regex](#exclude-frames-per-regex)
* [Omit Throwable Messages](#omit-throwable-messages)
* [Maximum Depth per Throwable](#maximum-depth-per-throwable)
* [Maximum Trace Size (bytes)](#maximum-trace-size-bytes)
* [Classname Shortening](#classname-shortening)
* [Custom Line Separator](#custom-line-separator)
* [Root Cause First](#root-cause-first)
* [Conditional Output](#conditional-output)
* [Stack Hashes](#stack-hashes)
* [Using with PatternLayout](#using-with-patternlayout)
* [Registering Additional Providers](#registering-additional-providers)
* [Prefix/Suffix/Separator](#prefixsuffixseparator)
* [Composite Encoder/Layout](#composite-encoderlayout)
* [Providers common to LoggingEvents and AccessEvents](#providers-common-to-loggingevents-and-accessevents)
* [Providers for LoggingEvents](#providers-for-loggingevents)
* [Providers for AccessEvents](#providers-for-accessevents)
* [Nested JSON Provider](#nested-json-provider)
* [Pattern JSON Provider](#pattern-json-provider)
* [LoggingEvent patterns](#loggingevent-patterns)
* [AccessEvent patterns](#accessevent-patterns)
* [Custom JSON Provider](#custom-json-provider)
* [Status Listeners](#status-listeners)
* [Joran/XML Configuration](#joranxml-configuration)
* [Duration Property](#duration-property)
* [Comma separated list of values](#comma-separated-list-of-values)

## Including it in your project

Maven style:

```xml

net.logstash.logback
logstash-logback-encoder
9.0

ch.qos.logback
logback-classic
1.5.20

```

If you get `ClassNotFoundException`/`NoClassDefFoundError`/`NoSuchMethodError` at runtime,
then ensure the required dependencies (and appropriate versions) as specified in the pom file
from the maven repository exist on the runtime classpath.
Specifically, the following need to be available on the runtime classpath:

* jackson-databind / jackson-core / jackson-annotations >= 3.0.0
* logback-core >= 1.5.0
* logback-classic >= 1.5.0 (required for logging _LoggingEvents_)
* logback-access >= 2.0.0 (required for logging _AccessEvents_)
* slf4j-api (usually comes as a transitive dependency of logback-classic)
* java-uuid-generator (required if the `uuid` provider is used)

Older versions than the ones specified in the pom file _might_ work, but the versions in the pom file are what testing has been performed against.
Support for logback versions prior to 1.3.0 was removed in logstash-logback-encoder 7.4.
Support for jackson versions prior to 3.0.0 was removed in logstash-logback-encoder 9.0.

If you are using logstash-logback-encoder in a project (such as spring-boot) that also declares dependencies on any of the above libraries, you might need to tell maven explicitly which versions to use to avoid conflicts.
You can do so using maven's [dependencyManagement](https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Management) feature.
For example, to ensure that maven doesn't pick different versions of logback-core, logback-classic, and logback-access, add this to your project's pom.xml

```xml

1.5.20
2.0.6

ch.qos.logback
logback-core
${logback-classic.version}

ch.qos.logback
logback-classic
${logback-classic.version}

ch.qos.logback.access
logback-access-common
${logback-access.version}

```

## Java Version Requirements

| logstash-logback-encoder | Minimum Java Version supported |
|--------------------------|--------------------------------|
| 9.x | 17 |
| 8.x | 11 |
| 7.x | 8 |
| 6.x | 8 |
| 5.x | 7 |
| <= 4.x | 6 |

## Usage

To log using JSON format, you must configure logback to use either:

* an appender provided by the logstash-logback-encoder library, OR
* an appender provided by logback (or another library) with an encoder or layout provided by the logstash-logback-encoder library

The appenders, encoders, and layouts provided by the logstash-logback-encoder library are as follows:

| Format | Protocol | Function | LoggingEvent | AccessEvent |
|---------------|------------|----------|------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------|
| Logstash JSON | Syslog/UDP | Appender | [`LogstashUdpSocketAppender`](/src/main/java/net/logstash/logback/appender/LogstashUdpSocketAppender.java) | [`LogstashAccessUdpSocketAppender`](/src/main/java/net/logstash/logback/appender/LogstashAccessUdpSocketAppender.java) |
| Logstash JSON | TCP | Appender | [`LogstashTcpSocketAppender`](/src/main/java/net/logstash/logback/appender/LogstashTcpSocketAppender.java) | [`LogstashAccessTcpSocketAppender`](/src/main/java/net/logstash/logback/appender/LogstashAccessTcpSocketAppender.java) |
| any | any | Appender | [`LoggingEventAsyncDisruptorAppender`](/src/main/java/net/logstash/logback/appender/LoggingEventAsyncDisruptorAppender.java) | [`AccessEventAsyncDisruptorAppender`](/src/main/java/net/logstash/logback/appender/AccessEventAsyncDisruptorAppender.java) |
| Logstash JSON | any | Encoder | [`LogstashEncoder`](/src/main/java/net/logstash/logback/encoder/LogstashEncoder.java) | [`LogstashAccessEncoder`](/src/main/java/net/logstash/logback/encoder/LogstashAccessEncoder.java) |
| Logstash JSON | any | Layout | [`LogstashLayout`](/src/main/java/net/logstash/logback/layout/LogstashLayout.java) | [`LogstashAccessLayout`](/src/main/java/net/logstash/logback/layout/LogstashAccessLayout.java) |
| General JSON | any | Encoder | [`LoggingEventCompositeJsonEncoder`](/src/main/java/net/logstash/logback/encoder/LoggingEventCompositeJsonEncoder.java) | [`AccessEventCompositeJsonEncoder`](/src/main/java/net/logstash/logback/encoder/AccessEventCompositeJsonEncoder.java) |
| General JSON | any | Layout | [`LoggingEventCompositeJsonLayout`](/src/main/java/net/logstash/logback/layout/LoggingEventCompositeJsonLayout.java) | [`AccessEventCompositeJsonLayout`](/src/main/java/net/logstash/logback/encoder/AccessEventCompositeJsonLayout.java) |

These encoders/layouts can generally be used by any logback appender (such as `RollingFileAppender`).

The general _composite_ JSON encoders/layouts can be used to
output any JSON format/data by configuring them with various JSON _providers_.
The Logstash encoders/layouts are really just extensions of the general
composite JSON encoders/layouts with a pre-defined set of providers.

The logstash encoders/layouts are easier to configure if you want to use the standard logstash version 1 output format.
Use the [composite encoders/layouts](#composite-encoderlayout) if you want to heavily customize the output,
or if you need to use logstash version 0 output.

The `*AsyncDisruptorAppender` appenders are similar to logback's `AsyncAppender`,
except that a [LMAX Disruptor RingBuffer](https://lmax-exchange.github.io/disruptor/)
is used as the queuing mechanism, as opposed to a `BlockingQueue`.
These async appenders can delegate to any other underlying logback appender.

### UDP Appenders

To output JSON for LoggingEvents to a syslog/UDP channel,
use the `LogstashUdpSocketAppender` with a `LogstashLayout` or `LoggingEventCompositeJsonLayout`
in your `logback.xml`, like this:

```xml

MyAwesomeSyslogServer

514

```
You can further customize the JSON output by customizing the layout as described in later sections.

For example, to configure [global custom fields](#global-custom-fields), you can specify
```xml

MyAwesomeSyslogServer

514

{"appname":"myWebservice"}

```

To output JSON for AccessEvents over UDP, use a `LogstashAccessUdpSocketAppender`
with a `LogstashAccessLayout` or `AccessEventCompositeJsonLayout`
in your `logback-access.xml`, like this:

```xml

MyAwesomeSyslogServer

514

{"appname":"myWebservice"}

```

To receive syslog/UDP input in logstash, configure a [`syslog`](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-syslog.html) or [`udp`](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-udp.html) input with the [`json`](https://www.elastic.co/guide/en/logstash/current/plugins-codecs-json.html) codec in logstash's configuration like this:
```
input {
syslog {
codec => "json"
}
}
```

### TCP Appenders

To output JSON for LoggingEvents over TCP, use a `LogstashTcpSocketAppender`
with a `LogstashEncoder` or `LoggingEventCompositeJsonEncoder`
in your `logback.xml`, like this:

```xml

127.0.0.1:4560

```

To output JSON for AccessEvents over TCP, use a `LogstashAccessTcpSocketAppender`
with a `LogstashAccessEncoder` or `AccessEventCompositeJsonEncoder`
in your `logback-access.xml`, like this:

```xml

127.0.0.1:4560

```

The TCP appenders use an encoder, rather than a layout as the [UDP appenders](#udp-appenders) .
You can use a `Logstash*Encoder`, `*EventCompositeJsonEncoder`, or any other logback encoder.
All the output formatting options are configured at the encoder level.

Internally, the TCP appenders are asynchronous (using the [LMAX Disruptor RingBuffer](https://lmax-exchange.github.io/disruptor/)).
All the encoding and TCP communication is delegated to a single writer thread.
There is no need to wrap the TCP appenders with another asynchronous appender
(such as `AsyncAppender` or `LoggingEventAsyncDisruptorAppender`).

All the configuration parameters (except for sub-appender) of the [async appenders](#async-appenders) are valid for TCP appenders. For example, `waitStrategyType` and `ringBufferSize`.

By default, the TCP appenders will never block the logging thread - if the RingBuffer is full (e.g. due to slow network, etc), then events will be dropped. If desired, the appender can also be configured to block and wait for free space, see [RingBuffer Full](#ringbuffer-full) for more information.

The TCP appenders will automatically reconnect if the connection breaks. Multiple destinations can be configured to increase availability and reduce message lost. See [Multiple Destinations](#multiple-destinations) for more information.

To receive TCP input in logstash, configure a [`tcp`](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-tcp.html) input with the [`json_lines`](https://www.elastic.co/guide/en/logstash/current/plugins-codecs-json_lines.html) codec in logstash's configuration like this:

```
input {
tcp {
port => 4560
codec => json_lines
}
}
```

To guarantee that logged messages have had a chance to be processed by the TCP appender, you'll need to [cleanly shut down logback](http://logback.qos.ch/manual/configuration.html#stopContext) when your application exits.

#### Keep-Alive

If events occur infrequently, and the connection breaks consistently due to a server-side idle timeout,
then you can enable keep alive functionality by configuring a `keepAliveDuration` like this:

```xml

...
5 minutes

```

This setting accepts a Logback Duration value - see the section dedicated to [Duration Property](#duration-property) for more information about the valid values.

When the `keepAliveDuration` is set, then a keep alive message will be sent if an event has not occurred for the length of the duration.
The keep alive message defaults to unix line ending (`\n`), but can be changed by setting the `keepAliveMessage` property to the desired value. The following values have special meaning:

- ``: no keep alive
- `SYSTEM`: system's line separator
- `UNIX`: unix line ending (`\n`)
- `WINDOWS`: windows line ending (`\r\n`)

Any other value will be used as-is.

The keep alive message is encoded in `UTF-8` by default. This can be changed by setting the `keepAliveCharset` property to the name of the desired charset.

#### Multiple Destinations

The TCP appenders can be configured to try to connect to one of several destinations like this:

```xml

destination1.domain.com:4560
destination2.domain.com:4560
destination3.domain.com:4560

...

```

or this:

```xml

destination1.domain.com:4560,
destination2.domain.com:4560,
destination3.domain.com:4560

...

```

Destinations are expressed using the following format: `host[:port]` where:
- `host` can be a hostname (eg. `localhost`) , an IPv4 address (eg. `192.168.1.1`) or an IPv6 address enclosed between brackets (eg. `[2001:db8::1]`).
- `port` is optional and, if specified, must be prefixed by a colon (`:`). It must be a valid integer value between `0` and `65535`.

The appender uses a `connectionStrategy` to determine:

* the order in which destination connections are attempted, and
* when an established connection should be reestablished (to the next destination selected by the connection strategy).

Logs are only sent to one destination at a time (i.e. not all destinations).
By default, the appender will stay connected to the connected destination
until it breaks, or until the application is shut down.
Some connection strategies can force a reconnect (see below).
If a connection breaks, then the appender will attempt to connect
to the next destination selected by the connection strategy.

The available connection strategies are as follows:

Strategy
Description

preferPrimary
(default)
The first destination is considered the primary destination.
Each additional destination is considered a secondary destination.
This strategy prefers the primary destination, unless it is down.
The appender will attempt to connect to each destination in the order in which they are configured.
If a connection attempt fails, thes the appender will attempt to connect to the next destination.
If a connection succeeds, and then closes before the minConnectionTimeBeforePrimary
has elapsed, then the appender will attempt to connect to the next destination.
If a connection succeeds, and then closes after the minConnectionTimeBeforePrimary
has elapsed, then the appender will attempt to connect
to the destinations in the order in which they are configured,
starting at the first/primary destination.

The secondaryConnectionTTL can be set to gracefully close connections to secondary
destinations after a specific duration. This will force the
the appender to reattempt to connect to the destinations in order again.
The secondaryConnectionTTL value does not affect connections to the
primary destination.

The minConnectionTimeBeforePrimary (10 seconds by default) specifies
the minimum amount of time that a sucessfully established connection
must remain open before the next connection attempt will try the primary.
i.e. If a connection stays open less than this amount of time,
then the next connection attempt will attempt the next destination (instead of the primary).
This is used to prevent a connection storm to the primary in the case the
primary accepts a connection, and then immediately closes it.

Example:


  <appender name="stash" class="net.logstash.logback.appender.LogstashTcpSocketAppender">

      <destination>destination1.domain.com:4560</destination>

      <destination>destination2.domain.com:4560</destination>

      <destination>destination3.domain.com:4560</destination>

      <connectionStrategy>

          <preferPrimary>

              <secondaryConnectionTTL>5 minutes</secondaryConnectionTTL>

          </preferPrimary>

      </connectionStrategy>

  </appender>

roundRobin

This strategy attempts connections to the destination in round robin order.
If a connection fails, the next destination is attempted.

The connectionTTL can be set to gracefully close connections after a specific duration.
This will force the the appender to reattempt to connect to the next destination.

Example:


  <appender name="stash" class="net.logstash.logback.appender.LogstashTcpSocketAppender">

      <destination>destination1.domain.com:4560</destination>

      <destination>destination2.domain.com:4560</destination>

      <destination>destination3.domain.com:4560</destination>

      <connectionStrategy>

          <roundRobin>

              <connectionTTL>5 minutes</connectionTTL>

          </roundRobin>

      </connectionStrategy>

  </appender>

random

This strategy attempts connections to the destination in a random order.
If a connection fails, the next random destination is attempted.

The connectionTTL can be set to gracefully close connections after a specific duration.
This will force the the appender to reattempt to connect to the next random destination.

Example:


  <appender name="stash" class="net.logstash.logback.appender.LogstashTcpSocketAppender">

      <destination>destination1.domain.com:4560</destination>

      <destination>destination2.domain.com:4560</destination>

      <destination>destination3.domain.com:4560</destination>

      <connectionStrategy>

          <random>

              <connectionTTL>5 minutes</connectionTTL>

          </random>

      </connectionStrategy>

  </appender>

You can also use your own custom connection strategy by implementing the `DestinationConnectionStrategy` interface,
and configuring the appender to use it like this:

```xml

destination1.domain.com:4560
destination2.domain.com:4560
destination3.domain.com:4560

```

#### Reconnection Delay

By default, the TCP appender will wait 30 seconds between connection attempts to a single destination.
The time between connection attempts to each destination is tracked separately.

This amount of time to delay can be changed by setting the `reconnectionDelay` field.

```xml

...
1 second

```

This setting accepts a Logback Duration value - see the section dedicated to [Duration Property](#duration-property) for more information about the valid values.

#### Connection Timeout

By default, a connection timeout of 5 seconds is used when connecting to a remote destination.
You can adjust this by setting the appender's `connectionTimeout` configuration property to the desired value.

```xml

...
5 seconds

```

A value of `0` means "don't use a timeout and wait indefinitely" which often really means "use OS defaults".

This setting accepts a Logback Duration value - see the section dedicated to [Duration Property](#duration-property) for more information about the valid values.

#### Write Buffer Size

By default, a buffer size of `8192` bytes is used to buffer socket output stream writes.
You can adjust this by setting the appender's `writeBufferSize`.

```xml

...
16384

```

Buffering can be disabled by setting the `writeBufferSize` to `0`.
Consider disabling the write buffer if you are concerned about losing data from the buffer for flaky connections.
Disabling the buffer can potentially slow down the writer thread due to increased system calls,
but in some environments, this does not seem to affect overall performance.
See [this discussion](https://github.com/logfellow/logstash-logback-encoder/issues/342).

#### Write Timeout

If a destination stops reading from its socket input, but does not close the connection, then writes from the TCP appender will eventually backup, causing the ring buffer to backup, causing events to be dropped.

To detect this situation, you can enable a write timeout, so that "stuck" writes will eventually timeout, at which point the connection will be re-established.
When the [write buffer](#write-buffer-size) is enabled, any buffered data will be lost when the connection is reestablished.

By default there is no write timeout. To enable a write timeout, do the following:

```xml

...
1 minute

```

Note that since the blocking java socket output stream used to send events does not have a concept of a write timeout, write timeouts are detected using a task scheduled periodically with the same frequency as the write timeout.
For example, if the write timeout is set to 30 seconds, then a task will execute every 30 seconds to see if 30 seconds has elapsed since the start of the current write operation.
Therefore, it is recommended to use longer write timeouts (e.g. > 30s, or minutes), rather than short write timeouts, so that this task does not execute too frequently.
Also, this approach means that it could take up to two times the write timeout before a write timeout is detected.

The write timeout must be >0. A timeout of zero is interpreted as an infinite timeout which effecively means "no write timeout".

This setting accepts a Logback Duration value - see the section dedicated to [Duration Property](#duration-property) for more information about the valid values.

#### Initial Send Delay

The appender starts writing the events stored in the queue as soon as the connection is established. In some cases you may want to add an extra delay before sending the first events after the connection is established. This may come in handy in situations where the appender connects to an intermediate proxy that needs some time to establish a connection to the final destination. If the appender starts writing immediately, events may be lost in-flight if the proxy ultimately fails to connect to the final destination.

To enable this feature, set the `initialSendDelay` to the desired delay before the first event is sent after the connection is established. If the connection is lost before the delay expires, no event will be lost. The default value is `0` which means no delay and start flusing pending events immediately.

The following example configures a delay of 5 secondes before writing in the new connection:

```xml

...
5 secondes

```

This setting accepts a Logback Duration value - see the section dedicated to [Duration Property](#duration-property) for more information about the valid values.

#### SSL

To use SSL, add an `` sub-element within the `` element for the `LogstashTcpSocketAppender`
or `LogstashAccessTcpSocketAppender`.

See the [logback manual](http://logback.qos.ch/manual/usingSSL.html) for how to configure SSL.
SSL for the `Logstash*TcpSocketAppender`s are configured the same way as logback's `SSLSocketAppender`.

For example, to enable SSL using the JVM's default keystore/truststore, do the following:

```xml

...

```

To use a different truststore, do the following:

```xml

...

classpath:server.truststore
${server.truststore.password}

```

All the customizations that [logback](http://logback.qos.ch/manual/usingSSL.html) offers
(such as configuring cipher specs, protocols, algorithms, providers, etc.)
are supported by the `Logback*TcpSocketAppender`s.

See the logstash documentation for the [`tcp`](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-tcp.html) input for how to configure it to use SSL.

### Async Appenders

For example:

```xml

...

```

> [!WARNING]
> Logback 1.3 no longer allows declaring an `` inside another ``.
> The nested appender should instead be declared outside and `` must be used to refer to it.
>
> See [LOGBACK-1674](https://jira.qos.ch/browse/LOGBACK-1674) for more information.

#### RingBuffer Size

Logging events are first enqueued in a ring buffer before they are delivered to their final destination by a separate handler thread.
The buffer size is fixed, it does not grow or shrink at runtime. Its size is determined by the `ringBufferSize` configuration property set to `8192` by default.

If the handler thread is not as fast as the producing threads, then the ring buffer will eventually fill up, at which point events will be dropped (the default) or the producing threads are blocked depending on configured `appendTimeout` (see [RingBuffer Full](#ringbuffer-full).

#### RingBuffer Full

The async appenders will by default never block the logging thread.
If the RingBuffer is full (e.g. due to slow network, etc), then events will be dropped.

Alternatively, you can configure the appender to wait until space becomes available instead of dropping the events immediately. This may come in handy when you want to rely on the buffering and the async nature of the appender but don't want to loose any event in case of large logging bursts that exceed the size of the RingBuffer.

The behaviour of the appender when the RingBuffer is controlled by the `appendTimeout` configuration property:

| `appendTimeout` | Behaviour when RingBuffer is full |
|-----------------|------------------------------------------------------------------------|
| `< 0` | disable timeout and wait until space is available |
| `0` | no timeout, give up immediately and drop event (this is the *default*) |
| `> 0` | retry during the specified amount of time |

Logging threads waiting for space in the RingBuffer wake up periodically at a frequency starting at `1ns` and increasing exponentially up to `appendRetryFrequency` (default `5ms`).
Only one thread is allowed to retry at a time. If a thread is already retrying, additional threads are waiting on a lock until the first is finished. This strategy should help to limit CPU consumption while providing good enough latency and throughput when the ring buffer is at (or close) to its maximal capacity.

When the appender drops an event, it emits a warning status message every `droppedWarnFrequency` consecutive dropped events (`1000` by default, use `0` to turn off warnings). Another status message is emitted when the drop period is over and a first event is succesfully enqueued reporting the total number of events that were dropped.

#### Graceful Shutdown

To guarantees that logged messages have had a chance to be processed by asynchronous appenders (including the TCP appender) and ensure background threads have been stopped, you'll need to [cleanly shut down logback](http://logback.qos.ch/manual/configuration.html#stopContext) when your application exits.

When gracefully stopped, async appenders wait until all events in the buffer are processed and the buffer is empty.
The maximum time to wait is configured by the `shutdownGracePeriod` parameter and is set to `1 minute` by default.
Events still in the buffer after this period is elapsed are dropped and the appender is stopped.

#### Wait Strategy

By default, the [`BlockingWaitStrategy`](https://lmax-exchange.github.io/disruptor/docs/com/lmax/disruptor/BlockingWaitStrategy.html) is used by the worker thread spawned by this appender.
The `BlockingWaitStrategy` minimizes CPU utilization, but results in slower latency and throughput.
If you need faster latency and throughput (at the expense of higher CPU utilization), consider
a different [wait strategy](https://lmax-exchange.github.io/disruptor/docs/com/lmax/disruptor/WaitStrategy.html) offered by the disruptor.

> [!IMPORTANT]
> Whichever wait strategy you choose, be sure to test and monitor CPU utilization, latency, and throughput to ensure it meets your needs.
> For example, in some configurations, `SleepingWaitStrategy` can consume 90% CPU utilization at rest.

The wait strategy can be configured on the async appender using the `waitStrategyType` parameter, like this:

```xml

sleeping
...

```

The supported wait strategies are as follows:

Wait Strategy
Parameters
Implementation

blocking
none
BlockingWaitStrategy

busySpin
none
BusySpinWaitStrategy

liteBlocking
none
LiteBlockingWaitStrategy

yielding
none
YieldingWaitStrategy

sleeping{

  retries,

  sleepTimeNs

}

e.g.
sleeping
or
sleeping{500,1000}

retries - Number of times (integer) to spin before sleeping. (default = 200)

sleepTimeNs - Time in nanoseconds to sleep each iteration after spinning (default = 100)

SleepingWaitStrategy

phasedBackoff{

  spinTime,

  yieldTime,

  timeUnit,

  fallbackStrategy

}

e.g.
phasedBackoff{10,60,seconds,blocking}

spinTime - Time to spin before yielding

yieldTime - Time to yield before falling back to the fallbackStrategy

timeUnit - Units of time for spin and yield timeouts. String name of a TimeUnit value (e.g. seconds)

fallbackStrategy - String name of the wait strategy to fallback to after the timeouts have elapsed

PhasedBackoffWaitStrategy

timeoutBlocking{

  timeout,

  timeUnit

}

e.g.
timeoutBlocking{1,minutes}

timeout - Time to block before throwing an exception

timeUnit - Units of time for timeout. String name of a TimeUnit value (e.g. seconds)

TimeoutBlockingWaitStrategy

liteTimeoutBlocking{

  timeout,

  timeUnit

}

e.g.
liteTimeoutBlocking{1,minutes}

timeout - Time to block before throwing an exception

timeUnit - Units of time for timeout. String name of a TimeUnit value (e.g. seconds)

LiteTimeoutBlockingWaitStrategy

See [AsyncDisruptorAppender](/src/main/java/net/logstash/logback/appender/AsyncDisruptorAppender.java)
for other configuration parameters (such as `ringBufferSize`, `threadNamePrefix`, `daemon`, and `droppedWarnFrequency`)

### Appender Listeners

Listeners can be registered to an appender to receive notifications for the appender lifecycle and event processing.

See the two listener interfaces for the types of notifications that can be received:

* [`AppenderListener`](/src/main/java/net/logstash/logback/appender/listener/AppenderListener.java) - basic notifications for the [async appenders](#async-appenders) and [UDP appenders](#udp-appenders).
* [`TcpAppenderListener`](/src/main/java/net/logstash/logback/appender/listener/TcpAppenderListener.java) - extension of `AppenderListener` with additional TCP-specific notifications. Only works with the [TCP appenders](#tcp-appenders).

Some example use cases for a listener are:

* Monitoring metrics for events per second, event processing durations, dropped events, connections successes / failures, etc.
* Logging event processing errors to a different appender (that perhaps appends to a different destination).

A [`FailureSummaryLoggingAppenderListener`](src/main/java/net/logstash/logback/appender/listener/FailureSummaryLoggingAppenderListener.java)
is provided that will log a warning on the first success after a series of consecutive append/send/connect failures.
The message includes summary details of the failures that occurred (such as the number of failures, duration of the failures, etc).
To register it:

```xml

net.logstash.logback.appender.listener.FailureSummaryLoggingAppenderListener

```

You may also create your own listener by implementing the `*Listener` interface and register it to an appender using the `listener` xml element like this:

```xml

...

propertyValue

```

Multiple listeners can be registered by supplying multiple `listener` xml elements.

### Encoders / Layouts

You can use any of the encoders/layouts provided by the logstash-logback-encoder library with other logback appenders.

For example, to output LoggingEvents to a file, use the `LogstashEncoder`
with the `RollingFileAppender` in your `logback.xml` like this:

```xml

info

/some/path/to/your/file.log

/some/path/to/your/file.log.%d{yyyy-MM-dd}
30

```

To log AccessEvents to a file, configure your `logback-access.xml` like this:

```xml

/some/path/to/your/file.log

```

The `LogstashLayout` and `LogstashAccessLayout` can be configured the same way as
the `LogstashEncoder` and `LogstashAccessEncoder`. All the other examples
in this document use encoders, but the same options apply to the layouts as well.

To receive file input in logstash, configure a [`file`](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html) input in logstash's configuration like this:

```
input {
file {
path => "/some/path/to/your/file.log"
codec => "json"
}
}
```

### Threads and ThreadLocals

Asynchronous appenders use a separate set of threads to process logging events.

The encoders/layouts use ThreadLocals internally to improve performance.

In environments that support application reloading (e.g. web containers)
and in applications with many threads logging events (e.g. virtual threads),
it is recommended to:
1. use an asynchronous appender (such as `LogstashTcpSocketAppender`, `LoggingEventAsyncDisruptorAppender`, or [`AsyncAppender`](https://logback.qos.ch/manual/appenders.html#AsyncAppender)), and
2. [cleanly shut down logback](http://logback.qos.ch/manual/configuration.html#stopContext) when the application is stopped/reloaded.

Using an asynchronous appender ensures that a limited number of threads use the encoder/layout,
and therefore limits the number of ThreadLocal value instances
and minimizes resource contention.

Cleanly shutting down logback stops the threads used by the asynchronous appender,
and makes their ThreadLocal values eligible for garbage collection.

Failure to use an asynchronous appender may result in errors like the following when stopping/reloading an application in Tomcat:

```
The web application [webapp] created a ThreadLocal with key of type [java.lang.ThreadLocal.SuppliedThreadLocal]
(value [java.lang.ThreadLocal$SuppliedThreadLocal@6f75b07c]) and a value of type
[net.logstash.logback.util.ThreadLocalHolder.Holder] (value [net.logstash.logback.util.ThreadLocalHolder$Holder@7d940c3e])
but failed to remove it when the web application was stopped.
Threads are going to be renewed over time to try and avoid a probable memory leak.
```

Failure to cleanly shutdown logback may result in errors like the following when stopping/reloading an application in Tomcat:

```
The web application [webapp] appears to have started a thread named [logback-appender-ASYNC-2] but has failed to stop it.
This is very likely to create a memory leak.
```

## LoggingEvent Fields

The following sections describe the fields included in the JSON output by default for LoggingEvents written by the

* `LogstashEncoder`
* `LogstashLayout`, and
* the logstash appenders

If you are using the [composite encoders/layouts](#composite-encoderlayout), then the fields written will
vary by the providers you configure.

### Standard Fields

These fields will appear in every LoggingEvent unless otherwise noted.
The field names listed here are the default field names.
The field names can be customized (see [Customizing Standard Field Names](#customizing-standard-field-names)).

| Field | Description |
|---------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `@timestamp` | Time of the log event (`ISO_OFFSET_DATE_TIME`) - see [Customizing Timestamp](#customizing-timestamp) |
| `@version` | Logstash format version (e.g. `1`) - see [Customizing Version](#customizing-version) |
| `message` | Formatted log message of the event - see [Customizing Message](#customizing-message) |
| `logger_name` | Name of the logger that logged the event |
| `thread_name` | Name of the thread that logged the event |
| `level` | String name of the level of the event |
| `level_value` | Integer value of the level of the event |
| `stack_trace` | (Only if a throwable was logged) The stacktrace of the throwable. Stackframes are separated by line endings. |
| `tags` | (Only if tags are found) The names of any markers not explicitly handled. (e.g. markers from `MarkerFactory.getMarker` will be included as tags, but the markers from [`Markers`](/src/main/java/net/logstash/logback/marker/Markers.java) will not.) This can be fully disabled by specifying `false`, in the encoder/layout/appender configuration. |

### MDC fields

By default, `LogstashEncoder`/`LogstashLayout` will write each
[Mapped Diagnostic Context (MDC) (`org.slf4j.MDC`)](https://www.slf4j.org/api/org/slf4j/MDC.html)
entry to the output.

To disable writing MDC entries, add `false`
to the `LogstashEncoder`/`LogstashLayout` configuration.

You can also configure specific entries in the MDC to be included or excluded as follows:

```xml

key1ToInclude
key2ToInclude

```

```xml

key1ToExclude
key2ToExclude

```

When key names are specified for inclusion, then all other fields will be excluded.
When key names are specified for exclusion, then all other fields will be included.
It is a configuration error to specify both included and excluded key names.

By default, the MDC key is used as the field name in the output.
To use an alternative field name in the output for an MDC entry,
specify `mdcKeyName=fieldName`:

```xml

key1=alternateFieldNameForKey1

```

You can also manipulate the MDC entry values written to the JSON output.
By default, no manipulations are done and all MDC entry values are written as text.

Currently, the following MDC entry writers are supported:

```xml

keyPatternToInclude
keyPatternToExclude

```

To add your own MDC entry writer for other types or apply the manipulations only for specific fields
you can write your own implementation of [`MdcEntryWriter`](src/main/java/net/logstash/logback/composite/loggingevent/mdc/MdcEntryWriter.java).

You can also replace the default MDC JSON provider with your own class extending from
[`MdcJsonProvider`](src/main/java/net/logstash/logback/composite/loggingevent/MdcJsonProvider.java).
Configuring your class as a [Custom JSON Provider](#custom-json-provider) will then replace
the default `MdcJsonProvider`.

```xml

```

### Key Value Pair Fields

Slf4j 2's [fluent API](https://www.slf4j.org/manual.html#fluent) supports attaching key value pairs to the log event.

`LogstashEncoder`/`LogstashLayout` will write each key value pair as a field in the output by default.

To disable writing key value pairs, add `false`
to the `LogstashEncoder`/`LogstashLayout` configuration.

You can also configure specific key value pairs to be included or excluded as follows:

```xml

key1ToInclude
key2ToInclude

```

```xml

key1ToExclude
key2ToExclude

```

When key names are specified for inclusion, then all other keys will be excluded.
When key names are specified for exclusion, then all other keys will be included.
It is a configuration error to specify both included and excluded key names.

By default, the key is used as the field name in the output.
To use an alternative field name in the output for an key value pair,
specify`keyName=fieldName`:

```xml

key1=alternateFieldNameForKey1

```

### Context fields

By default, each property of Logback's Context (`ch.qos.logback.core.Context`)
will appear as a field in the LoggingEvent.
This can be disabled by specifying `false`
in the encoder/layout/appender configuration.

Note that logback versions prior to 1.1.10 included a `HOSTNAME` property by default in the context.
As of logback 1.1.10, the `HOSTNAME` property is lazily calculated (see [LOGBACK-1221](https://jira.qos.ch/browse/LOGBACK-1221)), and will no longer be included by default.

### Caller Info Fields
The encoder/layout/appender do not contain caller info by default.
This can be costly to calculate and should be switched off for busy production environments.

To switch it on, add the `includeCallerData` property to the configuration.
```xml

true

```
If the encoder is included inside an asynchronous appender, such as
`AsyncAppender`, `LoggingEventAsyncDisruptorAppender`, or `LogstashTcpSocketAppender`, then
`includeCallerData` must be set to true on the appender as well.

When switched on, the following fields will be included in the log event:

| Field | Description |
|----------------------|---------------------------------------------------------------|
| `caller_class_name` | Fully qualified class name of the class that logged the event |
| `caller_method_name` | Name of the method that logged the event |
| `caller_file_name` | Name of the file that logged the event |
| `caller_line_number` | Line number of the file where the event was logged |

### Custom Fields

In addition to the fields above, you can add other fields to the LoggingEvent either globally, or on an event-by-event basis.

#### Global Custom Fields

Add custom fields that will appear in every LoggingEvent like this :

```xml

{"appname":"myWebservice","roles":["customerorder","auth"],"buildinfo":{"version":"Version 0.1.0-SNAPSHOT","lastcommit":"75473700d5befa953c45f630c6d9105413c16fe1"}}

```

or in an AccessEvent like this :

```xml

{"appname":"myWebservice","roles":["customerorder","auth"],"buildinfo":{"version":"Version 0.1.0-SNAPSHOT","lastcommit":"75473700d5befa953c45f630c6d9105413c16fe1"}}

```

#### Event-specific Custom Fields

When logging a message, you can add additional fields to the JSON output by using

* (recommended) slf4j's [fluent API](https://www.slf4j.org/manual.html#fluent) for attaching key value pairs to the log event.
see [Key Value Pair Fields](#key-value-pair-fields).
* _structured arguments_ provided by
[`StructuredArguments`](/src/main/java/net/logstash/logback/argument/StructuredArguments.java), OR
* _markers_ provided by
[`Markers`](/src/main/java/net/logstash/logback/marker/Markers.java)

The difference between structured arguments and markers is that
* `StructuredArguments` are included in the log event's formatted message
(when the message has a parameter for the argument) _AND_ in the JSON output.
* `StructuredArguments` will be included in the JSON output if using `LogstashEncoder/Layout`
or if using [composite encoders/layouts](#composite-encoderlayout) with the `arguments` provider.
* `Markers` are only written to the JSON output, and _NEVER_ to the log event's formatted message.
* `Markers` will be included in the JSON output if using `LogstashEncoder/Layout`
or if using [composite encoders/layouts](#composite-encoderlayout) with the `logstashMarkers` provider.

You can use `StructuredArguments` even if the message does not contain a parameter
for the argument. However, in this case, the argument will only be written to the JSON output
and not the formatted message (which is effectively the same behavior that the Markers provide).
In general, you should use `StructuredArguments`, unless you have a static analyzer
that flags parameter count / argument count mismatches.

Both `StructuredArguments` and `Markers` require constructing additional objects.
Therefore, it is best practice to surround the log lines with `logger.isXXXEnabled()`,
to avoid the object construction if the log level is disabled.

Examples using `StructuredArguments`:

```java
import static net.logstash.logback.argument.StructuredArguments.*;

/*
* Add "name":"value" to the JSON output,
* but only add the value to the formatted message.
*
* The formatted message will be `log message value`
*/
logger.info("log message {}", value("name", "value"));

/*
* Add "name":"value" to the JSON output,
* and add name=value to the formatted message.
*
* The formatted message will be `log message name=value`
*/
logger.info("log message {}", keyValue("name", "value"));

/*
* Add "name":"value" ONLY to the JSON output.
*
* Since there is no parameter for the argument,
* the formatted message will NOT contain the key/value.
*
* If this looks funny to you or to static analyzers,
* consider using Markers instead.
*/
logger.info("log message", keyValue("name", "value"));

/*
* Add multiple key value pairs to both JSON and formatted message
*/
logger.info("log message {} {}", keyValue("name1", "value1"), keyValue("name2", "value2")));

/*
* Add "name":"value" to the JSON output and
* add name=[value] to the formatted message using a custom format.
*/
logger.info("log message {}", keyValue("name", "value", "{0}=[{1}]"));

/*
* In the JSON output, values will be serialized by Jackson's ObjectMapper.
* In the formatted message, values will follow the same behavior as logback
* (formatting of an array or if not an array `toString()` is called).
*
* Add "foo":{...} to the JSON output and add `foo.toString()` to the formatted message:
*
* The formatted message will be `log message `
*/
Foo foo = new Foo();
logger.info("log message {}", value("foo", foo));

/*
* Add "name1":"value1","name2":"value2" to the JSON output by using a Map,
* and add `myMap.toString()` to the formatted message.
*
* Note the values can be any object that can be serialized by Jackson's ObjectMapper
* (e.g. other Maps, JsonNodes, numbers, arrays, etc)
*/
Map myMap = new HashMap();
myMap.put("name1", "value1");
myMap.put("name2", "value2");
logger.info("log message {}", entries(myMap));

/*
* Add "array":[1,2,3] to the JSON output,
* and array=[1,2,3] to the formatted message.
*/
logger.info("log message {}", array("array", 1, 2, 3));

/*
* Add fields of any object that can be unwrapped by Jackson's UnwrappableBeanSerializer to the JSON output.
* i.e. The fields of an object can be written directly into the JSON output.
* This is similar to the @JsonUnwrapped annotation.
*
* The formatted message will contain `myobject.toString()`
*/
logger.info("log message {}", fields(myobject));

/*
* In order to normalize a field object name, static helper methods can be created.
* For example:
* public static StructuredArgument foo(Foo foo) {
* return StructuredArguments.value("foo", foo);
* }
*/
logger.info("log message {}", foo(foo));

```

Abbreviated convenience methods are available for all the structured argument types.
For example, instead of `keyValue(key, value)`, you can use `kv(key, value)`.

Examples using `Markers`:

```java
import static net.logstash.logback.marker.Markers.*;

/*
* Add "name":"value" to the JSON output.
*/
logger.info(append("name", "value"), "log message");

/*
* Add "name1":"value1","name2":"value2" to the JSON output by using multiple markers.
*/
logger.info(append("name1", "value1").and(append("name2", "value2")), "log message");

/*
* Add "name1":"value1","name2":"value2" to the JSON output by using a map.
*
* Note the values can be any object that can be serialized by Jackson's ObjectMapper
* (e.g. other Maps, JsonNodes, numbers, arrays, etc)
*/
Map myMap = new HashMap();
myMap.put("name1", "value1");
myMap.put("name2", "value2");
logger.info(appendEntries(myMap), "log message");

/*
* Add "array":[1,2,3] to the JSON output
*/
logger.info(appendArray("array", 1, 2, 3), "log message");

/*
* Add "array":[1,2,3] to the JSON output by using raw json.
* This allows you to use your own json seralization routine to construct the json output
*/
logger.info(appendRaw("array", "[1,2,3]"), "log message");

/*
* Add any object that can be serialized by Jackson's ObjectMapper
* (e.g. Maps, JsonNodes, numbers, arrays, etc)
*/
logger.info(append("object", myobject), "log message");

/*
* Add fields of any object that can be unwrapped by Jackson's UnwrappableBeanSerializer.
* i.e. The fields of an object can be written directly into the json output.
* This is similar to the @JsonUnwrapped annotation.
*/
logger.info(appendFields(myobject), "log message");

```

## AccessEvent Fields

The following sections describe the fields included in the JSON output by default for AccessEvents written by the

* `LogstashAccessEncoder`,
* `LogstashAccessLayout`, and
* the logstash access appenders.

If you are using the [composite encoders/layouts](#composite-encoderlayout), then the fields written will
vary by the providers you configure.

### Standard Fields

These fields will appear in every AccessEvent unless otherwise noted.
The field names listed here are the default field names.
The field names can be customized (see [Customizing Standard Field Names](#customizing-standard-field-names)).

| Field | Description |
|------------------|-------------------------------------------------------------------------------------------------------------------|
| `@timestamp` | Time of the log event. (`yyyy-MM-dd'T'HH:mm:ss.SSSZZ`) See [customizing timestamp](#customizing-timestamp). |
| `@version` | Logstash format version (e.g. `1`) See [customizing version](#customizing-version). |
| `message` | Message in the form `${remoteHost} - ${remoteUser} [${timestamp}] "${requestUrl}" ${statusCode} ${contentLength}` |
| `method` | HTTP method |
| `protocol` | HTTP protocol |
| `status_code` | HTTP status code |
| `requested_url` | Request URL |
| `requested_uri` | Request URI |
| `remote_host` | Remote host |
| `remote_user` | Remote user |
| `content_length` | Content length |
| `elapsed_time` | Elapsed time in millis |

### Header Fields

Request and response headers are not logged by default, but can be enabled by specifying a field name for them, like this:

```xml

request_headers
response_headers

```

See [Customizing Standard Field Names](#customizing-standard-field-names)) for more details.

To write the header names in lowercase (so that header names that only differ by case are treated the same),
set `lowerCaseFieldNames` to true, like this:

```xml

request_headers
response_headers

true

```

Headers can be filtered via configuring the `requestHeaderFilter` and/or the `responseHeaderFilter`
with a [`HeaderFilter`](/src/main/java/net/logstash/logback/composite/accessevent/HeaderFilter.java), such as the
[`IncludeExcludeHeaderFilter`](/src/main/java/net/logstash/logback/composite/accessevent/IncludeExcludeHeaderFilter.java).

The `IncludeExcludeHeaderFilter` can be configured like this:

```xml

request_headers

Content-Type

```

Custom filters implementing [`HeaderFilter`](/src/main/java/net/logstash/logback/composite/accessevent/HeaderFilter.java)
can be used by specifying the filter class like this:

```xml

```

## Customizing Jackson

Logstash-logback-encoder uses [Jackson](https://github.com/FasterXML/jackson) to encode log and access events.

Logstash-logback-encoder provides sensible defaults for Jackson, but gives you full control over the Jackson configuration.

For example, you can:
* specify the [data format](#data-format)
* customize the [`TokenStreamFactoryBuilder`, `MapperBuilder`, and `JsonGenerator`](#customizing-tokenstreamfactory-objectmapper-and-jsongenerator)
* register [jackson modules](#registering-jackson-modules)
* configure [character escapes](#customizing-character-escapes)

### Data Format

JSON is used by default, but other data formats supported by Jackson can be used.
* [text data formats](https://github.com/FasterXML/jackson-dataformats-text)
* [binary data formats](https://github.com/FasterXML/jackson-dataformats-binary)

> [!IMPORTANT]
> When using non-JSON data formats, you must include the appropriate jackson dataformat library on the runtime classpath,
> typically via a maven/gradle dependency (e.g. for Smile, include `jackson-dataformat-smile`).

The following data formats are directly supported:
* `cbor`
* `json` (the default)
* `smile`
* `yaml`

To use one these formats, specify the `` like this:

```xml

smile

```

Other data formats can be used by implementing
[`net.logstash.logback.dataformat.DataFormatFactory`](src/main/java/net/logstash/logback/decorate/DataFormatFactory.java),
and configuring it like this:

```xml

```

The following [decorators](#customizing-tokenstreamfactory-objectmapper-and-jsongenerator)
can be used to configure data-format-specific write features:
* [`CborWriteFeatureDecorator`](src/main/java/net/logstash/logback/decorate/cbor/CborWriteFeatureDecorator.java)
* [`JsonWriteFeatureDecorator`](src/main/java/net/logstash/logback/decorate/json/JsonWriteFeatureDecorator.java)
* [`SmileWriteFeatureDecorator`](src/main/java/net/logstash/logback/decorate/smile/SmileWriteFeatureDecorator.java)
* [`YamlWriteFeatureDecorator`](src/main/java/net/logstash/logback/decorate/yaml/YamlWriteFeatureDecorator.java)

For example:

```xml

smile

WRITE_HEADER

```

### Customizing TokenStreamFactory, ObjectMapper, and JsonGenerator

The `ObjectMapper`, `TokenStreamFactory` and `JsonGenerator` used to write output can be customized by instances of
* [`MapperBuilderDecorator`](/src/main/java/net/logstash/logback/decorate/MapperBuilderDecorator.java)
* [`TokenStreamFactoryBuilderDecorator`](/src/main/java/net/logstash/logback/decorate/TokenStreamFactoryBuilderDecorator.java)
* [`JsonGeneratorDecorator`](/src/main/java/net/logstash/logback/decorate/JsonGeneratorDecorator.java).

For example, you could enable pretty printing by using the
[PrettyPrintingDecorator](/src/main/java/net/logstash/logback/decorate/PrettyPrintingDecorator.java)

Or customize object mapping like this:

```java
public class MixInDecorator> implements MapperBuilderDecorator {

@Override
public B decorate(B mapperBuilder) {
return mapperBuilder.addMixIn(MyTarget.class, MyMixin.class);
}
}
```
and then specify the decorators in the logback.xml file like this:

```xml

```

Jackson features can be enabled/disabled by using the following feature decorators:

| Jackson Feature Enum | Feature Decorator |
|------------------------------|--------------------------------------------------------------------|
| `TokenStreamFactory.Feature` | `net.logstash.logback.decorate.TokenStreamFactoryFeatureDecorator` |
| `MapperFeature` | `net.logstash.logback.decorate.MapperFeatureDecorator` |
| `SerializationFeature` | `net.logstash.logback.decorate.SerializationFeatureDecorator` |
| `StreamWriteFeature` | `net.logstash.logback.decorate.StreamWriteFeatureDecorator` |
| `JsonNodeFeature` | `net.logstash.logback.decorate.JsonNodeFeatureDecorator` |
| `EnumFeature` | `net.logstash.logback.decorate.EnumFeatureDecorator` |
| `DateTimeFeature` | `net.logstash.logback.decorate.DateTimeFeatureDecorator` |
| `JsonWriteFeature` | `net.logstash.logback.decorate.json.JsonWriteFeatureDecorator` |
| `SmileWriteFeature` | `net.logstash.logback.decorate.smile.SmileWriteFeatureDecorator` |
| `YAMLWriteFeature` | `net.logstash.logback.decorate.yaml.YamlWriteFeatureDecorator` |
| `CBORWriteFeature` | `net.logstash.logback.decorate.cbor.CborWriteFeatureDecorator` |

For example:

```xml

INTERN_PROPERTY_NAMES

WRITE_NUMBERS_AS_STRINGS

```

See the [net.logstash.logback.decorate](/src/main/java/net/logstash/logback/decorate) package
and sub-packages for other decorators.

If you prefer pretty printing for easier interactive viewing of (error) logs, you
may also prefer to write the stacktrace as an array of strings where each string
is a stacktrace line:

```xml

true

true

```

### Registering Jackson Modules

By default, Jackson modules are dynamically registered via
`MapperBuilder.findAndAddModules()`.

Therefore, you just need to add jackson modules (e.g. jackson-datatypes-collections) to the classpath,
and they will be dynamically registered.

To disable automatic discovery, set `false` on the encoder/layout.

If you have a module that Jackson is not able to dynamically discover,
you can register it manually via a [`MapperBuilderDecorator`](#customizing-tokenstreamfactory-objectmapper-and-jsongenerator).

### Customizing Character Escapes

By default, when a string is written as a JSON string value, any character not allowed in a JSON string will be escaped.
For example, the newline character (ASCII 10) will be escaped as `\n`.

To customize these escape sequences, use the `net.logstash.logback.decorate.json.CharacterEscapesDecorator`.

For example, if you want to use something other than `\n` as the escape sequence for the newline character, you can do the following:

```xml

10
\u2028

```

You can also disable all the default escape sequences by specifying `false` on the `CharacterEscapesJsonFactoryDecorator`.
If you do this, then you will need to register custom escapes for each character that is illegal in JSON string values. Otherwise, invalid JSON could be written.

## Masking

The [`MaskingJsonGeneratorDecorator`](src/main/java/net/logstash/logback/mask/MaskingJsonGeneratorDecorator.java)
can be used to mask sensitive values (e.g. personally identifiable information (PII) or financial data).

Data to be masked can be identified by [path](#identifying-field-values-to-mask-by-path)
and/or by [value](#identifying-field-values-to-mask-by-value).

### Identifying field values to mask by path

Paths of fields to mask can be specified in several ways, as shown in the following example:

```xml

****

singleFieldName
/absolute/path/to/mask
partial/path/to/mask
partial/path/with/*/wildcard
tilde~0slash~1escapedPath

path1, path2, path3

some/path
some/other/path
[masked]

anotherFieldName,anotherFieldName2
**anotherCustomMask**

```

See [`PathBasedFieldMasker`](src/main/java/net/logstash/logback/mask/PathBasedFieldMasker.java)
for the path string format and more examples. But in general:

* Paths follow a format similar to (but not _exactly_ same as) a [JSON Pointer](http://tools.ietf.org/html/draft-ietf-appsawg-json-pointer-03).
* Absolute paths start with `/` and are absolute to the root of the JSON output event (e.g. `/@timestamp` would mask the default timestamp field)
* Partial paths do not start with `/` and match anywhere that path sequence is seen in the output.
* A path with a single token (i.e. no `/` characters) will match all occurrences of a field with the given name
* A wildcard token (`*`) will match anything at that location within the path
* Use `~1` to escape `/` within a token
* Use `~0` to escape `~` within a token

### Identifying field values to mask by value

Specific values to be masked can be specified in several ways, as seen in the following example:

```xml

****

^foo$
bar

^baz$,
^blah$

^(foo)-.*$
^(bar)-.*$
$1****

```

Identifying data to mask by value is much more expensive than identifying data to mask by [path](#identifying-field-values-to-mask-by-path).
Therefore, prefer identifying data to mask by path.

The value to mask is passed through every value masker, with the output of one masker passed as input to the next masker.
This allows each masker to mask specific substrings within the value.
The order in which the maskers are executed is not defined, and should not be relied upon.

When using regexes to identify strings to mask, all matches within each string field value will be replaced.
If you want to match the full string field value, then use the beginning of line (`^`) and end of line (`$`) markers.

## Customizing Standard Field Names

The standard field names above for LoggingEvents and AccessEvents can be customized by using the `fieldNames`configuration element in the encoder or appender configuration.

For example:

```xml

time
msg
stacktrace
...

```
Prevent a field from being output by setting the field name to `[ignore]`.

For LoggingEvents, see [`LogstashFieldNames`](/src/main/java/net/logstash/logback/fieldnames/LogstashFieldNames.java)
for all the field names that can be customized. Each java field name in that class is the name of the xml element that you would use to specify the field name (e.g. `logger`, `levelValue`). Additionally, a separate set of [shortened field names](/src/main/java/net/logstash/logback/fieldnames/ShortenedFieldNames.java) can be configured like this:

```xml

```

For LoggingEvents, log the caller info, MDC properties, and context properties
in sub-objects within the JSON event by specifying field
names for `caller`, `mdc`, and `context`, respectively.

For AccessEvents, see [`LogstashAccessFieldNames`](/src/main/java/net/logstash/logback/fieldnames/LogstashAccessFieldNames.java)
for all the field names that can be customized. Each java field name in that class is the name of the xml element that you would use to specify the field name (e.g. `fieldsMethod`, `fieldsProtocol`).

## Customizing Version

The version field value by default is the string value `1`.

The value can be changed like this:

```xml

```

The value can be written as a number (instead of a string) like this:

```xml

true

```

## Customizing Timestamp

By default, timestamps are written as string values in the format specified by
[`DateTimeFormatter.ISO_OFFSET_DATE_TIME`](https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/time/format/DateTimeFormatter.html#ISO_OFFSET_DATE_TIME)
(e.g. `2019-11-03T10:15:30.123+01:00`), in the default TimeZone of the host Java platform.

You can change the pattern like this:

```xml

yyyy-MM-dd'T'HH:mm:ss.SSS

```

The value of the `timestampPattern` can be any of the following:

* `[UNIX_TIMESTAMP_AS_NUMBER]` - timestamp written as a JSON number value of the milliseconds since unix epoch
* `[UNIX_TIMESTAMP_AS_STRING]` - timestamp written as a JSON string value of the milliseconds since unix epoch
* `[` _`constant`_ `]` - (e.g. `[ISO_OFFSET_DATE_TIME]`) timestamp written using the given `DateTimeFormatter` constant
* any other value - (e.g. `yyyy-MM-dd'T'HH:mm:ss.SSS`) timestamp written using a `DateTimeFormatter` created from the given pattern

The provider uses a standard Java DateTimeFormatter under the hood. However, special optimisations are applied when using one of the following standard ISO formats that make it nearly 7x faster and more GC friendly:

* `[ISO_OFFSET_DATE_TIME]`
* `[ISO_ZONED_DATE_TIME]`
* `[ISO_LOCAL_DATE_TIME]`
* `[ISO_DATE_TIME]`
* `[ISO_INSTANT]`

With logback 1.3+ the timestamp will have millisecond precision.

The formatter uses the default TimeZone of the host Java platform by default. You can change it like this:

```xml

UTC

```

The value of the `timeZone` element can be any string accepted by java's `TimeZone.getTimeZone(String id)` method.
For example `America/Los_Angeles`, `GMT+10` or `UTC`.
Use the special value `[DEFAULT]` to use the default TimeZone of the system.

## Customizing LoggingEvent Message

By default, LoggingEvent messages are written as JSON strings. Any characters not allowed in a JSON string, such as newlines, are escaped.
See the [Customizing Character Escapes](#customizing-character-escapes) section for details.

You can also write messages as JSON arrays instead of strings, by specifying a `messageSplitRegex` to split the message text.
This configuration element can take the following values:

* any valid regex pattern
* `SYSTEM` (uses the system-default line separator)
* `UNIX` (uses `\n`)
* `WINDOWS` (uses `\r\n`)

If you split the log message by the origin system's line separator, the written message does not contain any embedded line separators.
The target system can unambiguously parse the message without any knowledge of the origin system's line separators.

For example:

```xml

SYSTEM

```
```xml

\r?\n

```
```xml

```

## Customizing AccessEvent Message

By default, AccessEvent messages are written in the following format:

```
%clientHost - %user [%date] "%requestURL" %statusCode %bytesSent
```

To customize the message pattern, specify the `messagePattern` like this:

```xml

%clientHost [%date] "%requestURL" %statusCode %bytesSent

```

The pattern can contain any of the [AccessEvent conversion words](http://logback.qos.ch/manual/layouts.html#AccessPatternLayout).

## Customizing Logger Name Length

For LoggingEvents, you can shorten the logger name field length similar to the normal pattern style of `%logger{36}`.
Examples of how it is shortened can be found [here](https://logback.qos.ch/manual/layouts.html#logger).

```xml

```

The algorithm will shorten the logger name and attempt to reduce its size down to a maximum of number of characters.
It does so by reducing each part between dots to their first letter and gradually expand them starting from the right most element until the maximum size is reached.

To enable this feature, set the `shortenedLoggerNameLength` property to the desired value.
Setting the length to zero constitutes an exception and returns only the part of the logger name after last dot.
Use `-1` to disable shortening entirely.

The next table provides examples of the abbreviation algorithm in action.

| LENGTH | LOGGER NAME | SHORTENED |
|--------|----------------------------|----------------------------|
| 0 | `org.company.stack.Sample` | `Sample` |
| 5 | `org.company.stack.Sample` | `o.c.s.Sample` |
| 16 | `org.company.stack.Sample` | `o.c.stack.Sample` |
| 22 | `org.company.stack.Sample` | `o.company.stack.Sample` |
| 25 | `org.company.stack.Sample` | `org.company.stack.Sample` |

## Customizing Stack Traces

When [logging exceptions](https://www.baeldung.com/slf4j-log-exceptions),
stack traces are formatted using logback's `ExtendedThrowableProxyConverter` by default.
However, you can configure the encoder to use any [`ThrowableHandlingConverter`](https://logback.qos.ch/apidocs/ch/qos/logback/classic/pattern/ThrowableHandlingConverter.html) to format stacktraces.

Note that the `ThrowableHandlingConverter` only applies to the
[exception passed as an extra argument](https://www.baeldung.com/slf4j-log-exceptions)
to the log method, the way you normally log an exception in slf4j.
Do **NOT** use [structured arguments or markers](#event-specific-custom-fields) for exceptions.

A powerful [`ShortenedThrowableConverter`](/src/main/java/net/logstash/logback/stacktrace/ShortenedThrowableConverter.java) is included in the logstash-logback-encoder library to format stacktraces with the features listed in the next sections.
This converter can even be used within a `PatternLayout` to format stacktraces in any non-JSON logs you may have.

### Omit Common Frames

Nested stacktraces often contain redundant frames that can safely be omitted without loosing any valuable information.

The following example shows a standard stack trace of an exception with a single root cause:

```
java.lang.RuntimeException: Unable to invoke service
at org.company.stack.gen.StackGenerator.causedBy(StackGenerator.java:40)
at org.company.stack.gen.StackGenerator.generateCausedBy(StackGenerator.java:34)
at org.company.stack.framework.Dispatcher.invoke(Dispatcher.java:11)
at org.company.stack.framework.Dispatcher.dispatch(Dispatcher.java:8)
at org.company.stack.Sample.execute(Sample.java:36)
at org.company.stack.Sample.omitCommonFrames(Sample.java:22)
at org.company.stack.Sample.main(Sample.java:18)
Caused by: java.lang.RuntimeException: Destination unreachable
at org.company.stack.gen.StackGenerator.two(StackGenerator.java:58)
at org.company.stack.gen.StackGenerator.one(StackGenerator.java:55)
at org.company.stack.gen.StackGenerator.causedBy(StackGenerator.java:38)
at org.company.stack.gen.StackGenerator.generateCausedBy(StackGenerator.java:34)
at org.company.stack.framework.Dispatcher.invoke(Dispatcher.java:11)
at org.company.stack.framework.Dispatcher.dispatch(Dispatcher.java:8)
at org.company.stack.Sample.execute(Sample.java:36)
at org.company.stack.Sample.omitCommonFrames(Sample.java:22)
at org.company.stack.Sample.main(Sample.java:18)
```

As we can see, the exception and the root cause have the first 7 frames in common. The overall stack trace length can be reduced by omitting these redundant frames from the root cause, as shown below:

Common frames are omitted and replaced with a message indicating the number of frames dropped. Note that the last common frame remains as a visual cue to the reader.

This feature is enabled by default and can be disabled if desired by setting the `omitCommonFrames` property to `false`.

### Truncate after Regex

It is possible to use regular expressions to truncate the stacktrace after the first matching stacktrace element. The strings being matched against are in the form "fullyQualifiedClassName.methodName".

For example, to suppress everything below `org.company.stack.framework` package or after a call to the `StackGenerator.one()` method, configure the following:

```xml

^org\.company\.stack\.framework\..*
^\.StackGenerator\.one

```

This will produce something similar to the following:

Note how the stacktrace is truncated _after_ the matching frames.

Alternatively, multiple regex patterns can be specified at once using the `` configuration keyword. This property accepts an optional comma separated list of patterns. The previous example configuration can also be written as follows:

```xml

^org\.company\.stack\.fra

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/logfellow/logstash-logback-encoder

Awesome Lists containing this project

README