Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/jillesvangurp/kt-search-logback-appender
Log appender for Logback that bulk indexes to Elasticsearch/Opensearch using kt-search as the client. Intended for use with jvm based application servers like Spring Boot
https://github.com/jillesvangurp/kt-search-logback-appender
kotlin-library logback-appender logging spring-boot
Last synced: about 2 months ago
JSON representation
Log appender for Logback that bulk indexes to Elasticsearch/Opensearch using kt-search as the client. Intended for use with jvm based application servers like Spring Boot
- Host: GitHub
- URL: https://github.com/jillesvangurp/kt-search-logback-appender
- Owner: jillesvangurp
- License: mit
- Created: 2022-08-09T13:37:30.000Z (over 2 years ago)
- Default Branch: master
- Last Pushed: 2024-10-28T15:41:38.000Z (3 months ago)
- Last Synced: 2024-10-28T17:46:16.636Z (3 months ago)
- Topics: kotlin-library, logback-appender, logging, spring-boot
- Language: Kotlin
- Homepage:
- Size: 308 KB
- Stars: 0
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://github.com/jillesvangurp/kt-search-logback-appender/actions/workflows/ci.yml)
Log appender for Logback that bulk indexes to Elasticsearch using kt-search as the client.
The appender uses the provided `LogMessage` as a model class. This is an opinionated representation of log messages as produced by Logback. It passes on MDC variable, allows you pass on Logback context variables, and represents exception stack traces in a way that makes searching on exception classes or messages really easy.
Features
- You can let it create a datastream for you; or if you prefer you can manage your own mapping templates
- Can be easily configured to work with elastic cloud
- You can also use Opensearch but in that case you need to turn ilm off. You may want to explore the Opensearch state management for this which implements similar functionality.
Note. still a work in progress and not ready for general usage yet.
## Gradle
Add the `maven.tryformation.com` repository:
```kotlin
repositories {
mavenCentral()
maven("https://maven.tryformation.com/releases") {
content {
includeGroup("com.jillesvangurp")
}
}
}
```
And then the dependency:
```kotlin
// check the latest release tag for the current version
implementation("com.jillesvangurp:kt-search-logback-appender:0.1.4")
```
## Usage
After adding the dependency, add the appender to your logback configuration.
```xml
test-loggers
true
9999
```
## Settings
This is the list of writable properties and their defaults. TODO document properly but mostly does what it says on the tin. You can override any of these with the appropriate xml tag.
```kotlin
// you can override all the public properties via the logback xml config
/** Will log a lot of detail. Useful for debugging when the appender isn't working as expected. */
var verbose = false
/** Leave this off unless you have an issue with elasticsearch that you need to diagnose */
var logElasticSearchCalls = false
var host: String = "localhost"
var port: Int = 9200
var userName: String? = null
var password: String? = null
var ssl: Boolean = false
/** maximum time to wait until flushing messages to Elasticsearch */
var flushSeconds: Int = 5
/** maximum bulk request page size before flushing. */
var bulkMaxPageSizw: Int = 200
/** attempt to (re) create templates and datas treams. Leave to false if you want to control this manually. */
var manageDataStreamAndTemplates: Boolean = false
/** Elasticsearch only feature, leave disabled for opensearch and set up the os equivalent manually */
var configureIlm = false
var dataStreamName = "applogs"
// ILM settings below
var hotRollOverGb = 5
var hotMaxAge = "1d"
var numberOfReplicas = 1
var numberOfShards = 1
var warmMinAgeDays = 3
var deleteMinAgeDays = 30
var warmShrinkShards = 1
var warmSegments = 1
var contextVariableFilterRe = ""
/** comma separated list of mdc fields (without mdc prefix), will be coerced to Long in the json */
var coerceMdcFieldsToLong = ""
var coerceMdcFieldsToDouble = ""
```
## Elastic Cloud
When using the appender with elastic cloud and the `manageDataStreamAndTemplates` setting enabled you can either use a user with full privileges or create a user with at least these privileges:
Cluster:
- manage_index_templates
- manage_pipeline
- monitor
- manage_ilm
Index:
Grant these permissions on the `applogs*` prefix
- create
- index
- create_doc
- create_index
- write
- view_index_metadata