https://github.com/romanow/log4j2-masking
Mask log output to fluent-bit and console
https://github.com/romanow/log4j2-masking
elk-stack log4j2 masking
Last synced: about 2 months ago
JSON representation
Mask log output to fluent-bit and console
- Host: GitHub
- URL: https://github.com/romanow/log4j2-masking
- Owner: Romanow
- License: mit
- Created: 2024-05-14T13:52:19.000Z (12 months ago)
- Default Branch: master
- Last Pushed: 2024-12-02T21:31:49.000Z (5 months ago)
- Last Synced: 2025-01-17T09:31:09.228Z (3 months ago)
- Topics: elk-stack, log4j2, masking
- Language: Kotlin
- Homepage: https://romanow.github.io/log4j2-masking/
- Size: 114 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://github.com/Romanow/logs-masking-example/actions/workflows/build.yml)
[](https://codecov.io/gh/Romanow/logs-masking-example)
[](https://github.com/pre-commit/pre-commit)
# Masking logs
## Подключение
### Maven
```xml
ru.romanow.logging
log-masking-lib
${log-masking-lib.version}
```
### Gradle
```groovy
testImplementation "ru.romanow.logging:log-masking-lib:$logMaskingVersion"
```
## Реализация
Для маскирования логов используется
плагин [`MaskingConverter`](log-masking-lib/src/main/kotlin/ru/romanow/logging/filter/MaskingConverter.kt).
Для его подключения используется annotation processor `org.apache.logging.log4j:log4j-core`, который собирает плагины
в [`$buildDir/tmp/kapt3/classes/main/META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat`](build/tmp/kapt3/classes/main/META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat).
В шаблоне этот конвертер вызывается как `%mask{ %msg }`:
```xml
```
Отправка в ELK выполняется в формате JSON через fluent-bit (по адресу localhost:5170), для сборки сообщения
используется `JsonTemplateLayout` (шаблон [layout.json](log-masking-lib/src/main/resources/logging/layout.json), для маскирования
используется pattern resolver как `%mask{ %msg }`.
## Тестирование
```shell
$ docker compose up -d
$ ./gradlew bootRun --args='--spring.profiles.active=docker'
$ curl http://localhost:8080/api/v1/send \
-H 'Content-Type: application/json' \
-d '{"lastName": "Alex"}'
$ docker compose logs -f fluent-bit
```
Вывод fluent-bit:
```
fluent-bit | {"localTime"=>"2024-05-14T20:47:40.589Z", "level"=>"INFO", "text"=>"Initializing Servlet 'dispatcherServlet'", "levelInt"=>6, "loggerName"=>"org.springframework.web.servlet.DispatcherServlet", "threadName"=>"http-nio-8080-exec-1", "appName"=>"log-masking", "appType"=>"JAVA"}]
fluent-bit | {"localTime"=>"2024-05-14T20:47:40.590Z", "level"=>"INFO", "text"=>"Completed initialization in 1 ms", "levelInt"=>6, "loggerName"=>"org.springframework.web.servlet.DispatcherServlet", "threadName"=>"http-nio-8080-exec-1", "appName"=>"log-masking", "appType"=>"JAVA"}]
fluent-bit | {"localTime"=>"2024-05-14T20:47:40.620Z", "level"=>"INFO", "text"=>"{"lastName":""}", "levelInt"=>6, "loggerName"=>"ru.romanow.logging.web.LogSender", "threadName"=>"http-nio-8080-exec-1", "appName"=>"log-masking", "appType"=>"JAVA"}]
```
## Правила маскирования
Тип поля:
* `email` –> `[email protected]` -> `r**********@mail.ru`
* `firstName` –> не маскируется
* `lastName`, `middleName` –> if (name.length > 7) `Ab*****c` else `A*****`
* `text` –> в зависимости от длины:
* 1 -> `*`
* 1..4 -> `1234` -> `1***`
* 5..9 -> `123456789` -> `1******89` (length * 60%)
* 10..15 -> `12345678901` -> `12*******01` (length * 60%)
* 16 + -> `12345678901234567` -> `123***********567` (length * 60%)
Конфигурация:
```yaml
application:
masking:
- field: fullName
type: FULL_NAME
- field: lastName
type: LAST_NAME
- field: email
type: EMAIL
- field: JWT
type: TEXT
- regex: Authorization\s*:\s*(\S+)
type: TEXT
```