https://github.com/teragrep/rlp_06
Teragrep Syslog-RELP Developer Example for Java
https://github.com/teragrep/rlp_06
java syslog teragrep
Last synced: about 2 months ago
JSON representation
Teragrep Syslog-RELP Developer Example for Java
- Host: GitHub
- URL: https://github.com/teragrep/rlp_06
- Owner: teragrep
- License: apache-2.0
- Created: 2023-05-16T13:50:00.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2025-04-07T11:50:18.000Z (about 1 year ago)
- Last Synced: 2025-04-07T12:39:39.001Z (about 1 year ago)
- Topics: java, syslog, teragrep
- Language: Java
- Homepage: https://teragrep.com
- Size: 34.2 KB
- Stars: 0
- Watchers: 2
- Forks: 1
- Open Issues: 3
-
Metadata Files:
- Readme: README.adoc
- License: LICENSE
Awesome Lists containing this project
README
= Java Syslog-RELP Developer Example
== Overview
Java syslog-RELP Developer Example project guides through design considerations with an actual example. The example uses https://www.rfc-editor.org/rfc/rfc5424[syslog] with Reliable Event Logging Protocol (https://en.wikipedia.org/wiki/Reliable_Event_Logging_Protocol[RELP]) in https://en.wikipedia.org/wiki/Java_(programming_language)[Java] programming language.
The project also utilizes Teragrep Java RELP Library (https://github.com/teragrep/rlp_01[rlp_01]). Rlp_01 ensures reliable record processing in Java applications. It uses optional https://en.wikipedia.org/wiki/Transport_Layer_Security[TLS] encryption and public-key https://en.wikipedia.org/wiki/Public_key_certificate[certificate] verification.
== Background
Syslog is a record format standard for message logging. It allows separation of:
* the software that produces message records,
* the system that stores them, and
* the software that reports and analyzes them.
When you use syslog as a record format, you can ensure the produced records are accessible.
RELP is a transport protocol. It allows reliable, encrypted and robust transmission of syslog records. With RELP, you can integrate with many syslog based systems. The mechanism is reliable for transporting the syslog messages over network.
Syslog and RELP allow a real-time record streaming, from an application to an archival and analytics system. Take https://github.com/teragrep/teragrep/[Teragrep] as an example.
=== Example about application logging
Application is producing many record streams containing valuable information about:
* requests, including transaction processing
* authentication
* authorization
* session handling
* functional errors
* technical errors
* debugging
* runtime metric data
All these are separate record streams from a single application. The application could, for example, handle web-shop orders.
Syslog record format allows capturing all these into their own streams. Meanwhile, the origin of the information is kept in the record metadata.
=== Example records for preceding example
*Requests*
[source,text]
<15>1 2023-01-01T00:00:00.123456+00:00 app-server1.example.com web-shop-requests - - - 127.0.0.1 - - [01/Jan/2023:00:00:00 +0300] "GET /webshop/ HTTP/1.1" 200 1995 "https://referer.example.com" "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0"
*Authentication*
[source,text]
<15>1 2023-01-01T00:00:00.123456+00:00 app-server1.example.com web-shop-auth - - - {"event": "login", "source": "127.0.0.1:38238", "username": "user1", "authentication method": "password"}
*Authorization*
[source,text]
<15>1 2023-01-01T00:00:00.123456+00:00 app-server1.example.com web-shop-authz - - - {"SUBJECT": "user:admin1", "VERB": "ASSUME", "OBJECT": "role:Administrators", "STATUS": "assume-ok"}
*Session handling*
[source,text]
<16>1 2023-01-01T00:00:00.123456+00:00 app-server1.example.com web-shop-session - - - login <[user1]>, operation , object_type
*Functional error*
[source,text]
<12>1 2023-01-01T00:00:00.123456+00:00 app-server1.example.com web-shop-logic-err - - - Checkout amount negative, calculated as <-100€>. discount coupons given: "2023FREE" by user: "user1" rejecting purchase;
*Technical error*
[source,text]
<13>1 2023-01-01T00:00:00.123456+00:00 app-server1.example.com web-shop-tech-err - - - Database connection to 127.0.0.2:3306 timed out.
*Debugging*
[source,text]
<17>1 2023-01-01T00:00:00.123456+00:00 app-server1.example.com web-shop-debug - - - Ensuring my code works properly enuf
*Runtime metric data*
[source,text]
<17>1 2023-01-01T00:00:00.123456+00:00 app-server1.example.com web-shop-metric - - - "time_taken":348
As seen above, syslog is able to capture all information. At the same time, the payload part of the record keeps information as human-readable. Delivery and retention of this kind of information is essential for the business.
By following this guide, you can ensure that records:
* are handled in a proper manner,
* contain essential metadata and
* are delivered without loss and in real-time from the application for further processing.
== Ready-Made Application Integrations for Syslog-RELP
For logging use, Teragrep ships following libraries:
* https://github.com/teragrep/jla_01[Relp Logging plugin for Logback]
* https://github.com/teragrep/jla_04[Java Util Logging RELP Handler]
* Log4j 1.x https://github.com/teragrep/jla_05[Log4j RELP Plugin]
* Log4j 2.x https://github.com/teragrep/jla_06[Log4j2 RELP Plugin]
You can use these libraries as part of the relevant logging framework. They integrate with their relevant logging framework by extending existing capabilities.
NOTE: Ready made libraries work well for the logging use. However, you may want more control over the syslog and RELP in other integrations (i.e. send the set the timestamp or https://en.wikipedia.org/wiki/Universally_unique_identifier[uuid] of the record). For this, see the next section.
== Syslog-RELP Integration Guidelines
=== General
* test cases must be executable
* errors (executions) must not be hidden
* server port needs to be configurable
* server address needs to be configurable
=== Syslog
==== Headers
* Wrap messages in syslog envelope
* Syslog-hostname must be configurable
* Syslog app-name must be configurable
NOTE: Sharing same syslog-hostname and syslog-app names allow parallel deployments of the application to share the same processing rules.
[source,java]
----
// see SyslogRecordTest.java
----
==== Structured-data
* Include origin data in syslog structure-data
* Include original uuid in syslog structured-data
NOTE: Structured-data is a managed schema. Adding new fields with our https://en.wikipedia.org/wiki/Private_enterprise_number[48577] is not a good idea.
[source,java]
----
// see SyslogRecordWithSDTest.java
----
=== RELP
==== Non-functional requirements
* Retry always and do not lose messages
* Processing mustn't hang infinitely due to technical issues, but rather retry new connection
[source,java]
----
// see CompleteExampleTest.java
----
==== RELP TLS
See https://github.com/teragrep/rlp_03/blob/main/src/test/java/com/teragrep/rlp_03/TlsClientTest.java[TlsClientTest.java] on rlp_03 repository.
== Contributing
// Change the repository name in the issues link to match with your project's name
You can involve yourself with our project by https://github.com/teragrep/rlp_06/issues/new/choose[opening an issue] or submitting a pull request.
Contribution requirements:
. *All changes must be accompanied by a new or changed test.* If you think testing is not required in your pull request, include a sufficient explanation as why you think so.
. Security checks must pass
. Pull requests must align with the principles and http://www.extremeprogramming.org/values.html[values] of extreme programming.
. Pull requests must follow the principles of Object Thinking and Elegant Objects (EO).
Read more in our https://github.com/teragrep/teragrep/blob/main/contributing.adoc[Contributing Guideline].
=== Contributor License Agreement
Contributors must sign https://github.com/teragrep/teragrep/blob/main/cla.adoc[Teragrep Contributor License Agreement] before a pull request is accepted to organization's repositories.
You need to submit the CLA only once. After submitting the CLA you can contribute to all Teragrep's repositories.