Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/teragrep/cfe_18

Integration main data management for Teragrep
https://github.com/teragrep/cfe_18

centralized-logging data-management integration-platform log-management teragrep

Last synced: about 1 year ago
JSON representation

Integration main data management for Teragrep

Awesome Lists containing this project

README 

          
= CFE_18, Integration main data management for Teragrep



CFE_18 takes advantage of relational-database standardization and MariaDB system-versioning through creating database schema according to integration data needs.



The project uses the Flyway migration strategy to ensure fault-free deployment and presents data through Swagger UI via RESTful endpoints. CFE_18 does have Spring Boot framework built-in which utilizes the framework only through endpoints. MyBatis works as a mapper between Spring boot and MariaDB database and creates queries specified within said mappers.



== Features



* CFE_18 can be used to manage integrations

* Schema structure can be modified to accommodate user needs



== Documentation



See the official documentation on https://docs.teragrep.com[docs.teragrep.com].



== Limitations



* Java 8



== How to Use



. Setup database according to end-user needs

. Can be deployed using https://tomcat.apache.org/tomcat-9.0-doc/deployer-howto.html[tomcat]/JBoss



== How to Run in a Container?



NOTE: Files are stored under `quadlet/docker` directory. They are viable for containerized deployment without Quadlet.



The project requires an external database to use. CFE_18 image uses https://docs.docker.com/build/building/multi-stage/[multi-stage] to build Rocky Linux 9 during the first stage as base image and installs Apache Tomcat on top of the previous stage. CFE_18.war is deployed via using Apache Tomcat.



Expected files are:



* `/etc/tomcat/Catalina/localhost/cfe-18/application.properties`

* `/etc/tomcat/Catalina/localhost/cfe-18/log4j2.xml`



Example files for configuration are:



* `/quadlet/docker/application.properties`

* `/quadlet/docker/log4j2.xml`



/quadlet/docker/application.properties requires configuring:



* spring.datasource.url = jdbc:mariadb://127.0.0.1:3306/cfe_18

* spring.datasource.username = username

* spring.datasource.password = password

* spring.datasource.driver-class-name = org.mariadb.jdbc.Driver



MariaDB must be used as the database solution for CFE_18 containerized deployment, when using CFE_18 image. It is required since CFE_18 takes advantage of MariaDB's https://mariadb.com/kb/en/system-versioned-tables/[system-versioned tables].



.Example run command

[source,console]

----

$ docker run -ti  -v /quadlet/docker/application.properties:/etc/tomcat/Catalina/localhost/cfe-18/application.properties:ro -v /quadlet/docker/log4j2.xml:/etc/tomcat/Catalina/localhost/cfe-18/log4j2.xml:ro  -p 8080:8080 ghcr.io/teragrep/cfe_18:latest

----

Please notice that in the example command above:



1. `application.properties` and `log4j2.xml` are mounted accordingly.

2. 8080:8080 are for external and internal ports.

3. ghcr.io/teragrep/cfe_18:latest image from Githubs image repository



=== How to run container with Quadlet



https://docs.podman.io/en/latest/markdown/podman-systemd.unit.5.html[See also the Quadlet's documentation.]



IMPORTANT: This is the recommended deployment style for containerized deployment. Running Podman containers without Quadlets results them not starting independently after host reboots.



==== Docker directory



Files inside the directory are required for Apache Tomcat to deploy CFE_18. They are explained in the section `How to Run in a Container?`



==== docker-entrypoint-initdb.d directory



Files inside the directory are required for MariaDB to initialize with the correct databases intact.

See also MariaDB https://hub.docker.com/_/mariadb[initialization script documentation for initializing the database contents].



==== cfe18.container



The following Quadlet file is required to deploy CFE_18 Podman container as a system service. The example below is from `quadlet/cfe18.container`.



The following Quadlet file is inserted under `/etc/containers/systemd` on the host machine. See more on https://docs.podman.io/en/latest/markdown/podman-systemd.unit.5.html[Podman Quadlet docs].



.Quadlet file for CFE_18 container

----

[Unit]

Description=CFE_18 container

Requires=cfe18mariadb.service



[Container]

Image=ghcr.io/teragrep/cfe_18:latest

Volume=/opt/teragrep/cfe_18/app:/etc/tomcat/Catalina/localhost/cfe_18:Z

PublishPort=8080:8080

# Static IP for cfe_18

IP=10.88.0.3



[Install]

# Start by default on boot

WantedBy=multi-user.target default.target

----



Keywords explained



- Requires -- Needs to have MariaDB service running before starting cfe18 container

- Image -- Pulls image from gchr.io to be used for container deployment

- Volume -- Mounts contents of  `quadlet/docker` under Tomcat configuration (`application.properties` and `log4j2.xml`)

- PublishPort -- Opens an internal and an external port for accessing CFE_18

- IP -- Declare IP in Podman subnet



==== cfe18mariadb.container



The following Quadlet file is required to deploy MariaDB Podman container as system service. Example below is from `quadlet/cfe18mariadb.container`.



The following Quadlet file is inserted under `/etc/containers/systemd` on the host machine. See more on https://docs.podman.io/en/latest/markdown/podman-systemd.unit.5.html[Podman Quadlet docs].



.Quadlet file for MariaDB container

----

[Unit]

Description=Database container for CFE_18



[Container]

Image=docker.io/mariadb:11.5

SecurityLabelDisable=true

# Should be stored as a secret when actually using to deploy.

Environment=MARIADB_ROOT_PASSWORD=password

# Static IP for cfe_18 to connect to

IP=10.88.0.2

# DB initialization script

Volume=/opt/teragrep/cfe_18/database/init:/docker-entrypoint-initdb.d

# Persistent Storage

Volume=/opt/teragrep/cfe_18/database/data:/var/lib/mysql



[Install]

WantedBy=multi-user.target default.target

----

Keywords explained



- Image -- Pulls image from gchr.io to be used for container deployment

- SecurityLabelDisable -- Turns off SELinux labeling

- Environment -- Env variable which is required by MariaDB image to install MariaDB inside the container

- IP -- Declare IP in Podman subnet

- Volume(1) -- Mounts contents of  `quadlet/docker-entrypoint-inidb.d` into the container for MariaDB to start database initialization

- Volume(2) -- Creates persistent storage which is saved on host under `/opt/teragrep/cfe_18/database/data`



== Contributing



You can involve yourself with our project by https://github.com/teragrep/cfe_18/issues/new/choose[opening an issue] or submitting a pull request.



Contribution requirements:



. *All changes must be accompanied by a new or changed test.* If you think testing is not required in your pull request, include a sufficient explanation as why you think so.

. Security checks must pass

. Pull requests must align with the principles and http://www.extremeprogramming.org/values.html[values] of extreme programming.

. Pull requests must follow the principles of Object Thinking and Elegant Objects (EO).



Read more in our https://github.com/teragrep/teragrep/blob/main/contributing.adoc[Contributing Guideline].



=== Contributor License Agreement



Contributors must sign https://github.com/teragrep/teragrep/blob/main/cla.adoc[Teragrep Contributor License Agreement] before a pull request is accepted to organization's repositories.



You need to submit the CLA only once.

After submitting the CLA you can contribute to all Teragrep's repositories.