https://github.com/schm1tz1/log-analysis-sqlite

Scripting tools for simple log analysis with Python and SQLite
https://github.com/schm1tz1/log-analysis-sqlite

Last synced: 14 days ago
JSON representation

Scripting tools for simple log analysis with Python and SQLite

• Host: GitHub
• URL: https://github.com/schm1tz1/log-analysis-sqlite
• Owner: Schm1tz1
• Created: 2023-04-04T14:16:35.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2023-04-04T14:16:41.000Z (almost 2 years ago)
• Last Synced: 2024-10-31T09:52:33.065Z (3 months ago)
• Language: Shell
• Size: 1.95 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# log-analysis-sqlite - scripting tools for simple log analysis with Python and SQLite

This is a small script using [sqlite-utils](https://sqlite-utils.datasette.io/en/stable/) to import logs into sqlite.

It is inspired by the blog post from [Jeqo](https://jeqo.github.io/notes/2022-09-24-ingest-logs-sqlite/) and was extended to parse logs from Confluent-for-Kubernetes.

I modified the regexp to analyze the CFK-based logs.

So far these 2 regular expressions are supported - feel free to add more:

- Java log4j: https://regex101.com/r/2AxwfE/1

- Confluent-for-Kubernetes: https://regex101.com/r/pMzqPU/1

Usage example:

`./sqlite-import.sh -i ./logs/kafka0.log -c kafka-0 -t kafka -f CFK -d ./logs.db`

A wrapper-script example is provided as `sqlite-import.sh` and some simple analysis statement can be found in `sqlite-examples.sql`.