https://github.com/vishva7/logly
A log analytics platform designed to collect, process and visualize log data in real time. Built as mini-project for Cloud Computing (UE22CS351B)
https://github.com/vishva7/logly
fastapi grafana kafka logging
Last synced: 2 months ago
JSON representation
A log analytics platform designed to collect, process and visualize log data in real time. Built as mini-project for Cloud Computing (UE22CS351B)
- Host: GitHub
- URL: https://github.com/vishva7/logly
- Owner: vishva7
- Created: 2025-04-01T09:30:36.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-04-01T10:23:43.000Z (about 1 year ago)
- Last Synced: 2025-04-01T11:27:39.524Z (about 1 year ago)
- Topics: fastapi, grafana, kafka, logging
- Language: Python
- Homepage:
- Size: 26.4 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Logly
Log Analytics Platform is designed to collect, process, and visualize log data in real time.
It leverages Grafana for visualization, Kafka for real-time log ingestion, and a relational database for storage.
Built for Cloud Computing course mini-project (UE22CS351B).
## Features
The platform visualizes the following metrics in Grafana:
- Request Count per Endpoint: Track the number of requests made to each API endpoint.
- Response Time Trends: Display response time patterns over different time periods.
- Most Frequent Errors in the Application: Identify and highlight recurring errors.
- Real-Time Logs: Provide a live feed of logs for monitoring purposes.
## Technology Stack
- Containerization: Docker
- Simulated server endpoints: FastAPI
- Message Broker: Apache Kafka for log ingestion
- Visualization: Grafana for querying and visualizing log data
## Running the Project
1. Start the Docker containers: `docker-compose up -d`
2. Access the API: API endpoints will be available at http://localhost:8000
3. Run the workload simulator (after containers are up):
`python workload_simulator.py --duration 300 --workers 5`
This will simulate traffic for 5 minutes with 5 concurrent workers.
4. Monitor Kafka (optional): Access Kafka UI at http://localhost:8080
5. View logs and metrics in Grafana: Access Grafana at http://localhost:3000
```language:none
Default credentials: admin/admin
You'll need to configure a PostgreSQL data source in Grafana:
Host: postgres:5432
Database: logly_db
User: logly
Password: logly_password
```
6. Shut down the project (when finished): `docker-compose down`