Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/prantadas/log-ingestor
A simple logging service that reports the error inside a http server. It uses Kafka for message queuing and MongoDB for storing the logs.
https://github.com/prantadas/log-ingestor
kafka microservice mongodb
Last synced: 24 days ago
JSON representation
A simple logging service that reports the error inside a http server. It uses Kafka for message queuing and MongoDB for storing the logs.
- Host: GitHub
- URL: https://github.com/prantadas/log-ingestor
- Owner: PrantaDas
- License: mit
- Created: 2024-09-16T19:55:31.000Z (3 months ago)
- Default Branch: main
- Last Pushed: 2024-09-18T06:12:58.000Z (3 months ago)
- Last Synced: 2024-10-14T12:09:32.655Z (2 months ago)
- Topics: kafka, microservice, mongodb
- Language: Go
- Homepage:
- Size: 31.3 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
## **Log Ingestor**
**Log Ingestor**
A simple logging service that reports the error inside a http server. It uses `Kafka` for message queuing and `MongoDB` for storing the logs.
### **Features**
* Simulates HTTP server errors.
* Publishes error logs to Kafka.
* Consumes error logs from Kafka.
* Stores error logs in MongoDB.
* Simple configuration for Kafka and MongoDB connection.
### **Prerequisites**
Before running the project, ensure you have the following installed:
* Go (version 1.18 or higher)
* Docker (for running Kafka and MongoDB in containers)
### **Usage**
##### 1. Clone the repository
```sh
https://github.com/PrantaDas/log-ingestor.git
```
##### 2. Change directory
```sh
cd log-ingestor
```
##### 3. Run docker compose file
```sh
docker compose up -d
```
##### 4. Verify kafka and zookeeper
```shell
nc -zv localhost 9092
nc -zv localhost 2181
```
##### 5. Push log to the message queue ( Send a request to the url to simulate error )
```sh
http://localhost:8080/simulate-error
```
### **Contributing**
Contributions are welcome! If you have suggestions for improvements or features, please create an issue or submit a pull request.
### **License**
This project is licensed under the MIT License. See the [LICENSE](https://github.com/PrantaDas/log-ingestor/blob/main/LICENSE) file for details.