https://github.com/jimmyyyeh/logger-master
A package for pushing serialized error log which generated by loguru to remote host.
https://github.com/jimmyyyeh/logger-master
efk fluentd logging mongodb python redis
Last synced: 2 months ago
JSON representation
A package for pushing serialized error log which generated by loguru to remote host.
- Host: GitHub
- URL: https://github.com/jimmyyyeh/logger-master
- Owner: jimmyyyeh
- Created: 2020-10-23T03:33:07.000Z (over 5 years ago)
- Default Branch: develop
- Last Pushed: 2023-03-31T14:58:03.000Z (about 3 years ago)
- Last Synced: 2025-05-17T02:07:09.180Z (about 1 year ago)
- Topics: efk, fluentd, logging, mongodb, python, redis
- Language: Python
- Homepage: https://pypi.org/project/logger-master/
- Size: 9.77 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# logger-master
**A package for pushing serialized error log which generated by [loguru](https://github.com/Delgan/loguru "loguru") to remote storage.**
## Description:
A package for pushing serialized error log which generated by [loguru](https://github.com/Delgan/loguru "loguru") to remote storage (including mongodb / redis / fluentd).
## How To Use:
### With MongoDB
* parameters:
- mongo_instance(object): mongodb instance
- mongo_db(str): database for storing log data
- mongo_collection(str): collection for storing log data
- terminal_displayed(bool): display log information on terminal screen or not
- rotation(str): [loguru](https://github.com/Delgan/loguru#modern-string-formatting-using-braces-style) rotation
- serialize(bool): data serialized or not
- log_format(str): [loguru](https://github.com/Delgan/loguru#modern-string-formatting-using-braces-style) format
- log_path(str): file path for storing log data
- custom_func(function): function for custom serialized data
```python
import json
from pymongo import MongoClient
from logger_master.logger import MongoLogger
# pushing log to mongodb instance
mongo_uri = 'mongodb://root:root@localhost:27017/?authMechanism=SCRAM-SHA-1'
mongo_instance = MongoClient(mongo_uri)
# basic usage for pushing log with specific database and collection
logger = MongoLogger(mongo_instance=mongo_instance,
mongo_db='my_log',
mongo_collection='my_log_collection')
# pushing serialize log to ./my_log/log_file
logger = MongoLogger(mongo_instance=mongo_instance,
mongo_db='my_log',
mongo_collection='my_log_collection',
serialize=True,
log_path='./my_log/log_file',
rotation='1 week')
# disable display log on screen
logger = MongoLogger(mongo_instance=mongo_instance,
mongo_db='my_log',
mongo_collection='my_log_collection',
log_path='./my_log/log_file',
terminal_displayed=False)
# pushing serialized log with custom format
def custom_function(serialized_data):
if not isinstance(serialized_data, dict):
serialized_data = json.loads(serialized_data)
serialized_data.update({'new_key': 'new_value'})
return serialized_data
logger = MongoLogger(mongo_instance=mongo_instance,
mongo_db='my_log',
mongo_collection='my_log_collection',
custom_func=custom_function)
try:
100 / 0
except Exception as e:
logger.error(str(e))
```
### With Redis
* parameters:
- redis_instance(object): redis instance
- key_prefix(str): prefix for redis key
- terminal_displayed(bool): display log information on terminal screen or not
- rotation(str): [loguru](https://github.com/Delgan/loguru#modern-string-formatting-using-braces-style) rotation
- serialize(bool): data serialized or not
- log_format(str): [loguru](https://github.com/Delgan/loguru#modern-string-formatting-using-braces-style) format
- log_path(str): file path for storing log data
- custom_func(function): function for custom serialized data
```python
import json
import redis
from logger_master.logger import RedisLogger
# basic usage for pushing log to redis instance
redis_instance = redis.StrictRedis(host='localhost',
password='root',
port='root')
# pushing log to mongodb with 'my_error_log' key prefix
logger = RedisLogger(redis_instance=redis_instance,
key_prefix='my_error_log')
# pushing serialize log to ./my_log/log_file
logger = RedisLogger(redis_instance=redis_instance,
key_prefix='my_error_log',
serialize=True,
log_path='./my_log/log_file',
rotation='1 week')
# disable display log on screen
logger = RedisLogger(redis_instance=redis_instance,
key_prefix='my_error_log',
log_path='./my_log/log_file',
terminal_displayed=False)
# pushing serialized log with custom format
def custom_function(serialized_data):
if not isinstance(serialized_data, dict):
serialized_data = json.loads(serialized_data)
serialized_data.update({'new_key': 'new_value'})
return serialized_data
logger = RedisLogger(redis_instance=redis_instance,
key_prefix='my_error_log',
custom_func=custom_function)
try:
100 / 0
except Exception as e:
logger.error(str(e))
```
### With Fluentd
* parameters:
- hostname(str): fluentd hostname
- port(int): fluentd port
- key_prefix(str): prefix for fluentd key
- terminal_displayed(bool): display log information on terminal screen or not
- rotation(str): [loguru](https://github.com/Delgan/loguru#modern-string-formatting-using-braces-style) rotation
- serialize(bool): data serialized or not
- log_format(str): [loguru](https://github.com/Delgan/loguru#modern-string-formatting-using-braces-style) format
- log_path(str): file path for storing log data
- custom_func(function): function for custom serialized data
```python
import json
from logger_master import FluentdLogger
# pushing log log to fluentd
# basic usage for pushing log data to fluentd
logger = FluentdLogger(hostname='localhost',
port=24224,
key_prefix='mongo')
# pushing serialize log to ./my_log/log_file
logger = FluentdLogger(hostname='localhost',
port=24224,
key_prefix='mongo',
serialize=True,
log_path='./my_log/log_file',
rotation='1 week')
# disable display log on screen
logger = FluentdLogger(hostname='localhost',
port=24224,
key_prefix='mongo',
log_path='./my_log/log_file',
terminal_displayed=False)
# pushing serialized log with custom format
def custom_function(serialized_data):
if not isinstance(serialized_data, dict):
serialized_data = json.loads(serialized_data)
serialized_data.update({'new_key': 'new_value'})
return serialized_data
logger = FluentdLogger(hostname='localhost',
port=24224,
key_prefix='mongo',
custom_func=custom_function)
try:
1 / 0
except Exception as e:
logger.error(msg=str(e))
```
**you can also push log to both fluentd and mongodb with [fluent-plugin-mongo](https://docs.fluentd.org/output/mongo).**
---
**Buy me a coffee, if you like it!**