https://github.com/cybergrind/safe_logger
Safe TimedRotatingFileHandler for concurrent writing processes
https://github.com/cybergrind/safe_logger
Last synced: 2 months ago
JSON representation
Safe TimedRotatingFileHandler for concurrent writing processes
- Host: GitHub
- URL: https://github.com/cybergrind/safe_logger
- Owner: cybergrind
- License: mit
- Created: 2014-11-10T10:50:51.000Z (over 10 years ago)
- Default Branch: master
- Last Pushed: 2019-06-17T11:47:07.000Z (almost 6 years ago)
- Last Synced: 2024-03-15T01:03:48.957Z (about 1 year ago)
- Language: Python
- Size: 12.7 KB
- Stars: 15
- Watchers: 2
- Forks: 3
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- jimsghstars - cybergrind/safe_logger - Safe TimedRotatingFileHandler for concurrent writing processes (Python)
README
# safe_logger
## Description
This handler is designed to provide correct log files rotation when multiple processes writing to file.
Heavilly tested on production systems with up to 50 writers.
**Caveat**: this logger has been extracted from other system, so can have issues cause by copy-pasting
## Pitfalls
* Naive aproach fails because concurrent processes do independent rollovers and finally you will have zero-lenght log file: so we need locking
* Naive locking fails when you have some processes that not write often and them can rotate you file in case you archive old, or etc.: so we need check file that want to move by compare inodes
* Naive approach to start handler have issues, when you open file that in rollover - you can write data to rotated file: so we need locking on open file
## Usage
```python
import os
import logging
from safe_logger import TimedRotatingFileHandlerSafe
class NullHandler(logging.Handler):
def emit(self, record):
pass
def write(self, *args, **kwargs):
pass
LOG_FILE = '/tmp/debug.log'
ERR_FILE = '/tmp/error.log'
FORMAT = '[%(asctime)s] [%(levelname)s] [PID: '+str(os.getpid())+'] [%(name)s]: %(message)s'
FORMATTER = logging.Formatter(FORMAT)
logging.basicConfig(level=logging.DEBUG, stream=NullHandler())
root = logging.root
log_handler = TimedRotatingFileHandlerSafe(LOG_FILE, when='MIDNIGHT')
log_handler.setLevel(logging.DEBUG)
log_handler.setFormatter(FORMATTER)
root.addHandler(log_handler)
err_handler = TimedRotatingFileHandlerSafe(ERR_FILE, when='MIDNIGHT')
err_handler.setLevel(logging.ERROR)
err_handler.setFormatter(FORMATTER)
root.addHandler(err_handler)
```