https://github.com/toddlers/logpurge
Logpurge Solution
https://github.com/toddlers/logpurge
logrotate-utility python
Last synced: 3 months ago
JSON representation
Logpurge Solution
- Host: GitHub
- URL: https://github.com/toddlers/logpurge
- Owner: toddlers
- Created: 2014-08-11T11:01:11.000Z (almost 11 years ago)
- Default Branch: master
- Last Pushed: 2014-08-13T10:06:00.000Z (almost 11 years ago)
- Last Synced: 2023-08-03T00:01:51.782Z (almost 2 years ago)
- Topics: logrotate-utility, python
- Language: Python
- Size: 168 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# logpurge
Manage logs on the servers easily. I have written this after fiddling with logrotate utility on linux boxes.
Some of the problems which I encountered with logorotate utility are :
- There is no regex support in logrotate utility. Only globs are available
- You have to restart your application to point to new log file.
- Managing multiple actions for your logs e.g. you want to upload your logs to s3 selectively and then delete them from the disk
## Install the dependecies before using :
``` sh
sudo yum install python-boto.noarch PyYAML.x86_64 python-httplib2.noarch -y
```
## Usage
### General help
``` sh
λ: python logpurge.py --help
Usage: logpurge.py [options]
Options:
-h, --help show this help message and exit
--cfg=CONFIG read options from config file
--list list all sections
--only=ONLY only process the specified section from the file
```
- Using only option you can process a specific filegroup from the config file
- List option will process all the sections in the config file and provide the files list for action
## Example config
```
AWS_ACCESS_KEY: 'AKIAIN4A4SHXQ'
AWS_SECRET_ACCESS_KEY: 'ALO5UUvZsSvbDn9kTqAGn9Z'
filegroup1:
path: '/mnt/ephemeral/jetty/logs/'
files: "server.log.*"
dateregex: '(\d{4}-\d{2}-\d{2})'
upto: "1 days ago"
action:
- s3
- delete
bucket: 'logs_testing'
filegroup2:
path: '/mnt/ephemeral/jetty/logs/'
files: "activity.log.*"
dateregex: '(\d{4}-\d{2}-\d{2})'
upto: "1 days ago"
action:
- move
dest: "/tmp/"
filegroup3:
path: '/mnt/ephemeral/jetty/logs/'
files: "*.start.log"
dateregex: '(\d{4}-\d{2}-\d{2})'
upto: "1 days ago"
action:
- delete
```
- Three types of actions are supported
- S3 will upload your logs to the mentioned s3 bucket with structure as "bucket/instance_ID/logfilename"
- DELETE will delete the logs from the disk
- MOVE will the log files to the mentioned destination. It will create the destination if doesn't exists
- You need to provide the date regex which is in the log file e.g. 2014-03-12.start.log
- Provide file glob . e.g. for file name 2014-03-12.start.log glob will be *.start.log
- Age is specified by "upto". Specify the maxage like "x days ago"