Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/maxpert/drep
dynamic regular expression print
https://github.com/maxpert/drep
filtering logging rust rust-lang tool
Last synced: 2 months ago
JSON representation
dynamic regular expression print
- Host: GitHub
- URL: https://github.com/maxpert/drep
- Owner: maxpert
- License: mit
- Created: 2020-06-20T23:40:40.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2022-07-10T03:52:39.000Z (over 2 years ago)
- Last Synced: 2024-11-01T18:27:56.603Z (3 months ago)
- Topics: filtering, logging, rust, rust-lang, tool
- Language: Rust
- Homepage:
- Size: 35.2 KB
- Stars: 89
- Watchers: 4
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://github.com/maxpert/drep/actions) [](https://github.com/maxpert/drep/releases)
# drep is dynamic regular expression print
`drep` is `grep` with dynamic reloadable filter expressions. This allows filtering stream of
logs/lines, while changing filters on the fly.
Filter is either a regex or plain text match, provided via input file.
Here is an example usage:
```bash
tail -f /var/log/nginx/error.log | drep -f /etc/drep/filters
```
Usually you will end up using this with your servers:
```bash
java -jar my-server.jar | drep -f server-filters
```
or
```bash
./uwsgi -s :8080 -w my_app | drep -f server-filters
```
## Filter file syntax
Each line of the filters file is an expression that starts with `~`, `=`, `!=`, or `!~`. The matches will be done
in the order filters written in the file, and if a filter matches subsequent filters won't be executed.
- Any line that starts with `!~` implies does not match regex, e.g: `!~"time": \d+.\d{0,2}`
- Any line that starts with `~` implies match regex, e.g: `~"time": \d+.\d{3,}`
- Any line that starts with `!=` implies does not contain text, e.g: `!=INFO`
- Any line that starts with `=` implies contain text, e.g: `="total-duration"`
Everything else is ignored, as you can see from plain text.
For regular expression documentation please refer to [this document](https://docs.rs/regex/1.3.9/regex/).
## Why?
While `grep --line-buffered` can do something similar changing regex on the fly is not possible.
Change filter regex on the fly is extremely useful in server/process environments where it's not possible to restart
the process just to change the `grep` filter.
Building on unix philosophy `drep` does only one job well, given bunch of filter from an input file
it can filter input lines to stdout.
## Features
- Lightweight on CPU, and memory (~3MB memory foot print, and 2 threads in total).
- Watch and reload filters file.
- No GC pauses and memory safe (Written in Rust).
- Plain text & regex matching (with negation support).
## Usage example
Given following simple `fizzbuzz.py`:
```python
import time
i = 1
while True:
fb = ""
if i % 3 == 0:
fb = "fizz"
if i % 5 == 0:
fb = "{}buzz".format(fb)
if fb:
print("{}. {}".format(i, fb), flush=True)
i = i + 1
time.sleep(0.1)
```
We can launch and pipe it's output `python fizzbuzz.py | drep -f filters`. Now if the contents of `filters` are:
```
~\sfizz\n
```
drep will only emit logs with fizz. e.g.
```
642. fizz
648. fizz
651. fizz
654. fizz
...
```
While keeping the process running without exiting you can just modify `filters` to:
```
~\sbuzz\n
```
This will change the drep output on the fly to only emit buzz:
```
805. buzz
815. buzz
820. buzz
...
```
## Building
Just clone the repo and run `cargo build --release`.