https://github.com/peppelinux/pyjq
A simple Python package to Query over Json Data
https://github.com/peppelinux/pyjq
django json wazuh
Last synced: 8 months ago
JSON representation
A simple Python package to Query over Json Data
- Host: GitHub
- URL: https://github.com/peppelinux/pyjq
- Owner: peppelinux
- License: bsd-3-clause
- Created: 2020-04-07T01:13:46.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2020-04-09T14:05:31.000Z (about 6 years ago)
- Last Synced: 2024-12-24T20:37:32.188Z (over 1 year ago)
- Topics: django, json, wazuh
- Language: Python
- Size: 68.4 KB
- Stars: 3
- Watchers: 3
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# pyjq
A simple Python package to Query Json Data.
## Features
- Supports pure json files
- Supports multiple json objects in a file, delimited by newlines (/n)
- Supports gzipped files
- Supports customizabile filters
- Supports pure datetime range filters
## Todo
The filters could be extended easily, adopting [Python3 stdlib operator](https://docs.python.org/3/library/operator.html).
See `pyjq.PyJQ.filter` to extend ops mapping.
## Installation
````
pip install pyjq-ng
````
## Example data
See `example/alerts.json`.
pyjq works on lines by lines (splitted by \n).
It have been used for Wazuh alert json files and Django dumps.
````
pyjq -j examples/django_dump.json -limit 2 -filter 'fields__original_url == https://google.com'
pyjq -j examples/django_dump.json -limit 2 -filter 'model == urlshortener.urlshortener'
````
## Usage
'agent__name' it's an example of the namespace used by pyjq to access to nested childs. It other word it means `json['agent']['name']`.
It haven't limits on number of nested elements.
Apply some custom filters with AND and OR operators on Wazuh Alert file
````
pyjq -j ../Scaricati/alerts.json -filter 'agent__ip == 172.16.16.102 and agent__name == telegram-gw or agent__ip == 172.16.16.108'
````
Contains operator
````
pyjq -j ../Scaricati/alerts.json -filter 'rule__description in iptables and agent__name == dev-bastion'
````
Convert a specified filed to a pure datetime object and filter in a specified range
````
pyjq -j ../Scaricati/alerts.json -start_datetime 2020-04-06T10:22:00 -end_datetime 2020-04-06T13:22:00 -datetime_field timestamp
````
Realtime reading, it will only takes the latter entries, delimited by newline \n
````
pyjq -j /var/ossec/logs/alerts/alerts.json -datetime_field timestamp -realtime
````
Use a gzipped json file directly
````
pyjq -j ../Scaricati/alerts.json.gzip
````
Limit results to 2
````
pyjq -j ../Scaricati/alerts.json -limit 2
````
Realtime monitoring of a specific entity
````
pyjq -j /var/ossec/logs/alerts/alerts.json -realtime -filter 'agent__name == tinyurl and rule__level == 3'
````
Custom callback, usefull for bot integration and other pub/sub APIs
````
python3 pyjq -j examples/alerts.json -realtime -filter 'agent__name == tinyurl and rule__description in ssh' -callback 'examples.callback.things'
````
Reading from stdin
````
cat examples/alerts.json | python3 ./pyjq -filter 'rule__level > 3'
# continous processing
tail -f /tmp/alerts.json | python3 ./pyjq -filter 'location != osquery'
````
## Author
Giuseppe De Marco
## Credits
Wazuh SIEM group @GarrLab