Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/jasonish/py-idstools
idstools: Snort and Suricata Rule and Event Utilities in Python (Including a Rule Update Tool)
https://github.com/jasonish/py-idstools
ids intrusion-detection snort suricata unified2
Last synced: 5 days ago
JSON representation
idstools: Snort and Suricata Rule and Event Utilities in Python (Including a Rule Update Tool)
- Host: GitHub
- URL: https://github.com/jasonish/py-idstools
- Owner: jasonish
- License: other
- Created: 2013-07-12T22:14:52.000Z (over 11 years ago)
- Default Branch: master
- Last Pushed: 2023-11-01T17:41:50.000Z (about 1 year ago)
- Last Synced: 2024-11-12T14:44:22.909Z (about 1 month ago)
- Topics: ids, intrusion-detection, snort, suricata, unified2
- Language: Python
- Homepage:
- Size: 4.89 MB
- Stars: 276
- Watchers: 34
- Forks: 85
- Open Issues: 22
-
Metadata Files:
- Readme: README.rst
- License: LICENSE
Awesome Lists containing this project
- awesome-suricata - py-idstools - Snort and Suricata Rule and Event Utilities in Python (Including a Rule Update Tool). (Programming Libraries and Toolkits)
README
py-idstools |docs|
==================py-idstools is a collection of Python libraries for working with IDS
systems (typically Snort and Suricata).Note for `rulecat` Users
------------------------
Rulecat development has stalled. Future rule management work is now done in
Suricata-Update which is bundled with Suricata. Please consider switching to
Suricata-Update.Included Programs
-----------------
- rulecat - Basic Suricata rule management tool suitable as a
replacement for for Oinkmaster and Pulled Pork.
- eve2pcap - Convert packets and payloads in eve logs to pcap.
- u2json - Convert unified2 files or spool directories to JSON.
- gensidmsgmap - Easily create a sid-msg.map file from rule files,
directories or a rule tarball.
- dumpdynamicrules - Helper for dumping Snort SO dynamic rule stubs.
- u2eve - Convert unified2 files to EVE compatible JSON.Library Features
----------------- Snort/Suricata unified2 log file parsing.
- Continuous unified2 directory spool reading with bookmarking.
- Snort/Suricata rule parser.
- Parser and lookup maps for classification.config.
- Parser and lookup maps for gen-msg.map and sid-msg.map.Requirements
------------- Python 2.7 or newer.
- Currently only tested on Linux.Installation
------------Latest Release (Recommended)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~pip install idstools
or on Fedora and CentOS (with EPEL):
yum install python-idstools
Latest from Git
~~~~~~~~~~~~~~~pip install https://github.com/jasonish/py-idstools/archive/master.zip
Manually
~~~~~~~~The idstools programs do not have to be installed to be used, they can
be executable directly from the archive directory::./bin/idstools-rulecat
Or to install manually::
python setup.py install
Examples
--------Reading a Unified2 Spool Directory
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~The following code snippet will "tail" a unified log directory
returning each record as a dict-like object::from idstools import unified2
reader = unified2.SpoolRecordReader("/var/log/snort",
"unified2.log", follow=True)
for record in reader:
if isinstance(record, unified2.Event):
print("Event:")
elif isinstance(record, unified2.Packet):
print("Packet:")
elif isinstance(record, unified2.ExtraData):
print("Extra-Data:")
print(record)See the `idstools unified2
`_
documentation for more information on read and parsing unified2 files.Parse Suricata/Snort Rules
~~~~~~~~~~~~~~~~~~~~~~~~~~The following code snippet will parse all the rules in a rule file::
from idstools import rule
for rule in rule.parse_file(sys.argv[1]):
print("[%d:%d:%d] %s" % (
rule.gid, rule.sid, rule.rev, rule.msg))In addition to parsing `files
`_,
`file objects
`_
and `strings
`_
containing individual rules can be parsed.Update Suricata Rules
~~~~~~~~~~~~~~~~~~~~~The following command will update your Suricata rules with the latest
Emerging Threats Open ruleset for the version of Snort you have
installed::idstools-rulecat -o /etc/suricata/rules
See the `idstools-rulecat documentation
`_ for
more examples and options.Documentation
-------------Further documentation is located at http://idstools.readthedocs.org.
Changelog
---------0.6.5 - 2023-11-02
~~~~~~~~~~~~~~~~~~
- dumpdynamicrules: Python 3 fix, plus fix for handling directories:
https://github.com/jasonish/py-idstools/pull/91
- rulecat: Fix placement of .md5 extension:
https://github.com/jasonish/py-idstools/pull/82
- rules: allow config action to be used in local.rules:
https://github.com/jasonish/py-idstools/pull/88
- rules: add more header elements into Rule object:
https://github.com/jasonish/py-idstools/pull/87
- eve2pcap: ipv6 fix: https://github.com/jasonish/py-idstools/pull/86
- misc: replace warn with warning
- unified2: support for event type 3:
https://github.com/jasonish/py-idstools/pull/74
- dumpdynamicrules: repack fix for directories:
https://github.com/jasonish/py-idstools/pull/910.6.4 - 2020-08-02
~~~~~~~~~~~~~~~~~~
- eve2pcap: fix displaying of errors from libpcap
- eve2pcap: python3 fixes
- eve2pcap: print number of packets converted on exit
- rules: fix parsing of rules where the address or port list has a space
- `Commit log `_0.6.3 - 2017-11-20
~~~~~~~~~~~~~~~~~~
- eve2pcap: fix segfault when calling libpcap functions.
- rulecat: for Emerging Threat rule URLs, use the Suricata version as found
- rulecat: default to Suricata 4.0 if it can't be found.
- rule parser: fix case where rule option does not end in ; and is
last option (https://github.com/jasonish/py-idstools/issues/58)
- `Commit log `_0.6.2 - 2017-08-09
~~~~~~~~~~~~~~~~~~
- rulecat: ignore *deleted.rules by default. Provide --no-ignore
option to disable default ignores without having to add a new
ignore.
- rulecat: suppress progress bar if quiet
- rulecat: fix output filenaming for downloads that are a single rule
file
- rulecat: more python3/unicode fixes
- rule parser: if metadata is specified more than once, append to the
existing metadata list instead of replacing it
(https://github.com/jasonish/py-idstools/issues/57)
- `Commit log `_0.6.1 - 2017-05-25
~~~~~~~~~~~~~~~~~~
- idstools-rulecat: handle zip archive files
- rules: handle msg with escaped semicolons
- rulecat: don't generate report summary if its not going to be logged
anyways (https://github.com/jasonish/py-idstools/issues/49)
- rulecat: Python 3 fixes
- rules: speed up parsing
- `Commit log `_0.6.0 - 2017-03-29
~~~~~~~~~~~~~~~~~~
- idstools-u2eve - output packet records
- idstools-rulecat: allow --local to be specified multiple times
- idstools-rulecat: --ignore option to ignore filenames
- More python 3 fixups.
- unified2 - deprecate event readers, use record readers instead
(https://github.com/jasonish/py-idstools/issues/14)
- u2json: --packet-hex and --printable to print raw buffers as printable
chars and hex in addition to base64.
- u2eve: --packet-printable to include a "packet_printable" field
- u2eve: include Snort extra-data with printable data.
- `Commit log `_0.5.6
~~~~~
- idstools-rulecat: fix issue parsing Suricata version on Python 3
- idstools-rulecat: don't convert rules with noalert to drop
- idstools-rulecat: allow suricata version to be set on the command
line (https://github.com/jasonish/py-idstools/issues/38)
- `Commit log `_0.5.5
~~~~~
- unified2: fix reading of ipv6 events
- idstools-u2json: option to sort the keys
- u2spewfoo: IPv6 printing fixes
- idstools-rulecat: use ET "enhanced" rules by default
- idstools-rulecat: suricata inspired colour logging
- idstools-rulecat: handle URLs ending with query parameters
- `Commit log `_0.5.4
~~~~~- idstools: handle rules with no msg in rule parser
- idstools-rulecat: support a drop.conf for setting rules to drop
- idstools-eve2pcap: allow link type to be set on command line
- unified2: handle large appid buffer in newer versions of Snort.
- `Commit log `_0.5.3
~~~~~
- idstools-rulecat: better documentation
- idstools-rulecat: use ET Pro https URL
- `Commit log `_0.5.2
~~~~~
- idstools-u2json: fix --delete
- idstools-u2json: add --verbose flag for debug logging
- idstools-rulecat: allow multiple urls
- `Commit log `_0.5.1
~~~~~
- New tool: eve2pcap. Converts packets and payloads found in Suricata
EVE logs to pcap files.
- Rule parser: handle multi-line rules.
- `Commit log `_0.5.0
~~~~~
- New tool: idstools-dumpdynamicrules. A wrapper around Snort to dump
dynamic rule stubs and optionally repack the tarball with the new
stubs.
- New tool: idstools-u2eve. Basically a copy of the current u2json,
but will aim to keep a compatible eve output style. idstools-u2json
will probably become more of a basic example program.
- A basic packet decoding module.
- New tool: rulecat. A basic Suricata rule management tool.
- `Commit log `_0.4.4
~~~~~
- Fix reading of growing file on OS X.
- Fix error in parsing decoder rules introduced in 0.4.3.
- `Commit log `_0.4.3
~~~~~
- Make the rule direction an accessible field of the rule object.
- `Commit log `_0.4.2
~~~~~
- Fix issue loading signature map files (GitHub issue #2).
- `Commit log `_0.4.1
~~~~~
- Fix IPv6 address unpacking.
- In u2json, if the protocol number can't be converted to a string,
encode the number as a string for a consistent JSON data type.
- `Commit log `_0.4.0
~~~~~
- New tool, u2json to convert unified2 files to JSON.
- `Commit log `_0.3.1
~~~~~
- Support the new appid unified2 event types introduced in Snort
2.9.7.0.alpha.
- `Commit log `_.. |docs| image:: https://readthedocs.org/projects/idstools/badge/?version=latest
:alt: Documentation Status
:scale: 100%
:target: https://idstools.readthedocs.io/en/latest/?badge=latest