{"id":13530365,"url":"https://github.com/jasonish/py-idstools","last_synced_at":"2025-04-12T15:42:23.393Z","repository":{"id":9489288,"uuid":"11379148","full_name":"jasonish/py-idstools","owner":"jasonish","description":"idstools: Snort and Suricata Rule and Event Utilities in Python (Including a Rule Update Tool)","archived":false,"fork":false,"pushed_at":"2023-11-01T17:41:50.000Z","size":5127,"stargazers_count":281,"open_issues_count":25,"forks_count":85,"subscribers_count":33,"default_branch":"master","last_synced_at":"2025-04-03T15:12:43.188Z","etag":null,"topics":["ids","intrusion-detection","snort","suricata","unified2"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jasonish.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2013-07-12T22:14:52.000Z","updated_at":"2025-03-15T10:53:37.000Z","dependencies_parsed_at":"2022-08-04T18:30:24.686Z","dependency_job_id":"da5c6baa-722d-4553-b23d-8bb8e067ccf9","html_url":"https://github.com/jasonish/py-idstools","commit_stats":{"total_commits":302,"total_committers":9,"mean_commits":33.55555555555556,"dds":"0.026490066225165587","last_synced_commit":"458cedcebd12483c829b30df259ccfb93b7bade7"},"previous_names":[],"tags_count":23,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jasonish%2Fpy-idstools","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jasonish%2Fpy-idstools/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jasonish%2Fpy-idstools/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jasonish%2Fpy-idstools/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jasonish","download_url":"https://codeload.github.com/jasonish/py-idstools/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248590917,"owners_count":21129912,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ids","intrusion-detection","snort","suricata","unified2"],"created_at":"2024-08-01T07:00:48.792Z","updated_at":"2025-04-12T15:42:23.372Z","avatar_url":"https://github.com/jasonish.png","language":"Python","funding_links":[],"categories":["Programming Libraries and Toolkits"],"sub_categories":[],"readme":"py-idstools |docs|\n==================\n\npy-idstools is a collection of Python libraries for working with IDS\nsystems (typically Snort and Suricata).\n\nNote for `rulecat` Users\n------------------------\nRulecat development has stalled. Future rule management work is now done in\nSuricata-Update which is bundled with Suricata. Please consider switching to\nSuricata-Update.\n\nIncluded Programs\n-----------------\n- rulecat - Basic Suricata rule management tool suitable as a\n  replacement for for Oinkmaster and Pulled Pork.\n- eve2pcap - Convert packets and payloads in eve logs to pcap.\n- u2json - Convert unified2 files or spool directories to JSON.\n- gensidmsgmap - Easily create a sid-msg.map file from rule files,\n  directories or a rule tarball.\n- dumpdynamicrules - Helper for dumping Snort SO dynamic rule stubs.\n- u2eve - Convert unified2 files to EVE compatible JSON.\n\nLibrary Features\n----------------\n\n- Snort/Suricata unified2 log file parsing.\n- Continuous unified2 directory spool reading with bookmarking.\n- Snort/Suricata rule parser.\n- Parser and lookup maps for classification.config.\n- Parser and lookup maps for gen-msg.map and sid-msg.map.\n\nRequirements\n------------\n\n- Python 2.7 or newer.\n- Currently only tested on Linux.\n\nInstallation\n------------\n\nLatest Release (Recommended)\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n    pip install idstools\n\nor on Fedora and CentOS (with EPEL):\n\n    yum install python-idstools\n\nLatest from Git\n~~~~~~~~~~~~~~~\n\n    pip install https://github.com/jasonish/py-idstools/archive/master.zip\n\nManually\n~~~~~~~~\n\nThe idstools programs do not have to be installed to be used, they can\nbe executable directly from the archive directory::\n\n  ./bin/idstools-rulecat\n\nOr to install manually::\n\n  python setup.py install\n\nExamples\n--------\n\nReading a Unified2 Spool Directory\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nThe following code snippet will \"tail\" a unified log directory\nreturning each record as a dict-like object::\n\n  from idstools import unified2\n\n  reader = unified2.SpoolRecordReader(\"/var/log/snort\",\n      \"unified2.log\", follow=True)\n  for record in reader:\n      if isinstance(record, unified2.Event):\n          print(\"Event:\")\n      elif isinstance(record, unified2.Packet):\n          print(\"Packet:\")\n      elif isinstance(record, unified2.ExtraData):\n          print(\"Extra-Data:\")\n      print(record)\n\nSee the `idstools unified2\n\u003chttp://idstools.readthedocs.io/en/latest/unified2.html\u003e`_\ndocumentation for more information on read and parsing unified2 files.\n\nParse Suricata/Snort Rules\n~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nThe following code snippet will parse all the rules in a rule file::\n\n  from idstools import rule\n\n  for rule in rule.parse_file(sys.argv[1]):\n      print(\"[%d:%d:%d] %s\" % (\n          rule.gid, rule.sid, rule.rev, rule.msg))\n\nIn addition to parsing `files\n\u003chttp://idstools.readthedocs.io/en/latest/apidoc/idstools.rule.html#idstools.rule.parse_file\u003e`_,\n`file objects\n\u003chttp://idstools.readthedocs.io/en/latest/apidoc/idstools.rule.html#idstools.rule.parse_fileobj\u003e`_\nand `strings\n\u003chttp://idstools.readthedocs.io/en/latest/apidoc/idstools.rule.html#idstools.rule.parse\u003e`_\ncontaining individual rules can be parsed.\n\nUpdate Suricata Rules\n~~~~~~~~~~~~~~~~~~~~~\n\nThe following command will update your Suricata rules with the latest\nEmerging Threats Open ruleset for the version of Snort you have\ninstalled::\n\n  idstools-rulecat -o /etc/suricata/rules\n\nSee the `idstools-rulecat documentation\n\u003chttp://idstools.readthedocs.io/en/latest/tools/rulecat.html\u003e`_ for\nmore examples and options.\n\nDocumentation\n-------------\n\nFurther documentation is located at http://idstools.readthedocs.org.\n\nChangelog\n---------\n\n0.6.5 - 2023-11-02\n~~~~~~~~~~~~~~~~~~\n- dumpdynamicrules: Python 3 fix, plus fix for handling directories:\n  https://github.com/jasonish/py-idstools/pull/91\n- rulecat: Fix placement of .md5 extension:\n  https://github.com/jasonish/py-idstools/pull/82\n- rules: allow config action to be used in local.rules:\n  https://github.com/jasonish/py-idstools/pull/88\n- rules: add more header elements into Rule object:\n  https://github.com/jasonish/py-idstools/pull/87\n- eve2pcap: ipv6 fix: https://github.com/jasonish/py-idstools/pull/86\n- misc: replace warn with warning\n- unified2: support for event type 3:\n  https://github.com/jasonish/py-idstools/pull/74\n- dumpdynamicrules: repack fix for directories:\n  https://github.com/jasonish/py-idstools/pull/91\n\n0.6.4 - 2020-08-02\n~~~~~~~~~~~~~~~~~~\n- eve2pcap: fix displaying of errors from libpcap\n- eve2pcap: python3 fixes\n- eve2pcap: print number of packets converted on exit\n- rules: fix parsing of rules where the address or port list has a space\n- `Commit log \u003chttps://github.com/jasonish/py-idstools/compare/0.6.3...0.6.4\u003e`_\n\n0.6.3 - 2017-11-20\n~~~~~~~~~~~~~~~~~~\n- eve2pcap: fix segfault when calling libpcap functions.\n- rulecat: for Emerging Threat rule URLs, use the Suricata version as found\n- rulecat: default to Suricata 4.0 if it can't be found.\n- rule parser: fix case where rule option does not end in ; and is\n  last option (https://github.com/jasonish/py-idstools/issues/58)\n- `Commit log \u003chttps://github.com/jasonish/py-idstools/compare/0.6.2...0.6.3\u003e`_\n\n0.6.2 - 2017-08-09\n~~~~~~~~~~~~~~~~~~\n- rulecat: ignore *deleted.rules by default. Provide --no-ignore\n  option to disable default ignores without having to add a new\n  ignore.\n- rulecat: suppress progress bar if quiet\n- rulecat: fix output filenaming for downloads that are a single rule\n  file\n- rulecat: more python3/unicode fixes\n- rule parser: if metadata is specified more than once, append to the\n  existing metadata list instead of replacing it\n  (https://github.com/jasonish/py-idstools/issues/57)\n- `Commit log \u003chttps://github.com/jasonish/py-idstools/compare/0.6.1...0.6.2\u003e`_\n\n0.6.1 - 2017-05-25\n~~~~~~~~~~~~~~~~~~\n- idstools-rulecat: handle zip archive files\n- rules: handle msg with escaped semicolons\n- rulecat: don't generate report summary if its not going to be logged\n  anyways (https://github.com/jasonish/py-idstools/issues/49)\n- rulecat: Python 3 fixes\n- rules: speed up parsing\n- `Commit log \u003chttps://github.com/jasonish/py-idstools/compare/0.6.0...0.6.1\u003e`_\n\n0.6.0 - 2017-03-29\n~~~~~~~~~~~~~~~~~~\n- idstools-u2eve - output packet records\n- idstools-rulecat: allow --local to be specified multiple times\n- idstools-rulecat: --ignore option to ignore filenames\n- More python 3 fixups.\n- unified2 - deprecate event readers, use record readers instead\n  (https://github.com/jasonish/py-idstools/issues/14)\n- u2json: --packet-hex and --printable to print raw buffers as printable\n  chars and hex in addition to base64.\n- u2eve: --packet-printable to include a \"packet_printable\" field\n- u2eve: include Snort extra-data with printable data.\n- `Commit log \u003chttps://github.com/jasonish/py-idstools/compare/0.5.6...0.6.0\u003e`_\n\n0.5.6\n~~~~~\n- idstools-rulecat: fix issue parsing Suricata version on Python 3\n- idstools-rulecat: don't convert rules with noalert to drop\n- idstools-rulecat: allow suricata version to be set on the command\n  line (https://github.com/jasonish/py-idstools/issues/38)\n- `Commit log \u003chttps://github.com/jasonish/py-idstools/compare/0.5.5...0.5.6\u003e`_\n\n0.5.5\n~~~~~\n- unified2: fix reading of ipv6 events\n- idstools-u2json: option to sort the keys\n- u2spewfoo: IPv6 printing fixes\n- idstools-rulecat: use ET \"enhanced\" rules by default\n- idstools-rulecat: suricata inspired colour logging\n- idstools-rulecat: handle URLs ending with query parameters\n- `Commit log \u003chttps://github.com/jasonish/py-idstools/compare/0.5.4...0.5.5\u003e`_\n\n0.5.4\n~~~~~\n\n- idstools: handle rules with no msg in rule parser\n- idstools-rulecat: support a drop.conf for setting rules to drop\n- idstools-eve2pcap: allow link type to be set on command line\n- unified2: handle large appid buffer in newer versions of Snort.\n- `Commit log \u003chttps://github.com/jasonish/py-idstools/compare/0.5.3...0.5.4\u003e`_\n\n0.5.3\n~~~~~\n- idstools-rulecat: better documentation\n- idstools-rulecat: use ET Pro https URL\n- `Commit log \u003chttps://github.com/jasonish/py-idstools/compare/0.5.2...0.5.3\u003e`_\n\n0.5.2\n~~~~~\n- idstools-u2json: fix --delete\n- idstools-u2json: add --verbose flag for debug logging\n- idstools-rulecat: allow multiple urls\n- `Commit log \u003chttps://github.com/jasonish/py-idstools/compare/0.5.1...0.5.2\u003e`_\n\n0.5.1\n~~~~~\n- New tool: eve2pcap. Converts packets and payloads found in Suricata\n  EVE logs to pcap files.\n- Rule parser: handle multi-line rules.\n- `Commit log \u003chttps://github.com/jasonish/py-idstools/compare/0.5.0...0.5.1\u003e`_\n\n0.5.0\n~~~~~\n- New tool: idstools-dumpdynamicrules. A wrapper around Snort to dump\n  dynamic rule stubs and optionally repack the tarball with the new\n  stubs.\n- New tool: idstools-u2eve. Basically a copy of the current u2json,\n  but will aim to keep a compatible eve output style.  idstools-u2json\n  will probably become more of a basic example program.\n- A basic packet decoding module.\n- New tool: rulecat. A basic Suricata rule management tool.\n- `Commit log \u003chttps://github.com/jasonish/py-idstools/compare/0.4.4...0.5.0\u003e`_\n\n0.4.4\n~~~~~\n- Fix reading of growing file on OS X.\n- Fix error in parsing decoder rules introduced in 0.4.3.\n- `Commit log \u003chttps://github.com/jasonish/py-idstools/compare/0.4.3...0.4.4\u003e`_\n\n0.4.3\n~~~~~\n- Make the rule direction an accessible field of the rule object.\n- `Commit log \u003chttps://github.com/jasonish/py-idstools/compare/0.4.2...0.4.3\u003e`_\n\n0.4.2\n~~~~~\n- Fix issue loading signature map files (GitHub issue #2).\n- `Commit log \u003chttps://github.com/jasonish/py-idstools/compare/0.4.1...0.4.2\u003e`_\n\n0.4.1\n~~~~~\n- Fix IPv6 address unpacking.\n- In u2json, if the protocol number can't be converted to a string,\n  encode the number as a string for a consistent JSON data type.\n- `Commit log \u003chttps://github.com/jasonish/py-idstools/compare/0.4.0...0.4.1\u003e`_\n\n0.4.0\n~~~~~\n- New tool, u2json to convert unified2 files to JSON.\n- `Commit log \u003chttps://github.com/jasonish/py-idstools/compare/0.3.1...0.4.0\u003e`_\n\n0.3.1\n~~~~~\n- Support the new appid unified2 event types introduced in Snort\n  2.9.7.0.alpha.\n- `Commit log \u003chttps://github.com/jasonish/py-idstools/compare/0.3.0...0.3.1\u003e`_\n\n.. |docs| image:: https://readthedocs.org/projects/idstools/badge/?version=latest\n   :alt: Documentation Status\n   :scale: 100%\n   :target: https://idstools.readthedocs.io/en/latest/?badge=latest\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjasonish%2Fpy-idstools","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjasonish%2Fpy-idstools","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjasonish%2Fpy-idstools/lists"}