{"id":43061168,"url":"https://github.com/wagga40/zircolite","last_synced_at":"2026-04-06T10:02:24.710Z","repository":{"id":37698827,"uuid":"343939740","full_name":"wagga40/Zircolite","owner":"wagga40","description":"A standalone SIGMA-based detection tool for EVTX, Auditd and Sysmon for Linux logs","archived":false,"fork":false,"pushed_at":"2026-01-31T13:50:38.000Z","size":58030,"stargazers_count":777,"open_issues_count":1,"forks_count":108,"subscribers_count":26,"default_branch":"master","last_synced_at":"2026-01-31T22:51:19.435Z","etag":null,"topics":["auditd","detection","evtx","evtxtract","forensics","forensics-tools","pysigma","python3","sigma","sigma-rules","sysmon"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wagga40.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2021-03-02T23:17:06.000Z","updated_at":"2026-01-31T15:20:55.000Z","dependencies_parsed_at":"2023-02-15T06:01:17.859Z","dependency_job_id":"46e893b3-9786-4221-b5f5-ec839c38d80b","html_url":"https://github.com/wagga40/Zircolite","commit_stats":null,"previous_names":[],"tags_count":40,"template":false,"template_full_name":null,"purl":"pkg:github/wagga40/Zircolite","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wagga40%2FZircolite","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wagga40%2FZircolite/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wagga40%2FZircolite/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wagga40%2FZircolite/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wagga40","download_url":"https://codeload.github.com/wagga40/Zircolite/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wagga40%2FZircolite/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29179561,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-06T22:12:24.066Z","status":"ssl_error","status_checked_at":"2026-02-06T22:12:09.859Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["auditd","detection","evtx","evtxtract","forensics","forensics-tools","pysigma","python3","sigma","sigma-rules","sysmon"],"created_at":"2026-01-31T12:07:40.975Z","updated_at":"2026-04-06T10:02:24.695Z","avatar_url":"https://github.com/wagga40.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# \u003cp align=\"center\"\u003e![](pics/zircolite_400.png)\u003c/p\u003e\n\n## Standalone SIGMA-Based Detection Tool for EVTX, Auditd, Sysmon for Linux, XML, CSV, or JSONL/NDJSON Logs \n![](pics/Zircolite-v3-cli.webp)\n\n[![python](https://img.shields.io/badge/python-3.10-blue)](https://www.python.org/)\n![version](https://img.shields.io/badge/Architecture-64bit-red)\n\n**Zircolite** is a standalone tool written in Python 3 that allows you to use SIGMA rules on:\n\n- MS Windows EVTX (EVTX, XML, and JSONL formats)\n- Auditd logs\n- Sysmon for Linux\n- EVTXtract\n- CSV and XML logs\n- JSON Array logs\n\n### Key Features\n\n- **Automatic Log Type Detection**: Automatically identifies log formats and timestamp fields using magic bytes, content analysis, and regex-based fallback -- no need to specify format flags in most cases.\n- **Multiple Input Formats**: Supports various log formats including EVTX, JSON Lines, JSON Arrays, CSV, XML, and more. Compressed or archived logs (gzip, bzip2, ZIP, 7-Zip) are supported; use `--archive-password` for encrypted ZIP/7z.\n- **Native Sigma Support**: Zircolite can directly use native Sigma rules (YAML) by converting them with pySigma.\n- **SIGMA Backend**: It is based on a SIGMA backend (SQLite) and does not use internal SIGMA-to-something conversion.\n- **Advanced Log Manipulation**: It can manipulate input logs by splitting fields and applying transformations, allowing for more flexible and powerful log analysis.\n- **Field Transforms**: Apply custom Python transformations to fields during processing (e.g., Base64 decoding, hex-to-ASCII conversion).\n- **Flexible Export**: Zircolite can export results to multiple formats using Jinja [templates](templates), including JSON, CSV, JSONL, Splunk, Elastic, Zinc, Timesketch, and more.\n- **Rich Terminal Output**: Detection results displayed in severity-sorted tables with MITRE ATT\u0026CK technique IDs, ATT\u0026CK tactics heatmap, rule coverage metrics, clickable output file links, and contextual post-run suggestions.\n\n**You can use Zircolite directly with Python.** \n\n**Documentation is available [here](https://wagga40.github.io/Zircolite/) (dedicated site) or [here](docs) (repository directory).**\n\n## Requirements / Installation\n\nThe project has been tested with Python 3.10 and above. Install dependencies with: `pip3 install -r requirements.txt`.\n\n### Dependencies\n\n- **Required**: `orjson`, `xxhash`, `rich`, `RestrictedPython`, `requests`, `pySigma`, `evtx` (pyevtx-rs), `jinja2`, `lxml`, `psutil`, `pyyaml`, `py7zr` (for 7-Zip archives)\n\n:warning: On some systems (Mac, ARM, etc.), the `evtx` Python library may require Rust and Cargo to be installed.\n\n## Quick Start\n\nCheck out tutorials made by others (EN, ES, and FR) [here](#tutorials).\n\n### EVTX Files\n\nHelp is available with:\n\n```shell\npython3 zircolite.py -h\n```\n\nIf your EVTX files have the extension \".evtx\":\n\n```shell\n# python3 zircolite.py --evtx \u003cEVTX FOLDER or EVTX FILE\u003e --ruleset \u003cSIGMA RULESET\u003e [--ruleset \u003cOTHER RULESET\u003e]\npython3 zircolite.py --evtx sysmon.evtx --ruleset rules/rules_windows_merged.json\n```\n\n### Using Native Sigma Rules (YAML)\n\nYou can use native Sigma rules (YAML) directly:\n\n```shell\n# Single YAML rule\npython3 zircolite.py --evtx sample.evtx --ruleset path/to/rule.yml\n\n# Directory of Sigma rules\npython3 zircolite.py --evtx sample.evtx --ruleset ./sigma/rules/windows/process_creation\n\n# With pySigma pipelines\npython3 zircolite.py --evtx sample.evtx --ruleset rule.yml --pipeline sysmon --pipeline windows-logsources\n```\n\n### Other Log Formats\n\nZircolite **auto-detects** the log format in most cases, so explicit format flags are optional:\n\n```shell\n# Auto-detection (recommended) - Zircolite identifies the format automatically\npython3 zircolite.py --events auditd.log --ruleset rules/rules_linux.json\npython3 zircolite.py --events sysmon.log --ruleset rules/rules_linux.json\npython3 zircolite.py --events \u003cJSON_FOLDER_OR_FILE\u003e --ruleset rules/rules_windows_merged.json\n\n# Explicit format flags (override auto-detection)\npython3 zircolite.py --events auditd.log --ruleset rules/rules_linux.json --auditd\npython3 zircolite.py --events sysmon.log --ruleset rules/rules_linux.json --sysmon4linux\npython3 zircolite.py --events \u003cJSON_FOLDER_OR_FILE\u003e --ruleset rules/rules_windows_merged.json --jsononly\npython3 zircolite.py --events \u003cJSON_FOLDER_OR_FILE\u003e --ruleset rules/rules_windows_merged.json --json-array\npython3 zircolite.py --events \u003cCSV_FOLDER_OR_FILE\u003e --ruleset rules/rules_windows_merged.json --csv-input\npython3 zircolite.py --events \u003cXML_FOLDER_OR_FILE\u003e --ruleset rules/rules_windows_merged.json --xml-input\n```\n\n- The `--events` argument can be a file or a folder. If it is a folder, all log files in the current folder and subfolders will be selected (use `--no-recursion` to disable).\n- Use `--file-pattern` to specify a custom glob pattern for file selection.\n- Use `--no-auto-detect` to disable automatic format detection.\n\n\u003e [!TIP]\n\u003e If you want to try the tool, you can test with [EVTX-ATTACK-SAMPLES](https://github.com/sbousseaden/EVTX-ATTACK-SAMPLES) (EVTX files).\n\n### Running with Docker\n\n```bash\n# Pull the Docker image\ndocker pull wagga40/zircolite:latest\n# If your logs and rules are in a specific directory\ndocker run --rm --tty \\\n    -v $PWD:/case/input:ro \\\n    -v $PWD:/case/output \\\n    wagga40/zircolite:latest \\\n    -e /case/input \\\n    -o /case/output/detected_events.json \\\n    -r /case/input/a_sigma_rule.yml\n```\n\n- Replace `$PWD` with the directory (absolute path only) where your logs and rules/rulesets are stored.\n\n### Automatic Processing Optimization\n\nZircolite automatically optimizes processing based on your workload. When you run Zircolite with multiple files, it:\n\n1. **Analyzes your files** - counts files, measures sizes, checks available RAM\n2. **Selects optimal database mode** - unified (all files in one DB) vs. per-file (separate DB per file)\n3. **Enables parallel processing** - when beneficial, automatically processes files in parallel\n\n```shell\npython3 zircolite.py --evtx ./logs/ --ruleset rules/rules_windows_merged.json\n```\n\nYou can control this behavior:\n\n```shell\n# Disable automatic mode selection (force per-file mode)\npython3 zircolite.py --evtx ./logs/ --ruleset rules/rules_windows_merged.json --no-auto-mode\n\n# Force unified database mode (enables cross-file correlation)\npython3 zircolite.py --evtx ./logs/ --ruleset rules/rules_windows_merged.json --unified-db\n\n# Disable parallel processing\npython3 zircolite.py --evtx ./logs/ --ruleset rules/rules_windows_merged.json --no-parallel\n\n# Specify maximum workers manually\npython3 zircolite.py --evtx ./logs/ --ruleset rules/rules_windows_merged.json --parallel-workers 4\n```\n\nThe parallel processor automatically:\n- Calculates optimal worker count based on available memory, CPU cores, and file sizes\n- Monitors memory usage and throttles if approaching limits\n- Falls back to sequential processing if parallel isn't beneficial\n\n### Using YAML Configuration Files\n\nFor complex or repeated analysis workflows, use a YAML configuration file:\n\n```shell\n# Generate a default configuration file\npython3 zircolite.py --generate-config my_config.yaml\n\n# Run with a configuration file\npython3 zircolite.py --yaml-config my_config.yaml\n\n# CLI arguments override config file settings\npython3 zircolite.py --yaml-config my_config.yaml --evtx ./other_logs/\n```\n\nExample configuration file (`config/zircolite_example.yaml`):\n\n```yaml\ninput:\n  path: ./logs/\n  format: evtx\n  recursive: true\n\nrules:\n  rulesets:\n    - rules/rules_windows_merged.json\n  pipelines:\n    - sysmon\n\noutput:\n  file: detected_events.json\n  format: json\n\nprocessing:\n  streaming: true      # Single-pass processing (default: enabled)\n  unified_db: false    # Per-file databases (default)\n  auto_mode: true      # Automatic mode selection (default: enabled)\n\nparallel:\n  enabled: true        # Parallel processing (auto-enabled when beneficial)\n  max_workers: null    # Auto-detect based on CPU/memory\n  memory_limit_percent: 85.0\n```\n\n### Updating Default Rulesets\n\n```shell\npython3 zircolite.py -U\n```\n\nAlternatively, if you use [Task](https://taskfile.dev/) (go-task), run `task update-rules` from the project root to update rules from [Zircolite-Rules-v2](https://github.com/wagga40/Zircolite-Rules-v2). See [docs](docs/README.md) for other tasks (Docker build, clean, etc.).\n\n\u003e [!IMPORTANT]  \n\u003e Please note that these rulesets are provided to use Zircolite out of the box, but [you should generate your own rulesets](#why-you-should-build-your-own-rulesets) as they can be noisy or slow. These auto-updated rulesets are available in the dedicated repository: [Zircolite-Rules-v2](https://github.com/wagga40/Zircolite-Rules-v2).\n\n### Field Splitting\n\nField splitting extracts key-value pairs from fields. For example, Sysmon logs contain a `Hashes` field like:\n\n```\nSHA1=abc123,MD5=def456,SHA256=789xyz\n```\n\nWith field splitting configured in `config/config.yaml`:\n\n```yaml\nsplit:\n  Hashes:\n    separator: \",\"\n    equal: \"=\"\n```\n\nThe event becomes:\n\n```json\n{\n  \"SHA1\": \"abc123\",\n  \"MD5\": \"def456\",\n  \"SHA256\": \"789xyz\",\n  \"Hashes\": \"SHA1=abc123,MD5=def456,SHA256=789xyz\"\n}\n```\n\nNow you can write rules that match on `SHA256` or `MD5` directly.\n\n### Field Transforms\n\nTransforms apply Python code to field values during processing. They can decode data, extract IOCs, or detect attack patterns.\n\n**Example: Base64 Decoding**\n\nWhen a command line contains `powershell -enc SGVsbG8gV29ybGQ=`, the transform:\n\n```yaml\ntransforms:\n  CommandLine:\n    - info: \"Base64 decode\"\n      type: python\n      code: |\n        def transform(param):\n            import base64\n            import re\n            match = re.search(r'-[eE]nc(?:odedcommand)?\\s+([A-Za-z0-9+/=]+)', param)\n            if match:\n                try:\n                    return base64.b64decode(match.group(1)).decode('utf-16-le')\n                except:\n                    return \"\"\n            return \"\"\n      alias: true\n      alias_name: \"CommandLine_b64decoded\"\n```\n\nCreates a new field `CommandLine_b64decoded` containing `Hello World`.\n\n**Example: LOLBin Detection**\n\n```yaml\ntransforms:\n  Image:\n    - info: \"Detect Living Off The Land Binaries\"\n      type: python\n      code: |\n        def transform(param):\n            import re\n            lolbins = ['certutil', 'mshta', 'regsvr32', 'rundll32', 'bitsadmin']\n            exe_name = param.lower().split('\\\\')[-1].replace('.exe', '')\n            for lolbin in lolbins:\n                if exe_name == lolbin:\n                    return f\"LOLBIN:{lolbin}\"\n            return \"\"\n      alias: true\n      alias_name: \"Image_LOLBinMatch\"\n```\n\nWhen `Image` is `C:\\Windows\\System32\\certutil.exe`, creates `Image_LOLBinMatch` = `LOLBIN:certutil`.\n\nSee [Advanced documentation](docs/Advanced.md#field-transforms) for all available transforms and detailed configuration.\n\n## Documentation\n\nComplete documentation is available [here](docs).\n\n## Mini-GUI\n\nThe Mini-GUI can be used completely offline. It allows you to display and search results. You can automatically generate a Mini-GUI \"package\" with the `--package` option. Use `--package-dir` to specify the output directory. To learn how to use the Mini-GUI, check the documentation [here](docs/Advanced.md#mini-gui).\n\n### Detected Events by MITRE ATT\u0026CK® Techniques and Criticality Levels\n\n![](pics/gui.webp)\n\n### Detected Events Timeline\n\n![](pics/gui-timeline.webp)\n\n### Detected Events by MITRE ATT\u0026CK® Techniques Displayed on the Matrix \n\n![](pics/gui-matrix.webp)\n\n## Tutorials, References, and Related Projects\n\n### Tutorials\n\n- **English**: [Russ McRee](https://holisticinfosec.io) has published a detailed [tutorial](https://holisticinfosec.io/post/2021-09-28-zircolite/) on SIGMA and Zircolite on his blog.\n\n- **Spanish**: **César Marín** has published a tutorial in Spanish [here](https://derechodelared.com/zircolite-ejecucion-de-reglas-sigma-en-ficheros-evtx/).\n\n- **French**: [IT-connect.fr](https://www.it-connect.fr/) has published [an extensive tutorial](https://www.it-connect.fr/) on Zircolite in French.\n\n- **French**: [IT-connect.fr](https://www.it-connect.fr/) has also published a [Hack the Box challenge write-up](https://www.it-connect.fr/hack-the-box-sherlocks-tracer-solution/) using Zircolite.\n\n### References \n\n- [Florian Roth](https://github.com/Neo23x0/) cited Zircolite in his [**SIGMA Hall of Fame**](https://github.com/Neo23x0/Talks/blob/master/Sigma_Hall_of_Fame_20211022.pdf) during his talk at the October 2021 EU ATT\u0026CK Workshop.\n- Zircolite has been cited and presented during [JSAC 2023](https://jsac.jpcert.or.jp/archive/2023/pdf/JSAC2023_workshop_sigma_jp.pdf).\n- Zircolite has been cited and used in multiple research papers:\n  - **CIDRE Team**:\n    - [PWNJUTSU - Website](https://pwnjutsu.irisa.fr)\n    - [PWNJUTSU - Academic Paper](https://hal.inria.fr/hal-03694719/document)\n    - [CERBERE: Cybersecurity Exercise for Red and Blue Team Entertainment, Reproducibility](https://centralesupelec.hal.science/hal-04285565/file/CERBERE_final.pdf)\n  - **Universidad de la República**:\n    - [A Process Mining-Based Method for Attacker Profiling Using the MITRE ATT\u0026CK Taxonomy](https://journals-sol.sbc.org.br/index.php/jisa/article/view/3902/2840)\n\n---\n\n## License\n\n- All the **code** of the project is licensed under the [GNU Lesser General Public License](https://www.gnu.org/licenses/lgpl-3.0.en.html).\n- `evtx_dump` is under the MIT license.\n- The rules are released under the [Detection Rule License (DRL) 1.0](https://github.com/SigmaHQ/Detection-Rule-License/blob/main/LICENSE.Detection.Rules.md).\n\n---\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwagga40%2Fzircolite","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwagga40%2Fzircolite","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwagga40%2Fzircolite/lists"}