{"id":15044313,"url":"https://github.com/elastic/securitylabs-thrunting-tools","last_synced_at":"2025-10-19T20:31:27.974Z","repository":{"id":65978336,"uuid":"566872183","full_name":"elastic/securitylabs-thrunting-tools","owner":"elastic","description":"A collection of  utilities to help with analysis on the command line.","archived":false,"fork":false,"pushed_at":"2024-08-09T04:25:31.000Z","size":78,"stargazers_count":16,"open_issues_count":0,"forks_count":3,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-01-28T20:58:37.825Z","etag":null,"topics":["analysis","command-line","cti","cybersecurity","elasticsearch","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/elastic.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-16T15:40:46.000Z","updated_at":"2025-01-13T20:25:55.000Z","dependencies_parsed_at":"2024-09-25T02:03:26.881Z","dependency_job_id":null,"html_url":"https://github.com/elastic/securitylabs-thrunting-tools","commit_stats":{"total_commits":29,"total_committers":1,"mean_commits":29.0,"dds":0.0,"last_synced_commit":"1d4bcd34d198a96f4dee7621744b0c9ea81a9d6a"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastic%2Fsecuritylabs-thrunting-tools","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastic%2Fsecuritylabs-thrunting-tools/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastic%2Fsecuritylabs-thrunting-tools/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastic%2Fsecuritylabs-thrunting-tools/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/elastic","download_url":"https://codeload.github.com/elastic/securitylabs-thrunting-tools/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":237215417,"owners_count":19273548,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["analysis","command-line","cti","cybersecurity","elasticsearch","python"],"created_at":"2024-09-24T20:50:26.043Z","updated_at":"2025-10-19T20:31:27.635Z","avatar_url":"https://github.com/elastic.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Elastic Security Labs thrunting-tools\n\nHave you ever been threat hunting (hereafter known as \"thrunting\") in Kibana and thought\n\"Gee! I wish I could take these results and do some automation on the command line!\".\nWell look no further, fellow thrunter! This repo has just what you need to make your\nautomation adventures a bit easier.\n\nthrunting-tools is a collection of command line utilities for working with data.\n\nThe current list of tools are:\n\n- `eql-query`, a tool to let you perform EQL searches from your shell!\n- `lucene-query`, a tool to let you perform Lucene searches against Elasticsearch in your\n  comfort zone, the command line.\n- `from-charcode`, a tool to convert a character code in a given base to the ASCII character\n- `to-charcode`, a tool to convert an ASCII character to a given base\n- `url-decode`, a tool to decode urlencoded strings\n- `url-encode`, a tool to encode common character or all special characters to urlencoded strings\n- `zlib-compress`, a tool to perform zlib compression/deflation on the command line\n- `zlib-decompress`, a tool to perform zlib decompression/inflation on the command line\n- `zlib-deflate`, an alias for zlib-compress\n- `zlib-decompress`, an alias for zlib-decompress\n- `unmap-pe`, processes a PE binary, removing the memory mapping. Useful for analyzing process memory dumps\n\n## Installation\n\nThe easiest way to install thrunting-tools is with [pipx](https://pypa.github.io/pipx/). Once\nyou have pipx installed, to install these tools on your path, simply install the latest release\nwith:\n\n```shell\npipx install thrunting-tools\n```\n\nAlternatively, if you'd like to install with pip and you have your own Python environment, you can\ndo that too.\n\n```shell\npip3 install thrunting-tools\n```\n\nYou can now check that each command was installed.\n\n```shell\neql-query --version\nlucene-query --version\n```\n\n### Docker Usage\n\nLastly, if you want to use a container runtime environment, you can use the latest release from\nthe repository GitHub Container Repository. Currently, we're publishing AMD64 and ARM64 images.\n\n```shell\ndocker pull ghcr.io/elastic/securitylabs-thrunting-tools:main\n```\n\nThen, you can run the container and pass your local configuration in to the default\nlocation used by the container, `/config.yml`. (NOTE: the `:z` part of the volume\nspecification is only needed if you use SELinux)\n\n```shell\ndocker run -ti -v \"${HOME}/.config/thrunting-tools/config.yml\":/config.yml:ro,z \\\n    --rm ghcr.io/elastic/securitylabs-thrunting-tools:latest eql-query --help\n```\n\n## Usage\n\nEach of the commands provide a usage when called with `--help`.\n\n```shell\n$ eql-query --help\n\n Usage: eql-query [OPTIONS] QUERY\n\n╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────╮\n│ *    query      TEXT  Query specified using EQL (See https://ela.st/eql) [required]         │\n╰─────────────────────────────────────────────────────────────────────────────────────────────╯\n╭─ Options ───────────────────────────────────────────────────────────────────────────────────╮\n│ --index        -i      TEXT     Index pattern to search. Defaults to                        │\n│                                 '.alerts-security.alerts-default,apm-*-transaction*,logs-*' │\n│ --since        -s      TEXT     Earliest time filter using datemath or datetime             │\n│                                 [default: now-30d/d]                                        │\n│ --before       -b      TEXT     Latest time filter using datemath or datetime               │\n│                                 [default: now]                                              │\n│ --compact      -c               Output one event/sequence per line                          │\n│ --fields       -f      TEXT     Comma separated list of fields to display [default: None]   │\n│ --size         -s      INTEGER  Specify maximum size of result set [default: 100]           │\n│ --config               PATH     Optional path to YAML configuration with settings for       │\n│                                 Elasticsearch                                               │\n│                                 [default:                                                   │\n│                                 /home/user/.config/thrunting-tools/config.yml]           │\n│ --environment  -e      TEXT     Environment name to use from config file (if present)       │\n│                                 [default: default]                                          │\n│ --help                          Show this message and exit.                                 │\n╰─────────────────────────────────────────────────────────────────────────────────────────────╯\n```\n\n## Configuration\n\nThere are two ways to pass configuration to the tools: environment variables and configuration files.\n\nThe tools default to looking for the YAML configuration file in the platform-specific\nconfiguration directory (see the `--help` output). If present, configuration groups are\ntop-level keys (e.g. `elasticsearch`), which contain a list of environments. All scripts will\ncheck for the first environment with the name attribute set to `default`  if none is specified\non the command line.\n\nExample configuration file:\n\n```yaml\nelasticsearch:\n  - name: default\n    cloud_id: \"security-cluster:dXMtd2VzdDEuZ2NwLmNsb3VkLmVzLmlvJGFiY2R=\"\n    cloud_auth: \"elastic:changeme\"\n```\n\n## Examples\n\nRun query using `devel` environment configuration\n\n```shell\neql-query -e devel 'malware where event.kind: \"alert\"'\n```\n\nUsing `jq` and `wc` to get the number of alert events where `EXCEL.EXE` was the parent process.\n\n```shell\neql-query 'any where event.kind: \"alert\"' -c | \\\n    jq 'select(._source.process.parent.name == \"EXCEL.EXE\")' -c | wc -l\n```\n\nFind the unique event categories of all events in the last day that triggered based upon a\nrule using the 'sandbox' environment\n\n```shell\n$ lucene-query --since 'now-1d' 'rule: *' -e sandbox -c | \\\n    jq '._source.event.category[]' -c -r | sort -u\nnetwork\n```\n\nFind the unique dynamic DNS subdomains of a particular domain resolved in our network in the\nlast month\n\n```shell\nlucene-query --since 'now-1M' 'dns.question.name: *.duckdns.org' -c \\\n    | jq '._source.dns.question.name' -r | sort -u\n...\n```\n\nFind a list of all the unique agent IDs that resolved a known malware domain within the last 12 months.\n\n```shell\n$ lucene-query --since 'now-12M' 'dns.question.name: puerto2547.duckdns.org' -c \\\n    | jq '._source.agent.id' -r | sort -u\nec82f608-3d1b-4651-900e-b970c68bbeef\n```\n\nExtract a single binary using Elastic Defend integration with\n[optional sample collection](https://www.elastic.co/security-labs/collecting-cobalt-strike-beacons-with-the-elastic-stack) enabled.\nNote that additional shell scripting would be needed to loop over a set of results.\n\n```shell\neql-query 'process where ?process.Ext.memory_region.bytes_compressed_present == true' \\\n    --size 1 \\\n    --fields 'process.Ext.memory_region.bytes_compressed' | \\\n    jq -r '.process.Ext.memory_region.bytes_compressed' | \\\n    base64 -d | zlib-decompress \u003e captured_sample.bin\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felastic%2Fsecuritylabs-thrunting-tools","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Felastic%2Fsecuritylabs-thrunting-tools","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felastic%2Fsecuritylabs-thrunting-tools/lists"}