Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/NVISOsecurity/ee-outliers
Open-source framework to detect outliers in Elasticsearch events
https://github.com/NVISOsecurity/ee-outliers
anomaly-detection cirt ee-outliers machine-learning ml netsec outlier-detection outliers security-monitoring security-operations siem statistical-analysis statistics threat-hunting
Last synced: about 2 months ago
JSON representation
Open-source framework to detect outliers in Elasticsearch events
- Host: GitHub
- URL: https://github.com/NVISOsecurity/ee-outliers
- Owner: NVISOsecurity
- License: gpl-3.0
- Archived: true
- Created: 2018-12-11T15:49:39.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2023-05-22T21:36:43.000Z (over 1 year ago)
- Last Synced: 2024-05-18T22:22:00.711Z (4 months ago)
- Topics: anomaly-detection, cirt, ee-outliers, machine-learning, ml, netsec, outlier-detection, outliers, security-monitoring, security-operations, siem, statistical-analysis, statistics, threat-hunting
- Language: Python
- Homepage: https://blog.nviso.eu
- Size: 3.92 MB
- Stars: 204
- Watchers: 21
- Forks: 35
- Open Issues: 33
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
- awesome-elasticsearch - ee-outliers - ee-outliers is a framework to detect outliers in events stored in an Elasticsearch cluster. (Elasticsearch plugins / Cluster)
README
# ee-outliers
Framework to easily detect outliers in Elasticsearch events.
*Developed in Python and fully dockerized!*
## Documentation
### Introduction
- [What is ee-outliers?](#what-is-ee-outliers)
- [Why ee-outliers?](#why-ee-outliers)
- [How it works](#how-it-works)
### Using ee-outliers
- [Getting started](documentation/GETTING_STARTED.md)
- [Configuration parameters](documentation/CONFIG_PARAMETERS.md)
- [Example use cases](use_cases/examples)
- [Building detection use cases](documentation/CONFIG_OUTLIERS.md)
- [Whitelisting outliers](documentation/WHITELIST.md)
- [Notifications](documentation/NOTIFICATIONS.md)
- [Information for developers](documentation/DEVELOPMENT.md)
### Misc.
- [Contact](#contact)
- [Acknowledgements](#acknowledgements)
- [License](#license)
- [Screenshots](documentation/SCREENSHOTS.md)
## What is ee-outliers?
ee-outliers is a framework to detect statistical outliers in events stored
in an Elasticsearch cluster. It uses easy to write user-defined configuration files
to decide which & how events should be analysed for outliers.
The framework was developed for the purpose of detecting anomalies in
security events, however it could just as well be used for the detection
of outliers in other data.
The only thing you need is Docker and an Elasticsearch cluster and you are
ready to start your hunt for outlier events!
## Why ee-outliers?
Although we love Elasticsearch, its search language is still lacking support
for complex queries that allow for advanced analysis and detection of outliers -
features we came to love while using other tools such as Splunk.
This framework tries to solve these limitations by allowing the user to write
simple use cases that can help in spotting outliers in your data using statistical
and models. Machine learning models are under development.
## How it works
The framework makes use of statistical models that are easily defined by the user
in a configuration file. In case the models detect an outlier, the relevant
Elasticsearch events are enriched with additional outlier fields. These fields
can then be dashboarded and visualized using the tools of your choice
(Kibana or Grafana for example).
The possibilities of the type of anomalies you can spot using ee-outliers
is virtually limitless. A few examples of types of outliers we have detected
ourselves using ee-outliers during threat hunting activities include:
- Detect beaconing (DNS, TLS, HTTP, etc.)
- Detect geographical improbable activity
- Detect obfuscated & suspicious command execution
- Detect fileless malware execution
- Detect malicious authentication events
- Detect processes with suspicious outbound connectivity
- Detect malicious persistence mechanisms (scheduled tasks, auto-runs, etc.)
- …
Visit the page [Getting started](documentation/GETTING_STARTED.md) to get
started with outlier detection in Elasticsearch yourself!
## Contact
ee-outliers is developed & maintained by NVISO Labs.
You can reach out to the developers of ee-outliers by creating an issue in github.
For any other communication, you can reach out by sending us an e-mail
at [[email protected]](mailto:[email protected]).
We write about our research on our blog: https://blog.nviso.eu
You can follow us on twitter: https://twitter.com/NVISO_Labs
Thank you for using ee-outliers and we look forward to your feedback! 🐀
## License
ee-outliers is released under the GNU GENERAL PUBLIC LICENSE v3 (GPL-3).
[LICENSE](LICENSE)
## Acknowledgements
We are grateful for the support received by
[INNOVIRIS](https://innoviris.brussels/) and the Brussels region in
funding our Research & Development activities.
