https://github.com/hq969/cyber-threat-intelligence-analyzer
Cybersecurity Threat Intelligence Analyzer — Log analysis tool with rule-based detections and ML anomaly detection, featuring a Streamlit dashboard and CLI support.
https://github.com/hq969/cyber-threat-intelligence-analyzer
cybersecurity docker machine-learning pytest security-monitoring streamlit threat-intelligence
Last synced: about 2 months ago
JSON representation
Cybersecurity Threat Intelligence Analyzer — Log analysis tool with rule-based detections and ML anomaly detection, featuring a Streamlit dashboard and CLI support.
- Host: GitHub
- URL: https://github.com/hq969/cyber-threat-intelligence-analyzer
- Owner: hq969
- License: mit
- Created: 2025-08-23T09:35:59.000Z (10 months ago)
- Default Branch: main
- Last Pushed: 2025-08-23T11:38:52.000Z (10 months ago)
- Last Synced: 2025-11-01T04:21:36.301Z (8 months ago)
- Topics: cybersecurity, docker, machine-learning, pytest, security-monitoring, streamlit, threat-intelligence
- Language: Python
- Homepage: http://localhost:8501
- Size: 72.3 KB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# 🔒 Cybersecurity Threat Intelligence Analyzer
A Python-based **threat intelligence tool** that ingests logs (CSV, JSON, Apache access logs), normalizes them, applies **rule-based detections** (brute-force, error spikes, suspicious UAs/paths, request floods), and runs **ML anomaly detection** (IsolationForest) to highlight unusual IP behavior.
Includes:
- 📊 **Streamlit dashboard** for interactive analysis
- ⚡ **CLI mode** for headless batch processing
- 🐳 **Dockerfile** for containerized deployment
- ✅ **Tests** with pytest
- 📁 **Sample logs** for quick testing
---
## 🚀 Features
- **Multiple log formats** supported (CSV, JSON, Apache combined log format)
- **Rule-based detections**:
- Brute-force login attempts (failed-login bursts)
- High error ratios (e.g. HTTP 5xx floods)
- Suspicious paths (`/wp-admin`, `/phpmyadmin`, `.env`, etc.)
- Suspicious user agents (e.g., `sqlmap`, `nmap`, curl, requests)
- Request floods (per-IP spikes)
- **ML anomaly detection**:
- IsolationForest on per-IP request features
- Flags outliers in traffic patterns
- **Exportable results** (CSV outputs)
- **Interactive dashboard** with charts
---
## 📦 Installation
```bash
git clone https://github.com/your-username/cyber-threat-intel-analyzer.git
cd cyber-threat-intel-analyzer
# Setup venv (optional)
python -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
# Install dependencies
pip install -r requirements.txt
````
---
## 🎛️ Usage
### 1. Streamlit UI (Dashboard)
```bash
streamlit run cyber_threat_intel_analyzer.py
```
Open [http://localhost:8501](http://localhost:8501) in your browser.
Upload logs and explore results visually.
---
### 2. CLI Mode (Headless Batch Processing)
```bash
python cli_ingest.py --in samples/sample_access.log --out results/
```
Outputs:
* `normalized_events.csv` – normalized logs
* `rule_hits.csv` – rule detections
* `ml_anomalies.csv` – ML-based anomalies
---
### 3. Docker Deployment
```bash
docker build -t threat-analyzer .
docker run --rm -p 8501:8501 -v "$PWD:/data" threat-analyzer
```
---
## 🧪 Running Tests
```bash
pytest tests/
```
---
## 📁 Repository Structure
```
.
├── cyber_threat_intel_analyzer.py # Streamlit app
├── cli_ingest.py # CLI tool
├── requirements.txt
├── Dockerfile
├── README.md
├── tests/
│ └── test_rules.py # Rule-based detection tests
└── samples/
└── sample_access.log # Example Apache log
```
---
## 📊 Example Detection Output
From the included `samples/sample_access.log`:
* 🚨 Brute-force login attempts (401 spam from `203.0.113.5`)
* 🚨 Suspicious paths (`/wp-admin`, `/phpmyadmin`, `.env`)
* 🚨 Suspicious UAs (`sqlmap`, `nmap`, `requests`)
* ⚠️ Request floods from aggressive scanners
---
## 🔮 Roadmap
* [ ] Add **Syslog & NGINX error log parsing**
* [ ] Geo-IP enrichment (map attacker IPs)
* [ ] Sigma rule integration
* [ ] Slack/Email alerting for rule hits
---
## 📜 License
MIT License © 2025
---
### 👨💻 Author
Harsh Sonkar
🚀 Data Science • Cybersecurity • Cloud Engineer
```