{"id":44869330,"url":"https://github.com/tiraten-bot/anomalyzer","last_synced_at":"2026-04-28T12:06:36.587Z","repository":{"id":305948261,"uuid":"1023931709","full_name":"tiraten-bot/Anomalyzer","owner":"tiraten-bot","description":"A real-time anomaly detection and automated remediation system with AI/ML.","archived":false,"fork":false,"pushed_at":"2025-07-22T19:20:22.000Z","size":33,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-02-17T16:05:09.219Z","etag":null,"topics":["anomaly-detection","artificial-intelligence","bash-script","docker","kafka","machine-learning","ollama","prometheus","ubuntu","z-score"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tiraten-bot.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-21T23:45:52.000Z","updated_at":"2025-07-22T19:20:25.000Z","dependencies_parsed_at":"2025-07-22T21:26:17.987Z","dependency_job_id":null,"html_url":"https://github.com/tiraten-bot/Anomalyzer","commit_stats":null,"previous_names":["ozturkeniss/anomalyzer","tiraten-bot/anomalyzer"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/tiraten-bot/Anomalyzer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tiraten-bot%2FAnomalyzer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tiraten-bot%2FAnomalyzer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tiraten-bot%2FAnomalyzer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tiraten-bot%2FAnomalyzer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tiraten-bot","download_url":"https://codeload.github.com/tiraten-bot/Anomalyzer/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tiraten-bot%2FAnomalyzer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32379680,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-28T11:25:28.583Z","status":"ssl_error","status_checked_at":"2026-04-28T11:25:05.435Z","response_time":56,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anomaly-detection","artificial-intelligence","bash-script","docker","kafka","machine-learning","ollama","prometheus","ubuntu","z-score"],"created_at":"2026-02-17T12:21:47.123Z","updated_at":"2026-04-28T12:06:36.553Z","avatar_url":"https://github.com/tiraten-bot.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Anomalyzer\n\nA real-time anomaly detection and automated remediation system using AI/ML, Kafka, and microservices architecture.\n\n\n\n\nhttps://github.com/user-attachments/assets/b5682691-bbc7-4976-a2a0-14914fd2efb8\n\n\n\n\n\n\n## Architecture Overview\n\n```mermaid\ngraph TB\n    subgraph \"Data Sources\"\n        A[Ubuntu App] --\u003e B[Prometheus]\n        B --\u003e C[Server Backend]\n    end\n    \n    subgraph \"Processing Pipeline\"\n        C --\u003e D[Kafka]\n        D --\u003e E[AI Engine]\n        E --\u003e F[Ollama LLM]\n        F --\u003e E\n        E --\u003e G[Solution Topic]\n        G --\u003e C\n    end\n    \n    subgraph \"Actions\"\n        C --\u003e H[Scale Up]\n        C --\u003e I[Scale Down]\n        C --\u003e J[Restart Service]\n        C --\u003e K[Rate Limit]\n        C --\u003e L[Alarm]\n    end\n    \n    style A fill:#ff9999\n    style B fill:#ffcc99\n    style C fill:#87ceeb\n    style D fill:#ff99cc\n    style E fill:#90ee90\n    style F fill:#dda0dd\n    style G fill:#ff99cc\n    style H fill:#ff9999\n    style I fill:#ff9999\n    style J fill:#ff9999\n    style K fill:#ff9999\n    style L fill:#ff9999\n```\n\n## System Components\n\n```mermaid\ngraph LR\n    subgraph \"Monitoring\"\n        A[Ubuntu App\u003cbr/\u003ePort 8000] --\u003e B[Prometheus\u003cbr/\u003ePort 9090]\n    end\n    \n    subgraph \"Orchestration\"\n        C[Server Backend\u003cbr/\u003ePort 9000] --\u003e D[Kafka\u003cbr/\u003ePort 9092]\n        D --\u003e E[AI Engine\u003cbr/\u003ePort 5000]\n    end\n    \n    subgraph \"AI/ML\"\n        E --\u003e F[Ollama LLM\u003cbr/\u003ePort 11434]\n        F --\u003e G[Anomaly Detection]\n        G --\u003e H[Solution Generation]\n    end\n    \n    subgraph \"Actions\"\n        C --\u003e I[Kubernetes Actions]\n        C --\u003e J[Alerts]\n        C --\u003e K[Logs]\n    end\n    \n    style A fill:#ff9999\n    style B fill:#ffcc99\n    style C fill:#87ceeb\n    style D fill:#ff99cc\n    style E fill:#90ee90\n    style F fill:#dda0dd\n    style G fill:#90ee90\n    style H fill:#90ee90\n    style I fill:#87ceeb\n    style J fill:#87ceeb\n    style K fill:#87ceeb\n```\n\n## Data Flow\n\n```mermaid\nsequenceDiagram\n    participant U as Ubuntu App\n    participant P as Prometheus\n    participant S as Server Backend\n    participant K as Kafka\n    participant A as AI Engine\n    participant O as Ollama LLM\n    \n    U-\u003e\u003eP: Generate Metrics\n    P-\u003e\u003eS: Query Metrics\n    S-\u003e\u003eK: Publish Metrics\n    K-\u003e\u003eA: Consume Metrics\n    A-\u003e\u003eA: Detect Anomaly\n    A-\u003e\u003eO: Request Analysis\n    O-\u003e\u003eA: Return Solution\n    A-\u003e\u003eK: Publish Solution\n    K-\u003e\u003eS: Consume Solution\n    S-\u003e\u003eS: Execute Action\n```\n\n## Features\n\n- **Real-time Anomaly Detection**: Z-score based spike detection\n- **AI-Powered Analysis**: LLM-based explanation and solution generation\n- **Automated Remediation**: Kubernetes-native action execution\n- **Microservices Architecture**: Scalable and maintainable design\n- **Event-Driven Processing**: Kafka-based message queuing\n- **Monitoring Integration**: Prometheus metrics collection\n\n## Quick Start\n\n### Prerequisites\n\n- Docker and Docker Compose\n- 8GB+ RAM (for Ollama LLM)\n- Linux/macOS/Windows with Docker support\n\n### Installation\n\n1. **Clone the repository**\n   ```bash\n   git clone https://github.com/yourusername/anomalyzer.git\n   cd anomalyzer\n   ```\n\n2. **Start the system**\n   ```bash\n   docker-compose up -d\n   ```\n\n3. **Verify all services are running**\n   ```bash\n   docker-compose ps\n   ```\n\n4. **Test the system**\n   ```bash\n   # Test Prometheus metrics\n   curl http://localhost:9000/metrics-test\n   \n   # Send test anomaly\n   curl -X POST http://localhost:9000/send-test-metric\n   \n   # Check server logs\n   docker logs anomalyzer-server-1 --tail 20\n   ```\n\n## Service Details\n\n### Ubuntu App (Port 8000)\n- **Technology**: Go\n- **Purpose**: Generates test metrics for Prometheus\n- **Metrics**: `app_requests_total`, `http_request_duration_seconds`, `app_requests_errors_total`\n\n### Prometheus (Port 9090)\n- **Technology**: Prometheus\n- **Purpose**: Metrics collection and storage\n- **Configuration**: `prometheus.yml`\n\n### Server Backend (Port 9000)\n- **Technology**: Go\n- **Purpose**: Orchestration and action execution\n- **Endpoints**:\n  - `/health` - Health check\n  - `/metrics-test` - Query Prometheus metrics\n  - `/send-test-metric` - Send test anomaly\n  - `/action/*` - Execute remediation actions\n\n### Kafka (Port 9092)\n- **Technology**: Apache Kafka\n- **Purpose**: Message queuing between services\n- **Topics**:\n  - `metrics-topic` - Raw metrics from server\n  - `ai-solutions-topic` - AI-generated solutions\n\n### AI Engine (Port 5000)\n- **Technology**: Python\n- **Purpose**: Anomaly detection and AI analysis\n- **Components**:\n  - Anomaly Detector (Z-score based)\n  - LLM Integration (Ollama)\n  - Solution Generator\n\n### Ollama LLM (Port 11434)\n- **Technology**: Ollama\n- **Purpose**: Large Language Model for analysis\n- **Model**: `llama2:7b-chat`\n\n## Anomaly Detection\n\nThe system uses statistical methods to detect anomalies:\n\n```mermaid\ngraph TD\n    A[Raw Metrics] --\u003e B[Z-Score Calculation]\n    B --\u003e C{Threshold Check}\n    C --\u003e|\u003e 1.5| D[Anomaly Detected]\n    C --\u003e|≤ 1.5| E[Normal]\n    D --\u003e F[AI Analysis]\n    E --\u003e G[No Action]\n    F --\u003e H[Solution Generation]\n    H --\u003e I[Action Execution]\n    \n    style A fill:#ff9999\n    style B fill:#90ee90\n    style C fill:#ffcc99\n    style D fill:#ff6666\n    style E fill:#90ee90\n    style F fill:#87ceeb\n    style G fill:#90ee90\n    style H fill:#90ee90\n    style I fill:#87ceeb\n```\n\n## Solution Types\n\nThe AI engine can generate the following solution types:\n\n- **scale_up**: Increase service replicas\n- **scale_down**: Decrease service replicas\n- **restart_service**: Restart the service\n- **rate_limit**: Apply rate limiting\n- **alarm**: Send alert notifications\n- **check_logs**: Analyze service logs\n- **investigate**: Deep investigation\n- **no_action**: No action required\n\n## Development\n\n### Project Structure\n\n```\nanomalyzer/\n├── ai-engine/           # Python AI/ML service\n│   ├── anomaly/         # Anomaly detection algorithms\n│   ├── kafka/          # Kafka producer/consumer\n│   ├── llm/            # LLM integration\n│   └── main.py         # Main application\n├── server/             # Go orchestration service\n│   ├── cmd/            # Application entry point\n│   ├── internal/       # Internal packages\n│   └── go.mod          # Go dependencies\n├── ubuntu/             # Go test application\n├── docker-compose.yml  # Service orchestration\n└── README.md          # This file\n```\n\n### Building Services\n\n```bash\n# Build all services\ndocker-compose build\n\n# Build specific service\ndocker-compose build ai-engine\ndocker-compose build server\n```\n\n### Testing\n\n```bash\n# Test anomaly detection\ncurl -X POST http://localhost:9000/send-test-metric\n\n# Test manual action\ncurl -X POST http://localhost:9000/action/scale-up\n\n# Check AI engine logs\ndocker logs anomalyzer-ai-engine-1\n\n# Check server logs\ndocker logs anomalyzer-server-1\n```\n\n## Monitoring\n\n### Prometheus Metrics\n\n- `app_requests_total`: Total request count\n- `http_request_duration_seconds`: Request duration\n- `app_requests_errors_total`: Error count\n\n### Logs\n\n```bash\n# View all logs\ndocker-compose logs\n\n# View specific service logs\ndocker-compose logs ai-engine\ndocker-compose logs server\n```\n\n## Production Deployment\n\n### Kubernetes\n\nThe system is designed to be deployed on Kubernetes:\n\n```mermaid\ngraph TB\n    subgraph \"Kubernetes Cluster\"\n        A[Prometheus Operator] --\u003e B[Prometheus]\n        C[Kafka Operator] --\u003e D[Kafka]\n        E[AI Engine Deployment] --\u003e F[AI Engine Pods]\n        G[Server Backend Deployment] --\u003e H[Server Pods]\n        I[Ollama Deployment] --\u003e J[Ollama Pods]\n    end\n    \n    subgraph \"Actions\"\n        H --\u003e K[kubectl scale]\n        H --\u003e L[kubectl rollout]\n        H --\u003e M[Webhook Alerts]\n    end\n    \n    style A fill:#87ceeb\n    style B fill:#ffcc99\n    style C fill:#87ceeb\n    style D fill:#ff99cc\n    style E fill:#87ceeb\n    style F fill:#90ee90\n    style G fill:#87ceeb\n    style H fill:#87ceeb\n    style I fill:#87ceeb\n    style J fill:#dda0dd\n    style K fill:#ff9999\n    style L fill:#ff9999\n    style M fill:#ff9999\n```\n\n### Environment Variables\n\n```bash\n# Kafka Configuration\nKAFKA_BROKERS=kafka:9092\nKAFKA_TOPIC_METRICS=metrics-topic\nKAFKA_TOPIC_SOLUTIONS=ai-solutions-topic\n\n# Ollama Configuration\nOLLAMA_HOST=http://ollama:11434\nOLLAMA_MODEL=llama2:7b-chat\n\n# Prometheus Configuration\nPROMETHEUS_URL=http://prometheus:9090\n```\n\n## Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Make your changes\n4. Add tests\n5. Submit a pull request\n\n## License\n\nThis project is licensed under the MIT License - see the LICENSE file for details.\n\n## Support\n\nFor issues and questions:\n- Create an issue on GitHub\n- Check the documentation\n- Review the logs for debugging\n\n---\n\n**Anomalyzer** - Intelligent anomaly detection and automated remediation for modern applications. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftiraten-bot%2Fanomalyzer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftiraten-bot%2Fanomalyzer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftiraten-bot%2Fanomalyzer/lists"}