{"id":30188131,"url":"https://github.com/nevil-ing/apache_log_agent","last_synced_at":"2026-06-22T19:31:32.274Z","repository":{"id":306137952,"uuid":"1025138657","full_name":"nevil-ing/apache_log_agent","owner":"nevil-ing","description":"A lightweight, cross-platform Python-based agent for collecting, parsing, and forwarding Apache web server logs via an API endpoint to security analytics platforms.","archived":false,"fork":false,"pushed_at":"2025-09-29T10:52:34.000Z","size":16,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-29T12:32:15.856Z","etag":null,"topics":["apache","apache-logs","bash","filebeat","log-aggregation","logs","logstash","logstash-outputs","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nevil-ing.png","metadata":{"files":{"readme":"README.MD","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-07-23T19:33:12.000Z","updated_at":"2025-09-29T10:52:38.000Z","dependencies_parsed_at":"2025-07-23T22:08:17.559Z","dependency_job_id":"5ee844ea-b23a-415d-b00f-a1bd6a3c5b07","html_url":"https://github.com/nevil-ing/apache_log_agent","commit_stats":null,"previous_names":["nevil-ing/apache_log_agent"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nevil-ing/apache_log_agent","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nevil-ing%2Fapache_log_agent","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nevil-ing%2Fapache_log_agent/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nevil-ing%2Fapache_log_agent/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nevil-ing%2Fapache_log_agent/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nevil-ing","download_url":"https://codeload.github.com/nevil-ing/apache_log_agent/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nevil-ing%2Fapache_log_agent/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34663524,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-22T02:00:06.391Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache","apache-logs","bash","filebeat","log-aggregation","logs","logstash","logstash-outputs","python"],"created_at":"2025-08-12T16:43:50.801Z","updated_at":"2026-06-22T19:31:32.238Z","avatar_url":"https://github.com/nevil-ing.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Apache Log Agent\n\nA lightweight, cross-platform **Python-based agent** for collecting, parsing, and forwarding **Apache web server logs** via an API endpoint to security analytics platforms.\n\nThis agent wraps and orchestrates the powerful **ELK Stack components (Filebeat and Logstash)** to provide enterprise-grade log processing capabilities with minimal overhead.\n\n---\n\n## Purpose\n\nThis agent serves as a complete **log collection and forwarding solution** specifically designed for **Apache HTTP Server** environments.\n\nRather than reinventing log parsing logic, it **leverages battle-tested tools** while providing a simplified deployment and management interface through Python.\n\n---\n\n##  Core Components\n\n- **Filebeat**  \n  Lightweight log shipper that tails Apache access and error logs in real-time\n\n- **Logstash**  \n  Powerful log parser and transformer that applies Grok patterns, GeoIP lookups, and custom field mappings\n\n- **Python Wrapper**  \n  Central management interface for installation, configuration, and service orchestration\n\n---\n\n##  Key Features\n\n### Smart Log Processing\n\n- Parses Apache Combined Log Format with precision\n- Extracts client IP, user agent, response codes, and request details\n- Performs automatic GeoIP enrichment for threat analysis\n- Normalizes timestamps to UTC with millisecond precision\n\n---\n\n### Security-Oriented Payload Construction\n\n- Builds structured **JSON payloads optimized for SIEM/XDR ingestion**\n- Includes security metadata fields like `is_anomaly`, `threat_type`, and `anomaly_score`\n- Creates nested data object containing raw log fields for detailed analysis\n- Standardizes source identification and timestamp formatting\n\n---\n\n### Easy Deployment \u0026 Management\n\n- **Single-command** installation and configuration\n- Automatic service management (`start` / `stop` / `status`)\n- Cross-distribution compatibility (**systemd-based Linux systems**)\n- Self-contained package with all required configurations\n\n---\n\n### Flexible Output Options\n\n- Forwards parsed logs to **remote APIs/SIEM solutions via HTTP**\n- Console output for debugging and development\n- Ready for integration with **Sentinel XDR**, **Elastic SIEM**, or custom backends\n\n---\n\n## Configuration Highlights\n\n### Filebeat Input Configuration\n\n- Monitors `/var/log/httpd/*.log` for Apache logs\n- Automatically tags events with Apache module metadata\n- Supports log rotation and multi-line log handling\n\n---\n\n### Logstash Processing Pipeline\n\n- Grok parsing using `%{COMBINEDAPACHELOG}` pattern\n- Date parsing and `@timestamp` normalization\n- GeoIP enrichment for client IP addresses\n- Ruby-based payload restructuring for security use cases\n- HTTP output to configured backend APIs\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnevil-ing%2Fapache_log_agent","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnevil-ing%2Fapache_log_agent","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnevil-ing%2Fapache_log_agent/lists"}