{"id":50647137,"url":"https://github.com/emredeveloper/mini_siem","last_synced_at":"2026-06-07T13:30:56.951Z","repository":{"id":356964620,"uuid":"1234782524","full_name":"emredeveloper/mini_siem","owner":"emredeveloper","description":"Mini SIEM platform for log ingestion, threat detection, alerting, and security event monitoring","archived":false,"fork":false,"pushed_at":"2026-05-10T16:32:21.000Z","size":15,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-10T18:27:35.670Z","etag":null,"topics":["cybersecurity","docker","fastapi","security-monitoring","siem"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/emredeveloper.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-10T16:28:45.000Z","updated_at":"2026-05-10T16:38:09.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/emredeveloper/mini_siem","commit_stats":null,"previous_names":["emredeveloper/mini_siem"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/emredeveloper/mini_siem","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emredeveloper%2Fmini_siem","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emredeveloper%2Fmini_siem/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emredeveloper%2Fmini_siem/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emredeveloper%2Fmini_siem/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/emredeveloper","download_url":"https://codeload.github.com/emredeveloper/mini_siem/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emredeveloper%2Fmini_siem/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34023902,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-07T02:00:07.652Z","response_time":124,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cybersecurity","docker","fastapi","security-monitoring","siem"],"created_at":"2026-06-07T13:30:52.777Z","updated_at":"2026-06-07T13:30:56.940Z","avatar_url":"https://github.com/emredeveloper.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Mini SIEM\n\nThis demo project establishes the following data flow:\n\n`FastAPI backend -\u003e JSON log file -\u003e Filebeat -\u003e Elasticsearch -\u003e Kibana`\n\n**Purpose:**\n\n- Centrally collect backend logs\n- View errors and traffic anomalies through Kibana\n- Monitor request durations and problematic endpoints\n\n## Folder Structure\n\n- `backend/`: Sample API and JSON log generation\n- `filebeat/`: Agent configuration that ships logs to Elasticsearch\n- `kibana/`: Dashboard creation steps\n- `scripts/`: Demo traffic generation helpers\n\n## Requirements\n\n- Docker Desktop or Docker Engine + Docker Compose\n\n## 1. Start the Stack\n\nIn this directory:\n\n```powershell\ndocker compose up --build\n```\n\nServices:\n\n- Backend: `http://localhost:8000`\n- Elasticsearch: `http://localhost:9200`\n- Kibana: `http://localhost:5601`\n\n## 2. Generate Sample Traffic\n\nIn a new terminal:\n\n```powershell\npowershell -ExecutionPolicy Bypass -File .\\scripts\\generate-traffic.ps1\n```\n\nOr test manually if you prefer:\n\n```powershell\nInvoke-RestMethod http://localhost:8000/health\nInvoke-RestMethod http://localhost:8000/simulate/slow\nInvoke-RestMethod http://localhost:8000/simulate/error\n```\n\n## 3. Verify Logs in Elasticsearch\n\n```powershell\nInvoke-RestMethod \"http://localhost:9200/mini-siem-logs-*/_search?size=5\u0026sort=%40timestamp:desc\"\n```\n\nExpected fields:\n\n- `@timestamp`\n- `message`\n- `log.level`\n- `event.category`\n- `event.type`\n- `http.request.method`\n- `http.response.status_code`\n- `url.path`\n- `client.ip`\n- `labels.duration_ms`\n\n## 4. Kibana Dashboard\n\nFor setup steps, see:\n\n- `kibana/dashboard-guide.md`\n\nWith this guide, you'll create the following visualizations:\n\n- Total error count\n- Traffic per minute chart\n- 5xx status endpoints\n- Slow requests table\n\n## Why This Architecture?\n\nIn this initial version, the backend doesn't write logs directly to Elasticsearch. Instead, logs are first written as JSON to disk, then Filebeat ships them to Elasticsearch. This approach is more resilient in real-world projects:\n\n- The application doesn't depend directly on Elasticsearch connectivity\n- Log forwarding is decoupled from application code\n- Log formatting can be enhanced at the Filebeat level\n\n## Next Steps\n\nIn future iterations, we could add:\n\n1. Automated Kibana dashboard import via saved objects\n2. Slack / email alerting mechanism\n3. Log enrichment with Logstash instead of Filebeat\n4. API authentication and authorization\n5. Request rate limiting and quota management\n6. Per-user activity tracking and monitoring fields\n4. Auth, rate limit ve kullanici bazli izleme alanlari\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femredeveloper%2Fmini_siem","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Femredeveloper%2Fmini_siem","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femredeveloper%2Fmini_siem/lists"}