{"id":25456240,"url":"https://github.com/sultanfariz/logging-challenge","last_synced_at":"2026-04-13T22:34:04.629Z","repository":{"id":277848041,"uuid":"933715016","full_name":"sultanfariz/logging-challenge","owner":"sultanfariz","description":"just another boring simple project to practice logging instrumentation on Go app and Nginx, utilizing Fluentbit, Loki, and Grafana.","archived":false,"fork":false,"pushed_at":"2025-05-30T15:03:03.000Z","size":3779,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-30T21:18:40.379Z","etag":null,"topics":["fluent-bit","fluentbit","go","golang","grafana","log","logging","loki","nginx"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sultanfariz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-02-16T14:31:48.000Z","updated_at":"2025-05-30T15:03:07.000Z","dependencies_parsed_at":null,"dependency_job_id":"3ff723b0-72f5-4bcf-8ca2-87b6e6a32a0b","html_url":"https://github.com/sultanfariz/logging-challenge","commit_stats":null,"previous_names":["sultanfariz/logging-challenge"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sultanfariz/logging-challenge","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sultanfariz%2Flogging-challenge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sultanfariz%2Flogging-challenge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sultanfariz%2Flogging-challenge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sultanfariz%2Flogging-challenge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sultanfariz","download_url":"https://codeload.github.com/sultanfariz/logging-challenge/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sultanfariz%2Flogging-challenge/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31774052,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-13T20:17:16.280Z","status":"ssl_error","status_checked_at":"2026-04-13T20:17:08.216Z","response_time":93,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fluent-bit","fluentbit","go","golang","grafana","log","logging","loki","nginx"],"created_at":"2025-02-18T01:29:34.770Z","updated_at":"2026-04-13T22:34:04.614Z","avatar_url":"https://github.com/sultanfariz.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Logging Challenge\n\nThis project demonstrates a comprehensive logging pipeline using modern observability tools. It features a Go HTTP service with structured logging, which along with Nginx access logs, are collected and processed through Fluent Bit, stored in Loki, and visualized using Grafana dashboards.\n\n## Architecture Overview\n\n```\nNginx/App -\u003e Fluent Bit -\u003e Loki -\u003e Grafana\n```\n\n### Components\n\n1. **Source Services**\n   - Go HTTP Service (port 8080)\n     - Structured logging using zerolog\n     - Request ID tracking\n     - HTTP method, path, and status code logging\n   - Nginx (port 80)\n     - Access logs forwarded to Fluent Bit\n\n2. **Log Collection (Fluent Bit)**\n   - Collects logs from multiple sources:\n     - Tails Go service logs from `/app/logs/app.log`\n     - Receives Nginx logs through Forward protocol (port 24224)\n   - Parses and processes logs\n   - Forwards to Loki with appropriate labels\n\n3. **Log Storage (Loki)**\n   - Stores logs with labels for efficient querying\n   - Handles log aggregation and indexing\n   - Exposes API for Grafana integration\n\n4. **Visualization (Grafana)**\n   - Real-time log visualization\n   - Pre-configured dashboards\n   - Query and analyze logs using LogQL\n\n## Getting Started\n\n### Prerequisites\n\n- Docker\n- Docker Compose\n\n### Running the Project\n\n1. Clone the repository\n\n2. Start the services:\n   ```bash\n   docker-compose up -d\n   ```\n\n3. Access the services:\n   - Go HTTP Service: http://localhost:8080\n   - Grafana: http://localhost:3000\n   - Nginx: http://localhost:80\n\n## API Endpoints\n\n### Greeting Service\n```\nGET /?name=YourName\n```\nReturns a greeting message for the provided name.\n\n## Logging Pipeline Details\n\n### 1. Application Logging\n\nThe Go service uses structured logging with the following features:\n- Request ID tracking for request tracing\n- HTTP method and path logging\n- Response status code capture\n- Error logging with context\n\nExample log output:\n```json\n{\n  \"level\": \"info\",\n  \"request_id\": \"uuid\",\n  \"method\": \"GET\",\n  \"path\": \"/\",\n  \"status\": 200,\n  \"message\": \"request completed\"\n}\n```\n\n### 2. Fluent Bit Configuration\n\nFluent Bit is configured to:\n- Tail the application log file\n- Receive Nginx logs via forward protocol\n- Parse and process logs\n- Forward to Loki with appropriate labels\n\nKey configurations:\n```ini\n[INPUT]\n    Name  tail\n    Path  /app/logs/app.log\n    Tag   http-service\n\n[OUTPUT]\n    name        loki\n    match       http-service\n    host        loki\n    port        3100\n    labels      app=http-service\n```\n\n### 3. Loki Integration\n\nLoki receives logs from Fluent Bit and:\n- Labels logs based on source (http-service, nginx)\n- Enables efficient log querying\n- Provides data source for Grafana\n\n### 4. Grafana Visualization\n\nGrafana is pre-configured with:\n- Loki data source\n- Dashboards for viewing application and Nginx logs\n- LogQL queries for log analysis\n\n## Monitoring and Debugging\n\n1. View logs in Grafana:\n   - Navigate to http://localhost:3000\n   - Use the pre-configured dashboards\n   - Write custom LogQL queries\n\n2. Direct Loki queries:\n   - Loki API available at http://localhost:3100\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsultanfariz%2Flogging-challenge","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsultanfariz%2Flogging-challenge","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsultanfariz%2Flogging-challenge/lists"}