{"id":28739794,"url":"https://github.com/hyperdxio/sample-fluentbit","last_synced_at":"2026-02-01T19:32:41.536Z","repository":{"id":298730683,"uuid":"1000911955","full_name":"hyperdxio/sample-fluentbit","owner":"hyperdxio","description":"Sample of how to send logs to HyperDX via fluentbit","archived":false,"fork":false,"pushed_at":"2025-06-12T14:28:44.000Z","size":5,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-06-12T15:37:45.211Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hyperdxio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-12T14:07:31.000Z","updated_at":"2025-06-12T14:28:48.000Z","dependencies_parsed_at":"2025-06-12T15:40:43.335Z","dependency_job_id":null,"html_url":"https://github.com/hyperdxio/sample-fluentbit","commit_stats":null,"previous_names":["hyperdxio/sample-fluentbit"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/hyperdxio/sample-fluentbit","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyperdxio%2Fsample-fluentbit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyperdxio%2Fsample-fluentbit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyperdxio%2Fsample-fluentbit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyperdxio%2Fsample-fluentbit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hyperdxio","download_url":"https://codeload.github.com/hyperdxio/sample-fluentbit/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyperdxio%2Fsample-fluentbit/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28987278,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-01T18:17:03.387Z","status":"ssl_error","status_checked_at":"2026-02-01T18:16:57.287Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-16T06:10:52.985Z","updated_at":"2026-02-01T19:32:41.522Z","avatar_url":"https://github.com/hyperdxio.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Fluent Bit + HyperDX Integration Demo\n\nThis demo shows how to run Fluent Bit v4.x with HyperDX integration using Docker Compose. The setup includes log collection, processing with OpenTelemetry formatting, and forwarding to HyperDX for observability.\n\nFor other Fluent Bit versions:\n- **Fluent Bit 2.x**: See [HyperDX Fluent Bit 2.x documentation](https://www.hyperdx.io/docs/install/fluentbit#fluentbit-2x)\n- **Other versions**: This configuration is compatible with Fluent Bit 3.1.x-4.x series\n\n## What This Demo Does\n\n- Runs Fluent Bit in a Docker container\n- Collects logs from multiple sources:\n  - Dummy logs (for testing)\n  - HTTP endpoint (for real log ingestion)\n- Processes logs with OpenTelemetry envelope format\n- Forwards processed logs to HyperDX\n- Provides monitoring endpoints\n\n## Prerequisites\n\n- Docker and Docker Compose installed\n- A HyperDX account and API key\n- Basic familiarity with Docker\n\n## Quick Start\n\n### 1. Get Your HyperDX API Key\n\n1. Log into your HyperDX account\n2. Navigate to your team settings\n3. Copy your Ingestion API key\n\n### 2. Set Up Environment Variable\n\nYou have two options to provide your API key:\n\n**Option A: Export environment variable**\n```bash\nexport HYPERDX_API_KEY=your_actual_api_key_here\n```\n\n**Option B: Create a .env file (recommended for demos)**\n```bash\necho \"HYPERDX_API_KEY=your_actual_api_key_here\" \u003e .env\n```\n\n### 3. Run the Demo\n\n```bash\ndocker-compose -f docker-compose.fluentbit-hyperdx.yml up\n```\n\nTo run in the background:\n```bash\ndocker-compose -f docker-compose.fluentbit-hyperdx.yml up -d\n```\n\n## Testing the Setup\n\n### 1. Check if Fluent Bit is Running\n\nThe container should start and you'll see logs indicating Fluent Bit is running. Look for messages like:\n```\n[info] [fluent bit] version=X.X.X\n[info] [input:dummy:dummy.0] initializing\n[info] [input:http:http.1] listening on 0.0.0.0:9880\n```\n\n### 2. Send Test Logs via HTTP\n\nYou can send logs to the HTTP input endpoint:\n\n```bash\ncurl -X POST http://localhost:9880/test \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"timestamp\":\"2024-01-01T12:00:00Z\",\"level\":\"info\",\"message\":\"Test log from HTTP endpoint\"}'\n```\n\n### 3. Check HyperDX Dashboard\n\n1. Log into HyperDX\n2. Navigate to the search page\n3. You should see logs from the `my-service` service appearing\n\n## Available Ports\n\n- **24224**: Forward input port (for Docker log driver)\n- **9880**: HTTP input port (for sending logs via HTTP)\n\n## Configuration Details\n\n### Log Processing Pipeline\n\nThe setup includes:\n1. **OpenTelemetry Envelope**: Wraps logs in OpenTelemetry format\n2. **Content Modifier**: Adds service name metadata\n3. **Dual Output**: Logs go to both stdout (for debugging) and HyperDX\n\n### Service Attribution\n\nAll logs are automatically tagged with `service.name: my-service`. You can modify this in the `fluent-bit-4.yaml` configuration file.\n\n## Troubleshooting\n\n### Common Issues\n\n**API Key Not Set**\n```\nError: Authorization header missing or invalid\n```\n- Ensure `HYPERDX_API_KEY` environment variable is set\n- Verify the API key is correct and active\n\n**Connection Issues**\n```\nError: failed to connect to in-otel.hyperdx.io\n```\n- Check internet connectivity\n- Verify firewall allows outbound HTTPS on port 443\n\n**Container Won't Start**\n```\nError: port already in use\n```\n- Check if ports 24224 or 9880 are already in use\n- Stop other services using these ports or modify the docker-compose file\n\n### Viewing Logs\n\nTo see Fluent Bit logs:\n```bash\ndocker-compose -f docker-compose.fluentbit-hyperdx.yml logs -f\n```\n\nTo see only errors:\n```bash\ndocker-compose -f docker-compose.fluentbit-hyperdx.yml logs | grep -i error\n```\n\n## Stopping the Demo\n\n```bash\ndocker-compose -f docker-compose.fluentbit-hyperdx.yml down\n```\n\nTo also remove the built image:\n```bash\ndocker-compose -f docker-compose.fluentbit-hyperdx.yml down --rmi local\n```\n\n## Customization\n\n### Changing the Service Name\n\nEdit `fluent-bit-4.yaml` and modify the `value` under `content_modifier`:\n```yaml\n- name: content_modifier\n  context: otel_resource_attributes\n  action: upsert\n  key: service.name\n  value: your-custom-service-name  # Change this\n```\n\n### Adding More Log Sources\n\nYou can extend the configuration by adding more inputs to the `fluent-bit-4.yaml` file. See the [Fluent Bit documentation](https://docs.fluentbit.io/manual/pipeline/inputs) for available input plugins.\n\n## Support\n\nFor issues with:\n- **Fluent Bit configuration**: Check the [Fluent Bit documentation](https://docs.fluentbit.io/)\n- **HyperDX integration**: Contact HyperDX support\n- **This demo setup**: Review the troubleshooting section above or open a GitHub issue\n\n## Next Steps\n\nOnce you've verified the demo works:\n1. Integrate with your actual applications\n2. Customize the log processing pipeline for your needs\n3. Set up proper monitoring and alerting in HyperDX\n4. Consider scaling the setup for production use\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhyperdxio%2Fsample-fluentbit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhyperdxio%2Fsample-fluentbit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhyperdxio%2Fsample-fluentbit/lists"}