{"id":30850922,"url":"https://github.com/rhobs/rhobs-synthetics-agent","last_synced_at":"2026-01-20T17:11:07.575Z","repository":{"id":307001728,"uuid":"1004524328","full_name":"rhobs/rhobs-synthetics-agent","owner":"rhobs","description":"RHOBS Synthetic Monitoring Agent","archived":false,"fork":false,"pushed_at":"2026-01-14T22:37:17.000Z","size":326,"stargazers_count":2,"open_issues_count":1,"forks_count":7,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-01-15T04:39:07.161Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rhobs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-06-18T19:01:03.000Z","updated_at":"2026-01-14T22:37:21.000Z","dependencies_parsed_at":"2026-01-08T14:08:08.171Z","dependency_job_id":null,"html_url":"https://github.com/rhobs/rhobs-synthetics-agent","commit_stats":null,"previous_names":["rhobs/rhobs-synthetics-agent"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/rhobs/rhobs-synthetics-agent","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rhobs%2Frhobs-synthetics-agent","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rhobs%2Frhobs-synthetics-agent/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rhobs%2Frhobs-synthetics-agent/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rhobs%2Frhobs-synthetics-agent/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rhobs","download_url":"https://codeload.github.com/rhobs/rhobs-synthetics-agent/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rhobs%2Frhobs-synthetics-agent/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28607624,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-20T16:10:39.856Z","status":"ssl_error","status_checked_at":"2026-01-20T16:10:39.493Z","response_time":117,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-09-07T06:40:20.708Z","updated_at":"2026-01-20T17:11:07.568Z","avatar_url":"https://github.com/rhobs.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# RHOBS Synthetics Agent\n\nA synthetic monitoring agent for the Red Hat Observability Service (RHOBS) ecosystem that retrieves synthetic probe configurations from the RHOBS Probes API and reconciles them locally by creating `probe.monitoring.rhobs` Custom Resources for automated instantiation of synthetic monitoring probes.\n\n## Overview\n\nThe RHOBS Synthetics Agent provides:\n- **Multi-API Integration**: Polls multiple RHOBS Probes APIs simultaneously to retrieve probe configurations\n- **URL Validation**: Validates target URLs before creating monitoring resources\n- **Custom Resource Management**: Creates Probe CRs in Kubernetes (auto-detects `monitoring.rhobs/v1` or `monitoring.coreos.com/v1`)\n- **Status Tracking**: Updates probe status (active/failed) via API calls across all configured APIs\n- **Label-based Filtering**: Uses configurable label selectors to target specific probes\n- **Probe Deduplication**: Automatically removes duplicate probes when fetching from multiple APIs\n\n## Quick Start\n\n### Building the Application\n\n```bash\n# Build the binary\nmake build\n\n# Build and run immediately\nmake run\n\n# Build with specific configuration\n./rhobs-synthetics-agent start --config example-config.yaml\n```\n\n### Running Tests\n\n```bash\n# Run all tests with coverage\nmake test\n\n# Generate detailed coverage report (creates coverage.html)\nmake coverage\n\n# Run tests with race detection\nmake test-race\n```\n\n### Code Quality\n\n```bash\n# Run linter (auto-installs golangci-lint if needed)\nmake lint\n\n# Auto-fix linting issues\nmake lint-fix\n\n# Run linter for CI environments\nmake lint-ci\n```\n\n## Core Functionality\n\n### Probe Reconciliation Workflow\n\n1. **Poll APIs**: Retrieves probe configurations from multiple `/api/metrics/v1/{tenant}/probes` endpoints\n2. **Deduplicate**: Removes duplicate probes by URL when multiple APIs return probes monitoring the same endpoint\n3. **Filter Probes**: Uses label selectors (e.g., `private=false,`)\n4. **Validate URLs**: Checks if target URLs are ready for monitoring\n5. **Create Resources**: Generates Probe Custom Resources (auto-detects API group)\n6. **Update Status**: Reports success/failure back to all relevant APIs via PATCH calls\n\n### Label Selector Support\n\nThe agent uses configurable label selectors to filter probes:\n```bash\n# Example: Only process non-private, pending probes\nLABEL_SELECTOR=\"private=false,\"\n```\n\n### URL Validation\n\nBefore creating monitoring resources, the agent validates target URLs to prevent false positive alerts:\n- Checks URL format and scheme (HTTP/HTTPS only)\n- Performs HEAD requests to verify accessibility\n- Handles server errors appropriately (5xx = validation failure, 4xx = acceptable)\n\n## Configuration\n\n### Full Configuration Example\n\n```yaml\n# config.yaml\nlog_level: debug\nlog_format: json\npolling_interval: 30s\ngraceful_timeout: 60s\n\n# API Configuration\napi_base_urls:\n  - \"https://observatorium-api-1.example.com\"\n  - \"https://observatorium-api-2.example.com\"\n  - \"https://observatorium-api-3.example.com\"\n\napi_tenant: \"my-rhobs-tenant\"\nlabel_selector: \"private=false,\"\n\n# Kubernetes Configuration\nnamespace: \"monitoring\"\n\n# Prometheus Configuration\n# remote_write_url defaults to http://thanos-receive-router-rhobs.\u003cnamespace\u003e.svc.cluster.local:19291/api/v1/receive\nprometheus:\n  remote_write_url: \"\"  # Uses namespace-aware default, or set explicitly for cross-cluster\n  remote_write_tenant: \"my-tenant\"\n  cpu_requests: \"200m\"\n  cpu_limits: \"1000m\"\n  memory_requests: \"512Mi\"\n  memory_limits: \"1Gi\"\n  managed_by_operator: \"observability-operator\"\n```\n\n### Environment Variables\n\n```bash\n# Core settings\nexport LOG_LEVEL=debug\nexport LOG_FORMAT=json\nexport POLLING_INTERVAL=30s\nexport GRACEFUL_TIMEOUT=60s\n\n# API configuration - comma-separated list of complete URLs\nexport API_URLS=\"https://api1.example.com/api/metrics/v1/my-rhobs-tenant/probes,https://api2.example.com/api/metrics/v1/my-rhobs-tenant/probes,https://api3.example.com/api/metrics/v1/my-rhobs-tenant/probes\"\nexport LABEL_SELECTOR=\"private=false,\"\n\n# Kubernetes settings\nexport NAMESPACE=\"monitoring\"\n\n# Prometheus configuration\n# PROMETHEUS_REMOTE_WRITE_URL defaults to http://thanos-receive-router-rhobs.\u003cnamespace\u003e.svc.cluster.local:19291/api/v1/receive\n# Only set explicitly for cross-cluster deployments:\n# export PROMETHEUS_REMOTE_WRITE_URL=\"https://rhobs.example.com/api/metrics/v1/tenant/api/v1/receive\"\nexport PROMETHEUS_REMOTE_WRITE_TENANT=\"my-tenant\"\nexport PROMETHEUS_CPU_REQUESTS=\"200m\"\nexport PROMETHEUS_CPU_LIMITS=\"1000m\"\nexport PROMETHEUS_MEMORY_REQUESTS=\"512Mi\"\nexport PROMETHEUS_MEMORY_LIMITS=\"1Gi\"\nexport PROMETHEUS_MANAGED_BY_OPERATOR=\"observability-operator\"\n\n./rhobs-synthetics-agent start\n```\n\n### Command Line Flags\n\n```bash\n# Basic usage (uses namespace-aware default for remote write URL)\n./rhobs-synthetics-agent start \\\n  --config /path/to/config.yaml \\\n  --log-level debug \\\n  --interval 30s \\\n  --graceful-timeout 60s \\\n  --namespace \"rhobs-stage\" \\\n  --api-urls \"https://api1.example.com/api/metrics/v1/my-tenant/probes\" \\\n  --prometheus-remote-write-tenant \"my-tenant\"\n\n# Cross-cluster deployment (explicit remote write URL)\n./rhobs-synthetics-agent start \\\n  --config /path/to/config.yaml \\\n  --api-urls \"https://rhobs.example.com/api/metrics/v1/my-tenant/probes\" \\\n  --prometheus-remote-write-url \"https://rhobs.example.com/api/metrics/v1/my-tenant/api/v1/receive\" \\\n  --prometheus-remote-write-tenant \"my-tenant\"\n```\n\n## Prometheus Configuration\n\nThe agent can automatically create and manage a Prometheus instance for synthetic monitoring when running in a Kubernetes cluster. This Prometheus instance will scrape probe resources and send metrics to a remote write endpoint (typically Thanos).\n\n### Configuration Options\n\n| Flag | Environment Variable | Config File Key | Default | Description |\n|------|---------------------|----------------|---------|-------------|\n| `--prometheus-remote-write-url` | `PROMETHEUS_REMOTE_WRITE_URL` | `prometheus.remote_write_url` | `\"\"` (uses `http://thanos-receive-router-rhobs.\u003cnamespace\u003e.svc.cluster.local:19291/api/v1/receive`) | Thanos remote write endpoint URL |\n| `--prometheus-remote-write-tenant` | `PROMETHEUS_REMOTE_WRITE_TENANT` | `prometheus.remote_write_tenant` | `hcp` | Thanos tenant identifier |\n| `--prometheus-cpu-requests` | `PROMETHEUS_CPU_REQUESTS` | `prometheus.cpu_requests` | `100m` | CPU requests for Prometheus pod |\n| `--prometheus-cpu-limits` | `PROMETHEUS_CPU_LIMITS` | `prometheus.cpu_limits` | `500m` | CPU limits for Prometheus pod |\n| `--prometheus-memory-requests` | `PROMETHEUS_MEMORY_REQUESTS` | `prometheus.memory_requests` | `256Mi` | Memory requests for Prometheus pod |\n| `--prometheus-memory-limits` | `PROMETHEUS_MEMORY_LIMITS` | `prometheus.memory_limits` | `512Mi` | Memory limits for Prometheus pod |\n| `--prometheus-managed-by-operator` | `PROMETHEUS_MANAGED_BY_OPERATOR` | `prometheus.managed_by_operator` | `observability-operator` | Value for app.kubernetes.io/managed-by label on Prometheus resources |\n\n## Architecture\n\n### Components\n\n- **API Client** (`internal/api`): Handles communication with RHOBS Probes API\n- **Probe Manager** (`internal/k8s`): Manages Custom Resource creation and URL validation\n- **Worker** (`internal/agent`): Orchestrates the reconciliation process\n- **Configuration** (`internal/agent`): Handles all configuration sources\n\n### Data Flow\n\n```\nMultiple APIs → Agent → Deduplication → URL Validation → Custom Resource Creation → Status Update\n```\n\n### Custom Resource Format\n\nGenerated Probe resources include (example shows `monitoring.rhobs/v1`, but `monitoring.coreos.com/v1` is also supported):\n\n```yaml\napiVersion: monitoring.rhobs/v1\nkind: Probe\nmetadata:\n  name: probe-{id}\n  namespace: monitoring\n  labels:\n    rhobs.monitoring/probe-id: \"{id}\"\n    rhobs.monitoring/cluster-id: \"{cluster_id}\"\n    rhobs.monitoring/management-cluster: \"{management_cluster_id}\"\n    rhobs.monitoring/managed-by: \"rhobs-synthetics-agent\"\nspec:\n  interval: \"30s\"\n  module: \"http_2xx\"\n  prober_url: \"synthetics-blackbox-prober-default-service:9115\"\n  targets:\n    staticConfig:\n      static:\n        - \"{target_url}\"\n      labels:\n        apiserver_url: \"{target_url}\"\n        cluster_id: \"{cluster_id}\"\n        management_cluster_id: \"{management_cluster_id}\"\n        private: \"{private_flag}\"\n```\n\n## Development\n\n### Setup Dependencies\n\n```bash\n# Download Go modules\nmake go-mod-download\n\n# Clean up dependencies\nmake go-mod-tidy\n```\n\n### Development Cycle\n\n```bash\n# Make your changes, then run quality checks\nmake lint-fix\nmake test\nmake build\n\n# Run with development settings\n./rhobs-synthetics-agent start --log-level debug --interval 10s\n```\n\n### Testing\n\n```bash\n# Run all tests\nmake test\n\n# Run with coverage report\nmake coverage\n\n# Run specific package tests\ngo test -v ./internal/api -run TestClient\n\n# Run integration tests\ngo test -v ./internal/agent -run TestWorker_FullIntegration\n\n# Test with race detection\nmake test-race\n```\n\n### Test Coverage\n\nCurrent test coverage:\n- **Internal/Agent**: 67.3% (core functionality, worker processes)\n- **Internal/API**: 90.9% (API client, error handling)\n- **Internal/K8s**: 93.1% (resource creation, validation)\n\nSee `TEST_COVERAGE_SUMMARY.md` for detailed coverage information.\n\n## API Reference\n\n### RHOBS Probes API Endpoints\n\n- `GET /api/metrics/v1/{tenant}/probes?label_selector={selectors}` - Retrieve probes\n- `PATCH /api/metrics/v1/{tenant}/probes/{id}` - Update probe status\n\n### Probe Data Model\n\n```json\n{\n  \"id\": \"probe-123\",\n  \"static_url\": \"https://api.example.com/health\",\n  \"labels\": {\n    \"cluster_id\": \"cluster-456\",\n    \"management_cluster_id\": \"mgmt-789\",\n    \"private\": \"false\"\n  },\n  \"status\": \"pending\"\n}\n```\n\n## Deployment\n\n### Docker Build\n\n```bash\n# Build Docker image\nmake docker-build\n\n# Push to registry\nmake docker-push\n```\n\n### Kubernetes Deployment\n\nThe agent is designed to run as a Kubernetes Deployment with:\n- Service account with permissions to create Custom Resources\n- ConfigMap or Secret for API credentials\n- Appropriate RBAC for Probe resources (monitoring.rhobs/v1 and/or monitoring.coreos.com/v1)\n\n## Troubleshooting\n\n### Common Issues\n\n1. **API Connection Failures**\n   - Verify `API_URLS` and network connectivity\n   - Check API credentials and ensure URLs include the correct tenant\n\n2. **URL Validation Failures**\n   - Review target URLs in probe configurations\n   - Check firewall/network policies for outbound requests\n\n3. **Custom Resource Creation Issues**\n   - Verify Kubernetes permissions and RBAC\n   - Check if CRDs are installed: `kubectl get crd probes.monitoring.rhobs probes.monitoring.coreos.com`\n\n### Debug Mode\n\n```bash\n# Enable debug logging\n./rhobs-synthetics-agent start --log-level debug\n\n# Use shorter polling interval for testing\n./rhobs-synthetics-agent start --interval 10s --log-level debug\n```\n\n## Contributing\n\n### Code Style\n\n- Follow Go conventions and best practices\n- Run `make lint-fix` before submitting changes\n- Ensure all tests pass: `make test`\n- Add tests for new functionality\n\n### Pull Request Process\n\n1. Fork the repository\n2. Create a feature branch\n3. Make changes with appropriate tests\n4. Run quality checks: `make lint test`\n5. Submit pull request with clear description\n\n## License\n\nThis project follows the same license as the RHOBS ecosystem.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frhobs%2Frhobs-synthetics-agent","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frhobs%2Frhobs-synthetics-agent","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frhobs%2Frhobs-synthetics-agent/lists"}