{"id":36753821,"url":"https://github.com/openshift-hyperfleet/hyperfleet-sentinel","last_synced_at":"2026-01-20T17:19:53.285Z","repository":{"id":320535832,"uuid":"1081820562","full_name":"openshift-hyperfleet/hyperfleet-sentinel","owner":"openshift-hyperfleet","description":"HyperFleet Sentinel Operator - Kubernetes operator that polls HyperFleet API, makes orchestration decisions, and publishes    events. Features configurable backoff strategies, horizontal sharding via SentinelConfig CRD, and broker abstraction (GCP    Pub/Sub, RabbitMQ, Stub). Centralized reconciliation logic.","archived":false,"fork":false,"pushed_at":"2026-01-09T18:40:38.000Z","size":384,"stargazers_count":0,"open_issues_count":2,"forks_count":7,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-01-12T18:54:53.217Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openshift-hyperfleet.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-23T10:36:40.000Z","updated_at":"2026-01-09T18:40:42.000Z","dependencies_parsed_at":"2025-10-24T11:24:15.932Z","dependency_job_id":"12da8a30-75b0-4b56-a893-db3ffc18be72","html_url":"https://github.com/openshift-hyperfleet/hyperfleet-sentinel","commit_stats":null,"previous_names":["openshift-hyperfleet/hyperfleet-sentinel"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/openshift-hyperfleet/hyperfleet-sentinel","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openshift-hyperfleet%2Fhyperfleet-sentinel","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openshift-hyperfleet%2Fhyperfleet-sentinel/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openshift-hyperfleet%2Fhyperfleet-sentinel/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openshift-hyperfleet%2Fhyperfleet-sentinel/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openshift-hyperfleet","download_url":"https://codeload.github.com/openshift-hyperfleet/hyperfleet-sentinel/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openshift-hyperfleet%2Fhyperfleet-sentinel/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28607626,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-20T16:10:39.856Z","status":"ssl_error","status_checked_at":"2026-01-20T16:10:39.493Z","response_time":117,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-12T12:47:41.147Z","updated_at":"2026-01-20T17:19:53.259Z","avatar_url":"https://github.com/openshift-hyperfleet.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# hyperfleet-sentinel\n\nHyperFleet Sentinel Service - Kubernetes service that polls HyperFleet API, makes orchestration decisions, and publishes events. Features configurable max age intervals, horizontal sharding via SentinelConfig CRD, and broker abstraction (GCP Pub/Sub, RabbitMQ, Stub). Centralized reconciliation logic.\n\n## Development Setup\n\n### Prerequisites\n\n- Go 1.25 or later\n- Docker or Podman\n- Make\n\n### Getting Started\n\n1. **Clone the repository**:\n   ```bash\n   git clone https://github.com/openshift-hyperfleet/hyperfleet-sentinel.git\n   cd hyperfleet-sentinel\n   ```\n\n2. **Generate the OpenAPI client**:\n   ```bash\n   make generate\n   ```\n\n   This will:\n   - Download the official OpenAPI spec from [hyperfleet-api](https://github.com/openshift-hyperfleet/hyperfleet-api) (main branch)\n   - Generate Go client code in `pkg/api/openapi/`\n\n   Both the downloaded spec and generated client code are **not committed** to git and must be regenerated locally.\n\n3. **Download dependencies**:\n   ```bash\n   make download\n   ```\n\n4. **Build the binary**:\n   ```bash\n   make build\n   ```\n\n5. **Run tests**:\n   ```bash\n   make test\n   ```\n\n### Common Make Targets\n\n- `make help` - Show all available make targets\n- `make generate` - Generate OpenAPI client from spec (Docker/Podman-based)\n- `make build` - Build the sentinel binary\n- `make test` - Run unit tests with coverage\n- `make test-integration` - Run integration tests (requires Docker/Podman)\n- `make test-all` - Run all tests (unit + integration)\n- `make fmt` - Format Go code\n- `make lint` - Run golangci-lint (requires golangci-lint installed)\n- `make clean` - Remove build artifacts and generated code\n\n### Testing\n\nThe project uses a hybrid testing approach:\n\n- **Unit tests**: Fast, isolated tests using mocks\n- **Integration tests**: End-to-end tests with real message brokers via testcontainers\n\n```bash\n# Run only unit tests (fast)\nmake test\n\n# Run integration tests (requires Docker or Podman)\nmake test-integration\n\n# Run all tests\nmake test-all\n```\n\nIntegration tests automatically work with both Docker and Podman. For troubleshooting and advanced configuration, see [docs/testcontainers.md](docs/testcontainers.md).\n\nFor instructions on running Sentinel locally or on GKE, see [docs/running-sentinel.md](docs/running-sentinel.md).\n\n### OpenAPI Client Generation\n\nThis project follows the [rh-trex](https://github.com/openshift-online/rh-trex) pattern for OpenAPI client generation. The OpenAPI specification is automatically downloaded from the official [hyperfleet-api](https://github.com/openshift-hyperfleet/hyperfleet-api) repository (main branch by default) during `make generate`.\n\nThe client is generated using Docker/Podman to ensure consistency across development environments.\n\nTo use a different branch or tag:\n```bash\nmake generate OPENAPI_SPEC_REF=v1.0.0    # Use a specific tag\nmake generate OPENAPI_SPEC_REF=develop   # Use a branch\n```\n\nFor detailed information about OpenAPI client generation, see [openapi/README.md](openapi/README.md).\n\n## Configuration\n\nThe Sentinel service uses YAML-based configuration with environment variable overrides for sensitive data (broker credentials).\n\n### Configuration File\n\nCreate a configuration file based on the examples in the `configs/` directory:\n\n- **`configs/gcp-pubsub-example.yaml`** - GCP Pub/Sub configuration\n- **`configs/rabbitmq-example.yaml`** - RabbitMQ configuration\n- **`configs/dev-example.yaml`** - Development configuration\n\n### Configuration Schema\n\n#### Required Fields\n\n| Field | Type | Description | Example |\n|-------|------|-------------|---------|\n| `resource_type` | string | Resource to watch (clusters, nodepools) | `clusters` |\n| `hyperfleet_api.endpoint` | string | HyperFleet API base URL (k8s service) | `http://hyperfleet-api.hyperfleet-system.svc.cluster.local:8080` |\n\n#### Optional Fields with Defaults\n\n| Field | Type | Default | Description |\n|-------|------|---------|-------------|\n| `poll_interval` | duration | `5s` | How often to poll the API for resource updates |\n| `max_age_not_ready` | duration | `10s` | Max age interval for resources not ready |\n| `max_age_ready` | duration | `30m` | Max age interval for ready resources |\n| `hyperfleet_api.timeout` | duration | `5s` | Request timeout for API calls |\n| `resource_selector` | array | `[]` | Label selectors for filtering resources (enables sharding) |\n| `message_data` | map | `{}` | Template fields for CloudEvents data payload |\n\n#### Resource Selector (Sharding)\n\nThe `resource_selector` field enables horizontal scaling by having multiple Sentinel instances watch different resource subsets:\n\n```yaml\nresource_selector:\n  - label: shard\n    value: \"1\"\n  - label: region\n    value: us-east-1\n```\n\nAn empty or omitted `resource_selector` means watch all resources. Multiple selectors use AND logic (all labels must match).\n\nFor detailed instructions on deploying multiple Sentinel instances with different resource selectors, see [docs/multi-instance-deployment.md](docs/multi-instance-deployment.md).\n\n#### Message Data Templates\n\nDefine custom fields to include in CloudEvents using Go template syntax. Both `.field` and `{{.field}}` formats are supported:\n\n```yaml\nmessage_data:\n  resource_id: .id\n  resource_type: .kind\n  href: .href\n  generation: .generation\n```\n\nTemplates can reference any field from the Resource object returned by the API. The example above follows the `ObjectReference` pattern (id, kind, href) with generation for reconciliation tracking.\n\n### Broker Configuration\n\nBroker configuration is managed by the [hyperfleet-broker library](https://github.com/openshift-hyperfleet/hyperfleet-broker). You can configure the broker using either:\n\n1. **broker.yaml file** (see `broker.yaml` in project root for example)\n2. **BROKER_CONFIG_FILE environment variable** (path to your broker config file)\n3. **Direct environment variables** (listed below)\n\n#### Environment Variables (Override broker.yaml)\n\n**RabbitMQ:**\n| Variable | Description | Example |\n|----------|-------------|---------|\n| `BROKER_RABBITMQ_URL` | Complete connection URL | `amqp://user:pass@localhost:5672/vhost` |\n\n**Google Pub/Sub:**\n| Variable | Description | Example |\n|----------|-------------|---------|\n| `BROKER_GOOGLEPUBSUB_PROJECT_ID` | GCP project ID | `my-gcp-project` |\n| `GOOGLE_APPLICATION_CREDENTIALS` | Service account key path (optional, uses ADC if not set) | `/path/to/key.json` |\n\n**Topic Configuration:**\n\n| Variable | Description | Example |\n|----------|-------------|---------|\n| `BROKER_TOPIC` | Topic name for publishing events | `hyperfleet-dev-clusters` |\n\nThe `BROKER_TOPIC` environment variable sets the full topic name where events will be published. When using Helm, the default topic is `{namespace}-{resourceType}` (e.g., `hyperfleet-dev-clusters`, `hyperfleet-dev-nodepools`). This enables isolation between different environments or tenants sharing the same broker. See [Naming Strategy](https://github.com/openshift-hyperfleet/architecture/blob/main/hyperfleet/components/sentinel/sentinel-naming-strategy.md) for details.\n\nFor detailed broker configuration options, see the [hyperfleet-broker documentation](https://github.com/openshift-hyperfleet/hyperfleet-broker).\n\n### Running Sentinel\n\nFor detailed instructions on running Sentinel locally or deploying to GKE, see [docs/running-sentinel.md](docs/running-sentinel.md).\n\nFor Helm chart documentation and configuration options, see [deployments/helm/sentinel/README.md](deployments/helm/sentinel/README.md).\n\n### Configuration Validation\n\nThe service validates configuration at startup and will fail fast on errors:\n\n- **Required fields present**: `resource_type`, `hyperfleet_api.endpoint`\n- **Valid enums**: `resource_type` must be clusters/nodepools\n- **Valid durations**: All interval fields must be positive\n- **Valid templates**: All `message_data` templates must be valid Go templates\n- **Broker configuration**: Managed by hyperfleet-broker library (see broker.yaml)\n\n### Configuration Examples\n\n#### Minimal Configuration\n\n```yaml\nresource_type: clusters\nhyperfleet_api:\n  endpoint: http://localhost:8000\n```\n\nThis uses all defaults. Broker configuration is managed via `broker.yaml` or environment variables (see Broker Configuration section).\n\n#### Production Configuration with Sharding\n\n```yaml\nresource_type: clusters\npoll_interval: 5s\nmax_age_not_ready: 10s\nmax_age_ready: 30m\n\nresource_selector:\n  - label: shard\n    value: \"1\"\n\nhyperfleet_api:\n  endpoint: http://hyperfleet-api.hyperfleet-system.svc.cluster.local:8080\n  timeout: 30s\n\nmessage_data:\n  resource_id: .id\n  resource_type: .kind\n  href: .href\n  generation: .generation\n```\n\n## Observability\n\nThe Sentinel service exposes Prometheus metrics on port 8080 at `/metrics` for monitoring and alerting.\n\n### Metrics\n\nSentinel provides 6 core metrics for comprehensive observability:\n\n| Metric | Type | Description |\n|--------|------|-------------|\n| `hyperfleet_sentinel_pending_resources` | Gauge | Number of resources pending reconciliation |\n| `hyperfleet_sentinel_events_published_total` | Counter | Total events published to message broker |\n| `hyperfleet_sentinel_resources_skipped_total` | Counter | Resources skipped (preconditions not met) |\n| `hyperfleet_sentinel_poll_duration_seconds` | Histogram | Duration of each polling cycle |\n| `hyperfleet_sentinel_api_errors_total` | Counter | Errors when calling HyperFleet API |\n| `hyperfleet_sentinel_broker_errors_total` | Counter | Errors when publishing to message broker |\n\nAll metrics include `resource_type` and `resource_selector` labels for filtering.\n\n**For detailed metric descriptions, example queries, and alerting rules**, see [docs/metrics.md](docs/metrics.md).\n\n### Grafana Dashboard\n\nA pre-built Grafana dashboard is available at `deployments/dashboards/sentinel-metrics.json` with 8 visualization panels covering all metrics.\n\nTo import:\n1. Navigate to Grafana → Dashboards → Import\n2. Upload `deployments/dashboards/sentinel-metrics.json`\n3. Select your Prometheus datasource\n\n### GKE Integration with Google Cloud Managed Prometheus\n\nSentinel integrates with Google Cloud Managed Prometheus (GMP) for automated metrics collection:\n\n```bash\n# Deploy with PodMonitoring enabled (default)\nhelm install sentinel ./deployments/helm/sentinel \\\n  --namespace hyperfleet-system \\\n  --create-namespace\n\n# Verify metrics in Google Cloud Console\n# Navigate to: Monitoring → Metrics Explorer\n# Query: hyperfleet_sentinel_pending_resources\n```\n\nGMP automatically discovers the PodMonitoring resource and begins scraping metrics. No additional configuration required.\n\n#### Alerting\n\nConfigure alerts in **Google Cloud Console → Monitoring → Alerting** using the PromQL expressions provided in [docs/metrics.md](docs/metrics.md).\n\nRecommended alerts:\n- SentinelHighPendingResources - High number of pending resources\n- SentinelAPIErrorRateHigh - High API error rate\n- SentinelBrokerErrorRateHigh - High broker error rate\n- SentinelSlowPolling - Slow polling cycles\n- SentinelNoEventsPublished - No events published despite pending resources\n- SentinelHighSkipRatio - High ratio of skipped resources\n- SentinelDown - Sentinel service is down\n\nSee [docs/metrics.md](docs/metrics.md) for complete alerting rules documentation.\n\n### Accessing Metrics\n\nAccess metrics through Google Cloud Console:\n1. Navigate to **Monitoring → Metrics Explorer**\n2. Select resource type: **Prometheus Target**\n3. Query: `hyperfleet_sentinel_pending_resources`\n\nMetrics are automatically collected by Google Cloud Managed Prometheus via the PodMonitoring resource.\n\n## Repository Access\n\nAll members of the **hyperfleet** team have write access to this repository.\n\n### Steps to Apply for Repository Access\n\nIf you're a team member and need access to this repository:\n\n1. **Verify Organization Membership**: Ensure you're a member of the `openshift-hyperfleet` organization\n2. **Check Team Assignment**: Confirm you're added to the hyperfleet team within the organization\n3. **Repository Permissions**: All hyperfleet team members automatically receive write access\n4. **OWNERS File**: Code reviews and approvals are managed through the OWNERS file\n\nFor access issues, contact a repository administrator or organization owner.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenshift-hyperfleet%2Fhyperfleet-sentinel","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenshift-hyperfleet%2Fhyperfleet-sentinel","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenshift-hyperfleet%2Fhyperfleet-sentinel/lists"}