{"id":47351989,"url":"https://github.com/hops-ops/aws-observe-stack","last_synced_at":"2026-05-23T07:13:39.121Z","repository":{"id":340454505,"uuid":"1166061978","full_name":"hops-ops/aws-observe-stack","owner":"hops-ops","description":"Observe Stack with AWS integrations (PodIdentity for OpenCost, etc)","archived":false,"fork":false,"pushed_at":"2026-04-02T23:57:20.000Z","size":106,"stargazers_count":0,"open_issues_count":8,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-03T00:40:46.579Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"KCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hops-ops.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-24T20:52:23.000Z","updated_at":"2026-03-30T20:20:46.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/hops-ops/aws-observe-stack","commit_stats":null,"previous_names":["hops-ops/stack-aws-observe","hops-ops/aws-observe-stack"],"tags_count":20,"template":false,"template_full_name":null,"purl":"pkg:github/hops-ops/aws-observe-stack","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hops-ops%2Faws-observe-stack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hops-ops%2Faws-observe-stack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hops-ops%2Faws-observe-stack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hops-ops%2Faws-observe-stack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hops-ops","download_url":"https://codeload.github.com/hops-ops/aws-observe-stack/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hops-ops%2Faws-observe-stack/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31571601,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-08T14:31:17.711Z","status":"ssl_error","status_checked_at":"2026-04-08T14:31:17.202Z","response_time":54,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-03-18T00:27:56.123Z","updated_at":"2026-04-22T07:06:00.991Z","avatar_url":"https://github.com/hops-ops.png","language":"KCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# aws-observe-stack\n\nDeploys the ObserveStack with AWS Pod Identity for OpenCost billing access.\n\n## Overview\n\nComposes the base `hops.ops.com.ai/ObserveStack` XRD with `aws.hops.ops.com.ai/PodIdentity`.\nAutomatically provisions IAM role and Pod Identity association for OpenCost's service account\nwith Cost Explorer, Pricing, CUR, and EC2 describe permissions.\n\n## Usage\n\n```yaml\napiVersion: aws.hops.ops.com.ai/v1alpha1\nkind: ObserveStack\nmetadata:\n  name: observe\n  namespace: default\nspec:\n  clusterName: my-cluster\n  aws:\n    region: us-east-1\n```\n\nDefault storage is pvc-backed (`storage.type: pvc`) for both Loki and Tempo with auto-created `gp3` StorageClasses (`loki`, `tempo`).\n\nYou can set global StorageClass behavior and override per component:\n\n```yaml\nspec:\n  storageClassDefaults:\n    reclaimPolicy: Retain\n  loki:\n    storage:\n      type: pvc\n      size: 20Gi\n      storageClass:\n        name: logs\n  tempo:\n    storage:\n      type: pvc\n      storageClass:\n        name: traces\n```\n\nWith custom values:\n\n```yaml\napiVersion: aws.hops.ops.com.ai/v1alpha1\nkind: ObserveStack\nmetadata:\n  name: observe\n  namespace: default\nspec:\n  clusterName: production-cluster\n  namespace: monitoring\n  loki:\n    storage:\n      type: s3\n      s3:\n        retentionDays: 30\n  tempo:\n    storage:\n      type: s3\n      s3:\n        retentionDays: 14\n  k8sMonitoring:\n    values:\n      opencost:\n        enabled: true\n  aws:\n    region: us-west-2\n    rolePrefix: prod-\n```\n\n### Dedicated NodePool\n\nIsolate observe workloads on a dedicated Karpenter NodePool (opt-in):\n\n```yaml\nspec:\n  clusterName: my-cluster\n  aws:\n    region: us-east-1\n  nodePool:\n    enabled: true\n```\n\nWhen enabled:\n- A Karpenter NodePool (`\u003cclusterName\u003e-observe`) is created with broad spot + on-demand instance selection\n- Non-daemonset pods (Prometheus, Loki, Tempo, Grafana, OpenCost, VPA, Goldilocks) are scheduled to the NodePool via `nodeSelector` and `tolerations`\n- Daemonsets (alloy-metrics, alloy-logs) continue to run on **all** nodes, including the observe NodePool\n\nCustom NodePool settings:\n\n```yaml\nspec:\n  nodePool:\n    enabled: true\n    nodeClassName: custom-nodeclass\n    limits:\n      nodes: 20\n    requirements:\n    - key: kubernetes.io/arch\n      operator: In\n      values: [amd64]\n    - key: karpenter.sh/capacity-type\n      operator: In\n      values: [spot]\n    disruption:\n      consolidationPolicy: WhenEmpty\n      consolidateAfter: \"120s\"\n```\n\n### Spot Instance Pricing\n\nEnable accurate spot pricing in OpenCost by pointing it at a [SpotFeed](https://github.com/hops-ops/aws-spot-feed) bucket:\n\n```yaml\nspec:\n  clusterName: my-cluster\n  aws:\n    region: us-east-1\n  spotFeed:\n    enabled: true\n    bucketName: hops-root-account-spot-feed\n    region: us-east-1\n```\n\nWhen enabled:\n- OpenCost PodIdentity gets S3 read permissions for the spot feed bucket\n- OpenCost exporter is configured with `spot_data_bucket`, `spot_data_region`, and spot node labels (`karpenter.sh/capacity-type=spot`)\n- Optional `prefix` field if the spot feed uses an S3 key prefix\n\n\u003e **Prerequisite:** Deploy a `SpotFeed` XRD in the same AWS account first. See [aws-spot-feed](https://github.com/hops-ops/aws-spot-feed).\n\n### Cloud Costs via CUR/Athena\n\nEnable full AWS cloud cost ingestion in OpenCost by referencing a separately managed `CostReport` XR:\n\n```yaml\napiVersion: aws.hops.ops.com.ai/v1alpha1\nkind: CostReport\nmetadata:\n  name: hops-root-account\n  namespace: default\nspec:\n  accountId: \"123456789012\"\n  region: us-east-2\n---\napiVersion: aws.hops.ops.com.ai/v1alpha1\nkind: ObserveStack\nmetadata:\n  name: observe\n  namespace: default\nspec:\n  clusterName: production\n  namespace: monitoring\n  aws:\n    region: us-east-2\n  cloudCosts:\n    enabled: true\n    costReportRef:\n      enabled: true\n      name: hops-root-account\n      accountId: \"123456789012\"\n      region: us-east-2\n```\n\nWhen enabled:\n- ObserveStack derives CUR/Athena names from the referenced `CostReport`\n- OpenCost is wired through the chart's native `cloudIntegrationSecret` support\n- The OpenCost PodIdentity gets Athena, Glue, and S3 permissions for the CUR bucket/workgroup\n- The generated `cloud-integration.json` uses OpenCost's legacy flat AWS format with the Athena query-results location `s3://{bucketName}/athena-results`\n\n`CostReport` is account-level and should be managed separately, one per AWS account. If your `CostReport` uses non-default names, set the corresponding overrides under `cloudCosts.costReportRef`. The legacy manual `cloudCosts` fields still work as a fallback.\n\n## What Gets Created\n\n1. **Helm Releases** - Prometheus, Loki, Tempo, k8s-monitoring/OpenCost, Grafana Operator, VPA, Goldilocks\n2. **PodIdentity** - IAM role + Pod Identity association with billing permissions for OpenCost\n3. **Usage** - deletion ordering: Observe deleted before PodIdentity\n4. **StorageClasses** - gp3 EBS classes for Prometheus, Loki, Tempo (when using PVC storage)\n5. **NodePool** *(opt-in)* - dedicated Karpenter NodePool for observe workloads\n\n## Development\n\n```bash\nmake render\nmake validate\nmake test\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhops-ops%2Faws-observe-stack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhops-ops%2Faws-observe-stack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhops-ops%2Faws-observe-stack/lists"}