{"id":24637954,"url":"https://github.com/upbound/configuration-observability-oss","last_synced_at":"2025-10-28T17:32:03.087Z","repository":{"id":203841587,"uuid":"706983704","full_name":"upbound/configuration-observability-oss","owner":"upbound","description":"A Crossplane Observability Reference Platform For Open Source Software Integrations","archived":false,"fork":false,"pushed_at":"2025-05-08T18:41:47.000Z","size":1124,"stargazers_count":8,"open_issues_count":7,"forks_count":3,"subscribers_count":17,"default_branch":"main","last_synced_at":"2025-05-08T23:45:00.112Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Makefile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/upbound.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-19T01:52:21.000Z","updated_at":"2025-03-15T13:21:24.000Z","dependencies_parsed_at":"2023-11-14T01:32:45.821Z","dependency_job_id":"3e764653-f554-41bc-a3f7-9a1c8f9378ac","html_url":"https://github.com/upbound/configuration-observability-oss","commit_stats":null,"previous_names":["upbound/configuration-observability-oss"],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/upbound%2Fconfiguration-observability-oss","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/upbound%2Fconfiguration-observability-oss/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/upbound%2Fconfiguration-observability-oss/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/upbound%2Fconfiguration-observability-oss/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/upbound","download_url":"https://codeload.github.com/upbound/configuration-observability-oss/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253166473,"owners_count":21864467,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-25T10:12:37.173Z","updated_at":"2025-10-18T10:24:58.053Z","avatar_url":"https://github.com/upbound.png","language":"Makefile","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Observability OSS Configuration\n\nThis repository contains an Upbound project, tailored for users establishing comprehensive observability for their Crossplane control planes with [Upbound](https://cloud.upbound.io). This configuration deploys a fully managed observability stack using open source tools like Prometheus and Grafana to monitor Crossplane platform performance.\n\n## Overview\n\nObservability is a measure of how well platform performance can be inferred from knowledge of its metrics, logs and traces outputs. This configuration provides monitoring for Crossplane components, providers, and managed resources.\n\nThe core components of a custom API in [Upbound Project](https://docs.upbound.io/learn/control-plane-project/) include:\n\n- **CompositeResourceDefinition (XRD):** Defines the API's structure.\n- **Composition(s):** Configures the Functions Pipeline\n- **Embedded Function(s):** Encapsulates the Composition logic and implementation within a self-contained, reusable unit\n\nIn this specific configuration, the API contains:\n\n- **an [Oss](/apis/oss/definition.yaml) custom resource type.**\n- **Composition:** Configured in [/apis/oss/composition.yaml](/apis/oss/composition.yaml), it provisions Prometheus, Grafana, and monitoring components.\n- **Embedded Function:** The Composition logic is encapsulated within [embedded function](/functions/xoss/main.k)\n\n## Important Disclaimers\n\n**⚠️ Management Cluster Resource Use**  \nPrometheus and Grafana may require significant cluster resources in relation to the amount of metrics scraped, processed and visualized. This may impact cluster operations. Consult the respective Prometheus Operator and Grafana documentation for set up guidance prior to using this configuration on mission critical Crossplane management clusters.\n\n**⚠️ Metric Stability**  \nCrossplane has no concept of metric stability. This implies that metrics used in this configuration may be absent in future versions of Crossplane and / or its providers.\n\n## Deployment\n\n- Execute `up project run`\n- Alternatively, install the Configuration from the [Upbound Marketplace](https://marketplace.upbound.io/configurations/upbound/configuration-observability-oss)\n- Check [examples](/examples/) for example XR (Composite Resource)\n\n## Testing\n\nThe configuration can be tested using:\n\n- `up project build` to build the project and embedded functions\n- `up test run tests/*` to run composition tests in `tests/test-xoss/`\n- `up test run tests/* --e2e` to run end-to-end tests in `tests/e2etest-xoss/`\n\n## Usage\n\nApply the resource claim as follows to deploy the observability stack:\n```bash\nkubectl apply -f examples/oss.yaml\n```\n\nTo load dashboards that are part of this configuration repository, apply the following dashboard resource claims:\n```bash\nkubectl apply -f examples/folder-grafana.yaml\nkubectl apply -f examples/dashboard-grafana-crossplane-health.yaml\nkubectl apply -f examples/dashboard-grafana-crossplane-mr.yaml\nkubectl apply -f examples/dashboard-grafana-crossplane-resources-ttr.yaml\nkubectl apply -f examples/dashboard-grafana-crossplane-sli-metrics.yaml\n```\n\nUse the following to forward localhost:9090 to the Prometheus pod.\n```\nPROMETHEUS_POD_NAME=$(k -n operators get pods|\\\n    awk '{print $1}'|\\\n    tail +2|\\\n    grep prometheus-0)\nkubectl -n operators port-forward ${PROMETHEUS_POD_NAME} 9090\n```\n\nUse the following to forward localhost:3000 to the Grafana pod.\n```\nGRAFANA_POD_NAME=$(k -n operators get pods|\\\n    awk '{print $1}'|\\\n    tail +2|\\\n    grep grafana)\nkubectl -n operators port-forward ${GRAFANA_POD_NAME} 3000\n```\n\nLog in to Grafana at http://localhost:3000 with the credentials\nobtained from running the following.\n```\nscripts/grafana-get-creds.sh\n```\n\nSee example dashboards below.\n\n#### Crossplane MR Dashboard\n![Crossplane MR Dashboard](docs/media/crossplane-mr-dashboard.png)\n\n#### Controller Runtime Panels From Crossplane Dashboard\n![Controller Runtime Panels From Crossplane Dashboar](docs/media/crossplane-controller-runtime-panels.png)\n\n#### Resources TTR Dashboard\n![Resources TTR Dashboard](docs/media/resoures-ttr-dashboard.png)\n\n### Crossplane Observability In Action\nOnce your cluster has been bootstrapped, and that prometheus and grafana\nendpoints have been forwarded, what's next?\n\nInstall a kubernetes secret with your provider credentials or use IRSA or\nyour own preferred way to provide the providers with the permissions to\ncreate and reconcile cloud resources.\n\nIf you use AWS, One way would be to add your credentials to\n`~/.aws/credentials`, and to run\n```\nkubectl create secret generic aws-creds \\\n    -n upbound-system \\\n    --from-file=credentials=~/.aws/credentials\n```\nNote that your shell may need a fully qualified path versus `~` above.\n\nApply a provider configuration as follows.\n```\ncat \u003c\u003cEOF | kubectl -f -\napiVersion: aws.upbound.io/v1beta1\nkind: ProviderConfig\nmetadata:\n  name: default\nspec:\n  credentials:\n    source: Secret\n    secretRef:\n      namespace: upbound-system\n      name: aws-creds\n      key: credentials\nEOF\n```\n\nApply resource claims and see information on the\nloaded dashboards. For example you can create an AWS VPC as follows, and\nyou can use your own compositions and any of our\n[marketplace configurations](https://marketplace.upbound.io/configurations).\n\n```\napiVersion: ec2.aws.upbound.io/v1beta1\nkind: VPC\nmetadata:\n  name: sample-vpc\n  annotations:\n    meta.upbound.io/example-id: ec2/v1beta1/vpc\nspec:\n  forProvider:\n    region: us-west-1\n    cidrBlock: 172.16.0.0/16\n    tags:\n      Name: SampleVpc\n```\n\n## Next steps\n\nThis repository serves as a foundational step for comprehensive Crossplane observability. To enhance the configuration, consider:\n\n1. Create new API definitions for monitoring additional components\n2. Customize the existing observability stack to meet your specific monitoring requirements\n3. Integrate with additional data sources or alerting systems\n4. Extend monitoring to cover your custom compositions and managed resources\n\n## Purpose\n\nThe goal for configuration-observability-oss is to complement other configurations such as [configuration-caas](https://marketplace.upbound.io/configurations). See the [Upbound Marketplace](https://marketplace.upbound.io/) for additional configurations.\n\n## Community\n\nFeel free to join the [SIG Observability Slack Channel](https://crossplane.slack.com/archives/C061GNH3LA0) to participate in the Crossplane observability journey.\n\nTo learn more about how to build APIs for your managed control planes in Upbound, read the guide on [Upbound's docs](https://docs.upbound.io/concepts/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fupbound%2Fconfiguration-observability-oss","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fupbound%2Fconfiguration-observability-oss","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fupbound%2Fconfiguration-observability-oss/lists"}