{"id":28420703,"url":"https://github.com/redhat-developer/observability-operator","last_synced_at":"2026-01-11T22:48:55.267Z","repository":{"id":37089359,"uuid":"318151012","full_name":"redhat-developer/observability-operator","owner":"redhat-developer","description":"Operator installing the Telemetry stack in a Kubernetes cluster and installing the metrics and alerts","archived":false,"fork":false,"pushed_at":"2023-11-03T10:47:07.000Z","size":1632,"stargazers_count":17,"open_issues_count":3,"forks_count":32,"subscribers_count":24,"default_branch":"main","last_synced_at":"2025-06-05T05:30:05.494Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/redhat-developer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-12-03T10:08:32.000Z","updated_at":"2024-08-01T17:42:41.000Z","dependencies_parsed_at":"2024-06-19T00:05:35.228Z","dependency_job_id":"8b034330-7f7b-4475-8a17-76dd6e7e77b7","html_url":"https://github.com/redhat-developer/observability-operator","commit_stats":{"total_commits":375,"total_committers":21,"mean_commits":"17.857142857142858","dds":0.544,"last_synced_commit":"8fe3a7465d02849d010bd664a1be4fcf8027352c"},"previous_names":["bf2fc6cc711aee1a0c2a/observability-operator"],"tags_count":25,"template":false,"template_full_name":null,"purl":"pkg:github/redhat-developer/observability-operator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redhat-developer%2Fobservability-operator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redhat-developer%2Fobservability-operator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redhat-developer%2Fobservability-operator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redhat-developer%2Fobservability-operator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/redhat-developer","download_url":"https://codeload.github.com/redhat-developer/observability-operator/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redhat-developer%2Fobservability-operator/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262113215,"owners_count":23260981,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-05T03:49:16.034Z","updated_at":"2026-01-11T22:48:55.230Z","avatar_url":"https://github.com/redhat-developer.png","language":"Go","readme":"# Observability Operator\n\n\u003c!-- we could/should add unit/integration/'go report' type badges IMO /--\u003e\n\n\n## What Is It?\n\nThe Observability Operator deploys \u0026 maintains a common platform for Application Services to share and utilize to aid in monitoring \u0026 reporting on their service components.\nIt integrates with the [Observatorium](https://github.com/observatorium) project for pushing metrics and logs to a central location.\n\n\n## What's included?    \n\n### Prometheus\n\n[Prometheus](https://prometheus.io/) is a free software application used for event monitoring and alerting. \nIt records real-time metrics in a time series database built using a HTTP pull model, with flexible queries \nand real-time alerting.\n\nPrometheus's [Alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/) is also supported, which \nhandles alerts sent by Prometheus \u0026 takes care of deduplicating, grouping, and routing them to the correct \nreceiver integration (such as, in our case, [PagerDuty](https://www.pagerduty.com/docs/guides/prometheus-integration-guide/)). \nIt also takes care of silencing and inhibition of alerts.\n\n### Grafana\n\n[Grafana](https://grafana.com/) is a multi-platform open source analytics and interactive visualization web \napplication. When configured with supported data sources, Grafana provides charts, graphs, and other visualizations \nvia UI dashboards.\n\n### Promtail\n\n[Promtail](https://grafana.com/docs/loki/latest/clients/promtail/) is a log aggregator for Loki, Grafana's platform for collecting and analyzing logs.\nLoki is the logging backend used in Observatorium.\n\n## How do we integrate our service with Observability?\n\nIf you find yourself asking, chances are good a conversation's already been started in that regard, but if not, \nreach out! It'll likely be a bit more involved than plug-and-play and we'd like to make sure we support your \nneeds. That being said (and know that what follows is absolutely subject to change)...\n\nObservability Operator is intended to support multiple application services, each of which will be responsible for \nmaintaining their own configuration repository \u0026 instantiating a ConfigMap containing a bit of information the \noperator needs in order to read from it. At current, it's expected that configuration repos reside within our \n['bf2' organization](https://github.com/bf2fc6cc711aee1a0c2a) as we use a special read-only mechanism limited to \nwithin our organization for access. \n\nAs an example, first take a look at the [configuration repository](https://github.com/bf2fc6cc711aee1a0c2a/observability-resources-mk) \nfor the first service we've onboarded, Managed Kafka. There you'll find an index file and various configuration files referenced from within. In order to use this config \nrepo, the Observability Operator must be told about it via a `Secret`:\n\n```yaml\nkind: Secret\napiVersion: v1\nmetadata:\n  name: kafka-observability-configuration\n  namespace: kafka-observability\n  labels:\n    configures: observability-operator\ndata:\n  access_token: '\u003ctoken here\u003e'\n  channel: 'resources'\n  repository: 'https://api.github.com/repos/bf2fc6cc711aee1a0c2a/observability-resources-mk/contents'\n  tag: \u003ctag or branch\u003e\n```\n\nThe Observability Operator doesn't care too much about what namespace the Secret resides in (that's not to say that \nwe won't have an opinion, though!). Instead, it scans all namespaces for any Secrets matching a particular label set \nas specified in the Observability CR (more on that in a bit):\n```yaml\n  configurationSelector:\n    matchLabels:\n      configures: \"observability-operator\"\n```\n\n## What's supported via external config?\n\nWithin a given resources folder an [index.json file](https://github.com/bf2fc6cc711aee1a0c2a/observability-resources-mk/blob/main/resources/index.json) \ncontaining, at a minimum, `id` and `config` fields must exist:\n ```yaml \n {\n   \"id\": \"shiny-managed-service-development\",\n   \"config\": {...}\n }\n```\nThe `id` field (specifying both the service and channel) is used by the observability operator to label \u0026 track\nvarious generated resources.\n\nThe `config` field may contain various entries as appropriate for your service:\n* `config.grafana.dashboards` expects an array list of `subdirectory/file.yaml` entries, each pointing to a complete Grafana \nDashboard YAML definition file:\n   ```yaml\n    \"grafana\": {\n     \"dashboards\": [\n       \"grafana/foo-dashboard.yaml\",\n       \"grafana/bar-dashboard.yaml\",\n     ]\n    },\n   }\n   ```\n* `config.promtail` specifies whether Promtail should be used and, if so, a namespace label selector for matching:\n  ```yaml\n    \"promtail\": {\n      \"observatorium\": \"default\"\n      \"enabled\": true,\n      \"namespaceLabelSelector\": {\n        \"app\": \"strimzi\"\n      }\n    },\n  ``` \n* `config.promtail.observatorium` specifies the `id` of the Observatorium config to forward logs to \n* `config.alertmanager` indicates the name of two prerequisite secrets assumed to pre-exist on the cluster for configuration \nof Prometheus PagerDuty \u0026 Alertmanager integrations:\n  ```yaml\n    \"alertmanager\": {\n      \"pagerDutySecretName\": \"pagerduty\",\n      \"deadmansSnitchSecretName\": \"deadmanssnitch\"\n    },\n  ```\n* `config.prometheus.pod_monitors` expects an array list of `sub/directory/file.yaml` entries, each pointing to a complete \nPrometheus [PodMonitor YAML definition](https://docs.openshift.com/container-platform/4.6/rest_api/monitoring_apis/podmonitor-monitoring-coreos-com-v1.html) \nfile:\n  ```yaml\n    \"prometheus\": {\n        \"pod_monitors\": [\n          \"prometheus/pod_monitors/foo-monitor.yaml\",\n          \"prometheus/pod_monitors/bar-monitor.yaml\",\n    ],\n  ```\n* `config.prometheus.rules` expects an array list of `sub/directory/file.yaml` entries, each pointing to a complete \n[PrometheusRule YAML definition](https://docs.openshift.com/container-platform/4.6/rest_api/monitoring_apis/prometheusrule-monitoring-coreos-com-v1.html) \nfile: \n  ```yaml\n    \"rules\": [\n      \"prometheus/prometheus-rules.yaml\"\n    ],\n  ```\n* `config.prometheus.federation` expects a single `subdirectory/file.yaml` location pointing to a file containing an \narray of regex patterns to be concatenated \u0026 used in instantiating a Prometheus [additional scrape config secret](https://github.com/prometheus-operator/prometheus-operator/blob/master/Documentation/additional-scrape-config.md):\n  ```yaml\n  \"federation\": \"prometheus/federation-config.yaml\",\n  ```\n\n* `config.prometheus.observatorium` specifies the `id` of the Observatorium config to forward metrics to\n\n* `config.prometheus.remoteWrite` expects a single `subdirectory/file.yaml` location pointing to a file containing an \narray of regex patterns to be concatenated \u0026 used in instantiating the Prometheus operand (CR): \n  ```yaml\n    \"remoteWrite\": \"prometheus/remote-write.yaml\"\n  ```\n\n* `config.observatoria` an array of observatorium configs, each with an id referenced by prometheus and/or promtail:\n  ```yaml\n    [{\n        \"id\": \"default\",\n        \"secretName\": \"observatorium-configuration-red-hat-sso\"\n      }\n    }, ...]\n  ```\n\nAdditionally, an empty ConfigMap can be created in a target namespace to prevent an Observability operand (CR) from being created in that namespace.\n* The ConfigMap requires the `name` to be set to `observability-operator-no-init` and the target `namespace` to be specified:\n  ```yaml\n    kind: ConfigMap\n    apiVersion: v1\n    metadata:\n      name: observability-operator-no-init\n      namespace: my-target-namespace\n  ```\n\n## What's required in the Observability operand (CR)?\nYour spec will be required to specify, at a minumum, two things:\n* a `resyncPeriod` to indicate how often external config should be re-fetched\n* a `retention` to configure the lifetime of stored data\n* a `configurationSelector` indicating what labels to match when scanning for external config info ConfigMaps as \npreviously mentioned \n```yaml\n  apiVersion: observability.redhat.com/v1\n  kind: Observability\n  metadata:\n    name: observability-sample\n  spec:\n    resyncPeriod: 1h\n    retention: 45d\n    configurationSelector:\n      matchLabels:\n        configures: \"observability-operator\"\n```\n\n## What's optionally supported via the Observability operand (CR)?\n\n* Prometheus storage config\n  ```yaml\n  spec:\n    storage:\n      prometheus:\n        volumeClaimTemplate:\n          spec:\n            storageClassName: ssd\n            resources:\n              requests:\n                storage: 40Gi\n  ```\n* Node Tolerations\n  ```yaml\n  spec:\n    tolerations:\n    - effect: NoSchedule\n      key: node-role.kubernetes.io/infra\n      operator: Exists\n  ```\n* Node Affinities\n  ```yaml\n  spec:\n    affinity:\n      nodeAffinity:\n        requiredDuringSchedulingIgnoredDuringExecution:\n          nodeSelectorTerms:\n            - matchExpressions:\n              - key: node-role.kubernetes.io/infra\n                operator: Exists\n  ```\n\n\n## Running Locally\n\n### Prerequisite Tools\n\n* golang 1.19+\n* operator-sdk v1.4.2\n* [CodeReady Containers](https://github.com/code-ready/crc) 1.18 (OCP 4.6.1) or later\n* [OpenShift command line tool](https://developers.redhat.com/openshift/command-line-tools)\n\n### Setting up your cluster\n\n- We recommend targeting a local CRC instance for daily development. Though we're hedging our bets a bit with these \namounts, we do feel additional memory config params are essential in having a stable platform to run against:\n```\ncrc start --pull-secret-file $HOME/.crc/cache/pull-secret.txt --cpus 6 --memory 24576\n```\n- Once your cluster is up and running, determine which (or create a new) namespace to target with various generated \nfiles. The operator will look in this namespace for an Observability operand (CR) and if not found, generate its own. \nIn order for it to do so, you need to indicate the namespace in one of two ways:\n  - specify the `WATCH_NAMESPACE=foo` environment variable when running\n  - create a file containing only the namespace name at `/var/run/secrets/kubernetes.io/serviceaccount/namespace`\n  to emulate a pod environment \u0026 prevent having to supply the env var every run.  \n\n### Adding requisite resources\nNote that some files require values that are not supplied here or within sample files for security reasons - please \nreview the content of each!\n\n* PagerDuty secret:\n  ```\n  oc apply -f config/samples/secrets/pagerduty.yaml\n  ```\n\n* DeadmansSnitch secret:\n  ```\n  oc apply -f config/samples/secrets/deadmanssnitch.yaml\n  ```\n\n* Sendgrid secret:\n  ```\n  oc apply -f config/samples/secrets/sendgrid.yaml\n  ```\n\n* Auth provider config secret:\n  \n  Users can choose between two auth configurations; dex or Red Hat SSO \n  \n  * Dex config secret\n   ```\n   oc apply -f config/samples/secrets/observatorium-dex-credentials.yaml\n   ```\n  * Red Hat SSO secret\n   ```\n   oc apply -f config/samples/secrets/observatorium-configuration-red-hat-sso.yaml\n\n   ```\n\n* External config repo secret:\n    * The Observability stack requires a Personal Access Token to read externalized configuration from within the bf2 organization. For development cycles, you will need to generate a personal token for your own GitHub user (with bf2 access) and place the value in the Secret. \n    * To generate a new token:\n        * Follow the steps [found here](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token), making sure to check **ONLY** the `repo` box at the top of the scopes/permissions list (which will check each of the subcategory boxes beneath it).\n        * Copy the value of your Personal Access Token to a secure private location. Once you leave the page, you cannot access the value again \u0026 you will be forced to reset the token to receive a new value should you lose the original.\n        * Take care not to push your PAT to any repository as if you do, GitHub will automatically revoke your token as soon as you push, and you'll need to follow this process again to generate a new token.\n    * Apply the Secret with token value substituted in:\n      ```\n      oc apply -f config/samples/secrets/observability_secret.yaml\n      ```\n\n* The required secrets listed above can be configured and deployed using\n  ```\n  make deploy/secrets\n  ```\n\n    * The following parameters can be modified:\n      * NAMESPACE: Defaults to current namespace in use.\n      * OBSERVATORIUM_TENANT:       Defaults to ``\n      * OBSERVATORIUM_GATEWAY:      Defaults to `https://observatorium-mst.api.stage.openshift.com`\n      * OBSERVATORIUM_AUTH_TYPE:    Defaults to `redhat`\n      * OBSERVATORIUM_RHSSO_URL:    Defaults to `https://sso.redhat.com/auth/`\n      * OBSERVATORIUM_RHSSO_REALM:  Defaults to `redhat-external`\n\n      * OBSERVATORIUM_RHSSO_METRICS_CLIENT_ID: **No default provided**\n      * OBSERVATORIUM_RHSSO_METRICS_SECRET: **No default provided**\n      * OBSERVATORIUM_RHSSO_LOGS_CLIENT_ID: **No default provided**\n      * OBSERVATORIUM_RHSSO_LOGS_SECRET: **No default provided**\n      * GITHUB_ACCESS_TOKEN: **No default provided**\n    * More information about configurable parameters can be found by running:\n    ```\n    oc process --parameters -f ./templates/secrets-template.yml\n    ```\n\n* For users deploying to CRC, an additional secret is required in the `openshift-monitoring` namespace for grafana datasources. This secret can be deployed with the command:\n  ```\n  make deploy/crc/secret\n  ```\n\n* Priority Class\n  * If the Observability Operator was not installed using OLM, you need to create the requried Priority Class yourself. Use the command to add the necessary Priority Class:\n  ```\n  oc apply -f config/samples/prioclass.yaml\n  ```\n\n### Running via `Makefile`\nIf this is the first time you've run against the cluster (or your CRD has changed and been uninstalled):\n```\nmake install\n```\n\nRun the operator as a local process: \n```\nWATCH_NAMESPACE=\u003cnamespace\u003e make run\n```\n\nalternatively, you can deploy the operator's latest image to your cluster:\n```\nmake deploy\n```\n\n### Running via IntelliJ\n![IntelliJ Debug Config](./readme-ide-run.png)\n\n* If you encounter errors regarding PrometheusRule validation webhooks, you can use the following:\n  ```\n  oc delete ValidatingWebhookConfiguration prometheusrules.openshift.io\n  ```\n\n### Running via VS Code\nAdd the following into your `launch.json` file to enable running and debugging.\n\n```     \n\"version\": \"0.2.0\",\n    \"configurations\": [{\n      \"name\": \"Observability Operator\",\n      \"type\": \"go\",\n      \"request\": \"launch\",\n      \"mode\": \"auto\",\n      \"program\": \"${workspaceFolder}/main.go\",\n      \"env\": {\n        \"WATCH_NAMESPACE\": \"kafka-observability\",\n        \"KUBERNETES_CONFIG\": \"~/.kube/config\",\n        \"OPERATOR_NAME\": \"observability-operator\"\n      },\n      \"cwd\": \"${workspaceFolder}\",\n      \"args\": []\n    }]\n  }\n```\n\n## Contributing\n\n1. Fork the Project\n2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`)\n3. Commit your Changes (`git commit -m 'Add some AmazingFeature'`)\n4. Push to the Branch (`git push origin feature/AmazingFeature`)\n5. Open a Pull Request\n\n\n## Issues\n\nIf you'd like to report an issue, feel free to use our [project Issues page](https://github.com/redhat-developer/observability-operator/v4/issues).\n\n\n## Contact\n\nIf all else fails, you can reach us at `mk-config-user@redhat.com`\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fredhat-developer%2Fobservability-operator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fredhat-developer%2Fobservability-operator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fredhat-developer%2Fobservability-operator/lists"}