{"id":27643297,"url":"https://github.com/matrixorigin/observability-charts","last_synced_at":"2026-02-27T17:03:53.352Z","repository":{"id":219676253,"uuid":"749620188","full_name":"matrixorigin/observability-charts","owner":"matrixorigin","description":null,"archived":false,"fork":false,"pushed_at":"2025-01-24T10:18:11.000Z","size":15321,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-24T00:12:00.658Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Mustache","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/matrixorigin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-29T03:43:08.000Z","updated_at":"2025-01-13T03:01:51.000Z","dependencies_parsed_at":"2024-05-10T05:21:04.728Z","dependency_job_id":"21f49d82-dc13-4935-b85d-d6e8ca794c1d","html_url":"https://github.com/matrixorigin/observability-charts","commit_stats":null,"previous_names":["matrixone-cloud/observability-charts","matrixorigin/observability-charts"],"tags_count":31,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matrixorigin%2Fobservability-charts","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matrixorigin%2Fobservability-charts/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matrixorigin%2Fobservability-charts/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matrixorigin%2Fobservability-charts/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/matrixorigin","download_url":"https://codeload.github.com/matrixorigin/observability-charts/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250535106,"owners_count":21446508,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-24T00:12:18.158Z","updated_at":"2026-02-27T17:03:48.316Z","avatar_url":"https://github.com/matrixorigin.png","language":"Mustache","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Observability\nMOCloud Observability Charts\n\n\n## 在已有k8s集群上部署mo-ob\n\n### 添加 Helm 仓库\n\n添加 Helm 仓库\n```\nhelm repo add mo-ob https://matrixorigin.github.io/observability-charts\n```\n更新仓库\n```\nhelm repo update\n```\n查看版本\n```\nhelm search repo mo-ob/mo-ruler-stack --versions --devel\nhelm search repo mo-ob/mo-ob-opensource --versions --devel\n```\n\n### 设置环境变量\n\n请指定 chart 版本 MO_RULER_STACK_VERSION 和 MO_OB_OPENSOURCE_VERSION\n\n```\nOBNS=mo-ob\nS3_ENDPOINT=\u003cyour-s3-endpoint\u003e\nS3_ACCESS_KEY=\u003cyour-s3-access-key\u003e\nS3_SECRET_KEY=\u003cyour-s3-secret-key\u003e\nS3_BUCKET=\u003cyour-bucket-name\u003e\nSTORAGE_CLASS=\u003cyour-storage-class\u003e\nPROM_STORAGE_SIZE=10Gi\nGRAFANA_USER=\u003cyour-admin-user\u003e\nGRAFANA_PWD=\u003cyour-grafana-pwd\u003e\nMO_RULER_STACK_VERSION=\u003chelm version\u003e\nMO_OB_OPENSOURCE_VERSION=\u003chelm version\u003e\n```\n\n### 部署 mo-ruler-stack\n安装\n\n```\nkubectl create namespace mo-ob\n\nhelm install -n ${OBNS} \\\n    --set grafana.persistence.storageClassName=${STORAGE_CLASS} \\\n    --set grafana.service.type=\"NodePort\" \\\n    --set grafana.adminUser=${GRAFANA_USER} \\\n    --set grafana.adminPassword=${GRAFANA_PWD} \\\n    --set alertmanager.persistence.enabled=\"false\" \\\n    mo-ruler-stack mo-ob/mo-ruler-stack --version ${MO_RULER_STACK_VERSION}\n```\n\n卸载\n\n```\nhelm uninstall -n ${OBNS} mo-ruler-stack\n```\n\n### 部署 mo-ob-opensource\n安装\n\n```\nhelm install -n ${OBNS} \\\n    --set loki.loki.storage.bucketNames.chunks=${S3_BUCKET} \\\n    --set loki.loki.storage.s3.endpoint=${S3_ENDPOINT} \\\n    --set loki.loki.storage.s3.accessKeyId=${S3_ACCESS_KEY} \\\n    --set loki.loki.storage.s3.secretAccessKey=${S3_SECRET_KEY} \\\n    --set loki.write.persistence.storageClass=${STORAGE_CLASS} \\\n    --set loki.write.replicas=2 \\\n    --set loki.write.resources.requests.memory=\"500Mi\" \\\n    --set loki.write.resources.requests.cpu=\"250m\" \\\n    --set loki.read.persistence.storageClass=${STORAGE_CLASS} \\\n    --set loki.read.resources.requests.memory=\"1Gi\" \\\n    --set loki.read.resources.requests.cpu=\"250m\" \\\n    --set loki.backend.persistence.storageClass=${STORAGE_CLASS} \\\n    --set loki.backend.resources.requests.memory=\"500Mi\" \\\n    --set loki.backend.resources.requests.cpu=\"250m\" \\\n    --set kube-prometheus-stack.prometheus.prometheusSpec.storageSpec.volumeClaimTemplate.spec.storageClassName=${STORAGE_CLASS} \\\n    --set kube-prometheus-stack.prometheus.prometheusSpec.storageSpec.volumeClaimTemplate.spec.resources.requests.storage=${PROM_STORAGE_SIZE} \\\n    --set kube-prometheus-stack.prometheus.prometheusSpec.resources.requests.memory=\"1Gi\" \\\n    --set kube-prometheus-stack.prometheus.prometheusSpec.resources.requests.cpu=\"250m\" \\\n    mo-ob-opensource ./charts/mo-ob-opensource --version ${MO_OB_OPENSOURCE_VERSION}\n```\n\n卸载\n\n```\nhelm uninstall -n ${OBNS} mo-ob-opensource\n```\n\n### 部署 dashboard-chart\n\nbuild\n\n```\ngit clone https://github.com/matrixorigin/ob-ops\nmake ctrl-res\n\n```\nmake之后将会生成 dashboard-chart\n\n安装\n\n\u003cpath-to-chart\u003e 是上面make的dashboard-chart\n\n```\nhelm install -n ${OBNS} controlplane-resources-chart \\\n--set policies.log.enabled=\"false\" \\\n--set policies.metric.enabled=\"false\" \\\n--set rules.log.enabled=\"false\" \\\n--set rules.metric.enabled=\"false\" \\\n\u003cpath-to-chart\u003e\n```\n\n卸载\n\n```\nhelm uninstall -n ${OBNS} controlplane-resources-chart\n```\n\n###\n\n获取grafana账号\n\n```\nkubectl get secret -n ${OBNS} grafana-admin-secret  -o jsonpath=\"{.data['admin-user']}\" | base64 -d\n```\n\n获取grafana密码\n\n```\nkubectl get secret -n ${OBNS} grafana-admin-secret  -o jsonpath=\"{.data['admin-password']}\" | base64 -d\n```\n\n\n# 进阶配置\n\n\n## mo-ob-opensource 从 1.0.8 升级到 1.0.9+\n\n从 1.0.9 开始引入了 victoria-metrics-k8s-stack 组件，如果需要从现有的 mo-ob 升级，则为了避免 [#4699](https://github.com/matrixorigin/MO-Cloud/issues/4699) 的问题，在 upgrade 前需要先手动 apply vm 的 crd\n\n\n![alt text](docs/chart/images/image.png)\n\netc/crds/vm-crd.yaml 来自 victoria-metrics-operator 0.40.1 内，且已经以在 namespace=mo-ob，release=mo-ob-opensource 下设置好了相应的 annotate 和 label，可以直接 apply，如上图所示，如果有定制需求，需要修改上图中的 namespace 和 release 的值\n\n\n```\nkubectl apply -f etc/crds/vm-crd.yaml\n```\n\n## 需要替换的镜像源\n\n如果部署的时候发现 image pull failed 等错误，需要替换镜像源（如阿里云），这些被替换的镜像都已经显示的写在了 chart 的 values.yaml 下，如 `charts/mo-ob-opensource/values.yaml` 下\n\n```\nalloy:\n  image:\n    registry: \"docker.io\"\n    repository: grafana/alloy\n    tag: v1.3.1\n```\n将对应的镜像 push 到需要替换的镜像源仓库即可部署\n\n\n## alertmanger 打开 web 鉴权\n\n1.在 `charts/mo-ruler-stack/values.yaml` 下设置 secretValue.alertmanager，alertmanager_web_auth_password_bcrypted 是 alertmanager_web_auth_password 的 bcrypt 加密\n\n```\n# secret value to create secret automatically\nsecretValue:\n  alertmanager: \n    # see: https://prometheus.io/docs/alerting/0.25/https\n    alertmanager_web_auth_user: admin\n    alertmanager_web_auth_password: admin\n    # need to be bcrypted, in bash: htpasswd -bnBC 10 \"\" \u003calertmanager_web_auth_password\u003e | tr -d ':\\n'\n    alertmanager_web_auth_password_bcrypted: $2y$10$Z3zgfm2IIeQqNmGWeqsrSecRuRmo/EAh4Srn0Mi0fG98dJZMn7RTS\n```\n\n2.在 `charts/mo-ruler-stack/values.yaml` 下启用 web.config.file:\n```\nalertmanager:\n  extraArgs:\n    web.config.file: /tmp/alertmanager-web-config/alertmanager-web-config.yaml\n```\n\n\n## 开启 alertmanager 鉴权与 alertmanager ha集群模式\n\n需要修改以下配置：\n\n1.在 `charts/mo-ruler-stack/values.yaml` 下修改 replicaCount:\n```\nalertmanager:\n  replicaCount: 3\n```\n\n\n2.在 `charts/mo-ob-opensource/values.yaml` 下修改 prometheus 的 alertingEndpoints 启用多个 alertmanager\n\n```\nkube-prometheus-stack:\n  prometheus:\n    pometheusSpec:\n      alertingEndpoints:\n      - name: \"mo-ob-alertmanager-0\"\n      - name: \"mo-ob-alertmanager-1\"\n      - name: \"mo-ob-alertmanager-2\"\n```\n\n3.在 `charts/mo-ob-opensource/values.yaml` 下修改 loki 的 alertmanager_url 启用多个 alertmanager\n\n```\nloki:\n  loki:\n    rulerConfig:\n      alertmanager_url: http://mo-ob-alertmanager-0.mo-ob:9093,http://mo-ob-alertmanager-1.mo-ob:9093,http://mo-ob-alertmanager-2.mo-ob:9093\n```\n\n即可启用 alertmanager ha集群\n\n# Scrape\n\n[Scrape List](./docs/scrape/README.md) \n\n## 如何添加新的采集任务\n\n 在业务代码中引入prometheus的指标抓取接口，详情请参考：[业务 metric 采集接入](https://github.com/matrixone-cloud/observability-charts/wiki/%E4%B8%9A%E5%8A%A1-metric-%E9%87%87%E9%9B%86%E6%8E%A5%E5%85%A5)\n\n为了便于prometheus的服务发现，在k8s上需要部署组件相对应的 `service` （推荐），部署好service后，可以去相应集群的grafana页面中看看是否已经有开始采集到数据（可能会有2-3分钟的延迟），不同集群的grafana环境以及账号请见：[Grafana 地址列表](https://doc.weixin.qq.com/doc/w3_AW0A-gb6AOIAWdUX2NbSWevRb4vhF?scode=AJsA6gc3AA8iTHdq3jAW0A-gb6AOI)\n\n\n# Alerts\n\n- [Alerts List](./docs/alerts/README.md)\n\n\n值得注意的是，所有在上面流程创建的新文件都需要在在 `.github/CODEOWNERS` 下以 `[文件名] [@github名]` 标注文件的 owner\n\n## 如何提交新的告警规则\n\n为了为你的应用接入监控并告警，需要完成以下工作：\n\n 1. 业务端暴露 /metrics 接口接入采集：[如何添加新的采集任务](#%E5%A6%82%E4%BD%95%E6%B7%BB%E5%8A%A0%E6%96%B0%E7%9A%84%E9%87%87%E9%9B%86%E4%BB%BB%E5%8A%A1)\n 2. 编写告警规则 \u0026 告警单元测试并验证：[编写告警规则与告警单元测试](https://github.com/matrixone-cloud/observability-charts/wiki/MO%E2%80%90OB-告警接入操作流程#3-编写告警规则与告警单元测试)\n 3. [可选] 添加 Alertmanager Receiver 配置并验证：[Alertmanager Receiver 配置与验证](https://github.com/matrixone-cloud/observability-charts/wiki/MO%E2%80%90OB-告警接入操作流程#4--可选-alertmanager-receiver-配置与验证)\n 4. [可选] Grafana Dashboard 本地调试：[Grafana 本地调试](https://github.com/matrixone-cloud/observability-charts/wiki/MO%E2%80%90OB-告警接入操作流程#4--可选-添加-grafana-dashboard-并本地调试)\n 5. [可选] 添加 Grafana Dashboard 信息：[如何提交新的 Dashboards 配置](#%E5%A6%82%E4%BD%95%E6%8F%90%E4%BA%A4%E6%96%B0%E7%9A%84-dashboards-%E9%85%8D%E7%BD%AE)\n 6. 提交 PR ，根据 PR 模板添加相关 README 说明\n\n详细 Workaround：[MO‐OB 告警接入操作流程](https://github.com/matrixone-cloud/observability-charts/wiki/MO%E2%80%90OB-%E5%91%8A%E8%AD%A6%E6%8E%A5%E5%85%A5%E6%93%8D%E4%BD%9C%E6%B5%81%E7%A8%8B)\n\n# Dashboards\n- [details](./docs/dashboards)\n\n- [dashboard config list](./charts/mo-ruler-stack/grafana/dashboards/README.md)\n\n## 如何提交新的 Dashboards 配置\n1. 绘制: 请在 grafana web UI上绘制你的 dashboard\n2. 下载dashboard.json配置: 请在 dashboard展示页 =\u003e setting =\u003e JSON Model =\u003e Save as =\u003e save on your PC.\n3. 提交PR: 请将 dashboard.json 提交至目录 [charts/mo-ruler-stack/grafana/dashboards](./charts/mo-ruler-stack/grafana/dashboards), 并追加 [dashboards list](./charts/mo-ruler-stack/grafana/dashboards/README.md)\n\n提交前请注意以下细节\n1. dashboard 标题, 推荐使用 `{模块}/{分类 or 服务}`\n    - 模块 取值: [ kubernetes, Node Exporter, Prometheus, MOCloud ]\n    - 分类 or 服务:  可参考目前取值 [dashboards list](./charts/mo-ruler-stack/grafana/dashboards/README.md)\n2. dashboard.json 文件名, 同样推荐使用 `{模块}/{分类 or 服务}.json`\n    - 例如: moc-auth-service.json 对应: `MOCloud / Auth-Service` metric指标\n    - 详见目前支持的 [dashboards list](./charts/mo-ruler-stack/grafana/dashboards/README.md)\n3. 在 `.github/CODEOWNERS` 下以 `[文件名] [@github名]` 标注dashboard文件的创建人\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatrixorigin%2Fobservability-charts","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmatrixorigin%2Fobservability-charts","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatrixorigin%2Fobservability-charts/lists"}