{"id":29899237,"url":"https://github.com/bilbyai/observability","last_synced_at":"2026-05-15T20:06:21.122Z","repository":{"id":305125618,"uuid":"1021192902","full_name":"bilbyai/observability","owner":"bilbyai","description":"Documentation on our observability dashboards","archived":false,"fork":false,"pushed_at":"2025-07-18T09:53:15.000Z","size":8291,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-18T13:39:00.280Z","etag":null,"topics":["documentation","elasticsearch","excalidraw","kibana"],"latest_commit_sha":null,"homepage":"https://bilby.kb.asia-southeast1.gcp.elastic-cloud.com/s/official-china/app/dashboards#/list","language":"Shell","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bilbyai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-17T03:25:40.000Z","updated_at":"2025-07-18T09:53:19.000Z","dependencies_parsed_at":"2025-07-18T13:39:53.822Z","dependency_job_id":"4996056f-6b09-4680-9318-9b83de05b949","html_url":"https://github.com/bilbyai/observability","commit_stats":null,"previous_names":["bilbyai/observability"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/bilbyai/observability","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bilbyai%2Fobservability","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bilbyai%2Fobservability/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bilbyai%2Fobservability/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bilbyai%2Fobservability/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bilbyai","download_url":"https://codeload.github.com/bilbyai/observability/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bilbyai%2Fobservability/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268221879,"owners_count":24215696,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-01T02:00:08.611Z","response_time":67,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["documentation","elasticsearch","excalidraw","kibana"],"created_at":"2025-08-01T12:19:32.317Z","updated_at":"2026-05-15T20:06:16.087Z","avatar_url":"https://github.com/bilbyai.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Data Observability on Elastic and Kibana\n\nThis is a documentation repository that provides information about how to access\nour data through Kibana, as well as observability dashboards that monitor,\nexplore, and ensure the quality of our data pipelines built on Elastic and\nKibana.\n\n## Overview\n\nOur observability solution consists of three dashboards and the Kibana Discover\npage designed to provide visibility into our data:\n\n| Page                                                                                                                                                                                                                                                | Documentation Link                                                                   | Purpose                                      | Description                                                                                                    |\n| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ | -------------------------------------------- | -------------------------------------------------------------------------------------------------------------- |\n| [Pipeline Health Dashboard](\u003chttps://bilby.kb.asia-southeast1.gcp.elastic-cloud.com/s/official-china/app/dashboards#/view/7770cea6-8100-4b25-ac13-be8f265624ad?_g=(filters:!(),refreshInterval:(pause:!t,value:60000),time:(from:now-2w,to:now))\u003e)  | [Documentation](https://github.com/bilbyai/observability/tree/main/pipeline-health)  | Quickly see the status of our pipelines.     | This dashboard answers the critical question within 30 seconds: \"Is the pipeline running ok?\"                  |\n| ~~Data QA Dashboard~~                                                                                                                                                                                                                               | (Not implemented yet due to lack of data)                                            | Determine the quality of our data pipelines. | This dashboard will help answer: \"Is the data produced by the pipeline ok?\"                                    |\n| [Data Exploration Dashboard](\u003chttps://bilby.kb.asia-southeast1.gcp.elastic-cloud.com/s/official-china/app/dashboards#/view/8288744a-964d-46c0-af9e-749ddea5ff51?_g=(filters:!(),refreshInterval:(pause:!t,value:60000),time:(from:now-4w,to:now))\u003e) | [Documentation](https://github.com/bilbyai/observability/tree/main/data-exploration) | Explore and understand our data.             | Get insights into data patterns, distributions, and characteristics to answer: \"What does the data look like?\" |\n| [Kibana Discover](https://bilby.kb.asia-southeast1.gcp.elastic-cloud.com/s/official-china/app/discover#/)                                                                                                                                           | [User Guide](https://github.com/bilbyai/observability/tree/main/kibana-discover)     | For in-depth investigations                  | This is used to find specific data points via filters.                                                         |\n\n\u003e 🚨 For in-depth investigations, please use\n\u003e [Kibana Discover](https://bilby.kb.asia-southeast1.gcp.elastic-cloud.com/s/official-china/app/discover#/)\n\u003e to directly query the data.\n\n### 1. ⚠️ Pipeline Health Dashboard\n\nMonitor the operational status of your data pipelines in real-time.\n\nThis dashboard answers the critical question: \"Is the pipeline running ok?\"\n\n### 2. 🎯 Data QA Dashboard (Coming Soon)\n\nEnsure data quality by validating the output of your pipelines.\n\nThis dashboard will help answer: \"Is the data produced by the pipeline ok?\"\n\n### 3. 📊 Data Exploration Dashboard\n\nExplore and understand your data through interactive visualizations. Get\ninsights into data patterns, distributions, and characteristics to answer: \"What\ndoes the data look like?\"\n\n### 4. 🔍 Kibana Discover\n\nFor detailed data exploration and custom queries, use\n[Kibana Discover](https://bilby.kb.asia-southeast1.gcp.elastic-cloud.com/s/official-china/app/discover#/)\nto directly search and analyze your data.\n\n## Architecture\n\nWe've decided to go with Elasticsearch and Kibana for the observability stack.\nThis allows us to create visualizations with our date pretty easily, and data\nanalysts and non-technical users can build visualizations on their own without\nneeding to think about data pipeline and infrastructure.\n\n## Contributing\n\nPlease refer to individual dashboard documentation for specific implementation\ndetails, as shown above.\n\nIf you'd like to build your own dashboard, please feel free! Tell us about it!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbilbyai%2Fobservability","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbilbyai%2Fobservability","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbilbyai%2Fobservability/lists"}