{"id":13522363,"url":"https://github.com/TongchengOpenSource/ckibana","last_synced_at":"2025-03-31T22:31:18.269Z","repository":{"id":214296539,"uuid":"721536727","full_name":"TongchengOpenSource/ckibana","owner":"TongchengOpenSource","description":"Visualizing data in ClickHouse using native Kibana.","archived":false,"fork":false,"pushed_at":"2025-03-07T09:29:02.000Z","size":12082,"stargazers_count":294,"open_issues_count":6,"forks_count":41,"subscribers_count":16,"default_branch":"main","last_synced_at":"2025-03-27T06:08:26.246Z","etag":null,"topics":["clickhouse","clickhouse-visualization","elasticsearch","kibana","kibana-visualization"],"latest_commit_sha":null,"homepage":"https://tongchengopensource.github.io/ckibana-docs/","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TongchengOpenSource.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-21T09:12:07.000Z","updated_at":"2025-03-23T11:32:50.000Z","dependencies_parsed_at":"2024-03-11T05:21:35.510Z","dependency_job_id":"18d84554-4bd6-442b-8806-5275518be729","html_url":"https://github.com/TongchengOpenSource/ckibana","commit_stats":null,"previous_names":["tongchengopensource/ckibana"],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TongchengOpenSource%2Fckibana","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TongchengOpenSource%2Fckibana/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TongchengOpenSource%2Fckibana/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TongchengOpenSource%2Fckibana/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TongchengOpenSource","download_url":"https://codeload.github.com/TongchengOpenSource/ckibana/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246552410,"owners_count":20795816,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clickhouse","clickhouse-visualization","elasticsearch","kibana","kibana-visualization"],"created_at":"2024-08-01T06:00:46.101Z","updated_at":"2025-03-31T22:31:13.246Z","avatar_url":"https://github.com/TongchengOpenSource.png","language":"Java","funding_links":[],"categories":["Java","ClickHouse","Integrations"],"sub_categories":["Data Visualization and Analysis"],"readme":"![CKibana Logo (Light)](img/logo-black.png#gh-light-mode-only)\n![CKibana Logo (Dark)](img/logo-white.png#gh-dark-mode-only)\n\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n[![Release](https://img.shields.io/github/release/TongchengOpenSource/ckibana.svg?color=brightgreen\u0026label=Release)](https://github.com/TongchengOpenSource/ckibana/releases)\n\n**ClickHouse adapter for Kibana**\n\n**ClickHouse proxy for Kibana**\n\n**Clickhouse visualization tool**\n\n#### English | [简体中文](README_cn.md)\n\n---\n\n- [CKibana Best Practices](https://tongchengopensource.github.io/ckibana-docs/en/blog/build-logging-platform-with-native-kibana-and-clickhouse/)\n- [kibana-visualization](https://warmdust.github.io/2024/07/20/005/)\n\n## Introduce\n\nCKibana is a service for convenient analysis of ClickHouse data using native Kibana.\n\nA typical use case is when nginx log storage is migrated from ElasticSearch to ClickHouse,users can seamlessly continue their log query and analysis with Kibana.\n\nIt is recommended to use version 6.x of Kibana. This version is used internally and has been more thoroughly tested.\n\n![tlb.gif](img/tlb.gif)\n\n## Features\n\n- Version Support: Compatible with ElasticSearch and Kibana versions 6.x, 7.x（\u003c=7.1.1）.\n- Syntax Support: Compatible with common ElasticSearch syntax (note: ip_range and date_range can only be queried using the querystring syntax in the search box).\n- Sampling Function: For queries with hit results exceeding the threshold, it supports dynamic calculation of sampling and result restoration to improve query performance and ensure that the basic trend of the chart is consistent with the real data trend (the larger the flow control threshold, the closer the real chart trend is).\n- Cache Function: Supports using ElasticSearch to cache results to improve the performance of repeated queries.\n- Time Round Function: Supports rounding query time, for example, round 20 seconds (example: if the query time is in seconds 0-19, it will automatically round to 0; if it is in seconds 20-39, it will automatically round to 20 seconds, for example, if the query time is from 01:50:15 to 05:52:47, it will automatically adjust to 01:50:00 to 05:52:40). Used in conjunction with the cache function, it can effectively alleviate the pressure on ClickHouse caused by concurrent queries of the same statement by multiple users.\n- Blacklist Function: Supports setting a blacklist to prevent complex statements from causing pressure on ClickHouse.\n- Query Template Function: Supports syntax monitoring after removing time to facilitate quick problem location. Used in conjunction with the blacklist function, it can effectively ensure the stability of ClickHouse.\n- Query Circuit Breaker: Supports advanced features such as configuring the maximum query time range.\n\n#### Supported Aggregation Syntax:\n\n| ES Syntax                  | Note       |\n|-----------------------|------------|\n| terms                   |            |\n| sum                     |            |\n| min                     |            |\n| max                    |            |\n| avg                     |            |\n| percentile_ranks         |            |\n| percentiles             |            |\n| filters                 | Currently, only the first level of aggregation is supported. |\n| filter item             |            |\n| cardinality             |            |\n\n\n#### UnSupported  Syntax:\nKibana Query Language(KQL)  is not supported, it is recommended to use the lucene instead。\n\n![img_1.png](img_close_kql.png)\n\n## Get started\n### quick start\nTo quickly experience Kibana, you can deploy it quickly using docker-compose.\n\n[ckibana-quick-start-doc](https://github.com/TongchengOpenSource/ckibana/blob/main/docker-compose/README.md)\n\n```shell\n# Enter the docker-compose directory.\ncd ckibana/docker-compose\n# Deploy\ndocker-compose up -d\n```\nAfter deployment is complete, access Kibana in a browser at： http://127.0.0.1:5601/\n\nA process with built-in mock data has been included. After importing the Kibana configuration file (dashboard, index-pattern, etc.), you can start experiencing it!\n\nConfiguration file path：docker-compose/quickstart-export.json\n![](docker-compose/image/dashboard-import.jpg)\nEffect display：\n![](docker-compose/image/dashboard.jpg)\n\n\n\n### Local run CKibana\nCKibana can be run on all major operating systems and requires Java JDK version 17 or higher to be installed. To check, please run:`java -version`:\n\n```shell\n$ java -version\njava version \"17.0.5\" \n```\n\nThe proxy service depends on the **ES, CK, and Kibana services**, which need to be prepared in advance. The following steps only include querying data from CK and do not include writing data to CK.\n\n**1）Create a database and a table**\n\nTo initialize database and table structures in ClickHouse, please refer to:[api-docs](https://tongchengopensource.github.io/ckibana-docs/en/docs/api-docs) Detailed explanation of creating a table\n\n**2）Start CKibana**\n\nTo modify the ES configuration in CKibana, you need to change it to your own ES information. The configuration file path is `src/main/resources/application.yml`\n```yaml\nmetadata-config:\n  hosts: your es metadata cluster hosts\n  headers:\n    headerKey: yourHeaderValue\n```\nPackage and run or run locally directly `com.ly.ckibana.Bootstrap`\n```shell\n### start ckibana\n$ nohup java -jar ckibana.jar \u003e run.out 2\u003e\u00261 \u0026\n\n### check whether ckibana is successfully started\n$ tail -f ~/logs/app.log\nTomcat started on port(s): 8080 (http) with context path ''\nStarted Bootstrap in 1.474 seconds\n```\n\n**3）start kibana**\n\nConfigure the elasticsearchHosts parameter in Kibana to the CKibana proxy address, so that it can be routed to the proxy service.\n\n```shell\neg：elasticsearchHosts=http://ip:port\n```\n\n**4）Configure the index pattern whitelist**\n\nOnly after configuring the corresponding whitelist can you proceed to create it in the next step `index pattern`，**The configured whitelist needs to be consistent with the table name**，please refer to:[api-docs](https://tongchengopensource.github.io/ckibana-docs/en/api-docs) Update the whitelist index interface\n\n**5）Create `index pattern` in Kibana**\n\n```shell\neg：If the CK table name is `table_test`, then create an index pattern `table_test`\n```\n\nAfter completing the above 5 steps, you can create a dashboard and start using it.\n\n## TODO\n\n- Segmented query: Supports automatically splitting query time, which can fully utilize the cache and improve query performance.\n\n## Documentation\n[CKibana-docs](https://tongchengopensource.github.io/ckibana-docs)  \n\n## Contributors\n\n\u003ca href=\"https://github.com/TongchengOpenSource/ckibana/graphs/contributors\"\u003e\u003cimg src=\"https://opencollective.com/ckibana/contributors.svg?width=890\" /\u003e\u003c/a\u003e\n\n## Join us\nThe group supports both Enterprise WeChat and personal WeChat accounts\n![](http://oss.17usoft.com/infra-github/ckibana/join-us.png)\n\n## HelloGithub计划\n\u003ca href=\"https://hellogithub.com/repository/0bcbf56bab0140c7983ecd57eb3dddde\" target=\"_blank\"\u003e\u003cimg src=\"https://api.hellogithub.com/v1/widgets/recommend.svg?rid=0bcbf56bab0140c7983ecd57eb3dddde\u0026claim_uid=G2Xcw3hN9ZkEloL\" alt=\"Featured｜HelloGitHub\" style=\"width: 250px; height: 54px;\" width=\"250\" height=\"54\" /\u003e\u003c/a\u003e\n\n## License\n\n[Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0.html) Copyright (C) Apache Software Foundation\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FTongchengOpenSource%2Fckibana","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FTongchengOpenSource%2Fckibana","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FTongchengOpenSource%2Fckibana/lists"}