Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/duyet/clickhouse-monitoring
Simple ClickHouse UI that relies on system tables to help monitor and provide overview of your cluster
https://github.com/duyet/clickhouse-monitoring
clickhouse duyet hacktoberfest monitoring nextjs vercel
Last synced: 15 days ago
JSON representation
Simple ClickHouse UI that relies on system tables to help monitor and provide overview of your cluster
- Host: GitHub
- URL: https://github.com/duyet/clickhouse-monitoring
- Owner: duyet
- License: gpl-3.0
- Created: 2023-11-16T13:18:24.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-11-21T09:31:49.000Z (21 days ago)
- Last Synced: 2024-11-21T10:25:22.958Z (21 days ago)
- Topics: clickhouse, duyet, hacktoberfest, monitoring, nextjs, vercel
- Language: TypeScript
- Homepage: https://clickhouse-monitoring.vercel.app
- Size: 13.2 MB
- Stars: 101
- Watchers: 3
- Forks: 11
- Open Issues: 11
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-clickhouse - duyet/clickhouse-monitoring - A simple UI for monitoring ClickHouse using a dashboard built with Next.js. (Integrations / Data Visualization and Analysis)
README
# ClickHouse Monitoring Dashboard
[](https://github.com/duyet/clickhouse-monitoring/actions/workflows/ci.yml)
[](https://duyet.github.io/uptime/history/clickhouse-monitoring-vercel-app)
The simple Next.js dashboard that relies on `system.*` tables to help monitor and provide an overview of your ClickHouse cluster.
Features:
- Query monitor: current queries, query history, query resources (memory, parts read, file_open, ...), most expensive queries, most used tables or columns, etc.
- Cluster monitor: total memory/CPU usage, distributed queue, global settings, mergetree settings, metrics, etc.
- Tables and parts information: size, row count, compression, part size, etc., at the column level detail.
- Useful tools: Zookeeper data exploration, query EXPLAIN, kill queries, etc.
- Visualization metric charts: queries and resource usage, number of merges/mutation, merge performance, query performance, etc.
**Demo** (_The ClickHouse server running on my homelab so can be slow sometimes_):
- [clickhouse-monitoring.vercel.app](https://clickhouse-monitoring.vercel.app)
- [clickhouse.duyet.net](https://clickhouse.duyet.net)
**Documentation**:
- https://duyet.github.io/clickhouse-monitoring/
**Screenshots**:
## Getting Started
To get the project up and running on your local machine, follow these steps:
1. Clone the repository
2. Install dependencies using `npm install` or `yarn install`
3. Create a `.env.local` file by copying the `.env.example` file and filling in the required environment variables:
- `CLICKHOUSE_HOST`: ClickHouse host(s), for example `http://localhost:8123` or `http://ch-1:8123,http://ch-2:8123`
- `CLICKHOUSE_NAME`: (Optional) Name of ClickHouse instance, must match the number of hosts in `CLICKHOUSE_HOST`, for example `localhost` or `ch-1,ch-2`.
- `CLICKHOUSE_USER`: ClickHouse user with permission to query the `system` database.
- `CLICKHOUSE_PASSWORD`: ClickHouse password for the specified user.
- `CLICKHOUSE_MAX_EXECUTION_TIME`: [`max_execution_time`](https://clickhouse.com/docs/en/operations/settings/query-complexity#max-execution-time) timeout in seconds. Default is `10`.
- `CLICKHOUSE_TZ`: ClickHouse server timezone. Default: `''`.
- `NEXT_QUERY_CACHE_TTL`: TTL of [`unstable_cache`](https://nextjs.org/docs/app/api-reference/functions/unstable_cache) - cache the results of most charts to speed up and reuse them across multiple requests. Default: `86400`.
- `NEXT_PUBLIC_LOGO`: (Optional) HTTP path to logo image.
- `EVENTS_TABLE_NAME`: The table name for storing dashboard self-tracking events. Default: `system.monitoring_events`
4. Run the development server with `npm run dev` or `yarn dev`
5. Open [http://localhost:3000](http://localhost:3000) in your browser to see the dashboard.
## ClickHouse Requirements
### 1. Monitoring user role
Suggested role for "monitoring" user must have these privileges on `system` database:
```xml
# File: users.d/monitoring_role.xml
monitoring_profile
::/0
GRANT monitoring_role
REVOKE ALL ON *.*
GRANT SELECT,SHOW,dictGet,REMOTE ON *.*
GRANT SELECT,INSERT,ALTER,CREATE,DROP,TRUNCATE,OPTIMIZE,SHOW,dictGet ON system.*
GRANT CREATE TEMPORARY TABLE ON *.*
```
`CREATE TEMPORARY TABLE` is needed because the UI using `FROM merge(system, '^query_log')` allows retrieving all the data from old tables that were renamed during the upgrade.
### 2. Monitoring user profile
```xml
# File: users.d/monitoring_profile.xml
1
1
50
0
save
save
```
## Deployment
### 1. Vercel
For easy deployment, use the [Vercel Platform](https://vercel.com/new?utm_medium=default-template&filter=next.js&utm_source=create-next-app&utm_campaign=create-next-app-readme), created by the makers of Next.js. Refer to the [Next.js deployment documentation](https://nextjs.org/docs/deployment) for more details.
### 2. Docker
Using the latest image here: https://github.com/duyet/clickhouse-monitoring/pkgs/container/clickhouse-monitoring
```bash
docker run -it \
-e CLICKHOUSE_HOST='http://localhost' \
-e CLICKHOUSE_USER='default' \
-e CLICKHOUSE_PASSWORD='' \
-e CLICKHOUSE_TZ='Asia/Ho_Chi_Minh' \
-e CLICKHOUSE_MAX_EXECUTION_TIME='15' \
-e NEXT_QUERY_CACHE_TTL='86400' \
--name clickhouse-monitoring \
ghcr.io/duyet/clickhouse-monitoring:main
```
### 3. Kubernetes Helm Chart
Using the latest helm chart here: https://github.com/duyet/charts/tree/master/clickhouse-monitoring
```bash
helm repo add duyet https://duyet.github.io/charts
cat <> values.yaml
env:
- name: CLICKHOUSE_HOST
value: http://localhost:8123
- name: CLICKHOUSE_USER
value: default
- name: CLICKHOUSE_PASSWORD
value: ''
- name: CLICKHOUSE_TZ
value: 'Asia/Ho_Chi_Minh'
- name: CLICKHOUSE_MAX_EXECUTION_TIME
value: '15'
- name: NEXT_QUERY_CACHE_TTL
value: '86400'
EOF
helm install -f values.yaml clickhouse-monitoring-release duyet/clickhouse-monitoring
```
## Feedback and Contributions
Feedback and contributions are welcome! Feel free to open issues or submit pull requests.
## License
See [LICENSE](LICENSE).
---
