https://github.com/frankchen021/bithon
An full stack observability platform
https://github.com/frankchen021/bithon
agent alerting apm diagnosis java logging metrics opentelemetry tracing
Last synced: about 1 month ago
JSON representation
An full stack observability platform
- Host: GitHub
- URL: https://github.com/frankchen021/bithon
- Owner: FrankChen021
- License: apache-2.0
- Created: 2021-03-09T08:39:02.000Z (about 4 years ago)
- Default Branch: master
- Last Pushed: 2025-04-10T15:04:59.000Z (about 1 month ago)
- Last Synced: 2025-04-11T01:13:23.941Z (about 1 month ago)
- Topics: agent, alerting, apm, diagnosis, java, logging, metrics, opentelemetry, tracing
- Language: Java
- Homepage:
- Size: 17.2 MB
- Stars: 20
- Watchers: 4
- Forks: 6
- Open Issues: 47
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://app.travis-ci.com/github/FrankChen021/bithon)
---
# Introduction
Bithon is a word combining binocular together with python.
It targets application metrics, logging, distributed tracing, alert and application risk governance under microservice environment.
## Architecture
The above pic illustrates the main components of this project, including:
- Agent, which collects metrics/tracing logs automatically from client application without any code modification at the application side
- Collector, which provides various interfaces (including OpenTelemetry GRPC interface) to receive metrics/tracing logs from clients
- Pipeline, which provides a flexible and robust way to hande small data scale to a very huge data scale for incoming metrics or tracing logs
- Storage, which provides an abstraction to underlying storages like H2, MySQL or Clickhouse
- Alerting, which allows us to set up alerts by using MetricSQL style expression on existing metrics or tracing logs
- Web, which provides a simple web portal for metrics/tracing log visualization
## Highlights
- Around 200 built-in metrics for various JDK or various Java middlewares like Apache Http Components
- Open Telemetry Tracing standard support and integration
- Built-in debugging diagnosis commands, including JMX bean realtime monitoring for target application
- Flexible deployment to adapt small data scale and huge data scale use cases
- Fast queries and very low storage cost benefit from ClickHouse
- PromQL style alerting expression support
Reference:
- [White Paper](doc/misc/white-paper.md)
- [How does the agent work?](doc/misc/rationale/index.md)
- [What's the difference between Jaeger and Bithon?](doc/misc/comparison/jaeger/index.md)
- [What's the difference between OpenTelemetry and Bithon?](doc/misc/comparison/opentelemetry/index.md)
# Preview
You can use the docker images in the Docker Hub to get a preview.
1. Start the backend service
```bash
docker run --network=host -e bithon_application_env=local -e JAVA_OPTS="-Dspring.profiles.active=all-in-one" -it bithon/server:latest
```
This starts the backend services with all-in-one profile, which includes all the services like tracing, event, metric, ctrl and alerting, using H2 database as back end storage.
And also the backend services is monitored by itself, you can see the metrics and tracing logs of the backend services itself.
2. Start the front-end service
```bash
docker run --network=host -e NEXT_PUBLIC_BITHON_API_SERVER_URL=http://localhost:9897 -itd bithon/web-app:latest
```
Change the environment variable `NEXT_PUBLIC_BITHON_API_SERVER_URL` to point to your own backend server if needed.
After the above steps, visit [http://localhost:3000](http://localhost:3000) to experience.
## Demo
A demo is provided by this [demo repo](https://github.com/FrankChen021/bithon-demo) with a docker-compose file.
You can follow the README on that demo repo to start the demo within just 3 steps.
# Build
## 1. Clone source code
After cloning this project along with all submodules by following commands
```bash
git clone https://github.com/FrankChen021/bithon.git
cd bithon && git submodule update --init
```
## 2. Configure JDK
JDK 17 and above are required to build this project.
If you have multiple JDKs on your machine, use `export JAVA_HOME={YOUR_JDK_HOME}` command to set correct JDK.
For example
```bash
export JAVA_HOME=/Library/Java/JavaVirtualMachines/openjdk-17.jdk/Contents/Home
```
## 3. Build the project
For the first time to build this project, use the following command to build dependencies first:
```bash
mvn clean install --activate-profiles shaded,jooq -T 1C
```
and then execute the following command to build the project.
```bash
mvn clean install -DskipTests -T 1C
```
After the first build, we don't need to build the dependencies anymore unless there are changes in these dependencies.
# Run
Once the project has been built, you could run the project in a standalone mode to evaluate this project.
## 1. Launch the backend services all in one
To launch server in evaluation mode, execute the following command:
```bash
java -Dspring.profiles.active=all-in-one -jar server/server-starter/target/server-starter.jar
```
By default, the application opens and listens on following ports at local
| Function | Port |
|----------|------|
| tracing | 9895 |
| event | 9896 |
| metric | 9898 |
| ctrl | 9899 |
| web | 9897 |
> Note:
> `-Dspring.profiles.include` parameter here is just for demo.
>
> You can make changes to `server/server-starter/src/main/resources/application.yml` to reflect your own settings.
>
> You can also use enable [Alibaba Nacos](doc/configuration/server/configuration-nacos.md) as your configuration storage center.
## 2. Attach agent to your target Java applications
Attach the agent to your java application so that your application can be managed the agent.
Add the the following VM arguments to your target Java application.
```bash
-javaagent:/agent/agent-distribution/target/agent-distribution/agent-main.jar -Dbithon.application.name= -Dbithon.application.env=
```
| Variable | Description |
|------------------------|--------------------------------------------------------------------------------------------------------------------------|
| YOUR_PROJECT_DIRECTORY | the directory where this project saves |
| YOUR_APPLICATION_NAME | the name of your application. It could be any string |
| YOUR_APPLICATION_ENV | the name of your environment to label your application. It could be any string. Usually it could be `dev`, `test`, `prd` |
By default, the agent connects collector running at local(127.0.0.1).
Collector address could be changed in file `agent/agent-main/src/main/resources/agent.yml`.
Make sure to re-build the project after changing the configuration file above.
# JDKs Compatibility
Even the project is built by JDK 17 and above, the agent is compatible with JDK 1.8+.
The following matrix lists the JDKs that are compatible with the agent on macOS.
And in theory, this matrix works both for Windows and Linux.
| JDK | Supported |
|---------------|-----------|
| JDK 1.8.0_291 | ✓ |
| JDK 9.0.4 | ✓ |
| JDK 10.0.2 | ✓ |
| JDK 11.0.12 | ✓ |
| JDK 12.0.2 | ✓ |
| JDK 13.0.2 | ✓ |
| JDK 14.0.2 | ✓ |
| JDK 15.0.2 | ✓ |
| JDK 16.02 | ✓ |
| JDK 17 | ✓ |
| JDK 21 | ✓ |
| JDK 22.0.2 | ✓ |
| JDK 23.0.2 | ✓ |
## JDK 11 and above
If the target application runs under JDK 11 and above, the following arguments should be added to JVM command to allow the agent to use Java Reflection on corresponding packages.
```text
--add-exports=java.base/jdk.internal.misc=ALL-UNNAMED --add-exports=java.base/sun.net.www=ALL-UNNAMED
```
# Supported Components
| Component | Min Version | Max Version | Metrics | Tracing |
|--------------------------------------------------------------|-------------|-------------|------------------------------------------------|---------|
| JVM | 1.8 | | ✓ | |
| JDK - Thread Pool | 1.8 | | ✓ | ✓ |
| JDK - HTTP Client | 1.8 | | [✓](doc/metrics/http-outgoing/README.md) | ✓ |
| Apache Druid(1) | 0.16 | 31.0 | | ✓ |
| Apache Kafka(2) | 0.10.0.0 | 3.9.0 | ✓ | ✓ |
| Apache OZone | 1.3.0 | | | ✓ |
| Apache ZooKeeper Client | 3.5 | 3.9 | ✓ | |
| Eclipse Glassfish | 2.34 | | | ✓ |
| GRPC | 1.57.0 | | ✓ | ✓ |
| Google Guice | 4.1.0 | | | ✓ |
| HTTP Client - Apache | 4.5.2 | 5.x | [✓](doc/metrics/http-outgoing/README.md) | ✓ |
| HTTP Client - Jetty | 9.4.6 | | [✓](doc/metrics/http-outgoing/README.md) | ✓ |
| HTTP Client - Netty | 3.10.6 | < 4.0 | [✓](doc/metrics/http-outgoing/README.md) | ✓ |
| HTTP Client - okhttp3 | 3.2 | 4.9 | [✓](doc/metrics/http-outgoing/README.md) | ✓ |
| HTTP Client - reactor-netty | 1.0.11 | | [✓](doc/metrics/http-outgoing/README.md) | ✓ |
| Jersey | 1.19.4 | | | ✓ |
| JDBC - Alibaba Druid | 1.0.28 | | ✓ | ✓ |
| JDBC - Apache Derby | 10.14.2 | | ✓ | ✓ |
| JDBC - H2 | 2.2.224 | | ✓ | ✓ |
| JDBC - MySQL | 5.x | 8.x | ✓ | ✓ |
| JDBC - PostgreSQL | 42.4.3 | | ✓ | ✓ |
| MongoDB | 3.4.2 | | ✓ | |
| Open Feign | 10.8 | | | ✓ |
| Quartz | 2.x | | ✓ | ✓ |
| Redis - Jedis | 2.9 | 5.x | ✓ | ✓ |
| Redis - Lettuce(3) | 5.1.2 | 6.x | ✓ | ✓ |
| Redis - Redisson | 3.19.0 | | ✓ | ✓ |
| Spring Boot | 1.5 | 3.0+ | | ✓ |
| [Spring Bean](doc/configuration/agent/plugin/spring-bean.md) | 4.3.12 | | | ✓ |
| Spring Open Feign | 10.8 | | | ✓ |
| Spring Rest Template | 4.3.12 | | | ✓ |
| Spring Scheduling | 4.3.12 | | | ✓ |
| Spring Gateway | 3.0.0 | | [✓](doc/metrics/http-outgoing/README.md) | ✓ |
| HTTP Server - Jetty | 9.4.41 | 12.0.x | ✓ | ✓ |
| HTTP Server - Netty | 2.0.0 | | | ✓ |
| HTTP Server - Tomcat | 8.5.20 | | ✓ | ✓ |
| HTTP Server - Undertow | 1.4.12 | | ✓ | ✓ |
| xxl-job | 2.3.0 | | | ✓ |
## Restrictions
1. For Apache Druid, the Jersey plugin is required to be enabled to collect query information.
2. From Apache Kafka clients 3.7, the consumer metrics only works when the `group.protocol` is configured as `classic` which is the default configuration of the consumer client.
3. For Lettuce, the tracing support is only available when it's used with Spring Data Redis API.
# User Doc
1. [Metrics](doc/metrics/README.md)
2. [Tracing](doc/tracing/README.md)
3. [Logging](doc/logging/README.md)
4. [Diagnosis](doc/diagnosis/README.md)
5. [Configuration](doc/configuration/configuration.md)
6. SDK
1. [Metrics](doc/sdk/metrics.md)
2. [Tracing](doc/sdk/tracing.md)
# Contribution
To develop for this project, intellij is recommended.
A code style template file(`dev/bithon_intellij_code_style`) must be imported into intellij for coding.
For more information, check the [development doc](doc/dev/development.md).
# License
[Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0)