https://github.com/guancecloud/datakit
DataKit is collection agent for Guance Cloud
https://github.com/guancecloud/datakit
datakit go
Last synced: about 1 year ago
JSON representation
DataKit is collection agent for Guance Cloud
- Host: GitHub
- URL: https://github.com/guancecloud/datakit
- Owner: GuanceCloud
- License: mit
- Created: 2021-05-31T11:35:13.000Z (about 5 years ago)
- Default Branch: main
- Last Pushed: 2025-03-27T00:56:14.000Z (about 1 year ago)
- Last Synced: 2025-04-12T01:07:54.624Z (about 1 year ago)
- Topics: datakit, go
- Language: Go
- Homepage:
- Size: 159 MB
- Stars: 273
- Watchers: 13
- Forks: 53
- Open Issues: 15
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://app.slack.com/client/T032YB4B6TA/)
[](LICENSE)
[](https://app.fossa.com/projects/git%2Bgithub.com%2FGuanceCloud%2Fdatakit?ref=badge_shield)
Website
•
Doc
## _Read this in other languages._
[
](README.zh_CN.md)
DataKit is an open source, integrated data collection agent, which provides full platform (Linux/Windows/macOS) support and has comprehensive data collection capability, covering various scenarios such as host, container, middleware, tracing, logging and security inspection.
## Key Features
- Support collection of metrics, logging and tracing
- Fully support Kubernetes ecology
- [Pipeline](https://docs.guance.com/pipeline/use-pipeline): Simple structured data extraction
- Supports third-party data import:
- [Telegraf](https://docs.guance.com/integrations/telegraf)
- [Prometheus](https://docs.guance.com/integrations/prom)
- [Statsd](https://docs.guance.com/integrations/statsd)
- [Fluentd](https://docs.guance.com/integrations/logstreaming-fluentd)
- [Filebeats](https://docs.guance.com/integrations/beats_output)
- Tracing
- [OpenTelemetry](https://docs.guance.com/integrations/opentelemetry)
- [DDTrace](https://docs.guance.com/integrations/ddtrace)
- [Zipkin](https://docs.guance.com/integrations/zipkin)
- [Jaeger](https://docs.guance.com/integrations/jaeger)
- [Skywalking](https://docs.guance.com/integrations/skywalking)
## Changelog
All DataKit changelog refers to [here](https://docs.guance.com/datakit/changelog).
## Minimal Requirements
| OS | Arch | Install Path |
| --- | --- | --- |
| Linux Kernel 2.6.23+ | amd64/386/arm/arm64 | `/usr/local/datakit` |
| macOS 10.12+([Why](https://github.com/golang/go/issues/25633)) | amd64 | `/usr/local/datakit` |
| Windows 7+/Server 2008R2+ | amd64/386 | 64-bit：`C:\Program Files\datakit`
32-bit：`C:\Program Files(32)\datakit` |
## Install DataKit
We can directly obtain the DataKit installation command from [guance cloud](http://guance.com). Most of the installation commands seems like that:
- Linux & Mac
```shell
DK_DATAWAY="https://openway.guance.com?token=" bash -c "$(curl -L https://static.guance.com/datakit/install.sh)"
```
- Windows
```powershell
Remove-Item -ErrorAction SilentlyContinue Env:DK_*;
$env:DK_DATAWAY="https://openway.guance.com?token=";
Set-ExecutionPolicy Bypass -scope Process -Force;
Import-Module bitstransfer;
start-bitstransfer -source https://static.guance.com/datakit/install.ps1 -destination .install.ps1;
powershell .install.ps1;
Remove-Item .install.ps1;
```
- [Kubernetes DaemonSet](https://docs.guance.com/datakit/datakit-daemonset-deploy)
For more documentations about DataKit installation, see [here](https://docs.guance.com/datakit/datakit-install).
## Build From Source
DataKit building relies on some external tools/libs, we must install them all before compile the source code.
> - **We recommend to build source on Ubuntu 20.04+**, other linux distribition may failed to install these dependencies. We do not support build DataKit on Windows.
> - Please build the project with `make`, we haven't testing with Golang/VSCode IDEs
### Setup Golang
Install and setup Golang(1.18.3+):
```shell
export GOPRIVATE=gitlab.jiagouyun.com/*
export GOPROXY=https://goproxy.cn,direct
export GOPATH=~/go # depends on your local settings
export GOROOT=~/golang-1.18.3 # depends on your local settings
export PATH=$GOROOT/bin:~/go/bin:$PATH
```
### Install other tools
> !!! Do not install these dependencies under datakit source code dir.
- make: `apt-get install make`
- gcc: `apt-get install gcc`
- gcc-multilib: `apt-get install -y gcc-multilib`
- tree: `apt-get install tree`
- packr2: `go install github.com/gobuffalo/packr/v2/packr2@v2.8.3`
- goyacc: `go install golang.org/x/tools/cmd/goyacc@latest`
- lint related:
- lint: `go install github.com/golangci/golangci-lint/cmd/golangci-lint@v1.46.2`
- eBPF related:
- clang 10.0+: `apt-get install clang`
- llvm 10.0+: `apt-get install llvm`
- kernel headers
- apt: `apt-get install -y linux-headers-$(uname -r)`
### Build
1. Clone code
```shell
$ mkdir -p $GOPATH/src/gitlab.jiagouyun.com/cloudcare-tools
$ cd $GOPATH/src/gitlab.jiagouyun.com/cloudcare-tools
$ git clone https://github.com/GuanceCloud/datakit.git # may be blocked by GFW
$ git clone https://jihulab.com/guance-cloud/datakit.git # jihulab mirror
$ cd datakit
```
2. Building
```shell
make
```
If building ok, all binaries are generated under *dist*:
```
dist
├── [4.0K] datakit-linux-amd64
│ ├── [ 72M] datakit
│ └── [4.0K] externals
│ ├── [ 14M] logfwd
│ └── [10.0M] oracle
├── [4.0K] local
│ ├── [ 26M] installer-linux-amd64
│ └── [ 228] version
└── [4.0K] standalone
└── [4.0K] datakit-ebpf-linux-amd64
└── [ 38M] datakit-ebpf
```
We can build all platforms(Linux/Mac/Windows) with following command:
```shell
make testing
```
## Basic Usage
We can use `help` command to see more usage of DataKit:
```shell
datakit help
# Or
./dist/datakit-linux-amd64/datakit help
```
## Contributing
Before contributing, check out some guideline of DataKit:
- Read [architecure introduciton](https://docs.guance.com/datakit/datakit-arch)
- Read [development guideline](https://docs.guance.com/datakit/development)
## Full Documentation
For full documents of DataKit, see
- [DataKit Doc](https://docs.guance.com/datakit)
## License
[](https://app.fossa.com/projects/git%2Bgithub.com%2FGuanceCloud%2Fdatakit?ref=badge_large)