{"id":20254601,"url":"https://github.com/guancecloud/datakit","last_synced_at":"2025-04-12T01:08:31.998Z","repository":{"id":40995649,"uuid":"372484538","full_name":"GuanceCloud/datakit","owner":"GuanceCloud","description":"DataKit is collection agent for Guance Cloud","archived":false,"fork":false,"pushed_at":"2025-03-27T00:56:14.000Z","size":166721,"stargazers_count":273,"open_issues_count":15,"forks_count":53,"subscribers_count":13,"default_branch":"main","last_synced_at":"2025-04-12T01:07:54.624Z","etag":null,"topics":["datakit","go"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/GuanceCloud.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-05-31T11:35:13.000Z","updated_at":"2025-04-09T09:36:15.000Z","dependencies_parsed_at":"2024-01-16T08:08:14.928Z","dependency_job_id":"ecc88875-3dc9-4d81-9305-9c4fde294073","html_url":"https://github.com/GuanceCloud/datakit","commit_stats":null,"previous_names":[],"tags_count":203,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GuanceCloud%2Fdatakit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GuanceCloud%2Fdatakit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GuanceCloud%2Fdatakit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GuanceCloud%2Fdatakit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/GuanceCloud","download_url":"https://codeload.github.com/GuanceCloud/datakit/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248501863,"owners_count":21114684,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["datakit","go"],"created_at":"2024-11-14T10:33:53.746Z","updated_at":"2025-04-12T01:08:26.988Z","avatar_url":"https://github.com/GuanceCloud.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg alt=\"datakit logo\" src=\"datakit-logo.png\" height=\"150\" /\u003e\n\u003c/p\u003e\n\n[![Slack Status](https://img.shields.io/badge/slack-join_chat-orange?logo=slack\u0026style=plastic)](https://app.slack.com/client/T032YB4B6TA/)\n[![MIT License](https://img.shields.io/badge/license-MIT-green?style=plastic)](LICENSE)\n[![FOSSA Status](https://app.fossa.com/api/projects/git%2Bgithub.com%2FGuanceCloud%2Fdatakit.svg?type=shield)](https://app.fossa.com/projects/git%2Bgithub.com%2FGuanceCloud%2Fdatakit?ref=badge_shield)\n\n\u003ch2\u003e\n  \u003ca href=\"https://www.guance.com/\"\u003eWebsite\u003c/a\u003e\n  \u003cspan\u003e • \u003c/span\u003e\n  \u003ca href=\"https://docs.guance.com/datakit/\"\u003eDoc\u003c/a\u003e\n\u003c/h2\u003e\n\n\n## _Read this in other languages._\n\u003ckbd\u003e[\u003cimg title=\"中文 (Simplified)\" alt=\"中文 (Simplified)\" src=\"https://cdn.staticaly.com/gh/hjnilsson/country-flags/master/svg/cn.svg\" width=\"22\"\u003e](README.zh_CN.md)\u003c/kbd\u003e\n\nDataKit is an open source, integrated data collection agent, which provides full platform (Linux/Windows/macOS) support and has comprehensive data collection capability, covering various scenarios such as host, container, middleware, tracing, logging and security inspection.\n\n## Key Features\n\n- Support collection of metrics, logging and tracing\n- Fully support Kubernetes ecology\n- [Pipeline](https://docs.guance.com/pipeline/use-pipeline): Simple structured data extraction\n- Supports third-party data import:\n    - [Telegraf](https://docs.guance.com/integrations/telegraf)\n    - [Prometheus](https://docs.guance.com/integrations/prom)\n    - [Statsd](https://docs.guance.com/integrations/statsd)\n    - [Fluentd](https://docs.guance.com/integrations/logstreaming-fluentd)\n    - [Filebeats](https://docs.guance.com/integrations/beats_output)\n    - Tracing \n        - [OpenTelemetry](https://docs.guance.com/integrations/opentelemetry)\n        - [DDTrace](https://docs.guance.com/integrations/ddtrace)\n        - [Zipkin](https://docs.guance.com/integrations/zipkin)\n        - [Jaeger](https://docs.guance.com/integrations/jaeger)\n        - [Skywalking](https://docs.guance.com/integrations/skywalking)\n\n## Changelog\n\nAll DataKit changelog refers to [here](https://docs.guance.com/datakit/changelog).\n\n## Minimal Requirements\n\n| OS                                                             | Arch                | Install Path                                                                   |\n| ---                                                            | ---                 | ---                                                                            |\n| Linux Kernel 2.6.23+                                           | amd64/386/arm/arm64 | `/usr/local/datakit`                                                           |\n| macOS 10.12+([Why](https://github.com/golang/go/issues/25633)) | amd64               | `/usr/local/datakit`                                                           |\n| Windows 7+/Server 2008R2+                                      | amd64/386           | 64-bit：`C:\\Program Files\\datakit`\u003cbr /\u003e32-bit：`C:\\Program Files(32)\\datakit` |\n\n\n## Install DataKit\n\nWe can directly obtain the DataKit installation command from [guance cloud](http://guance.com). Most of the installation commands seems like that:\n\n- Linux \u0026 Mac\n```shell\nDK_DATAWAY=\"https://openway.guance.com?token=\u003cYOUR-TOKEN\u003e\" bash -c \"$(curl -L https://static.guance.com/datakit/install.sh)\"\n```\n\n- Windows\n\n```powershell\nRemove-Item -ErrorAction SilentlyContinue Env:DK_*;\n$env:DK_DATAWAY=\"https://openway.guance.com?token=\u003cYOUR-TOKEN\u003e\";\nSet-ExecutionPolicy Bypass -scope Process -Force;\nImport-Module bitstransfer;\nstart-bitstransfer -source https://static.guance.com/datakit/install.ps1 -destination .install.ps1;\npowershell .install.ps1;\nRemove-Item .install.ps1;\n```\n\n- [Kubernetes DaemonSet](https://docs.guance.com/datakit/datakit-daemonset-deploy)\n\nFor more documentations about DataKit installation, see [here](https://docs.guance.com/datakit/datakit-install).\n\n## Build From Source\n\nDataKit building relies on some external tools/libs, we must install them all before compile the source code.\n\n\u003e - **We recommend to build source on Ubuntu 20.04+**, other linux distribition may failed to install these dependencies. We do not support build DataKit on Windows.\n\u003e - Please build the project with `make`, we haven't testing with Golang/VSCode IDEs\n\n### Setup Golang\n\nInstall and setup Golang(1.18.3+):\n\n```shell\nexport GOPRIVATE=gitlab.jiagouyun.com/*\nexport GOPROXY=https://goproxy.cn,direct\nexport GOPATH=~/go            # depends on your local settings\nexport GOROOT=~/golang-1.18.3 # depends on your local settings\nexport PATH=$GOROOT/bin:~/go/bin:$PATH\n```\n\n### Install other tools\n\n\u003e !!! Do not install these dependencies under datakit source code dir.\n\n- make: `apt-get install make`\n- gcc: `apt-get install gcc`\n- gcc-multilib: `apt-get install -y gcc-multilib`\n- tree: `apt-get install tree`\n- packr2: `go install github.com/gobuffalo/packr/v2/packr2@v2.8.3`\n- goyacc: `go install golang.org/x/tools/cmd/goyacc@latest`\n- lint related:\n  - lint: `go install github.com/golangci/golangci-lint/cmd/golangci-lint@v1.46.2`\n- eBPF related:\n\t- clang 10.0+: `apt-get install clang`\n\t- llvm 10.0+: `apt-get install llvm`\n\t- kernel headers\n\t\t- apt: `apt-get install -y linux-headers-$(uname -r)`\n\n### Build\n\n1. Clone code\n\n```shell\n$ mkdir -p $GOPATH/src/gitlab.jiagouyun.com/cloudcare-tools\n$ cd $GOPATH/src/gitlab.jiagouyun.com/cloudcare-tools\n\n$ git clone https://github.com/GuanceCloud/datakit.git   # may be blocked by GFW\n$ git clone https://jihulab.com/guance-cloud/datakit.git # jihulab mirror\n\n$ cd datakit\n```\n\n2. Building\n\n```shell\nmake\n```\n\nIf building ok, all binaries are generated under *dist*:\n\n```\ndist\n├── [4.0K]  datakit-linux-amd64\n│   ├── [ 72M]  datakit\n│   └── [4.0K]  externals\n│       ├── [ 14M]  logfwd\n│       └── [10.0M]  oracle\n├── [4.0K]  local\n│   ├── [ 26M]  installer-linux-amd64\n│   └── [ 228]  version\n└── [4.0K]  standalone\n    └── [4.0K]  datakit-ebpf-linux-amd64\n\t\t        └── [ 38M]  datakit-ebpf\n```\n\nWe can build all platforms(Linux/Mac/Windows) with following command:\n\n```shell\nmake testing\n```\n\n## Basic Usage\n\nWe can use `help` command to see more usage of DataKit:\n\n```shell\ndatakit help\n\n# Or\n\n./dist/datakit-linux-amd64/datakit help\n```\n\n## Contributing\n\nBefore contributing, check out some guideline of DataKit:\n\n- Read [architecure introduciton](https://docs.guance.com/datakit/datakit-arch)\n- Read [development guideline](https://docs.guance.com/datakit/development)\n\n## Full Documentation\n\nFor full documents of DataKit, see\n\n- [DataKit Doc](https://docs.guance.com/datakit)\n\n## License\n[![FOSSA Status](https://app.fossa.com/api/projects/git%2Bgithub.com%2FGuanceCloud%2Fdatakit.svg?type=large)](https://app.fossa.com/projects/git%2Bgithub.com%2FGuanceCloud%2Fdatakit?ref=badge_large)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fguancecloud%2Fdatakit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fguancecloud%2Fdatakit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fguancecloud%2Fdatakit/lists"}