Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/qiniu/logkit
Very powerful server agent for collecting & sending logs & metrics with an easy-to-use web console.
https://github.com/qiniu/logkit
collector-agent go log-collector logkit metrics-gathering pandora
Last synced: 7 days ago
JSON representation
Very powerful server agent for collecting & sending logs & metrics with an easy-to-use web console.
- Host: GitHub
- URL: https://github.com/qiniu/logkit
- Owner: qiniu
- License: apache-2.0
- Created: 2017-05-15T06:50:40.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2024-05-20T05:25:53.000Z (7 months ago)
- Last Synced: 2024-11-20T04:59:21.871Z (23 days ago)
- Topics: collector-agent, go, log-collector, logkit, metrics-gathering, pandora
- Language: Go
- Homepage: https://logkit-pro.qiniu.com
- Size: 25.6 MB
- Stars: 1,354
- Watchers: 58
- Forks: 255
- Open Issues: 55
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Audit: audit/audit.go
- Roadmap: ROADMAP.md
Awesome Lists containing this project
- my-awesome - qiniu/logkit - agent,go,log-collector,logkit,metrics-gathering,pandora pushed_at:2024-05 star:1.4k fork:0.3k Very powerful server agent for collecting & sending logs & metrics with an easy-to-use web console. (Go)
- go-awesome - logkit - Qiniu open source tool for collecting logs (Open source library / Logs)
- awesome-golang-repositories - logkit - to-use web console. (Repositories)
README
# logkit-community [](http://travis-ci.org/qiniu/logkit) [](https://goreportcard.com/report/github.com/qiniu/logkit) [](https://codecov.io/gh/qiniu/logkit/branch/master) [](https://godoc.org/github.com/qiniu/logkit)
[中文版](https://github.com/qiniu/logkit/blob/master/READMECN.md)
## Introduce
Very powerful server agent for collecting & sending logs & metrics with an easy-to-use web console.
### logkit-community Detail doc can be referred to[WIKI](https://github.com/qiniu/logkit/wiki)
## Support sources
* [File](https://github.com/qiniu/logkit/wiki/File-Reader): read data in file, including csv file，kafka-rest log，nginx log.
* [Elasticsearch](https://github.com/qiniu/logkit/wiki/ElasticSearch-Reader): read data in ElasticSearch.
* [MongoDB](https://github.com/qiniu/logkit/wiki/MongoDB-Reader): read data in MongoDB.
* [MySQL](https://github.com/qiniu/logkit/wiki/MySQL-Reader): read data in MySQL.
* [MicroSoft SQL Server](https://github.com/qiniu/logkit/wiki/MicroSoft-SQL-Server-Reader): read data in Microsoft SQL Server.
* [Postgre SQL](https://github.com/qiniu/logkit/wiki/PostgreSQL-Reader): read data in PostgreSQL.
* [Kafka](https://github.com/qiniu/logkit/wiki/Kafka-Reader): read data in Kafka.
* [Redis](https://github.com/qiniu/logkit/wiki/Redis-Reader): read data in Redis.
* [Socket](https://github.com/qiniu/logkit/wiki/Socket-Reader): read data via tcp\udp\unixsocket protocol.
* [Http](https://github.com/qiniu/logkit/wiki/Http-Reader): reveive data in post request as http server.
* [Script](https://github.com/qiniu/logkit/wiki/Script-Reader): support script and read data from the result.
* [Snmp](https://github.com/qiniu/logkit/wiki/Snmp-Reader): auto read data from Snmp service.
## Working method
logkit-community support multiple sources and can send kinds of data to Pandora, every data source relevant to a logic runner,a runner's workaround as follows:
## Contributing
Weclome to contribute to logkit:
* fix or [report bug](https://github.com/qiniu/logkit/issues/new)
* [pull issue](https://github.com/qiniu/logkit/issues/new) improve [wiki doc](https://github.com/qiniu/logkit/wiki)
* [review code](https://github.com/qiniu/logkit/pulls) or [create issues](https://github.com/qiniu/logkit/issues/new)
* contribute code (contribute kinds of modules including [reader](https://github.com/qiniu/logkit/wiki/Readers)、[parser](https://github.com/qiniu/logkit/wiki/Parsers)、[sender](https://github.com/qiniu/logkit/wiki/Senders) and [transformer](https://github.com/qiniu/logkit/wiki/Transformers)）
## Download
**lastest stable**：Go to [Download page](https://github.com/qiniu/logkit/wiki/Download)
**History**：Go to [Releases](https://github.com/qiniu/logkit/releases)
**Trial**：construct lastest logkit trial version every 5:00am (only for Linux 64 and Docker), you can download it (note: not include update of frontend).
* [Linux 64 download](https://pandora-dl.qiniu.com/nightly/logkit_nightly.tar.gz)
* Docker image nightly: `docker pull wonderflow/logkit:nightly`
## Install and Usage
### 1. Download&Decompress logkit-community tool
* Linux
``` sh
export LOGKIT_VERSION=
wget https://pandora-dl.qiniu.com/logkit_${LOGKIT_VERSION}.tar.gz && tar xvf logkit_${LOGKIT_VERSION}.tar.gz && rm logkit_${LOGKIT_VERSION}.tar.gz && cd _package_linux64/
```
* MacOS
``` sh
export LOGKIT_VERSION=
wget https://pandora-dl.qiniu.com/logkit_mac_${LOGKIT_VERSION}.tar.gz && tar xvf logkit_mac_${LOGKIT_VERSION}.tar.gz && rm logkit_mac_${LOGKIT_VERSION}.tar.gz && cd _package_mac/
```
* Windows
please download `https://pandora-dl.qiniu.com/logkit_windows_.zip` 并解压缩，go to directory
### 2. change logkit-community configuration
logkit.conf is logkit-community tool's configuration，mainly for specifing running resource and paths of runners.
Open `logkit.conf`, for example：
``` json
{
"max_procs": 8,
"debug_level": 1,
"clean_self_log":true,
"bind_host":"localhost:3000",
"static_root_path":"./public",
"confs_path": ["confs*"]
}
```
For simply use, you can only focus on three options:
1. `bind_host` port of logkit we。
1. `static_root_path` statistic resource path of logkit page, **recommand to use absolute path** note：old version moved to "public-old" directory。
1. `confs_path` including add conf in web, logkit also support monitor directory to add runners. (if you only need to add logkit runner in web, you can ignore this option)
### 3. startup logkit-community tool
``` sh
./logkit -f logkit.conf
```
### 4. Open logkit-community config page in web
the web url is the value of `bind_host` configured in step 2
## Contribute frontend code
refer to README file：[logkitweb/README.md](https://github.com/qiniu/logkit/blob/master/logkitweb/README.md)
## Install and startup from source code
``` sh
go build -o logkit logkit.go
./logkit -f logkit.conf
```
## startup logkit using docker
``` sh
docker pull wonderflow/logkit:
docker run -d -p 3000:3000 -v /local/logkit/dataconf:/app/confs -v /local/log/path:/logs/path logkit:
```
## Deploying logkit in Kubernetes
get configs deploying in Kubernetes
``` sh
curl -L -O https://raw.githubusercontent.com/qiniu/logkit/master/deploy/logkit_on_k8s.yaml
```
enjoy it！