Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/qiniu/logkit

Very powerful server agent for collecting & sending logs & metrics with an easy-to-use web console.
https://github.com/qiniu/logkit

collector-agent go log-collector logkit metrics-gathering pandora

Last synced: 17 days ago
JSON representation

Very powerful server agent for collecting & sending logs & metrics with an easy-to-use web console.

Lists

README 

        
# logkit-community [![Build Status](https://api.travis-ci.org/qiniu/logkit.svg)](http://travis-ci.org/qiniu/logkit) [![Go Report Card](https://goreportcard.com/badge/github.com/qiniu/logkit)](https://goreportcard.com/report/github.com/qiniu/logkit) [![codecov](https://codecov.io/gh/qiniu/logkit/branch/master/graph/badge.svg)](https://codecov.io/gh/qiniu/logkit/branch/master) [![GoDoc](https://godoc.org/github.com/qiniu/logkit?status.svg)](https://godoc.org/github.com/qiniu/logkit)



![logkit LOGO](https://raw.githubusercontent.com/qiniu/logkit/master/resources/logo.png)



[中文版](https://github.com/qiniu/logkit/blob/master/READMECN.md)



## Introduce



Very powerful server agent for collecting & sending logs & metrics with an easy-to-use web console.



### logkit-community Detail doc can be referred to[WIKI](https://github.com/qiniu/logkit/wiki)



## Support sources



* [File](https://github.com/qiniu/logkit/wiki/File-Reader): read data in file, including csv file，kafka-rest log，nginx log.

* [Elasticsearch](https://github.com/qiniu/logkit/wiki/ElasticSearch-Reader): read data in ElasticSearch.

* [MongoDB](https://github.com/qiniu/logkit/wiki/MongoDB-Reader): read data in MongoDB.

* [MySQL](https://github.com/qiniu/logkit/wiki/MySQL-Reader): read data in MySQL.

* [MicroSoft SQL Server](https://github.com/qiniu/logkit/wiki/MicroSoft-SQL-Server-Reader): read data in Microsoft SQL Server.

* [Postgre SQL](https://github.com/qiniu/logkit/wiki/PostgreSQL-Reader): read data in PostgreSQL.

* [Kafka](https://github.com/qiniu/logkit/wiki/Kafka-Reader): read data in Kafka.

* [Redis](https://github.com/qiniu/logkit/wiki/Redis-Reader): read data in Redis.

* [Socket](https://github.com/qiniu/logkit/wiki/Socket-Reader): read data via tcp\udp\unixsocket protocol.

* [Http](https://github.com/qiniu/logkit/wiki/Http-Reader): reveive data in post request as http server.

* [Script](https://github.com/qiniu/logkit/wiki/Script-Reader): support script and read data from the result.

* [Snmp](https://github.com/qiniu/logkit/wiki/Snmp-Reader): auto read data from Snmp service.



## Working method



logkit-community support multiple sources and can send kinds of data to Pandora, every data source relevant to a logic runner,a runner's workaround as follows:



![logkit workaround](https://raw.githubusercontent.com/qiniu/logkit/master/resources/logkit.png)



## Contributing



Weclome to contribute to logkit:



* fix or [report bug](https://github.com/qiniu/logkit/issues/new)

* [pull issue](https://github.com/qiniu/logkit/issues/new) improve [wiki doc](https://github.com/qiniu/logkit/wiki)

* [review code](https://github.com/qiniu/logkit/pulls) or [create issues](https://github.com/qiniu/logkit/issues/new)

* contribute code (contribute kinds of modules including [reader](https://github.com/qiniu/logkit/wiki/Readers)、[parser](https://github.com/qiniu/logkit/wiki/Parsers)、[sender](https://github.com/qiniu/logkit/wiki/Senders) and [transformer](https://github.com/qiniu/logkit/wiki/Transformers)）



## Download



**lastest stable**：Go to [Download page](https://github.com/qiniu/logkit/wiki/Download)



**History**：Go to [Releases](https://github.com/qiniu/logkit/releases)



**Trial**：construct lastest logkit trial version every 5:00am (only for Linux 64 and Docker), you can download it (note: not include update of frontend).



* [Linux 64 download](https://pandora-dl.qiniu.com/nightly/logkit_nightly.tar.gz)



* Docker image nightly:  `docker pull wonderflow/logkit:nightly`



## Install and Usage



### 1. Download&Decompress logkit-community tool



* Linux



``` sh

export LOGKIT_VERSION=

wget https://pandora-dl.qiniu.com/logkit_${LOGKIT_VERSION}.tar.gz && tar xvf logkit_${LOGKIT_VERSION}.tar.gz && rm logkit_${LOGKIT_VERSION}.tar.gz && cd _package_linux64/

```



* MacOS



``` sh

export LOGKIT_VERSION=

wget https://pandora-dl.qiniu.com/logkit_mac_${LOGKIT_VERSION}.tar.gz && tar xvf logkit_mac_${LOGKIT_VERSION}.tar.gz && rm logkit_mac_${LOGKIT_VERSION}.tar.gz && cd _package_mac/

```



* Windows



please download `https://pandora-dl.qiniu.com/logkit_windows_.zip` 并解压缩，go to directory



### 2. change logkit-community configuration



logkit.conf is logkit-community tool's configuration，mainly for specifing running resource and paths of runners.



Open `logkit.conf`, for example：



``` json

{

    "max_procs": 8,

    "debug_level": 1,

    "clean_self_log":true,

    "bind_host":"localhost:3000",

    "static_root_path":"./public",

    "confs_path": ["confs*"]

}

```



For simply use, you can only focus on three options:



1. `bind_host` port of logkit we。

1. `static_root_path` statistic resource path of logkit page, **recommand to use absolute path** note：old version moved to "public-old" directory。

1. `confs_path` including add conf in web, logkit also support monitor directory to add runners. (if you only need to add logkit runner in web, you can ignore this option)



### 3. startup logkit-community tool



``` sh

./logkit -f logkit.conf

```



### 4. Open logkit-community config page in web



the web url is the value of `bind_host` configured in step 2



## Contribute frontend code



refer to README file：[logkitweb/README.md](https://github.com/qiniu/logkit/blob/master/logkitweb/README.md)



## Install and startup from source code



``` sh

go build -o logkit logkit.go

./logkit -f logkit.conf

```



## startup logkit using docker



``` sh

docker pull wonderflow/logkit:

docker run -d -p 3000:3000 -v /local/logkit/dataconf:/app/confs -v /local/log/path:/logs/path logkit:

```



## Deploying logkit in Kubernetes



get configs deploying in Kubernetes



``` sh

curl -L -O https://raw.githubusercontent.com/qiniu/logkit/master/deploy/logkit_on_k8s.yaml

```



enjoy it！