Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/logrange/logrange
High performance data aggregating storage
https://github.com/logrange/logrange
database go golang log-aggregation logging pipelines stream-processing streams
Last synced: 7 days ago
JSON representation
High performance data aggregating storage
- Host: GitHub
- URL: https://github.com/logrange/logrange
- Owner: logrange
- License: apache-2.0
- Created: 2018-08-23T01:17:39.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2024-02-21T08:14:58.000Z (9 months ago)
- Last Synced: 2024-07-02T14:45:45.899Z (4 months ago)
- Topics: database, go, golang, log-aggregation, logging, pipelines, stream-processing, streams
- Language: Go
- Homepage:
- Size: 69.2 MB
- Stars: 193
- Watchers: 12
- Forks: 16
- Open Issues: 15
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome - logrange - High performance data aggregating storage (Go)
README
[](https://goreportcard.com/report/logrange/logrange) [](https://godoc.org/github.com/logrange/logrange/api) [](https://github.com/logrange/logrange/blob/master/LICENSE)
# Logrange - streaming database
[Logrange](https://www.logrange.io) is highly performant streaming database for aggregating streams of records like application logs, system metrics, audit logs etc. from thousands of sources. Logrange provides an API to do operations from simple search to the data analysis and machine learning.
__The product__
* _Size tolerant_. Logrange is able to store as much data as written there. The only limitation is the storage space available for the database.
* _Highly performant_. Writing and reading millions of records per second (hundered megabytes or gigabytes of data).
* _Write optimized_. Logrange persists the raw data in realtime. All other data processing like indexing can be done later.
* _Low latency_. Data becomes available for read within milliseconds after it is written
* _Scalable_. Supporting tens of thousands different streams of records (terrabytes of the data)
* _Highly available_. In clustering solution, data could be replicated between logrange nodes. Logrange will support data and load distribution policies.
* _Native for stream processing_. Merging, filtering and search using LQL (Logrange Query Language)
* _Open Source_. Logrange is 100% open source. It can be used for building trustworthy data storages.
* _Ready to use_. Basic installation includes pre-configured log processing tools: collector, forwarder, CLI tool and Logrange database service.
* _Easy installation_ either in a containerized (k8s or docker) or a custom environment.
## Quick start
Logrange shipment includes 2 executables - `logrange` server and `lr` - the logrange client. In the quick start you can use precompiled binaries to try logrange out within 1 minute:
### Step 1. Let's put everything into one dir
Make a directory and enter there:
```bash
mkdir lrquick
cd lrquick
```
### Step 2. Install logrange server and run it
```bash
curl -s https://logrange.io/download/install | bash -s logrange -d ./bin
./bin/logrange start --base-dir=./data --daemon
```
Normally, you have to see something like `Started. pid=12345`
### Step 3. Install logrange client and start collecting logs from the machine
```bash
curl -s https://logrange.io/download/install | bash -s lr -d ./bin
./bin/lr collect --storage-dir=./collector --daemon
```
The command above runs collector in background. It will send logs found in `/var/log` folder to the logrange server started in step 2.
### Step 4. Connect to the server, using CLI tool.
```bash
./bin/lr shell
...
```
In the logrange shell, you can try `select` to retrieve collected data:
```
> select limit 10
```
Or try `help` command to find out what commands are available.
## Quick stop
From the logrange folder (`lrquick`) type the following commands to stop collector and the logrange server:
```bash
./bin/lr stop-collect --storage-dir=./collector
./bin/logrange stop --base-dir=./data
```
Now, to clean up, just remove the `lrquick` folder:
```bash
cd ..
rm -rf ./lrquick/
```
## Documentation
[k8s Installation](https://github.com/logrange/k8s)
[The product](https://logrange.io/docs)
## Getting Help
- Found a bug or thinking about a new feature? [File an issue](https://github.com/logrange/logrange/issues/new)
- If you have any question or feedback regarding Logrange, send us e-mail to: [email protected]
## Contact us
Whether you have problems with log aggregating, processing or analysis, or wanting to build a secure data aggregating solution.
Reach out to [email protected]
Join our [gitter chat](https://gitter.im/logrange/community)
## License
Apache License 2.0, see [LICENSE](LICENSE).
## Acknowledgments
* GoLand IDE by [JetBrains]( https://www.jetbrains.com/?from=logrange) is used for the code development