Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/facebookarchive/beringei
Beringei is a high performance, in-memory storage engine for time series data.
https://github.com/facebookarchive/beringei
Last synced: about 1 month ago
JSON representation
Beringei is a high performance, in-memory storage engine for time series data.
- Host: GitHub
- URL: https://github.com/facebookarchive/beringei
- Owner: facebookarchive
- License: other
- Archived: true
- Created: 2016-10-11T00:45:02.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2018-07-11T17:33:58.000Z (over 6 years ago)
- Last Synced: 2024-05-21T01:05:51.257Z (6 months ago)
- Language: C++
- Homepage:
- Size: 894 KB
- Stars: 3,166
- Watchers: 201
- Forks: 296
- Open Issues: 13
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
- awesome-repositories - facebookarchive/beringei - Beringei is a high performance, in-memory storage engine for time series data. (C++)
- awesome-cloud-native - beringei - Beringei is a high performance, in-memory storage engine for time series data. (Database)
- awesome-starred - facebookarchive/beringei - Beringei is a high performance, in-memory storage engine for time series data. (others)
README
** THIS REPO HAS BEEN ARCHIVED AND IS NO LONGER BEING ACTIVELY MAINTAINED **
# Beringei [](https://circleci.com/gh/facebookincubator/beringei/tree/master)
A high performance, in memory time series storage engine
In the fall of 2015, we published the [paper “Gorilla: A Fast, Scalable, In-Memory Time Series Database”](http://www.vldb.org/pvldb/vol8/p1816-teller.pdf) at VLDB 2015. Beringei is the open source representation of the ideas presented in this paper.
Beringei is a high performance time series storage engine. Time series are commonly used as a representation of statistics, gauges, and counters for monitoring performance and health of a system.
## Features
Beringei has the following features:
* Support for very fast, in-memory storage, backed by disk for persistence. Queries to the storage engine are always served out of memory for extremely fast query performance, but backed to disk so the process can be restarted or migrated with very little down time and no data loss.
* Extremely efficient streaming compression algorithm. Our streaming compression algorithm is able to compress real world time series data by over 90%. The delta of delta compression algorithm used by Beringei is also fast - we see that a single machine is able to compress more than 1.5 million datapoints/second.
* Reference sharded service implementation, including a client implementation.
* Reference http service implementation that enables direct Grafana integration.
## How can I use Beringei?
Beringei can be used in one of two ways.
1. We have created a simple, sharded service, and reference client implementation, that can store and serve
time series query requests.
1. You can use Beringei as an embedded library to handle the low-level details of efficiently storing time series data. Using Beringei in this way is similar to [RocksDB](https://rocksdb.org) - the Beringei library can be the high performance storage system underlying your performance monitoring solution.
## Requirements
Beringei is tested and working on:
* Ubuntu 16.10
We also depend on these open source projects:
* [fbthrift](https://github.com/facebook/fbthrift)
* [folly](https://github.com/facebook/folly)
* [wangle](https://github.com/facebook/wangle)
* [proxygen](https://github.com/facebook/proxygen)
* [gtest](https://github.com/google/googletest)
* [gflags](https://github.com/gflags/gflags)
## Building Beringei
Our instructions are for Ubuntu 16.10 - but you will probably be able to modify
the install scripts and directions to work with other linux distros.
- Run `sudo ./setup_ubuntu.sh`.
- Build beringei.
```
mkdir build && cd build && cmake .. && make
```
- Generate a beringei configuration file.
```
./beringei/tools/beringei_configuration_generator --host_names $(hostname) --file_path /tmp/beringei.json
```
- Start beringei.
```
./beringei/service/beringei_main \
-beringei_configuration_path /tmp/beringei.json \
-create_directories \
-sleep_between_bucket_finalization_secs 60 \
-allowed_timestamp_behind 300 \
-bucket_size 600 \
-buckets $((86400/600)) \
-logtostderr \
-v=2
```
- Send data.
```
while [[ 1 ]]; do
./beringei/tools/beringei_put \
-beringei_configuration_path /tmp/beringei.json \
testkey ${RANDOM} \
-logtostderr -v 3
sleep 30
done
```
- Read the data back.
```
./beringei/tools/beringei_get \
-beringei_configuration_path /tmp/beringei.json \
testkey \
-logtostderr -v 3
```
## License
Beringei is BSD-licensed. We also provide an additional patent grant.