https://github.com/logv/sybil

columnar storage + NoSQL OLAP engine | https://logv.org
https://github.com/logv/sybil

columnar-storage golang olap

Last synced: about 1 month ago
JSON representation

columnar storage + NoSQL OLAP engine | https://logv.org

• Host: GitHub
• URL: https://github.com/logv/sybil
• Owner: logv
• License: other
• Created: 2016-01-27T03:50:27.000Z (over 9 years ago)
• Default Branch: master
• Last Pushed: 2024-08-24T18:45:31.000Z (9 months ago)
• Last Synced: 2024-08-24T19:21:43.408Z (9 months ago)
• Topics: columnar-storage, golang, olap
• Language: Go
• Homepage: https://logv.org
• Size: 1.27 MB
• Stars: 305
• Watchers: 15
• Forks: 26
• Open Issues: 30
• Metadata Files:
  • Readme: README.md
  • License: LICENSE.md

Awesome Lists containing this project

README

        

Sybil is an append only analytics datastore with no up front table schema

requirements; just log JSON records to a table and run queries. Written in Go,

sybil is designed for fast full table scans of multi-dimensional data on one or

more machines.

more documentation is available [on the wiki](http://github.com/logv/sybil/wiki)

and [in the repo](http://github.com/logv/sybil/blob/master/docs).

if sybil by itself is uninteresting (who wants to run command line queries,

anyways?), sybil is a supported backend for

[snorkel](http://github.com/logv/snorkel)

advantages

----------

  * Easy to setup and get data inside sybil - just pipe JSON on stdin to sybil

  * Supports histograms (and percentiles), standard deviations and time series roll ups

  * Runs fast full table queries ([performance notes](http://github.com/logv/sybil/wiki/Performance))

  * Lower disk usage through per column compression schemes

  * Serverless design with controlled memory usage

  * Per table retention policies (specify max age and/or size of tables)

  * Timestamp resistant query cache (optional) that avoids recomputation

disadvantages

-------------

  * JOINS not supported

  * No UPDATE operation on data - only writes

installation

------------

    go get github.com/logv/sybil

build status

------------

[![CircleCI](https://circleci.com/gh/logv/sybil.svg?style=svg)](https://circleci.com/gh/logv/sybil)

additional information

----------------------

* [command line tour](http://github.com/logv/sybil/wiki/Quick-Overview)

* [want to contribute?](http://github.com/logv/sybil/wiki/Contributing)

* [notes on performance](http://github.com/logv/sybil/wiki/Performance)

* [implementation notes](https://logv.org/sybil-implementation.html)

* [abadi survey of column stores](http://db.csail.mit.edu/pubs/abadi-column-stores.pdf)