https://github.com/preaction/tartarus
Scalable data warehouse platform
https://github.com/preaction/tartarus
Last synced: 9 months ago
JSON representation
Scalable data warehouse platform
- Host: GitHub
- URL: https://github.com/preaction/tartarus
- Owner: preaction
- License: other
- Created: 2016-01-03T01:41:27.000Z (over 10 years ago)
- Default Branch: master
- Last Pushed: 2016-11-12T04:04:25.000Z (over 9 years ago)
- Last Synced: 2024-04-17T21:17:03.618Z (about 2 years ago)
- Language: Perl
- Size: 32.2 KB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README
- Changelog: CHANGES
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
SYNOPSIS
# tartarus.yml
mounts:
user:
$class: Tartarus::DataSource::Postgres
host: localhost
stats:
$class: Tartarus::DataSource::RRDTool
root: /var/db/rrdtool
orders:
$class: Tartarus::DataSource::MySQL
host: db.example.com
transport:
$class: Tartarus::Transport::ZeroMQ
endpoint: tcp://*:7000
message:
$class: Tartarus::Message::Sereal
# Start the server
tartarus [-f ]
DESCRIPTION
Tartarus is a server that provides a unified API on to multiple data
sources, mounted in a tree. Users may make requests of the data,
reading and writing, or they can subscribe to messages as they pass
through the API, allowing for notification of updates.
Tartarus is built to help data warehouses be scalable and flexible. The
unified API makes simple data maintenance tasks trivial, allows for
easy migration of data between backend data platforms, and enables
scalability features like sharding and caching that the backend data
platform may not support.
Data Source
Tartarus collects multiple data sources, arranged in a tree. Each data
source can expose its data as a tree as well, mapping the data in a
logical, navigable structure.
Data sources can be relational databases like Postgres and MySQL,
document stores like MongoDB and ElasticSearch, and time series
databases like RRDTool and Graphite.
Tartarus can use other Tartarus servers as data sources, allowing for
sharding, failover, and other scalability patterns.
Custom data sources can take any form, allowing easy access to things
like SNMP data. Custom data sources can also filter and transform
existing data sources, allowing for on-the-fly transforms and analysis.
Transport
Tartarus allows incoming connections via a transport, like ZeroMQ,
nanomsg, HTTP, WebSockets, or plain TCP. Multiple transports can be
configured, allowing multiple types of clients.
Message
Tartarus messages can be serialized by one of many types of message
format, including Sereal, Protocol Buffers, MessagePack, Cap'n Proto,
JSON, BSON, and more.
Tartarus messages create a common data structure so that different data
sources can be used by the same program.
SEE ALSO
* Tartarus::DataSource
* Tartarus::Transport
* Tartarus::Message
message:
$class: Tartarus::Message::Sereal
The message type to use
transport:
$class: Tartarus::Transport::ZeroMQ
$args:
endpoint: tcp://127.0.0.1:7000
The transport layer to use
mounts:
/customers:
$class: Tartarus::DataSource::Postgres
$args:
host: localhost
port: 4660
/usage:
$class: Tartarus::DataSource::RRDTool
$args:
root_dir: /var/db/rrd
Mount a data source at a path.
$t->run();
Start the daemon.
POD ERRORS
Hey! The above document had some coding errors, which are explained
below:
Around line 86:
Unknown directive: =attr
Around line 93:
Unknown directive: =attr
Around line 102:
Unknown directive: =attr
Around line 117:
Unknown directive: =method