https://github.com/visemet/yocto-db

A data-stream management system
https://github.com/visemet/yocto-db

Last synced: 3 months ago
JSON representation

A data-stream management system

• Host: GitHub
• URL: https://github.com/visemet/yocto-db
• Owner: visemet
• License: bsd-3-clause
• Created: 2013-03-13T02:24:17.000Z (almost 12 years ago)
• Default Branch: v0.2
• Last Pushed: 2014-01-25T18:26:51.000Z (almost 11 years ago)
• Last Synced: 2024-10-02T16:41:24.934Z (3 months ago)
• Language: Erlang
• Homepage:
• Size: 3.94 MB
• Stars: 5
• Watchers: 4
• Forks: 1
• Open Issues: 11
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
yocto-db

========

A data-stream management system.

[![Build Status](https://travis-ci.org/visemet/yocto-db.png)](https://travis-ci.org/visemet/yocto-db)

Build & Run

-----------

    rebar clean compile

    erl -pz ebin deps/*/ebin

Test

----

    rebar eunit

Documentation

-------------

    rebar doc

Supervisor Hierarchy

--------------------

    +ydb_sup

    |

    +---+ydb_input_stream_sup

        |

        +---+ydb_input_stream

        |   |

        |   +---+ydb_{file,socket}_input

        |       |

        |       +ydb_branch_node

        |

        +ydb_query_sup

        |

        +---+ydb_query

The ydb\_input\_stream\_sup module supports registering input streams

either from a file or a socket. The input stream process also spawns

a branch node process and adds it as a subscriber.

IO Definitions

--------------

### File Input

    {ydb_file_input,

        child_node

      , filename :: string()

      , batch_size :: integer()

      , poke_freq :: integer()

    }

### File Output

    {ydb_file_output, child_node, filename :: string()}

### Socket Input

    {ydb_socket_input,

        child_node

      , port_no :: integer()

      , socket :: port()

      , acceptor :: pid()

    }

### Socket Output

    {ydb_socket_output,

        child_node

      , port_no :: integer()

      , socket :: port()

      , address :: term()

    }

Query Definitions

-----------------

Register a timer that pokes the intermediary plan nodes at each tick,

and store the reference in the supervisor state. This keeps the plan

nodes up to date on the current time so they how to store partial

results.

### Select

    {ydb_select, child_node, predicate :: ydb_clause()}

### Project

The column specification is either just the column name atom() or

a tuple containing the current column name and the new name for the

column {atom(), atom()}.

    {ydb_project, child_node, columns :: [atom() | {atom(), atom()}]}

### Joins

    {ydb_join_node, left_node, right_node, predicate :: ydb_clause()}

### Set Union

    {ydb_union_node, left_node, right_node}

### Set Difference

    {ydb_diff_node, left_node, right_node}

    

### Min

    {ydb_min, child_node, column :: atom() | {atom(), atom()}}

    

### Max

    {ydb_max, child_node, column :: atom() | {atom(), atom()}}

    

### Sum

    {ydb_sum, child_node, column :: atom() | {atom(), atom()}}

    

### Count

    {ydb_count, child_node, column :: atom() | {atom(), atom()}}

    

### Average

    {ydb_avg, child_node, column :: atom() | {atom(), atom()}}

    

### Variance

    {ydb_var, child_node, column :: atom() | {atom(), atom()}}

    

### Standard Deviation

    [ydb_stddev, child_node, column :: atom() | {atom(), atom()}}

Predicate Format

----------------

### Boolean Operators

    {ydb_and, clauses :: [ydb_clause()]}

    {ydb_or, clauses :: [ydb_clause()]}

    {ydb_not, clause :: ydb_clause()}

    ydb_clause() :: #ydb_cv{} | #ydb_cc{} | #ydb_and{} | #ydb_or{} | #ydb_not{}

### Comparison Operators

Comparing a column to a value.

    {ydb_cv,

        column :: atom()

      , operator :: compare()

      , value :: term()

    }

Comparing two columns together.

    {ydb_cc,

        left_col :: atom()

      , operator :: compare()

      , right_col :: atom()

    }

Comparison operators.

    compare() :: 'gt'  | 'lt'  | 'eq'

               | 'lte' | 'gte' | 'ne'

Table Formats

-------------

Rows in ETS tables are represented as tuples. All the rows in a

given ETS table should be in the same format, which will be one

of the following:

### Relation Table

    ydb_rel_tuple() :: {

        {'row_num', RowNum :: non_neg_integer()}

      , Tuple :: ydb_tuple()

    }.

RowNum serves as a unique id for each row.

### Synopsis Table

    ydb_syn_tuple() :: {

        {Op :: atom(), Timestamp :: non_neg_integer()}

      , Tuple :: ydb_tuple()

    }.

Op is the name of the aggregate (e.g. 'sum' or 'count').

### Diff Table

    ydb_diff_tuple() :: {

        {Diff :: diff(), Op :: atom(), Timestamp :: non_neg_integer()}

      , Tuple :: ydb_tuple()

    }.

Op is the name of the aggregate, as above (e.g. 'sum' or count').

Diff indicates whether the tuple is to be inserted or deleted.

    diff() :: '+' | '-'.