https://github.com/comboy/komoku
time series db with pubsub
https://github.com/comboy/komoku
Last synced: 10 months ago
JSON representation
time series db with pubsub
- Host: GitHub
- URL: https://github.com/comboy/komoku
- Owner: comboy
- Created: 2016-08-30T14:36:29.000Z (almost 10 years ago)
- Default Branch: master
- Last Pushed: 2017-03-08T09:30:35.000Z (over 9 years ago)
- Last Synced: 2025-08-04T09:08:52.573Z (11 months ago)
- Language: Elixir
- Homepage:
- Size: 69.3 KB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Komoku
### key value storage + time series database + pub sub
Komoku is a key value database for different types of data, which keeps track of all changes, and then can compact historical data in a clever way. You can subscribe to different key changes.
For now the API is going to be compatible with ruby version (https://github.com/comboy/komoku.ruby) but after I migrate my home automation to the new system I'm going to update it.
Why am I even writing this readme, it's way too early for that.
## Key types
HA denotes my personal use case in home automation as an example
### Numeric
Double. I'm considering using actual numeric type (specified precision, exact). Currently there's only gauge but I plan adding different types similar to what's known from rrdtool. It should be able to use different aggregations (for now, most of my cases are covered with gauge avg/min/max so I'm focusing on that)
HA: tracking temp, humidity light etc.
### Boolean
True and false. But. It represents the state changes that last. So with boolean you should be able to get stats like:
* what value was present at given time
* uptime-like stats e.g. 99.5% true during last month
* sum of time stats e.g. number of hours per day for which the state was false
* maybe common timespans stats (like avg distribution per hour during last month)
Boolean can be used for representing timespans between value changes.
HA: state of the light somewhere, is door open, is window open, how often am I at home
### Uptime
Boolean which automatically goes to false if not updated for `:max_time`
### String
Custom string.
* you can disable compacting and just use it as a log (better yet let's compact it, keeping in mind that it is a logfile), separate type may appear for that
* or you can be only assigning to it some set of strings representing state of something (stats then are boolean-alike but with more values)
* or use it to store some kind of config
HA: log, what's the alarm state (armed, disabled, waiting), selected radio station, what room I'm in
## TODO
### Features
* compacting historical data
* fetching historical data
* treating booleans as timespans
* user authentication
### Polishing and optimization
* better error handling, there are supervisors but incorrect user input shouldn't be resulting in any process crash
* unused key processes can be killed when there's like 100K+ of them
* add bench for 100K+ keys
* use benchfella
* add distillery
* beter docs
### Low priority features
* protobuf + tcp server
* http server
* komowku-web with graphs
* string opt to only allow specified set of values (careful with `update_key` and historical values that may not comply)
## Architecture
`Komoku.Server` is the main access point. You can call on it things like `get`, `put` or `list_keys`. It also starts outside interfacess like `Komoku.Server.Websocket` according to config.exs.
Each key is handled by a separate process (`KeyHandler`). It stores last value cache in its state, it's responsible for appropriate key value updates (e.g. change value to false for uptime type of key), consolidating data, and fetching it from `Storage` when necessary.
`Storage` is responsible for persistant value store. Basically a database interface.
`KeyMaster` manages keys. It keeps list of all the keys, creates, updates and removes them. It also manages `KeyHandler` processes.
`SubscriptionManager` - you won't guess! `KeyHandler` publishes changes of keys to it, and client processes created by e.g. `Komoku.Server.Websocket` can subscribe to these changes.