https://github.com/indix/abel
Business stats collection/aggregation
https://github.com/indix/abel
Last synced: 2 months ago
JSON representation
Business stats collection/aggregation
- Host: GitHub
- URL: https://github.com/indix/abel
- Owner: indix
- Created: 2017-05-26T10:35:28.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2018-02-07T09:15:34.000Z (almost 8 years ago)
- Last Synced: 2024-12-27T11:20:27.195Z (about 1 year ago)
- Language: Scala
- Size: 18.6 KB
- Stars: 1
- Watchers: 7
- Forks: 0
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
Abel - Aggregate stats from anywhere
------------------------------------
How it works
------------
Abel client emits metrics to a kafka (0.10) topic. Each metric has a *key* and a *value*. Every key has a name, tags and timestamp.
A value is of aggregate type. Aggregate is anything which (sort of commutative semigroup) can be folded with another aggregate of same type.
What abel consumer does is very simple, it regularly reads from kafka queue and merges all those metrics with same (exactly same) key and stores it to underlying rocksdb.
To use
------
Add the library to build.sbt (published to sonatype)
`"com.indix" %% "abel-core" % "1.1.8"`
And the client emission of metrics looks like,
```
val client = AbelClient(AbelConfig("", ""))
client.send(Metric(Key(name = "unique-upcs"), UniqueCount(product.getUpc)))
```
Abel currently supports
`Count` - Exact count of occurences
`UniqueCount` - Approximate counts using HLL+
`Average` - Exact average
`Pair` - Take 2 aggregates and club them together
`By` - Take enumerable number of key and aggregate of same type (Very useful to capture aggregate for various statuses which are queried together)
To query
`http:///metrics?name=&tags=&tags=&duration=&startTime=&endTime=`
If the metric is emitted with tags, it has to be queried with tags (Work in progress to add explosion rules to also aggregate without tags)
TODO:
----
- Support for percentiles with QDigest.
- Auto rollups for coarser timestamps.
- Integrate with distributed KV store.