https://github.com/kaltura/kanalony

Kaltura's next generation Analytics solution based on Spark, Cassandra and Kafka
https://github.com/kaltura/kanalony

Last synced: 3 months ago
JSON representation

Kaltura's next generation Analytics solution based on Spark, Cassandra and Kafka

• Host: GitHub
• URL: https://github.com/kaltura/kanalony
• Owner: kaltura
• License: agpl-3.0
• Created: 2015-09-17T07:54:13.000Z (about 9 years ago)
• Default Branch: v1.0
• Last Pushed: 2023-03-31T21:16:34.000Z (over 1 year ago)
• Last Synced: 2024-07-18T20:45:42.609Z (4 months ago)
• Language: Scala
• Size: 1.42 MB
• Stars: 12
• Watchers: 22
• Forks: 4
• Open Issues: 4
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# kAnalony Project

## Goals

The purpose of the kAnalony project is to create a robust and scalable infrastructure to accommodate the existing and future Analytical needs of Kaltura:

* Aggregate event-based time-series data using predefined configurable dimensions on predefined time resolutions

* Custom batch processing on aggregated data

* Custom batch processing on raw data

* Expose raw events data for preferred clients

## Background

There are currently 3 different Analytics systems in Kaltura:

1. Comprehensive metrics for VOD data, based on Pentaho's Kettle open source project and MySQL

2. Basic metrics for Live entries (including DVR) data, based on Apache Spark and Cassandra

3. Comprehensive metrics for OTT data, based on Sisense product

The new kAnalony project will create a scalable and robust infrastructure to facilitate the requirements of all current 3 systems.

It is based on [Apache Spark](https://spark.apache.org/), [Cassandra](http://cassandra.apache.org/) and [Kafka](http://kafka.apache.org/), 

written mostly with [Scala](http://www.scala-lang.org/).

## Why kAnalony?

The project name is a combination of the words Kaltura and Analytics, 

and sound like the word Cannelloni which is a cylindrical type of pasta generally served baked with a filling and covered by a sauce in Italian cuisine (Source: [Wikipedia](http://en.wikipedia.org/wiki/Cannelloni)).

## Components

* **kAnalony Receivers** - a group of Node.js servers behind a load balancer, which receive analytics events and push a transformation of them to a Kafka cluster.

## Installation

* [Kafka Cluster Installation](https://kaltura.atlassian.net/wiki/display/KANAL/Kafka+Cluster+Installation)

* [Spark-Cassandra Cluster Installation](https://kaltura.atlassian.net/wiki/display/KANAL/Spark-Cassandra+Cluster+Installation)