https://github.com/fzakaria/space-saving

Space Saving algorithm implementation (StreamSummary) in Java, used to solve heavy hitters / topk items.
https://github.com/fzakaria/space-saving

algorithm java topk

Last synced: 17 days ago
JSON representation

Space Saving algorithm implementation (StreamSummary) in Java, used to solve heavy hitters / topk items.

• Host: GitHub
• URL: https://github.com/fzakaria/space-saving
• Owner: fzakaria
• License: mit
• Created: 2016-08-24T23:00:33.000Z (about 8 years ago)
• Default Branch: master
• Last Pushed: 2016-08-24T23:05:51.000Z (about 8 years ago)
• Last Synced: 2024-10-11T23:34:23.772Z (about 1 month ago)
• Topics: algorithm, java, topk
• Language: Java
• Size: 15.6 KB
• Stars: 31
• Watchers: 4
• Forks: 9
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# SpaceSaving

Space Saving algorithm implementation in Java, also know as "HeavyHitter"

The purpose of the algorithm is to find the most frequently used items from an infinite stream.

In other words, even if you have millions of different values, the algorithm will try to find the most frequently used

within some acceptable error bound.

The datastructure used is described by the paper [Efficient Computation of Frequent and Top-k Elements in Data Streams](http://www.cse.ust.hk/~raywong/comp5331/References/EfficientComputationOfFrequentAndTop-kElementsInDataStreams.pdf)

by *Ahmed Metwally, Divyakant Agrawal, and Amr El Abbadi*

It is a a ϴ(1) "constant time" insert and  ϴ(k) lookup for the *topk* entries.

```

/**

 An epsilon of 0.001 will create 1000 counters and dictates the accuracy of the elements in the

 resulting topk based on the size of the stream.

 If your stream is 1,000,000 items an item is truly guarenteed to be in the top K elements if

 the frequency reported - error reported > 1000 [0.001 * 1,000,000 = 1000]

 Since top k only cares with skewed data, we generally care for elements in the stream that probably

 have way more than 1000 entries in the stream.

**/

final double epsilon = 0.001;

StreamSummary streamSummary = new StreamSummary<>(0.01);

streamSummary.offer(item);

```

## Aside

This repository is created because although other implementations existed, I was frustrated that

they did not include any unit tests or thorough documentation explaining some of the error bounds.