https://github.com/seiflotfy/count-min-log

Go implementation of Count-Min-Log
https://github.com/seiflotfy/count-min-log

Last synced: 27 days ago
JSON representation

Go implementation of Count-Min-Log

• Host: GitHub
• URL: https://github.com/seiflotfy/count-min-log
• Owner: seiflotfy
• License: mit
• Created: 2015-08-16T22:31:36.000Z (over 9 years ago)
• Default Branch: master
• Last Pushed: 2017-02-12T13:09:21.000Z (almost 8 years ago)
• Last Synced: 2024-07-31T20:47:59.614Z (4 months ago)
• Language: Go
• Homepage:
• Size: 34.2 KB
• Stars: 66
• Watchers: 4
• Forks: 5
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-go - count-min-log - Go implementation Count-Min-Log sketch: Approximately counting with approximate counters (Like Count-Min sketch but using less memory). (Data Structures and Algorithms / Miscellaneous Data Structures and Algorithms)
• awesome-go - count-min-log - Go implementation Count-Min-Log sketch: Approximately counting with approximate counters (Like Count-Min sketch but using less memory). (Data Structures and Algorithms / Miscellaneous Data Structures and Algorithms)
• awesome-go - count-min-log - Go implementation of Count-Min-Log - ★ 39 (Data Structures)
• awesome-go-extra - count-min-log - Min-Log|57|5|0|2015-08-16T22:31:36Z|2017-02-12T13:09:21Z| (Generators / Miscellaneous Data Structures and Algorithms)

README

        
# Count-Min-Log

[![GoDoc](https://godoc.org/github.com/seiflotfy/count-min-log?status.svg)](https://godoc.org/github.com/seiflotfy/count-min-log)

[Count-Min-Log sketch: Approximately counting with approximate counters - Guillaume Pitel & Geoffroy Fouquier](http://iswag-symposium.org/2015/pdfs/shortpaper1.pdf)

TL;DR: Count-Min-Log Sketch for improved Average Relative Error on low frequency events

Count-Min Sketch is a widely adopted algorithm for approximate event counting in large scale processing. However, the original version of the Count-Min-Sketch (CMS) suffers of some deficiences, especially if one is interested in the low-frequency items, such as in text- mining related tasks. Several variants of CMS have been proposed to compensate for the high relative error for low-frequency events, but the proposed solutions tend to correct the errors instead of preventing them. In this paper, we propose the Count-Min-Log sketch, which uses logarithm-based, approximate counters instead of linear counters to improve the average relative error of CMS at constant memory footprint.

This version implements the 16 bit register version. Will add back the 8-bit version soon.

## Example Usage

```go

import cml

...

sk, err := cml.NewDefaultSketch()

sk.IncreaseCount([]byte("scott pilgrim"))

...

sk.Frequency([]byte("scott pilgrim")) // ==> 1

```