{"id":17091430,"url":"https://github.com/jhurliman/node-streamcount","last_synced_at":"2026-03-08T16:34:20.249Z","repository":{"id":8632082,"uuid":"10278288","full_name":"jhurliman/node-streamcount","owner":"jhurliman","description":"Provides implementations of \"sketch\" algorithms for real-time counting of stream data","archived":false,"fork":false,"pushed_at":"2023-09-08T23:43:00.000Z","size":156,"stargazers_count":45,"open_issues_count":1,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-26T16:39:04.775Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jhurliman.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-05-25T01:50:49.000Z","updated_at":"2021-09-04T03:20:21.000Z","dependencies_parsed_at":"2022-09-26T18:51:33.788Z","dependency_job_id":null,"html_url":"https://github.com/jhurliman/node-streamcount","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jhurliman%2Fnode-streamcount","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jhurliman%2Fnode-streamcount/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jhurliman%2Fnode-streamcount/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jhurliman%2Fnode-streamcount/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jhurliman","download_url":"https://codeload.github.com/jhurliman/node-streamcount/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248487689,"owners_count":21112191,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-14T13:58:38.192Z","updated_at":"2026-03-08T16:34:15.210Z","avatar_url":"https://github.com/jhurliman.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# node-streamcount\n\n[![Build Status](https://travis-ci.org/jhurliman/node-streamcount.png)](https://travis-ci.org/jhurliman/node-streamcount)\n\nProvides implementations of \"sketch\" algorithms for real-time counting of\nstream data.\n\nFor an overview of the type of problems these algorithms solve, read\n[The Britney Spears Problem](http://www.americanscientist.org/issues/pub/the-britney-spears-problem)\nand Wikipedia's article on [Streaming algorithm](http://en.wikipedia.org/wiki/Streaming_algorithm).\n\nThe currently implemented algorithms include:\n\n* HyperLogLog\n* Count-Min sketch\n\n## Download\n\nThe source is available for download from\n[GitHub](http://github.com/jhurliman/node-streamcount).\nAlternatively, you can install using Node Package Manager (npm):\n\n    npm install streamcount\n\n## Quick Example\n\n```js\nvar streamcount = require('streamcount');\n\n// Create a stream counter to track unique visitors with a 1% margin of error.\nvar uniques = streamcount.createUniquesCounter(0.01);\n\n// Add some observations\nuniques.add('user1');\nuniques.add('user2');\nuniques.add('user3');\nuniques.add('user2');\n\n// Prints 3.000274691735112\nconsole.log(uniques.count());\n\n\n// Create a stream counter to track the top 3 pages viewed on our site.\nvar pageCounts = streamcount.createViewsCounter(3);\n\n// Add some observations\npageCounts.increment('/');\npageCounts.increment('/');\npageCounts.increment('/product1');\npageCounts.increment('/contact');\npageCounts.increment('/product3');\npageCounts.increment('/');\npageCounts.increment('/about');\npageCounts.increment('/about');\npageCounts.increment('/product2');\npageCounts.increment('/product1');\npageCounts.increment('/');\npageCounts.increment('/product1');\n\n// Prints [ [ 4, '/' ], [ 3, '/product1' ], [ 2, '/about' ] ]\nconsole.dir(pageCounts.getTopK());\n```\n\n## streamcount Documentation\n\n\u003ca name=\"createUniquesCounter\" /\u003e\n### createUniquesCounter\n\nCreates an object for tracking the approximate total number of unique IDs\nobserved. A common example is estimating the number of unique visitors to\na website. Returns a [HyperLogLog](#HyperLogLog) object.\n\n__Arguments__\n\n* stdError - (Optional) A value from (0-1) indicating the acceptable error\n  rate. This controls the accuracy / memory usage tradeoff. 0.01 is the\n  default.\n\n\u003ca name=\"createViewsCounter\" /\u003e\n### createViewsCounter\n\nCreates an object for tracking estimated top view counts for many unique\nIDs. A common example is tracking the most viewed products on a website.\nReturns a [CountMinSketch](#CountMinSketch) object.\n\n__Arguments__\n\n* topEntryCount - Maximum number of top entries to return view counts for. This\n  is the maximum size of the array returned by getTopK().\n* errFactor - (Optional) The estimated view counts returned by getTopK() can be\n  off by up to this percentage (0-1). This, combined with failRate, controls\n  the accuracy / memory usage tradeoff. 0.002 is the default.\n* failRate - (Optional) The probability of getting the answer for a query\n  completely wrong. From (0-1). This, combined with errFactor, controls the\n  accuracy / memory usage tradeoff. 0.0001 is the default.\n\n\u003ca name=\"getUniquesObjSize\" /\u003e\n### getUniquesObjSize\n\nReturns the serialized size of a uniques counter (HyperLogLog) object in\nbytes given a stdError. __NOTE:__ The memory usage will be higher than this\nnumber since we serialize 32-bit integers but JavaScript uses 64-bit numbers.\n\n__Arguments__\n\n* stdError - Parameter to createUniquesCounter() to estimate storage\n  requirements for.\n\n\u003ca name=\"getViewsObjSize\" /\u003e\n### getViewsObjSize\n\nReturns the serialized size of a views counter (CountMinSketch) object in\nbytes given an errFactor and failRate. __NOTE:__ This does not include the size\nof the serialized MinHeap which includes the size of each unique ID (up to a\nmax of topEntryCount) plus 5 bytes overhead per entry. __NOTE2:__ The memory\nusage will be higher than this number since we serialize 32-bit integers but\nJavaScript uses 64-bit numbers.\n\n__Arguments__\n\n* errFactor - Parameter to createViewsCounter() to estimate storage\n  requirements for.\n* failRate - Parameter to createViewsCounter() to estimate storage requirements\n  for.\n\n## HyperLogLog Documentation\n\n\u003ca name=\"HyperLogLog\" /\u003e\n### HyperLogLog\n\nInitializes a HyperLogLog object. Takes the same parameters as\n[createUniquesCounter](#createUniquesCounter).\n\n__Example__\n\n```js\nvar HyperLogLog = require('streamcount').HyperLogLog;\nvar uniques = new HyperLogLog();\n```\n\n### add\n\nAdd a member to the set.\n\n__Arguments__\n\n* key - String identifier to add to the set.\n\n### count\n\nCount the number of unique members in the set. Returns the estimated\ncardinality of the set.\n\n### serialize\n\nSerializes this data structure to a binary buffer. Returns a binary Buffer\nholding the serialized form of this structure.\n\n### HyperLogLog.deserialize\n\nStatic method to deserialize a binary buffer into a reconstituted HyperLogLog\nstructure.\n\n__Arguments__\n\n* buffer - Binary buffer holding the serialized structure.\n* start - Starting offset of the structure in the buffer.\n* length - Length of the serialized structure in the buffer.\n\n__Example__\n\n```js\nvar uniques = HyperLogLog.deserialize(bufferData);\n```\n\n### merge\n\nMerge another HyperLogLog structure of the same size into this one. This makes\nit possible to keep a local HyperLogLog object in memory on each webserver, and\nperiodically serialize-\u003esend-\u003edeserialize-\u003emerge the results into a single\ncount.\n\n__Arguments__\n\n* hyperLogLog - The other HyperLogLog object to merge in.\n\n## CountMinSketch Documentation\n\n\u003ca name=\"CountMinSketch\" /\u003e\n### CountMinSketch\n\nInitializes a CountMinSketch object. Takes the same parameters as\n[createViewsCounter](#createViewsCounter).\n\n__Example__\n\n```js\nvar CountMinSketch = require('streamcount').CountMinSketch;\nvar topten = new CountMinSketch(10);\n```\n\n### increment\n\nRecord an observation of the given key.\n\n__Arguments__\n\n* key - String identifier to increment the observation count for.\n\n### getTopK\n\nReturns a sorted list of tuples containing the estimated frequency count\nand key for the maxEntries top observed members. Returns an array of length\ntopEntryCount, containing arrays of length 2 where the first value is the\nestimated frequency count and the second value is the given key.\n\n### serialize\n\nSerializes this data structure to a binary buffer. Returns a binary Buffer\nholding the serialized form of this structure.\n\n### CountMinSketch.deserialize\n\nStatic method to deserialize a binary buffer into a reconstituted\nCountMinSketch structure.\n\n__Arguments__\n\n* buffer - Binary buffer holding the serialized structure.\n* start - Starting offset of the structure in the buffer.\n* length - Length of the serialized structure in the buffer.\n\n__Example__\n\n```js\nvar pageCounts = CountMinSketch.deserialize(bufferData);\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjhurliman%2Fnode-streamcount","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjhurliman%2Fnode-streamcount","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjhurliman%2Fnode-streamcount/lists"}