{"id":18267472,"url":"https://github.com/freaktechnik/twitchbots-stats","last_synced_at":"2025-04-09T02:26:46.054Z","repository":{"id":44160349,"uuid":"67546580","full_name":"freaktechnik/twitchbots-stats","owner":"freaktechnik","description":"Some word frequency analysis on usernames of bots from twitchbots.info","archived":false,"fork":false,"pushed_at":"2023-03-14T09:22:54.000Z","size":231,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-02-14T20:46:52.898Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/freaktechnik.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-09-06T21:01:36.000Z","updated_at":"2022-02-11T15:58:17.000Z","dependencies_parsed_at":"2022-09-05T14:41:07.934Z","dependency_job_id":null,"html_url":"https://github.com/freaktechnik/twitchbots-stats","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/freaktechnik%2Ftwitchbots-stats","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/freaktechnik%2Ftwitchbots-stats/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/freaktechnik%2Ftwitchbots-stats/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/freaktechnik%2Ftwitchbots-stats/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/freaktechnik","download_url":"https://codeload.github.com/freaktechnik/twitchbots-stats/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247964248,"owners_count":21025166,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-05T11:27:32.895Z","updated_at":"2025-04-09T02:26:46.029Z","avatar_url":"https://github.com/freaktechnik.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# twitchbots-stats\nCounts the occurence of substrings in a list of strings, in this project the\nusernames of all bots registered on [twitchbots.info](https://twitchbots.info).\n\nIt can count the occurence of substrings of a minimum length of 1 and up with\nthe efficiency staying about the same, though bigger minimum lengths will speed\nit up enormously, since the set of strings to operate on gets way smaller.\n\nThe occurences are counted by traversing a tree of existing substrings and\ncounting how many strings from the parent node still contain the longer\nsubstring. It also prunes shorter substrings with only one child if that has the\nsame occurence (i.e. no string has the shorter substring at the end of it).\n\nThe list of usernames is cached to the disk in a file called `bots.json`.\n\n## Usage\nAll parameters have to be tweaked within `index.js`, including the output\nformat. By default it prints a table with the twenty most common substrings.\n\n```shell\nnpm i\nnode index.js\n```\n\nThere is a (in theory) re-usable module for the occurence calculation in `count-occurences.js`.\n\n## Efficiency\nI'm not exactly sure what the complexity of it is, but it's probably somewhere\nin the `O(n * logn)` region.\n\nThe tree is traversed by generation, so all nodes on the same level are visited\nin one loop, then the next ones etc. This means the algorithm builds memory for\neach substring of strings that still included it and of letters that may follow\nit. I believe it is optimized so the only performance benefits to be gained may\nfit in the following categories:\n\n - Different traversal method\n - Less read/write intensive data structures\n - Optimization of V8 edge cases\n\nThis is a table of runtimes I have experienced (all with node 6) on 4234 strings:\n\nMinimum Length | Intel Pentium N3520 Time [s] | Intel Core i5 4460 Time [s]\n---------------|------------------------------|----------------------------\n3              | 1.8                          | 0.5\n5              | 1.8                          | 0.5\n11             | 1.0                          | 0.2\n14             | 0.4                          | 0.1\n\nThese numbers - despite both CPUs being able to adjust their clock speed - are\npretty stable. As can be seen, the big speed improvements happen with bigger\nnumbers where the amount of strings is reduced considerably - at minimum length\n11 there are about a quarter of the strings left.\n\n## Thanks\nThanks to liori from the tatoeba room for reminding me, that I should fix my\ntraversal order. Without them this would be way slower (like 1000x slower or\nsomething).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffreaktechnik%2Ftwitchbots-stats","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffreaktechnik%2Ftwitchbots-stats","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffreaktechnik%2Ftwitchbots-stats/lists"}