{"id":20390340,"url":"https://github.com/timvisee/wrdcntr","last_synced_at":"2025-04-12T11:21:29.434Z","repository":{"id":66230252,"uuid":"124603568","full_name":"timvisee/wrdcntr","owner":"timvisee","description":":dash: A simple yet very fast word counter witten in Rust","archived":false,"fork":false,"pushed_at":"2022-06-07T08:31:59.000Z","size":11851,"stargazers_count":7,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-26T06:11:13.648Z","etag":null,"topics":["concurrency","mapreduce","rayon","rust","wordcount"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/timvisee.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-03-09T23:19:55.000Z","updated_at":"2022-06-25T15:10:59.000Z","dependencies_parsed_at":null,"dependency_job_id":"7e09abe5-9dd6-4013-8728-d2370ca028ad","html_url":"https://github.com/timvisee/wrdcntr","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timvisee%2Fwrdcntr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timvisee%2Fwrdcntr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timvisee%2Fwrdcntr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timvisee%2Fwrdcntr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/timvisee","download_url":"https://codeload.github.com/timvisee/wrdcntr/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248558128,"owners_count":21124223,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["concurrency","mapreduce","rayon","rust","wordcount"],"created_at":"2024-11-15T03:24:25.721Z","updated_at":"2025-04-12T11:21:29.427Z","avatar_url":"https://github.com/timvisee.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# wrdcntr - a fast word counter in Rust\nThis is a simple yet very fast word counter written in [Rust][rust].\n\nThe counter reads all text from a given file, counts all words,\nand prints the number of occurrences for each word in alphabetical order.\n\nAll counting work is split across all CPU cores to count as quickly as possible,\nand is done in a [map/reduce][mapreduce]-like manner.\n\nWords contain any alpha numeric character, and are separated by any\nnon-alphanumeric character. Spaces, punctuation and so on are not counted.\n\nThe goal of this project is to show how powerful the Rust language can be with\nminimal effort.\n\n## Benchmark\nHere are some basic benchmarks with files from the [`samples`](samples/)\ndirectory.  \nThese benchmarks were done with the [hyperfine][hyperfine] tool,\neach timing 10 runs.\n\n```\n# 511KB with 115478 words\nhyperfine 'wrdcntr samples/book.txt --no-output'\n# Time (mean ± σ):   10.4 ms ±  1.4 ms\n# Range (min … max):  8.7 ms … 19.0 ms\n# [User: 22.0 ms, System: 2.4 ms]\n\n# 30MB with 7205401 words\nhyperfine 'wrdcntr samples/many_books.txt --no-output'\n# Time (mean ± σ):   344.6 ms ±   9.9 ms\n# Range (min … max): 333.9 ms … 370.2 ms\n# [User: 865.0 ms, System: 37.7 ms]\n\n# 35KB with 7074 words\nhyperfine 'wrdcntr LICENSE --no-output'\n# Time (mean ± σ):   2.4 ms ±  1.0 ms    \n# Range (min … max): 1.3 ms … 12.5 ms\n# [User: 3.2 ms, System: 1.2 ms]\n# Note: possibly inaccurate, because timings were less than 5ms\n```\n\nThese benchmarks were done on a machine running Linux with a\n4-core i5-4670K @4.1Ghz CPU and 16GB RAM.\n\nCounting files of 1GB is also fast, and nicely saturates all cores:\n\n![Counting 1GB of words on a 32-core server](cpu-usage.png)\n\n## Usage\nTo use the word counter, supply a file:\n```bash\n# Count words\nwrdcntr samples/book.txt\n```\n\nWhich outputs:\n\n```\n...\nAbout: 12\nAbove: 1\nAbsolutely: 1\nActually: 1\nAdd: 1\nAdmiration: 1\nAdrenaline: 1\nAfter: 37\nAgain: 1\n...\n```\n\n## Installation\nFor installation, the project must be compiled from source.\nGit and Rust cargo are required.\nInstall the latest version of Rust with [rustup][rustup].\n\nThen, clone and install `wrdcntr` with:\n\n```bash\n# Clone the project\ngit clone https://github.com/timvisee/wrdcntr.git\ncd wrdcntr\n\n# Install wrdcntr\ncargo install -f\n\n# Start using wrdcntr\nwrdcntr --help\n\n# or run it directly from Cargo\ncargo run --release -- --help\n```\n\nOr just build and invoke the binary directly (Linux/macOS):\n\n```bash\n# Clone the project\ngit clone https://github.com/timvisee/wrdcntr.git\ncd wrdcntr\n\n# Build the project (release version)\ncargo build --release\n\n# Start using wrdcntr\n./target/release/wrdcntr --help\n```\n\n## License\nThis project is released under the GNU GPL-3.0 license.\nCheck out the [LICENSE](LICENSE) file for more information.\n\n\n[hyperfine]: https://github.com/sharkdp/hyperfine\n[mapreduce]: https://en.wikipedia.org/wiki/MapReduce\n[rust]: https://rust-lang.org/\n[rustup]: https://rustup.rs/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimvisee%2Fwrdcntr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftimvisee%2Fwrdcntr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimvisee%2Fwrdcntr/lists"}