{"id":20122829,"url":"https://github.com/freaky/cw","last_synced_at":"2025-04-08T04:17:56.894Z","repository":{"id":57615010,"uuid":"164356380","full_name":"Freaky/cw","owner":"Freaky","description":"A Rust wc clone","archived":false,"fork":false,"pushed_at":"2022-10-06T03:39:48.000Z","size":68,"stargazers_count":103,"open_issues_count":2,"forks_count":5,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-24T09:19:56.994Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Freaky.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-01-07T00:02:17.000Z","updated_at":"2024-09-11T16:50:57.000Z","dependencies_parsed_at":"2022-08-27T03:24:34.656Z","dependency_job_id":null,"html_url":"https://github.com/Freaky/cw","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Freaky%2Fcw","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Freaky%2Fcw/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Freaky%2Fcw/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Freaky%2Fcw/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Freaky","download_url":"https://codeload.github.com/Freaky/cw/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247773722,"owners_count":20993639,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-13T19:41:20.846Z","updated_at":"2025-04-08T04:17:56.858Z","avatar_url":"https://github.com/Freaky.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# cw - Count Words\n\nA fast `wc` clone in Rust.\n\n## Synopsis\n\n```\n-% cw --help\ncw 0.5.0\nThomas Hurst \u003ctom@hur.st\u003e\nCount Words - word, line, character and byte count\n\nUSAGE:\n    cw [FLAGS] [OPTIONS] [input]...\n\nFLAGS:\n    -c, --bytes              Count bytes\n    -m, --chars              Count UTF-8 characters instead of bytes\n    -h, --help               Prints help information\n    -l, --lines              Count lines\n    -L, --max-line-length    Count bytes (default) or characters (-m) of the longest line\n    -V, --version            Prints version information\n    -w, --words              Count words\n\nOPTIONS:\n        --files0-from \u003cfiles0_from\u003e    Read input from the NUL-terminated list of filenames in the given file.\n        --files-from \u003cfiles_from\u003e      Read input from the newline-terminated list of filenames in the given file.\n        --threads \u003cthreads\u003e            Number of counting threads to spawn [default: 1]\n\nARGS:\n    \u003cinput\u003e...    Input files\n\n-% cw Dickens_Charles_Pickwick_Papers.xml\n 3449440 51715840 341152640 Dickens_Charles_Pickwick_Papers.xml\n```\n\n## Performance\n\nCounts of multiple files may be accelerated by use of the `--threads` option:\n\n```\n  'xargs \u003cfiles cw --threads=12' ran\n    2.01 ± 0.03 times faster than 'xargs \u003cfiles cw --threads=4'\n    7.07 ± 0.09 times faster than 'xargs \u003cfiles cw'\n   11.55 ± 0.15 times faster than 'xargs \u003cfiles wc'\n   17.31 ± 0.23 times faster than 'xargs \u003cfiles gwc'\n```\n\nLine counts are optimized using the [`bytecount`][bytecount] crate:\n\n```\n  'cw -l Dickens_Charles_Pickwick_Papers.xml' ran\n    3.44 ± 0.04 times faster than 'wc -l Dickens_Charles_Pickwick_Papers.xml'\n    4.17 ± 0.05 times faster than 'gwc -l Dickens_Charles_Pickwick_Papers.xml'\n```\n\nLine counts with line length are optimized using the [`memchr`][memchr] crate:\n\n```\n  'cw -lL Dickens_Charles_Pickwick_Papers.xml' ran\n    1.73 ± 0.01 times faster than 'wc -lL Dickens_Charles_Pickwick_Papers.xml'\n   15.07 ± 0.07 times faster than 'gwc -lL Dickens_Charles_Pickwick_Papers.xml'\n```\n\nNote without `-m` cw only operates on bytes, and it never cares about your locale.\n\n```\n  'cw Dickens_Charles_Pickwick_Papers.xml' ran\n    1.45 ± 0.01 times faster than 'wc Dickens_Charles_Pickwick_Papers.xml'\n    2.05 ± 0.00 times faster than 'gwc Dickens_Charles_Pickwick_Papers.xml'\n```\n\n`-m` enables UTF-8 processing, with a fast-path for just character length, again\nusing `bytecount`:\n\n```\n  'cw -m Dickens_Charles_Pickwick_Papers.xml' ran\n   30.21 ± 0.39 times faster than 'gwc -m Dickens_Charles_Pickwick_Papers.xml'\n   70.36 ± 0.91 times faster than 'wc -m Dickens_Charles_Pickwick_Papers.xml'\n```\n\n```\n  'cw -m test-utf-8.html' ran\n   84.74 ± 1.12 times faster than 'wc -m test-utf-8.html'\n  124.21 ± 1.64 times faster than 'gwc -m test-utf-8.html'\n```\n\nAnd another path for character and line length:\n\n```\n  'cw -mlL Dickens_Charles_Pickwick_Papers.xml' ran\n    3.88 ± 0.01 times faster than 'gwc -mlL Dickens_Charles_Pickwick_Papers.xml'\n    9.05 ± 0.02 times faster than 'wc -mlL Dickens_Charles_Pickwick_Papers.xml'\n```\n\n```\n  'cw -mlL test-utf-8.html' ran\n    9.42 ± 0.01 times faster than 'wc -mlL test-utf-8.html'\n   18.95 ± 0.03 times faster than 'gwc -mlL test-utf-8.html'\n```\n\nAnd a slow path for everything else:\n\n```\n  'cw -mLlw Dickens_Charles_Pickwick_Papers.xml' ran\n    1.35 ± 0.00 times faster than 'gwc -mLlw Dickens_Charles_Pickwick_Papers.xml'\n    3.15 ± 0.00 times faster than 'wc -mLlw Dickens_Charles_Pickwick_Papers.xml'\n```\n\nThese tests are on FreeBSD 12 on a 2.1GHz Westmere Xeon.  `gwc` is from GNU\ncoreutils 8.30 - note its performance here is rather pessimised in some areas by\nFreeBSD's rather weak `memchr` implementation.  YMMV.\n\nFor best results build with:\n\n```\ncargo build --release --features runtime-dispatch-simd\n```\n\nThis enables SIMD optimizations for line and character counting.  It has no\neffect if you count anything else.\n\n\n## Future\n\n * Test suite.\n * Factor internals out into a library. (#1)\n * Improve multibyte support.\n * Possibly implement locale.\n * Replace clap/structopt with something lighter.\n\n\n## See Also\n\n### [uwc]\n\n[uwc] focuses on following Unicode rules as precisely as possible, taking into\naccount less-common newlines, counting graphemes as well as codepoints, and\nfollowing Unicode word-boundary rules precisely.\n\nThe cost of this is currently a great deal of performance, with counts on my\nbenchmark file taking over a minute.\n\n\n### [rwc]\n\ncw was originally called [rwc] until I noticed this existed.  It's quite old and\ndoesn't appear to compile.\n\n\n### [linecount]\n\nA little library that only does plain newline counting, along with a binary\ncalled `lc`.  Version 0.2 will use the same algorithm as `cw`.\n\n\n[bytecount]: https://crates.io/crates/bytecount\n[memchr]: https://crates.io/crates/memchr\n[uwc]: https://crates.io/crates/uwc\n[rwc]: https://crates.io/crates/rwc\n[linecount]: https://crates.io/crates/linecount\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffreaky%2Fcw","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffreaky%2Fcw","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffreaky%2Fcw/lists"}