{"id":31552387,"url":"https://github.com/phreppo/ct","last_synced_at":"2025-10-04T19:54:54.137Z","repository":{"id":157564254,"uuid":"588114264","full_name":"phreppo/ct","owner":"phreppo","description":"⚡ The fastest line counter. At least on my computer.","archived":false,"fork":false,"pushed_at":"2025-07-05T11:11:37.000Z","size":743,"stargazers_count":16,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-07-05T11:44:13.211Z","etag":null,"topics":["blazingly-fast","cli","line-counter","zig"],"latest_commit_sha":null,"homepage":"","language":"Zig","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/phreppo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-12T11:06:50.000Z","updated_at":"2025-07-05T11:11:41.000Z","dependencies_parsed_at":"2024-04-28T23:26:52.136Z","dependency_job_id":"6ac4b44a-2d8b-478f-8d6c-be4c3553d5a0","html_url":"https://github.com/phreppo/ct","commit_stats":null,"previous_names":["phreppo/ct"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/phreppo/ct","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phreppo%2Fct","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phreppo%2Fct/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phreppo%2Fct/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phreppo%2Fct/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/phreppo","download_url":"https://codeload.github.com/phreppo/ct/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phreppo%2Fct/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278366632,"owners_count":25975090,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-04T02:00:05.491Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["blazingly-fast","cli","line-counter","zig"],"created_at":"2025-10-04T19:54:48.374Z","updated_at":"2025-10-04T19:54:54.132Z","avatar_url":"https://github.com/phreppo.png","language":"Zig","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CounT\n\n⚡ The fastest line counter on my computer ⚡\n\n![Benchmarks](./img/benchmarks-single-thread.png)\n\n## Usage\n\n![Usage example](img/usage.png)\n\nSee the usage with `ct --help`.\n```\nusage: ct [OPTIONS] [input]...\nOPTIONS\n\t--threads,-t \u003cthreads\u003e          Sets the number of threads to use. (default: 1)\n\t--chunks-size,-c \u003cchunks-size\u003e  Sets the size (in bytes) of the chunks allocated. (default: 2048Kb)\n\t--help,-h                       Prints the help message.\nARGS\n\t\u003cinput\u003e\t\tPath to the input file(s).\n```\n\n## The Algorithm\n\n`ct` uses a multithreaded algorithm that reads the input file in chunks.\nIt allocates a fixed amount of memory for each thread (that can be set with `--chunks-size`), and reuses it for the whole execution of the program.\nThis minimizes the number of allocations and makes the execution quite fast.\nSetting the `--chunk-size` parameter properly can have a huge impact on the performance.\n\n## Benchmarks\n\nThere is a script file to generate test data for comparison.\nRun `./scripts/gen.zig` to generate files of different sizes, up to 16Gb.\nThe files are saved in the directory `./files`.\nGenerating the files might take a while.\nThey are generated by a blazingly fast Zig script.\nYou can reproduce the benchmarks by running `make benchmark`.\nYou need `hyperfine` and `cw` installed.\nTo see how the results are obtained check `./scripts/benchmark.sh`.\n\n### Single Thread Benchmarks\n\nI compared `ct` to `wc -l`, [`cw -l`](https://github.com/Freaky/cw) and a trivial Python script.\nThe Python script (`./scripts/countlines.py`) fails on the 16Gb file, so that I do not include it in what follows.\n\n![Single thread results](img/benchmarks-single-thread.png)\n\n### Multithreaded Benchmarks\n\n`ct`'s algorithm is optimized for multithreading.\n`wc -l` is not multithreaded, so that we have to compare to sequential code.\nHere are the results using 4 threads in `ct` and `cw`:\n\n![Unfair results](./img/benchmarks-unfair.png)\n\nThe results are unfair, because `cw` does not allow multithreaded execution on a single file (at least in version `0.7.0`), as it puts each different input file on a different thread.\nThis implies that for a single file, `ct` is by far the fastest line counter out there!\nTo perform a fair comparison with `cw`, we have to use as many files as threads, so that the tool can parallelize optimally its computations.\nHere, I use 4 threads.\n\n![Fair multithreaded benchmarks](img/benchmarks-multithreaded.png)\n\nObserve that this case is particularly advantageous for `cw` because the tasks are perfectly balanced.\nBy giving as input files of different sizes `cw` does not scale lineraly.\n`ct` scales better in this case because it divides the overall work fairly over the threads.\n\n### Scalability\n`ct` scales reasonably well:\n\n![Scalability benchmarks](img/scalability.png)\n\n### My Machine\n\n* Ubuntu 20.04.6 LTS\n* Intel® Core™ i7-8650U CPU @ 1.90GHz × 8\n* 16Gb Memory\n* 512Gb SSD\n\n## Building\n\nTo build simply run `make`.\nIf you want to build the optimized version use `make release`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fphreppo%2Fct","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fphreppo%2Fct","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fphreppo%2Fct/lists"}