{"id":22843357,"url":"https://github.com/milahu/sum-parallel","last_synced_at":"2025-06-13T09:04:51.581Z","repository":{"id":206701063,"uuid":"717479357","full_name":"milahu/sum-parallel","owner":"milahu","description":"summarize numbers from a text file using multiple CPU cores","archived":false,"fork":false,"pushed_at":"2023-11-11T17:39:57.000Z","size":5,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-06T09:37:11.556Z","etag":null,"topics":["arithmetic","arithmetic-summation","arithmetics","parallel-computing","parallel-processing","summarization"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/milahu.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"license.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-11-11T15:46:40.000Z","updated_at":"2023-11-11T15:51:39.000Z","dependencies_parsed_at":"2023-11-11T18:26:07.604Z","dependency_job_id":"66287ab8-bc91-4929-942f-160d1232f2dd","html_url":"https://github.com/milahu/sum-parallel","commit_stats":null,"previous_names":["milahu/sum-parallel"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/milahu%2Fsum-parallel","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/milahu%2Fsum-parallel/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/milahu%2Fsum-parallel/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/milahu%2Fsum-parallel/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/milahu","download_url":"https://codeload.github.com/milahu/sum-parallel/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246418646,"owners_count":20773938,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arithmetic","arithmetic-summation","arithmetics","parallel-computing","parallel-processing","summarization"],"created_at":"2024-12-13T02:14:28.453Z","updated_at":"2025-03-31T05:11:48.274Z","avatar_url":"https://github.com/milahu.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# sum-parallel\n\nsummarize numbers from a text file using multiple CPU cores\n\n## status\n\nthe current [sum-parallel.c](sum-parallel.c) is slower than [sum.c](sum.c)\n\n```\nseq 10000000 \u003einput.txt\ngcc -o sum sum.c \u0026\u0026 time ./sum \u003cinput.txt\ngcc -o sum-parallel sum-parallel.c \u0026\u0026 time ./sum-parallel input.txt\n```\n\n```\n50000005000000\n\nreal    0m4.611s\nuser    0m4.553s\nsys     0m0.058s\n50000005000000\n\nreal    0m6.754s\nuser    0m12.317s\nsys     0m2.780s\n```\n\n## cpu-bound or io-bound\n\n[What do the terms \"CPU bound\" and \"I/O bound\" mean?](https://stackoverflow.com/questions/868568/what-do-the-terms-cpu-bound-and-i-o-bound-mean)\n\n\u003e adding two numbers takes a single CPU cycle, memory reads take about [100 CPU cycles](http://www.eecs.berkeley.edu/~rcs/research/interactive_latency.html) in 2016 hardware.\n\n```console\n$ pv -a input.txt \u003e/dev/null\n[1.52GiB/s]\n\n$ pv -a input.txt | ./sum\n[15.4MiB/s]\n50000005000000\n```\n\n## read multiple numbers with fscanf\n\ncurrently, i read only one number\n\n```c\nfscanf(fptr, \"%d\", \u0026n)\n```\n\nbut fscanf can read multiple values\n\n```c\nfscanf(fptr, \"%d\\n%d\\n%d\\n%d\", \u0026n1, \u0026n2, \u0026n3, \u0026n4)\n```\n\nbut no, fscanf is already buffered\n\nhttps://stackoverflow.com/a/9587245/10440128\n\n\u003e When you use fread or the other file I/O functions in the C standard library, memory is buffered in several places.\n\n\u003e The C library will usually create a buffer for every FILE* you have open. Data is read into this buffers in large chunks. This allows fread to satisfy many small requests without having to make a large number of system calls, which are expensive. This is what people mean when they say fread is buffered.\n\n\u003e The kernel will also buffer files that are being read in the disk cache. This reduces the time needed for the read system call, since if data is already in memory, your program won't have to wait while the kernel fetches it from the disk. The kernel will hold on to recently read files, and it may read ahead for files which are being accessed sequentially.\n\n## run multiple sum in parallel\n\nthis runs serial\n\n```console\n$ time { n=4; for i in $(seq $n); do seq $((1 + 10000000 / n * (i - 1))) $((10000000 / n * i)) | ./sum; done | ./sum; }\n50000005000000\n\nreal    0m5.251s\nuser    0m5.850s\nsys     0m0.442s\n```\n\nthis runs parallel\n\n```console\n$ time { n=2; t=$(mktemp); p=; for i in $(seq $n); do seq $((1 + 10000000 / n * (i - 1))) $((10000000 / n * i)) | ./sum \u003e\u003e$t \u0026 p+=\" $!\"; done; wait $p; ./sum \u003c$t; }\n50000005000000\n\nreal    0m3.190s\nuser    0m6.549s\nsys     0m0.577s\n\n$ time { n=4; t=$(mktemp); p=; for i in $(seq $n); do seq $((1 + 10000000 / n * (i - 1))) $((10000000 / n * i)) | ./sum \u003e\u003e$t \u0026 p+=\" $!\"; done; wait $p; ./sum \u003c$t; }\n50000005000000\n\nreal    0m3.338s\nuser    0m9.707s\nsys     0m0.874s\n```\n\nsuccess, this is 2x faster than serial\n\n`n=4` is not faster, because i have only 2 cpu cores\n\n```console\n$ grep -m1 \"^cpu cores\" /proc/cpuinfo\ncpu cores       : 2\n```\n\n## see also\n\n- [Shell command to sum integers, one per line?](https://stackoverflow.com/questions/450799/shell-command-to-sum-integers-one-per-line)\n- [Add up a column of numbers at the Unix shell](https://stackoverflow.com/questions/926069/add-up-a-column-of-numbers-at-the-unix-shell)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmilahu%2Fsum-parallel","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmilahu%2Fsum-parallel","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmilahu%2Fsum-parallel/lists"}