{"id":20770447,"url":"https://github.com/dannyvankooten/1brc","last_synced_at":"2026-03-08T11:32:10.534Z","repository":{"id":215413567,"uuid":"738599261","full_name":"dannyvankooten/1brc","owner":"dannyvankooten","description":"C11 implementation of the 1 Billion Rows Challenge. 1️⃣🐝🏎️ Runs in ~1.6 seconds on my not-so-fast laptop CPU w/ 16GB RAM.","archived":false,"fork":false,"pushed_at":"2024-06-15T12:10:31.000Z","size":52,"stargazers_count":94,"open_issues_count":8,"forks_count":42,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-12-08T19:21:49.481Z","etag":null,"topics":["1brc","c","c11"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dannyvankooten.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-03T15:48:12.000Z","updated_at":"2024-12-05T06:51:16.000Z","dependencies_parsed_at":"2024-06-13T17:26:41.938Z","dependency_job_id":"01e2c865-ac41-4eb9-a35f-780e96f17df6","html_url":"https://github.com/dannyvankooten/1brc","commit_stats":null,"previous_names":["dannyvankooten/1brc"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dannyvankooten%2F1brc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dannyvankooten%2F1brc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dannyvankooten%2F1brc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dannyvankooten%2F1brc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dannyvankooten","download_url":"https://codeload.github.com/dannyvankooten/1brc/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":229869738,"owners_count":18136931,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["1brc","c","c11"],"created_at":"2024-11-17T12:09:34.174Z","updated_at":"2026-03-08T11:32:10.505Z","avatar_url":"https://github.com/dannyvankooten.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 1️⃣🐝🏎️ The One Billion Row Challenge\n\n- Challenge blog post: https://www.morling.dev/blog/one-billion-row-challenge/\n- Challenge repository: https://github.com/gunnarmorling/1brc\n\nThe challenge: **compute simple floating-point math over 1 billion rows. As fast as possible, without dependencies.**\n\nImplemented in standard C11 with POSIX threads (however, no SIMD). `analyze.c` contains the fastest implementation, while `{1..7}.c` contain slower versions of the same program.\n\nI wrote up some implmentation details on my blog here: https://www.dannyvankooten.com/blog/2024/1brc/\n\n## Running the challenge\n\nFirst, compile the two programs using any capable C-compiler.\n\n```sh\nmake\n```\n\nTo compile in debug mode:\n\n```sh\nDEBUG=1 make\n```\n\nBy default, Make will attempt to find the number of threads to use from `nproc`\nor `sysctl`. To compile while specifying the number of threads to use explicitly:\n\n```sh\nNTHREADS=8 make\n```\n\n### Create the measurements file with 1B rows\n\n```\nbin/create-sample 1000000000\n```\n\nThis will create a 12 GB file with 1B rows named `measurements.txt` in your current working directory. The program to create this sample file will take a minute or two, but you only need to run it once.\n\n### Run the challenge:\n\n```\ntime bin/analyze measurements.txt \u003e/dev/null\n\nreal\t0m1.392s\nuser\t0m0.000s\nsys\t    0m0.010sys\n```\n\n**Note:** the performance difference between a warm and a hot pagecache is quite extreme. Run `echo 3 \u003e /proc/sys/vm/drop_caches` to drop your pagecache, then run the program twice in a row. It's not uncommon for the second run to be well over twice as fast.\n\n\n### Benchmarks\n\nSince I don't have access to a Hetzner CCX33 box, here are the reference times for the currently leading Java implementations from the official challenge when I run them on my machine.\n\n| # | Result (m:s.ms) | Implementation     | Language | Submitter     |\n|---|-----------------|--------------------|-----|---------------|\n| ? |        00:01.590 | [link](https://github.com/dannyvankooten/1brc/blob/main/analyze.c)| C | [Danny van Kooten](https://github.com/dannyvankooten)|\n| 1.|        00:06.131 | [link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_spullara.java)| 21.0.1-graalce| [Sam Pullara](https://github.com/spullara)|\n| 2.|        00:06.421 | [link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_royvanrijn.java)| 21.0.1-graalce   | [Roy van Rijn](https://github.com/royvanrijn)|\n\n\n### Progressions\n\nYou can find the average runtime (across 5 consecutive runs) for the various states of the program below, from baseline to the final and fully optimized version. Because I have no patience, this was run on a measurements file with only 100M rows.\n\n```\n1.c runtime=[ 55.86 59.09 64.28 63.63 56.08 ] average=59.79s   linear-search by city name (baseline)\n2.c runtime=[ 9.14 9.31 9.35 9.05 9.30 ] average=9.23s hashmap with linear probing\n3.c runtime=[ 4.27 4.51 4.47 4.28 4.25 ] average=4.36s custom temperature float parser instead of strod\n4.c runtime=[ 2.38 2.41 2.46 2.40 2.39 ] average=2.41s fread with 64MB chunks instead of line-by-line\n5.c runtime=[ 2.13 1.99 1.99 2.00 2.05 ] average=2.03s unroll parsing of city name and generating hash\n6.c runtime=[ 0.49 0.49 0.49 0.50 0.50 ] average=0.49s parallelize across 16 threads\n7.c runtime=[ 0.30 0.25 0.23 0.24 0.24 ] average=0.25s mmap entire file instead of fread in chunks\n```\n\nYou can run the benchmark script for all progressions by executing `./run-progressions.sh` (needs `bash`, `make`, `time` and `awk`).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdannyvankooten%2F1brc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdannyvankooten%2F1brc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdannyvankooten%2F1brc/lists"}