{"id":19282516,"url":"https://github.com/rrcook/brc","last_synced_at":"2025-04-22T01:31:55.933Z","repository":{"id":217414015,"uuid":"743247489","full_name":"rrcook/brc","owner":"rrcook","description":"Elixir implementation of the 1 billion row challenge","archived":false,"fork":false,"pushed_at":"2024-01-16T03:34:25.000Z","size":8,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-01-16T08:26:21.972Z","etag":null,"topics":["1brc","elixir"],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rrcook.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2024-01-14T19:07:39.000Z","updated_at":"2024-01-16T08:26:39.063Z","dependencies_parsed_at":"2024-01-16T08:26:36.340Z","dependency_job_id":"4a0cf835-8e36-4027-a1aa-41600fe3e3c5","html_url":"https://github.com/rrcook/brc","commit_stats":null,"previous_names":["rrcook/brc"],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rrcook%2Fbrc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rrcook%2Fbrc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rrcook%2Fbrc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rrcook%2Fbrc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rrcook","download_url":"https://codeload.github.com/rrcook/brc/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223886254,"owners_count":17219705,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["1brc","elixir"],"created_at":"2024-11-09T21:27:12.247Z","updated_at":"2024-11-09T21:27:12.830Z","avatar_url":"https://github.com/rrcook.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Brc\n\nUpdate 1\nThe main switch is from using File.stream to erlang's prim_file.\nFor some reason, and I want to investigate this, is that the File.stream based approach only used 8 of 16 processors. That's on my 8 physical CPU, 16 logical CPU machine.\n\nErlang's prim_file is very handy here. You can read a block of bytes, then at that stopping point you can read_line to get to the end of the line that could have been chopped off from the block read.\nThanks to icedragon200 for pointing me to prim_file.\n\nOriginal\n\nMy attempt at the billion row challenge in Elixir. \nElixir 1.16.0 - important because the argument order for File.stream changed between 1.15 and 1.16.\nVanilla elixir, no extra libraries to run. I am including eflambe to run \u0026 make flamegraphs to help tune performance.\n\nbrc_city was my first attempt. It used a process for every city. Don't do this, most of your app's time will be spent in process sleeping.\n\nbrc uses a pool of workers. Each worker receives a list of cities. Fewer workers with more work eliminates idle time.\n\nIf using eflambe, run something like\n\niex -S mix\n\n:eflambe.apply({Brc, :run_file_buf, [\"measurements.txt\"]}, [output_format: :brendan_gregg, open: :speedscope])\n\n\n## Installation\n\nAfter cloning, run 'mix deps.get' to get eflambe. Then 'mix escript.build', then './brc measurments.txt'\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frrcook%2Fbrc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frrcook%2Fbrc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frrcook%2Fbrc/lists"}