https://github.com/citizen-thayne/elixir-billion-row-challenge
Learning Elixir through doing the Billion Row Challenge
https://github.com/citizen-thayne/elixir-billion-row-challenge
benchmarking billion-rows-challenge elixir mix
Last synced: 6 months ago
JSON representation
Learning Elixir through doing the Billion Row Challenge
- Host: GitHub
- URL: https://github.com/citizen-thayne/elixir-billion-row-challenge
- Owner: Citizen-Thayne
- Created: 2024-09-12T20:02:53.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-09-12T20:17:28.000Z (almost 2 years ago)
- Last Synced: 2025-01-22T06:26:26.925Z (over 1 year ago)
- Topics: benchmarking, billion-rows-challenge, elixir, mix
- Language: Elixir
- Homepage:
- Size: 426 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Elixir Billion Row Challenge
The goal of this project is to tackle the challenge of processing extremely large datasets with Elixir, leveraging its concurrency and functional programming capabilities. This repository provides a practical example of how to manage and manipulate a billion rows of data efficiently.
## The Challenge
The Billion Row Challenge is a simple task that involves processing a large dataset. The dataset consists of a billion rows, each containing a city and a measurement of the weather temperature. The goal is to calculate the average temperature for each city.
# Dependencies
This project requires Elixir to be installed on your machine. You can find the installation instructions [here](https://elixir-lang.org/install.html).
# Installation
```bash
git clone https://github.com/Citizen-Thayne/elixir-billion-row-challenge.git
mix deps.get
```
# Running the project
Benchmarks built with Benchee are available as a mix task
**Generating the dataset**
```bash
mix benchmark generator
```
**Calculating the average temperature for each city**
```bash
mix benchmark average
```
# TODO
1. Collect series of benchmark results
2. Render benchmark results