https://github.com/adityachandak287/1brc

1 Billion Row Challenge
https://github.com/adityachandak287/1brc

Last synced: about 7 hours ago
JSON representation

1 Billion Row Challenge

• Host: GitHub
• URL: https://github.com/adityachandak287/1brc
• Owner: adityachandak287
• Created: 2024-02-05T17:12:02.000Z (10 months ago)
• Default Branch: main
• Last Pushed: 2024-08-26T20:43:35.000Z (3 months ago)
• Last Synced: 2024-08-27T00:39:28.321Z (3 months ago)
• Language: Java
• Size: 40 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# The One Billion Row Challenge

Learn more: https://github.com/gunnarmorling/1brc

## Getting Started

### 1. Create Measurements

```bash

bash scripts/create_measurements.sh 1000000

```

Above example will create measurements file `measurements-1000000.txt`

### 2. Run baseline

```bash

bash scripts/calculate_average_baseline_original_rounding.sh measurements-1000000.txt

```

Baseline average calculation output saved to `average_baseline.txt`

### 3. Run implementation

```bash

# Golang

go run ./go/main.go --file measurements-1000000.txt --impl track_aggregates_v2 > average.txt

# Bun (TypeScript)

bun run node/brute-force.ts measurements-1000000.txt > average.txt

# NodeJS (Javascript)

npm run build

node ./node/dist/v1.js measurements-1000000.txt > average.txt

# Python

python ./python/brute_force.py measurements-1000000.txt > average.txt

```

### 4. Test implementation output against baseline

```bash

bash scripts/test.sh

```

If output is incorrect, test script will exit with non-zero exit code and diff output from baseline.

## Results Comparison

### Run benchmark script

The [benchmark script](scripts/bench.sh) runs all implementations against different number of rows of inputs (10, 100, 1k, 10k, 100k, etc) using [hyperfine](https://github.com/sharkdp/hyperfine) to compare runtime among different implementations.

```bash

bash scripts/bench.sh

```

Refer to [results](results/) directory for comparison results.