https://github.com/1git2clone/polars-vs-pandas

CSV heading parsing speed comparison for a 1 million line randomly generated CSV file.
https://github.com/1git2clone/polars-vs-pandas

benchmarking pandas pandas-dataframe polars polars-dataframe

Last synced: 3 months ago
JSON representation

CSV heading parsing speed comparison for a 1 million line randomly generated CSV file.

• Host: GitHub
• URL: https://github.com/1git2clone/polars-vs-pandas
• Owner: 1Git2Clone
• License: mit
• Created: 2025-02-15T22:28:55.000Z (3 months ago)
• Default Branch: main
• Last Pushed: 2025-02-15T22:51:56.000Z (3 months ago)
• Last Synced: 2025-02-15T23:26:11.630Z (3 months ago)
• Topics: benchmarking, pandas, pandas-dataframe, polars, polars-dataframe
• Language: Rust
• Homepage:
• Size: 0 Bytes
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Pandas vs Polars

[![Build Icon]][Build Status] [![License Icon]][LICENSE]

[Build Icon]: https://img.shields.io/github/actions/workflow/status/1Git2Clone/polars-vs-pandas/full_run.yml?branch=main

[Build Status]: https://github.com/1git2clone/polars-vs-pandas/actions?query=branch%3Amain

[License Icon]: https://img.shields.io/badge/license-MIT-blue.svg

[LICENSE]: LICENSE

CSV heading parsing comparison for a 1 million line randomly generated CSV file.

## Table of contents

- [Pandas vs Polars](#pandas-vs-polars)

  - [Making the CSV file](#making-the-csv-file)

  - [Benchmarking Pandas and Polars](#benchmarking-pandas-and-polars)

  - [Results](#results)

    - [Specs](#specs)

## Making the CSV file

Run [`src/main.rs`](./src/main.rs) for CSV file generation.

## Benchmarking Pandas and Polars

```sh

cd pl_vs_pd

uv venv

source .venv/bin/activate # `.venv\Scripts\Activate` on Windows

uv pip install -r pyproject.toml

uv run python main.py

```

## Results

On [my machine](#specs) Polars is consistently 39~40 times faster than Pandas.

### Specs

- Ryzen 5 3600X

- Arch Linux `6.13.2-zen1-1-zen`

- Transcend NVMe SSD (`TS512GMTE220S`)