https://github.com/chris-santiago/polars-play

https://github.com/chris-santiago/polars-play

Last synced: 10 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/chris-santiago/polars-play
• Owner: chris-santiago
• Created: 2023-03-02T03:11:40.000Z (over 1 year ago)
• Default Branch: master
• Last Pushed: 2024-07-09T20:40:20.000Z (4 months ago)
• Last Synced: 2024-10-13T04:48:00.463Z (24 days ago)
• Language: Jupyter Notebook
• Size: 35.2 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Comparing Pandas, Polars and Vaex

Using 2019-2022 data from [NYC Taxi Data](https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page).

Number of rows: 179,807,942

Dataset size: 24.4GB

## Performance

|Framework|Read|Groupby|Total

|---|---|---|---|

|Pandas|dies|n/a|n/a|

|Polars|1ms|2.3s|2.3s|

|Vaex|416ms|5.3s|5.3s|

### Notes

- Data is stored in parquet files on local disk

- Using M1 MBA with 8GB RAM-- unsure how memory swap impacts result