https://github.com/przbadu/csv-import-stats
Performance Benchmark of CSV.for_each, CSV.parse and CSV.read
https://github.com/przbadu/csv-import-stats
Last synced: about 1 month ago
JSON representation
Performance Benchmark of CSV.for_each, CSV.parse and CSV.read
- Host: GitHub
- URL: https://github.com/przbadu/csv-import-stats
- Owner: przbadu
- Created: 2022-02-21T11:22:26.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2022-02-21T11:36:26.000Z (about 3 years ago)
- Last Synced: 2025-02-14T22:47:47.980Z (3 months ago)
- Language: Ruby
- Size: 250 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
## How to test
- [ ] [Install ruby](https://www.ruby-lang.org/en/documentation/installation/)
### Generate CSV file first
below command will generate `data.csv` file with 1,000,000 records
```ruby
ruby generate_csv.rb
```
### Check Benchmark
You will see 2 useful information for each method
- Time it consumed to read whole CSV records
- more importantly *Memory* it consumed to process above request.
**Check benchmark of `CSV.read` method**
```ruby
> ruby csv_read.rb
Sum: 499999500000
Time: 11.74
Memory: 1887.02 MB
```
**Check benchmark of `CSV.parse` method**
```ruby
> ruby csv_parse.rb
Sum: 499999500000
Time: 9.68
Memory: 2193.36 MB
```
**Check benchmark of `CSV.for_each` method**
```ruby
> ruby csv_for_each.rb
Sum: 499999500000
Time: 5.74
Memory: 1.8 MB
```
As you might see, there is no huge *Time* difference in each CSV methods to read csv, but
there is huge performance difference on how much memory each methods take.
Always use `CSV.for_each` if you know your CSV has lots of data in it, because it will always read csv data per row.