https://github.com/rasbt/datapipes-blog
Code for the DataPipes article
https://github.com/rasbt/datapipes-blog
deep-learning machine-learning python pytorch
Last synced: 8 months ago
JSON representation
Code for the DataPipes article
- Host: GitHub
- URL: https://github.com/rasbt/datapipes-blog
- Owner: rasbt
- License: bsd-3-clause
- Created: 2022-06-10T00:56:00.000Z (about 4 years ago)
- Default Branch: main
- Last Pushed: 2022-06-14T13:36:57.000Z (about 4 years ago)
- Last Synced: 2025-03-31T04:41:18.241Z (about 1 year ago)
- Topics: deep-learning, machine-learning, python, pytorch
- Language: Jupyter Notebook
- Homepage: https://sebastianraschka.com/blog/2022/datapipes.html
- Size: 20.5 KB
- Stars: 13
- Watchers: 2
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
## About
This is the complementary code for the article [Taking Datasets, DataLoaders, and PyTorch’s New DataPipes for a Spin](https://sebastianraschka.com/blog/2022/datapipes.html).
To recreate the environment used in this blogpost, you can use
```bash
conda create --name datapipes python=3.8
conda activate datapipes
pip install -r requirements.txt
```
The code was run using
```
Python version: 3.8.13
torch: 1.11.0
torchdata: 0.3.0
```
## Running the Code
Run the [`0_download-and-prep-data.ipynb`](0_download-and-prep-data.ipynb) notebook first to download the dataset. The data loading scripts are all independent and self-contained and can be run in any order
- [1_dataset-csv.py](1_dataset-csv.py): `python 1_dataset-csv.py`
- [2_imagefolder.py](2_imagefolder.py): `python 2_imagefolder.py`
- [3_datapipes-csv.py](3_datapipes-csv.py): `python 3_datapipes-csv.py`