{"id":17816960,"url":"https://github.com/eczy/make-datasetfolder","last_synced_at":"2025-04-02T08:26:56.506Z","repository":{"id":57439320,"uuid":"238094130","full_name":"eczy/make-datasetfolder","owner":"eczy","description":"A utility to create a PyTorch DatasetFolder from any .csv or .tsv file with file path and class data.","archived":false,"fork":false,"pushed_at":"2020-02-04T04:58:07.000Z","size":15,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-08T08:47:46.500Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eczy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-02-04T00:57:10.000Z","updated_at":"2020-02-04T04:58:09.000Z","dependencies_parsed_at":"2022-09-26T17:20:28.318Z","dependency_job_id":null,"html_url":"https://github.com/eczy/make-datasetfolder","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eczy%2Fmake-datasetfolder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eczy%2Fmake-datasetfolder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eczy%2Fmake-datasetfolder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eczy%2Fmake-datasetfolder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eczy","download_url":"https://codeload.github.com/eczy/make-datasetfolder/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246780280,"owners_count":20832560,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-27T16:39:50.778Z","updated_at":"2025-04-02T08:26:56.472Z","avatar_url":"https://github.com/eczy.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# make-datasetfolder\nA utility to create a PyTorch DatasetFolder from any .csv or .tsv file with file path and class data.\n\n## Installation\nFrom PyPI: `pip install make-datasetfolder`\n\nFrom GitHub:\n```\ngit clone https://github.com/eczy/make-datasetfolder\npip install -e make-datasetfolder\n```\n\n## Use Case\nIn PyTorch, the `DataFolder` and `ImageFolder` classes provide a convenient interface for computer vision datasets structured as such:\n\n```\nroot/class_x/xxx.ext\nroot/class_x/xxy.ext\nroot/class_x/xxz.ext\n\nroot/class_y/123.ext\nroot/class_y/nsdf3.ext\nroot/class_y/asd932_.ext\n```\n\nThis utility transforms any dataset with a table containing file paths and class labels into this format.\n\n## Example\nSuppse you have `dataset.csv` of the form:\n```\nsample,class,some_feature,another_feature\nimg-0001.jpg,0,foo,bar\nsome/relative/directory/img-0002.jpg,1,foo,bar\n...\n```\n\nRunning `make-dataset-folder -p sample -l class dataset.csv output` will create a folder `output` with the following structure:\n\n```\noutput/0/img-0001.jpg\noutput/1/img-0002.jpg\n...\n```\n\nUsing the `-m` flag will move images rather than copy them. This could be useful for large datasets that shouldn't be duplicated on disk.\n\n## Usage\n```\nusage: make-datasetfolder [-h] [-p PATH_COLUMN] [-l LABEL_COLUMN] [-m] [-f]\n                          [-t THREADS]\n                          input output\n\npositional arguments:\n  input                 Path to input .csv or .tsv\n  output                Path to output directory.\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -p PATH_COLUMN, --path-column PATH_COLUMN\n                        Column name or index with file paths (default: 0).\n  -l LABEL_COLUMN, --label-column LABEL_COLUMN\n                        Column name or index with labels (default: 1).\n  -m, --move            Move files instead of copying.\n  -f, --force           Overwrite output directory if it already exists.\n  -t THREADS, --threads THREADS\n                        Number of threads to use (default: number of CPU\n                        cores)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feczy%2Fmake-datasetfolder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feczy%2Fmake-datasetfolder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feczy%2Fmake-datasetfolder/lists"}