{"id":20410038,"url":"https://github.com/thumnlab/curbench","last_synced_at":"2025-09-24T04:31:30.459Z","repository":{"id":239322400,"uuid":"797103600","full_name":"THUMNLab/CurBench","owner":"THUMNLab","description":null,"archived":false,"fork":false,"pushed_at":"2024-10-11T08:12:55.000Z","size":1032,"stargazers_count":7,"open_issues_count":0,"forks_count":0,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-04-06T08:26:02.584Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/THUMNLab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-07T07:49:31.000Z","updated_at":"2025-03-11T09:11:18.000Z","dependencies_parsed_at":"2025-01-18T10:45:28.343Z","dependency_job_id":null,"html_url":"https://github.com/THUMNLab/CurBench","commit_stats":null,"previous_names":["thumnlab/curbench"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/THUMNLab/CurBench","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUMNLab%2FCurBench","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUMNLab%2FCurBench/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUMNLab%2FCurBench/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUMNLab%2FCurBench/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/THUMNLab","download_url":"https://codeload.github.com/THUMNLab/CurBench/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUMNLab%2FCurBench/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":276694558,"owners_count":25687767,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-24T02:00:09.776Z","response_time":97,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-15T05:44:40.969Z","updated_at":"2025-09-24T04:31:30.110Z","avatar_url":"https://github.com/THUMNLab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CurBench: A Curriculum Learning Benchmark\n\nA benchmark for Curriculum Learning.\n\nThe code of ICML 2024 paper **CurBench: A Curriculum Learning Benchmark**.\n\nThe paper can be downloaded from the [official website](https://openreview.net/pdf?id=Htw0bSgjXE) or in the [docs directory](https://github.com/THUMNLab/CurBench/tree/master/docs).\n\n\n## Environment\n\n1. python \u003e= 3.7  \n\n    [https://www.python.org/downloads/](https://www.python.org/downloads/)\n\n2. pytorch \u003e= 1.12\n\n    [https://pytorch.org/](https://pytorch.org/)\n\n3. torch_geometric\n\n    [https://pytorch-geometric.readthedocs.io/en/latest/install/installation.html](https://pytorch-geometric.readthedocs.io/en/latest/install/installation.html)\n\n4. other requirements\n\n```bash\npip install -r requirements.txt \n```\n\n\n## Dataset\n\n### Vision\n\n**CIFAR-10** and **CIFAR-100** will be downloaded automatically.\n\n**Tiny-ImageNet** is  a subset of the ILSVRC2012 version of ImageNet and consists of 64 × 64 × 3 down-sampled images. It needs to be downloaded manually from the [official website](https://image-net.org/download.php). \n\n``` bash\nCurBench\n└── data\n    ├── cifar-10-batches-py\n    │   ├── data_batch_1\n    │   ├── data_batch_2\n    │   ├── ...\n    │   └── test_batch\n    ├── cifar-100-python\n    │   ├── train\n    │   ├── test\n    │   └── meta\n    │   └── ...\n    └── tiny-imagenet-200\n        ├── train\n        ├── val\n        └── test\n\n# For easier data processing, we use a Tiny-ImageNet dataset utility class for pytorch: https://gist.github.com/lromor/bcfc69dcf31b2f3244358aea10b7a11b\n# After the processing, the directory becomes:\n\nCurBench\n└── data\n    └── tiny-imagenet-200\n        ├── train_batch\n        ├── val_batch\n        └── ...\n```\n\n### Text\n\n**GLUE** will be downloaded automatically and it consists of **cola**, **sst2**, **mrpc**, **qqp**, **stsb**, **mnli**, **qnli**, **rte**, ...\n\n### Graph\n\n**TUDataset** will be downloaded automatically and it consists of many datasets, among which we choose **MUTAG**, **PROTEINS**, **NCI1**\n\n**OGB** will be downloaded automatically and it consists of many datasets, among which we choose **molhiv**\n\n\n## Quick Start\n\n``` bash\n# 1. clone from the repository\ngit clone \ncd CurBench\n\n# 2. pip install local module: curbench\npip install -e .\n\n# 3. prepare dataset\n\n# 4. run the example code\npython examples/base.py\n```\n\n\n## Run \n\n### Single Run\n\n```bash\n# 1. vision standard\npython examples/base.py --data \u003ccifar10/cifar100/tinyimagenet\u003e --net \u003clenet/resnet18/vit\u003e --gpu \u003c0/1/2/...\u003e\n\n# 2. text standard\npython examples/base.py --data \u003crte/sst2/cola/...\u003e --net \u003clstm/bert/gpt\u003e --gpu \u003c0/1/2/...\u003e\n\n# 3. graph standard\npython examples/base.py --data \u003cmutag/proteins/nci1/molhiv\u003e --net \u003cgcn/gat/gin\u003e --gpu \u003c0/1/2/...\u003e\n\n# Note: Do not use LRE, MW-Net and DDS when backbone model is LSTM, which is not suitable for direct gradient calculation.\n```\n\n### Batch Run\n```bash\npython run.py\n```\n\n## Cite\n\nPlease cite our paper as follows if you find our code useful:\n\n```\n@inproceedings{zhoucurbench,\n  title={CurBench: Curriculum Learning Benchmark},\n  author={Zhou, Yuwei and Pan, Zirui and Wang, Xin and Chen, Hong and Li, Haoyang and Huang, Yanwen and Xiong, Zhixiao and Xiong, Fangzhou and Xu, Peiyang and Zhu, Wenwu and others},\n  booktitle={Forty-first International Conference on Machine Learning}\n}\n```\n\nYou may also find our [survey paper](https://arxiv.org/pdf/2010.13166.pdf) helpful:\n```\n@article{wang2021survey,\n  title={A survey on curriculum learning},\n  author={Wang, Xin and Chen, Yudong and Zhu, Wenwu},\n  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},\n  year={2021},\n  publisher={IEEE}\n}\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthumnlab%2Fcurbench","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthumnlab%2Fcurbench","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthumnlab%2Fcurbench/lists"}