{"id":28913256,"url":"https://github.com/zjunlp/DART","last_synced_at":"2025-06-21T20:34:28.371Z","repository":{"id":44802083,"uuid":"439525353","full_name":"zjunlp/DART","owner":"zjunlp","description":"[ICLR 2022] Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners","archived":false,"fork":false,"pushed_at":"2022-12-07T07:36:54.000Z","size":76,"stargazers_count":132,"open_issues_count":0,"forks_count":17,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-06-13T23:05:12.780Z","etag":null,"topics":["dart","few-shot-learning","iclr","iclr2022","language-models","pre-trained-language-models","prompt","prompt-learning","prompt-tuning","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zjunlp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-12-18T04:14:58.000Z","updated_at":"2025-06-03T07:42:29.000Z","dependencies_parsed_at":"2023-01-24T13:46:10.945Z","dependency_job_id":null,"html_url":"https://github.com/zjunlp/DART","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/zjunlp/DART","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjunlp%2FDART","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjunlp%2FDART/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjunlp%2FDART/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjunlp%2FDART/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zjunlp","download_url":"https://codeload.github.com/zjunlp/DART/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjunlp%2FDART/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261192470,"owners_count":23122875,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dart","few-shot-learning","iclr","iclr2022","language-models","pre-trained-language-models","prompt","prompt-learning","prompt-tuning","pytorch"],"created_at":"2025-06-21T20:34:22.141Z","updated_at":"2025-06-21T20:34:28.362Z","avatar_url":"https://github.com/zjunlp.png","language":"Python","readme":"# DART\nImplementation for ICLR2022 paper *[Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners](https://arxiv.org/pdf/2108.13161.pdf)*. \n- ❗NOTE: The code has been reorganized and we also provide a paper-list at [PromptKG](https://github.com/zjunlp/PromptKG).\n\n## Environment\n- python@3.6\n- Use `pip install -r requirements.txt` to install dependencies.\n- `wandb` account is required if the user wants to search for best hyper-parameter combinations.\n\n## Data source\n- 16-shot GLUE dataset from [LM-BFF](https://github.com/princeton-nlp/LM-BFF).\n- Generated data consists of 5 random splits (13/21/42/87/100) for a task, each has 16 samples.\n  - The generation process follows LM-BFF [here](https://github.com/princeton-nlp/LM-BFF/blob/main/tools/generate_k_shot_data.py).\n\n## How to run\n- To train / test on a data split from a single task with specific parameters, use `run.py`.\n  - For customized training \u0026 evaluation, you can modify based on the sample configuration file `config/sample.yml`.\n```bash\n$ python run.py -h  \nusage: run.py [-h] [--config CONFIG] [--do_train] [--do_test]\n\noptional arguments:\n  -h, --help            show this help message and exit\n  --config CONFIG, -c CONFIG\n                        Configuration file storing all parameters\n  --do_train\n  --do_test\n```\n- To search optimal hyper-parameters for each task and reproduce our result, please use `sweep.py`:\n  - Please refer to documentation for [WandB](https://docs.wandb.ai/) for more details.\n  - **❗NOTE: we follow [LM-BFF](https://github.com/princeton-nlp/LM-BFF) in that we search optimal sets of hyper-parameters on different data splits respectively.**\n```bash\n$ python sweep.py -h\nusage: sweep.py [-h] [--project_name PROJECT_NAME] --task_name TASK_NAME\n                [--data_split {13,21,42,87,100}]\n                [--pretrain_model PRETRAIN_MODEL] [--pet_method {pet,diffpet}]\n                [--random_seed RANDOM_SEED] [--max_run MAX_RUN]\n\noptional arguments:\n  -h, --help            show this help message and exit\n  --project_name PROJECT_NAME\n                        project name for sweep\n  --task_name TASK_NAME\n  --data_split {13,21,42,87,100}\n                        few-shot split-id for GLUE dataset\n  --pretrain_model PRETRAIN_MODEL\n                        name or path for pretrained model\n  --pet_method {pet,diffpet}\n                        prompt encoding method\n  --random_seed RANDOM_SEED\n                        random seed for training\n  --max_run MAX_RUN     maximum tries for sweep\n```\n\n## How to Cite\n```\n@inproceedings{\nzhang2022differentiable,\ntitle={Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners},\nauthor={Ningyu Zhang and Luoqiu Li and Xiang Chen and Shumin Deng and Zhen Bi and Chuanqi Tan and Fei Huang and Huajun Chen},\nbooktitle={International Conference on Learning Representations},\nyear={2022},\nurl={https://openreview.net/forum?id=ek9a0qIafW}\n}\n```\n","funding_links":[],"categories":["\u003cspan id=\"head1\"\u003e *Keywords* \u003c/span\u003e","Python"],"sub_categories":["[Prompt PEFT](#content)"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzjunlp%2FDART","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzjunlp%2FDART","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzjunlp%2FDART/lists"}