{"id":13753268,"url":"https://github.com/zhouj8553/FlipDA","last_synced_at":"2025-05-09T20:35:10.531Z","repository":{"id":47788449,"uuid":"379211366","full_name":"zhouj8553/FlipDA","owner":"zhouj8553","description":null,"archived":false,"fork":false,"pushed_at":"2024-05-11T14:49:45.000Z","size":983,"stargazers_count":64,"open_issues_count":1,"forks_count":15,"subscribers_count":2,"default_branch":"main-v2","last_synced_at":"2024-11-16T05:32:39.633Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zhouj8553.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-06-22T09:21:08.000Z","updated_at":"2024-11-13T07:29:09.000Z","dependencies_parsed_at":"2024-05-11T15:44:52.541Z","dependency_job_id":"e27315b4-47f0-4b29-872f-e099db665ec0","html_url":"https://github.com/zhouj8553/FlipDA","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhouj8553%2FFlipDA","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhouj8553%2FFlipDA/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhouj8553%2FFlipDA/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhouj8553%2FFlipDA/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zhouj8553","download_url":"https://codeload.github.com/zhouj8553/FlipDA/tar.gz/refs/heads/main-v2","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253321847,"owners_count":21890478,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T09:01:19.347Z","updated_at":"2025-05-09T20:35:05.416Z","avatar_url":"https://github.com/zhouj8553.png","language":"Python","funding_links":[],"categories":["文本分类"],"sub_categories":[],"readme":"# FlipDA\n\nThis repository contains the official code for FlipDA.\n\nWe provide an automatic data augmentation method based on T5 and self-training by flipping the label. We evaluate it on FewGLUE, and improve its performance.\n\n## Model\n\n![image](https://github.com/zhouj8553/FlipDA/blob/main-v2/img/model.png)\n\n## Setup\n\nInstall and setup environment with requirements.txt.\nAll our experiments are conducted on a single V100 with 32G VIDEO MEMORY.\nThe FewGLUE dataset is default placed in the folder _data_. The data could be downloaded in https://drive.google.com/file/d/1ibM0YikAza0R7v1HGDYIFO1WO-78432A/view?usp=sharing.\n\n## Step1: Run Baseline\n\nFirst, you should run the baseline. \u003ctask_name\u003e could be \"boolq\", \"rte\", \"cb\", \"copa\", \"wsc\", \"wic\", and \"multirc\". \u003cgpu_id\u003e could be 0,1,2,..., according to the number of your gpu.\n\n```bash\nbash scripts/run_pet.sh \u003ctask_name\u003e \u003cgpu_id\u003e baseline\n```\n\nFor example, to reproduce the baseline, you can run the commands as follows:\n\n```bash\nbash scripts/run_pet.sh boolq 0 baseline\nbash scripts/run_pet.sh rte 1 baseline\nbash scripts/run_pet.sh cb 2 baseline\nbash scripts/run_pet.sh multirc 3 baseline\nbash scripts/run_pet.sh copa 4 baseline\nbash scripts/run_pet.sh wsc 5 baseline\nbash scripts/run_pet.sh wic 6 baseline\nbash scripts/run_pet.sh record 7 baseline\n```\n\nIf you run the command and shell as default, the results will be in _results/baseline/pet/\u003ctask_name\u003e\\_albert\\_model/result_test.txt_.\n\n## Step2: Produce augmented files\n\nThe code to generate augmented examples by T5 model is in _genaug/total_gen_aug.py_.\n\nYou could use the command as follows to generate augmented examples. \u003ctask_name\u003e could be \"BoolQ\", \"RTE\", \"CB\", \"COPA\", \"WSC\", \"WiC\", \"MultiRC\", and \"ReCoRD\". \u003cmask_ratio\u003e could be arbitrary floating point number between 0 and 1, and in our experiments, we only tried 0.3, 0.5, and 0.8. \u003caug_type\u003e could be \"default\" or \"rand_iter_%d\", where %d could be any integers. \u003clabel_type\u003e could be \"flip\" or \"keep\". \"do_sample\" and \"num_beams\" controls the generation style (sample/greedy/beam search). \u003caug_num\u003e denotes the number of augmented samples to be generated for each sample, in our experiments, we choose \u003caug_num\u003e 10.\n\n```bash\nCUDA_VISIBLE_DEVICES=0 python -m genaug.total_gen_aug --task_name \u003ctask_name\u003e --mask_ratio \u003cmask_ratio\u003e --aug_type \u003caug_type\u003e --label_type \u003clabel_type\u003e --do_sample --num_beams \u003cnum_beams\u003e --aug_num \u003caug_num\u003e\n```\n\nFor example, to generate the augmented data we use in RTE, you could use the following command.\n\n```bash\nCUDA_VISIBLE_DEVICES=0 python -m genaug.total_gen_aug --task_name RTE --mask_ratio 0.5 --aug_type 'default' --label_type 'flip' --do_sample --num_beams 1  --aug_num 10\nCUDA_VISIBLE_DEVICES=1 python -m genaug.total_gen_aug --task_name RTE --mask_ratio 0.5 --aug_type 'default' --label_type 'keep' --do_sample --num_beams 1  --aug_num 10\n\n```\n\n## Alternative: Run baselines with augmented files without classifier\n\nIf you want to add all the augmented data in the augmented files into the model (you do not want to change the label according to the trained classifier or filter the augmented examples), you could run the command as follows.\n\n```bash\nbash scripts/run_pet.sh boolq 0 \u003caugmented_file_name\u003e\n```\n\n## Step3: Run FlipDA with augmented files\n\nIf the \u003caugmented_file_name\u003e has the corresponding version, we will load it. For example, if the filename is \"t5_flip_0.5_default_sample0_beam1_augnum10\" and we find \"t5_keep_0.5_default_sample0_beam1_augnum10\", we will load them both.\n\nIf you allow the classifier to correct the label of the augmented data, run the command as follows, where \u003caugmented_file_name\u003e is the augmented file name such as \"t5_flip_0.5_default_sample0_beam1_augnum10\".\n\n```bash\nbash scripts/run_pet.sh boolq 0 genaug_\u003caugmented_file_name\u003e_filter_max_eachla\n```\n\nIf you do not allow the classifier to correct the label of the augmented data, run the command as follows, where \u003caugmented_file_name\u003e is the augmented file name such as \"t5_flip_0.5_default_sample0_beam1_augnum10\".\n\n```bash\nbash scripts/run_pet.sh boolq 0 genaug_\u003caugmented_file_name\u003e_filter_max_eachla_sep\n```\n\nNote that which command to choose is based on the relative power of the augmentation model and the classification model. If the augmentation model is accurate enough, choosing the command with \"sep\" will be better. Otherwise, choose the first one. If you are not sure, just try them both.\n\nTo reproduce our result, command for AlBERT-xxlarge-v2 of FlipDA is:\n\n```bash\nbash scripts/run_pet.sh rte 0 genaug_t5_flip_0.5_default_sample1_beam1_augnum10_filter_max_eachla_sep\nbash scripts/run_pet.sh boolq 0 genaug_t5_flip_0.3_default_sample1_beam1_augnum10_filter_max_eachla\nbash scripts/run_pet.sh cb 0 genaug_t5_flip_0.5_default_sample1_beam1_augnum10_filter_max_eachla\nbash scripts/run_pet.sh copa 0 genaug_t5_flip_0.8_default_sample0_beam10_augnum10_filter_max_eachla_sep\nbash scripts/run_pet.sh wic 0 genaug_t5_flip_0.8_default_sample1_beam1_augnum10_filter_max_eachla_sep\nbash scripts/run_pet.sh wsc 0 genaug_t5_keep_0.3_default_sample0_beam1_augnum10wscaugtype_extra_filter_max_prevla\nbash scripts/run_pet.sh multirc 0 genaug_t5_flip_0.5_rand_iter_10_sample1_beam1_augnum10_filter_max_eachla_sep\nbash scripts/run_pet.sh record 0 genaug_t5_flip_0.3_rand_iter_10_sample0_beam10_augnum10_filter_max_eachla\n```\n\nNote that we do not try all the hyperparameter combinations to save time and avoid overfitting, and our main contribution is proving that label flipping is useful for few-shot data augmentation. We will not be surprised if you get a better result with more careful implementation and hyperparameter selection.\n\n## Citation\n\nThis paper will appear at ACL 2022 (Main Conference). Please cite us if FlipDA is useful in your work:\n\n```\n@inproceedings{DBLP:conf/acl/ZhouZTJY22,\n  author    = {Jing Zhou and\n               Yanan Zheng and\n               Jie Tang and\n               Li Jian and\n               Zhilin Yang},\n  title     = {FlipDA: Effective and Robust Data Augmentation for Few-Shot Learning},\n  booktitle = {{ACL} {(1)}},\n  pages     = {8646--8665},\n  publisher = {Association for Computational Linguistics},\n  year      = {2022}\n}\n```\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzhouj8553%2FFlipDA","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzhouj8553%2FFlipDA","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzhouj8553%2FFlipDA/lists"}