{"id":19611926,"url":"https://github.com/princeton-nlp/rationale-robustness","last_synced_at":"2025-04-27T22:33:48.802Z","repository":{"id":37814499,"uuid":"485431417","full_name":"princeton-nlp/rationale-robustness","owner":"princeton-nlp","description":"NAACL 2022: Can Rationalization Improve Robustness? https://arxiv.org/abs/2204.11790","archived":false,"fork":false,"pushed_at":"2022-11-21T16:06:09.000Z","size":22408,"stargazers_count":27,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-05T04:31:47.716Z","etag":null,"topics":["interpretability","nlp","robustness"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/princeton-nlp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-04-25T15:38:26.000Z","updated_at":"2024-10-30T06:06:39.000Z","dependencies_parsed_at":"2023-01-20T23:15:48.702Z","dependency_job_id":null,"html_url":"https://github.com/princeton-nlp/rationale-robustness","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/princeton-nlp%2Frationale-robustness","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/princeton-nlp%2Frationale-robustness/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/princeton-nlp%2Frationale-robustness/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/princeton-nlp%2Frationale-robustness/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/princeton-nlp","download_url":"https://codeload.github.com/princeton-nlp/rationale-robustness/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251219600,"owners_count":21554444,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["interpretability","nlp","robustness"],"created_at":"2024-11-11T10:45:05.594Z","updated_at":"2025-04-27T22:33:43.792Z","avatar_url":"https://github.com/princeton-nlp.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Can Rationalization Improve Robustness?\n\nThis repository contains the code for our NAACL'22 paper [Can Rationalization Improve Robustness?](https://arxiv.org/pdf/2204.11790.pdf).\nYou can also check out our slides [here](slides/naacl2022-slides.pdf) or the NAACL 2022 talk [here](https://aclanthology.org/2022.naacl-main.278.mp4).\n\n## Quick Links\n\n- [Overview](#overview)\n- [Main Results](#main-results)\n- [Run Code](#run-code)\n  - [Requirements](#requirements)\n  - [Training](#training)\n  - [Attacks](#attacks)\n- [Bugs or Questions?](#bugs-or-questions)\n- [Citation](#citation)\n- [Acknowledgment](#acknowledgment)\n\n## Overview\nWe explore the intersection between interpretability and robustness in the popular predict-then-rationalize framework. Extractive rationalization is a way to expose its inner reasoning process by selecting a subset of input text (upper part of the figure). Ideally, the rationalizer would select a set of tokens that \"explains\" the prediction. A good rationalizer should select non-spurious texts (e.g., stopwords or dataset specific biases) and hence is expected to fence off added attack/spurious text (bottom part of the figure). We examine a wide range of models and datasets and find that the rationalization may be promising in providing robustness benefit to regular full-context models. We also caustion the findings with analysis on cases where the the models are sensitive to the choice of the attack.\n\n[\u003cimg src=\"figures/main.png\" width=\"550\"/\u003e](main.png)\n\n\nBelow are some examples of our robustness tests. The blue sentences indicate the query/question, the red sentences indicate the attack (transformed from the query/question), and the yellow highlights indicating the gound truth rationales.\n\n[\u003cimg src=\"figures/examples.png\" width=\"1000\"/\u003e](examples.png)\n\n\n## Main Results\n\nBelow is the main result table of the original performance (Ori) and the attacked performance (Att). The models are:\n-  `FC` = full context model\n-  `VIB` = the [Variational Information Buttleneck](https://arxiv.org/pdf/2005.00652.pdf) model\n-  `SPECTRA` = the [Sparse Structured Text Rationalization](https://arxiv.org/pdf/2109.04552.pdf) model\n-  `*-sup` = models using the human provided rationales as supervision\n\n|               | FEVER (Ori / Att) | MultiRC (Ori / Att) | SQuAD (Ori / Att) | Beer (Ori / Att) | Hotel (Ori / Att) |\n|:--------------|:---------:|:---------:|:---------:|:---------:|:---------:|\n| FC       | 90.7 / 77.9 | 70.7 / 63.0 | 87.2 / 59.1 | 93.8 / 59.5 | 99.5 / 79.3 |\n| VIB      | 87.8 / 82.6 | 65.4 / 63.6 | 77.1 / 56.5 | 93.8 / 88.0 | 94.0 / 59.3 |\n| SPECTRA  | 84.0 / 76.5 | 63.8 / 63.3 | 65.5 / 45.5 | 95.4 / 89.7 | 94.5 / 51.3 |\n| FC-sup   | 91.9 / 77.1 | 71.5 / 64.0 | 87.0 / 57.3 | - | - |\n| VIB-sup  | 90.2 / 81.4 | 68.7 / 63.7 | 86.5 / 56.5 | - | - |\n\n\n## Run Code\n### Requirements\n\nTo install the dependencies, please run\n```\npip install -r requirements.txt\n```\n\nConfigure `rrtl/config.py` to point the base path to your repo.\n\n### Training\n\nTo train, simply run `run_{dataset}.sh {model_option}`. `{model_option}` takes one of these values `[fc | vib | spectra]`.\n\nFor example, to run the SPECTRA model on FEVER (make sure to set your repo path `base_path` in the run script):\n```\n./scripts/run_fever.sh spectra\n```\n\nBelow are the arguments specific to the VIB models:\n- `--tau`: temperature\n- `--pi`: rationale sparsity\n- `--beta`: coefficient of the KL term (strength of the KL regularization)\n- `--gamma`: coefficient for the human rationale supervision loss\n- `--use-gold-rationale`: if set to `True`, the VIB-sup model will be trained\n- `--use-neg-rationale`: if set to `True`, the ART model will be trained\n\nBelow are the arguments specific to the SPECTRA models:\n- `--budget`: (int) absolute number of tokens or sentences to select as rationales\n- `--budget_ratio`: (float) the ratio of rationales to predict over the number of sentences in the context\n- `--temperature`: temperature term\n- `--solver_iter`: number of iterations to run the LP-SparseMAP solver\n\n### Attacks\n#### Run the checkpoint on the attack dataset (FEVER/MultiRC):\n\nFirst run `python -m rr.attacks.aug_and_cache_data --dataset-name fever --aug-method addsent --attack-dir addsent --insert_pos 0` to cache the attacks (see `rr/attacks/aug_and_cache_data.py` for other options).\n```\npython -m rr.eval.run_eval --model-name {MODEL_CHECKPOINT_DIR} \\\n                           --bottleneck-type {vib | vib_semi | full | full_multitask} \\\n                           --exp-dir {fever | multirc} \\\n                           --attack-dir addsent_pos0 \\\n                           --output-to-tmp                           \n```\nHere `vib_semi` corresponds to VIB-sup, `full` to FC, and `full_multitask` to FC-sup in the paper.\n\n\n#### Run the checkpoint on the attack dataset (SQuAD):\n```\npython -m rrtl.run_squad_eval --dataset-name squad-addonesent --load-path /path/to/checkpoint.pt\n```\n\n#### Run the checkpoint on the attack dataset (Beer/Hotel)\nTo cache the attack, first run `python -m rrtl.attacks.sentiment_attack` to cache the attacks. Second, run:\n```\npython -m rrtl.analysis.eraser.run_token_level_analysis --dataset-name {beer | hotel} \\\n                                                        --eval-mode stats \\\n                                                        --load-path /path/to/checkpoint.pt\n```\n\n### Hyperparameters\n\n| Dataset | Epoch | Learning Rate | Batch Size |\n|:--------------|:---------:|:---------:|:---------:|\n|   FEVER | 10 | 5e-5 | 32 |\n| MultiRC | 10 | 5e-5 | 32 |\n|   SQuAD |  3 | 1e-5 | 32 |\n|    Beer | 20 | 5e-5 | 64 |\n|   Hotel | 20 | 5e-5 | 64 |\n\nModel variants such as `*-sup` models or `ART` training may have slightly different hyperparameters. Please refer to the training script for the detailed setting.\n\n## Bugs or Questions\n\nIf you have any questions about our paper, please reach out to Howard Chen (howardchen@cs.princeton.edu). Also, feel free to open an issue for bug reporting.\n\n## Citation\n\n```bibtex\n@inproceedings{chen2022RationaleRobustness,\n   title={Can Rationalization Improve Robustness?},\n   author={Chen, Howard and He, Jacqueline and Narasimhan, Karthik and Chen, Danqi},\n   booktitle={North American Chapter of the Association for Computational Linguistics (NAACL)},\n   year={2022}\n}\n```\n\n## Acknowledgment\nWe thank the authors of [An Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction](https://arxiv.org/abs/2005.00652) for their [implementations](https://github.com/bhargaviparanjape/explainable_qa) that we adapt in our VIB models.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprinceton-nlp%2Frationale-robustness","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprinceton-nlp%2Frationale-robustness","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprinceton-nlp%2Frationale-robustness/lists"}