{"id":19674067,"url":"https://github.com/frankaging/bert-lrp","last_synced_at":"2025-04-29T01:31:08.202Z","repository":{"id":52400858,"uuid":"304980710","full_name":"frankaging/BERT-LRP","owner":"frankaging","description":"On Explaining Your Explanations of BERT: An Empirical Study with Sequence Classification","archived":false,"fork":false,"pushed_at":"2022-11-30T10:43:17.000Z","size":5146,"stargazers_count":30,"open_issues_count":2,"forks_count":9,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-05T12:24:50.800Z","etag":null,"topics":["attribution-extraction","attribution-methods","bert","decision","gradient-hooks","mutli-gpu","ready-to-use","relevance-scores","saliency-detection","sensitivity-analysis","sentiment-analysis"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2101.00196","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/frankaging.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-10-17T22:33:43.000Z","updated_at":"2024-10-11T13:29:13.000Z","dependencies_parsed_at":"2023-01-23T15:32:37.469Z","dependency_job_id":null,"html_url":"https://github.com/frankaging/BERT-LRP","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frankaging%2FBERT-LRP","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frankaging%2FBERT-LRP/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frankaging%2FBERT-LRP/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frankaging%2FBERT-LRP/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/frankaging","download_url":"https://codeload.github.com/frankaging/BERT-LRP/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251415704,"owners_count":21585876,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["attribution-extraction","attribution-methods","bert","decision","gradient-hooks","mutli-gpu","ready-to-use","relevance-scores","saliency-detection","sensitivity-analysis","sentiment-analysis"],"created_at":"2024-11-11T17:17:08.838Z","updated_at":"2025-04-29T01:31:04.329Z","avatar_url":"https://github.com/frankaging.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# On Explaining Your Explanations of BERT: An Empirical Study with Sequence Classification\nPytorch implmentation of Gradient Sensitivity, Graident X Input, Layerwise Revelance Propagation and Layerwise Attention Tracing for BERT-based models in just a single `model.backward()` call.\n\n\n### Citation\n\n[Zhengxuan Wu](http://zen-wu.social) and [Desmond C. Ong](https://web.stanford.edu/~dco/). 2020. [On Explaining Your Explanations of BERT: An Empirical Study with Sequence Classification](https://arxiv.org/abs/2101.00196). Ms., Stanford University and National University of Singapore.\n\n```stex\n  @article{wu-ong-2020-explain,\n    title={On Explaining Your Explanations of BERT: An Empirical Study with Sequence Classification},\n    author={Wu, Zhengxuan and Ong, Desmond C.},\n    journal={arXiv preprint},\n    url={https://arxiv.org/abs/2101.00196},\n    year={2020}}\n```\n\n### Abstract\nBERT, as one of the pretrianed language models, attracts the most attention in recent years for creating new benchmarks across NLP tasks via fine-tuning. One pressing issue is to open up the blackbox and explain the decision makings of BERT. A number of attribution techniques have been proposed to explain BERT models, but are often limited to sequence to sequence tasks. In this paper, we adapt existing attribution methods on explaining decision makings of BERT in sequence classification tasks. We conduct extensive analyses of four existing attribution methods by applying them to four different datasets in sentiment analysis. We compare the reliability and robustness of each method via various ablation studies. Furthermore, we test whether attribution methods explain generalized semantics across semantically similar tasks. Our work provides solid guidance for using attribution methods to explain decision makings of BERT for downstream classification tasks.\n\n### Introduction\nRegular BERT training is for accuracy, but if you are interested in understanding why BERT made that decision, and which tokens contribute more towards that model decision, this repo will help you understand more about the BERT model. If you want to know the decision of your trained BERT, simple import our BERT model, and train with a gradient hook enabled. Your attribution scores will be calculated with a simple ``backward()`` call:\n```python\nfrom BERT import *\n# initialize model with our own BERT module\nmodel = BERT()\n# training loop\nmodel.train()\n# after training, simply call respective attribuiton method's backward function\nmodel.backward()\n```\n\n### Install Requirements\nYou will have to clone this repo, and install all the dependencies. You can skip this step if you have torch and cuda installed. That is all you need. You can also mannually install these without going through this installation headache that ``requirements.txt`` may give you.\n```bash\ncd BERT_LRP/code/\npip install -r requirements.txt\n```\n\n### Download Pretrained BERT Model\nYou will have to download pretrained BERT model in order to execute the fine-tune pipeline. We recommand to use models provided by the official release on BERT from [BERT-Base (Google's pre-trained models)](https://github.com/google-research/bert). Note that their model is in tensorflow format. To convert tensorflow model to pytorch model, you can use the helper script to do that. For example,\n```bash\ncd BERT_LRP/code/\npython convert_tf_checkpoint_to_pytorch.py \\\n--tf_checkpoint_path uncased_L-12_H-768_A-12/bert_model.ckpt \\\n--bert_config_file uncased_L-12_H-768_A-12/bert_config.json \\\n--pytorch_dump_path uncased_L-12_H-768_A-12/pytorch_model.bin\n```\n\n### BERT Model and Pretrain\nOur BERT model is adapted from [huggingface](https://github.com/huggingface/transformers) BERT model for text classification. If you want to take a look at the original model please search for [BertForSequenceClassification](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py). If you want to fine-tune or train a BERT classifier, you can either use their pipeline line or ours. Ours is tested against theirs before publishing. It is bug-free. To train a model, you can do something like this,\n```bash\ncd code/\nCUDA_VISIBLE_DEVICES=0.1,2 python run_classifier.py \\\n--model_type BERTPretrain \\\n--eval_test \\\n--do_lower_case \\\n--max_seq_length 512 \\\n--train_batch_size 8 \\\n--eval_batch_size 8 \\\n--learning_rate 2e-5 \\\n--num_train_epochs 3 \\\n--seed 123 \\\n--task_name SST5 \\\n--data_dir ../datasets/SST5/ \\\n--vocab_file ../models/BERT-Google/vocab.txt \\\n--bert_config_file ../models/BERT-Google/bert_config.json \\\n--output_dir ../results/SST5-NewSeed/ \\\n--init_checkpoint ../models/BERT-Google/pytorch_model.bin\n```\nTake a look at ``code/util/processor.py`` to see how we process different datasets. We currently supports almost 10 different dataset loadings. You can create your own within 1 minute for loading data. You can specify your directories info above in the command.\n\n### Analyze, Attribution, Relevance and More\nOnce you have your model ready, save it to a location that you know (e.g., ``../results/SST5/checkpoint.bin``). Our example code how to get relevance scores is in a jupyter notebook format, which is much easier to read. This is how you will open it,\n```bash\ncd code/notebook/\njupyter notebook\n```\nInside ``lrp_visualize``, we provide an example on how to get relevance scores! In short, it is really easy, when you create your BERT model, just provide an extra argument which enables a variety of gradient hooks, by setting ``init_lrp=True``. And then, in your evaluation loop, you can call ``model.backward_lrp(relevance_score)``.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrankaging%2Fbert-lrp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffrankaging%2Fbert-lrp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrankaging%2Fbert-lrp/lists"}