https://github.com/qdata/mncover
https://github.com/qdata/mncover
Last synced: 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/qdata/mncover
- Owner: QData
- License: mit
- Created: 2022-05-03T13:04:40.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2022-05-11T03:27:49.000Z (almost 3 years ago)
- Last Synced: 2025-01-11T11:26:31.912Z (4 months ago)
- Language: Python
- Size: 2.19 MB
- Stars: 3
- Watchers: 2
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# MNCOVER
This repository contains code for white box testing of NLP models as described in the following paper:
>[White-box Testing of NLP models with Mask Neuron Coverage](https://arxiv.org/abs/2004.11494)
> https://arxiv.org/abs/2205.05050
> Findings of Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2022
## Trained Masks and Initialization Files
All trained masks and initialization files can be found [here](https://drive.google.com/drive/folders/1u1a4gsUU4KGPQfUv_d1-MoaeL1UFEqo5?usp=sharing).
## Initializing Coverage
```
python initialize_coverage.py --seed 1 --bins-word 10 --bins-attention 10 --max-seq-len 128 --batch-size 128 --alpha 1.0 --test-name "change names" --suite sentiment --subset 1500 --base-model roberta-base --save-dir results/
```
## Filtering and Calculating Failure Rates
```
python calculate_coverage.py --seed 1 --bins-word 10 --bins-attention 10 --max-seq-len 128 --batch-size 128 --alpha 1.0 --test-name "change names" --suite sentiment --subset 1500 --base-model roberta-base --save-dir results/
```