https://github.com/qdata/awesome-robustness-testing-for-nlp
A curated list of papers on testing NLP.
https://github.com/qdata/awesome-robustness-testing-for-nlp
List: awesome-robustness-testing-for-nlp
Last synced: 4 months ago
JSON representation
A curated list of papers on testing NLP.
- Host: GitHub
- URL: https://github.com/qdata/awesome-robustness-testing-for-nlp
- Owner: QData
- Created: 2020-08-25T19:19:06.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2020-10-19T11:03:26.000Z (over 5 years ago)
- Last Synced: 2025-10-12T19:03:09.325Z (8 months ago)
- Homepage:
- Size: 80.1 KB
- Stars: 5
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Awesome Papers on Automated-Robustness-Testing-for-NLP
## Why is testing DNNs important?
DNNs are modern software being deployed everywhere. Like other software these must be tested for corner cases(when the software is likely to be problematic).
## Why is testing DNNs hard?
DNNs have too many parameters: too many neurons. Manually finding corner cases is too difficult. Need automated testing , i.e. generating automatically corner cases for large DNNs.
## General Intro Position Papers/ Blogs
1. [DeepMind Medium Blog](https://medium.com/@deepmindsafetyresearch/towards-robust-and-verified-ai-specification-testing-robust-training-and-formal-verification-69bd1bc48bda)
2. [General Survey of Testing in ML](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9000651)
### Neuron Coverage Based
1. [GrayBox Testing: DeepTest](https://arxiv.org/pdf/1708.08559.pdf)
2. [White Box Gradient Based Testing](https://arxiv.org/abs/1705.06640)
3. [DeepCT](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8668044)
4. [Concolic Testing for Deep Neural Networks](http://qav.comlab.ox.ac.uk/papers/swr+18.pdf)
5. [FuzzTesting](https://www.comp.nus.edu.sg/~abhik/pdf/ICSE20_Sensei.pdf) -- Augmentation
6. [Testing Deep Neural Networks- Symbolic Execution](https://arxiv.org/abs/1803.04792)
7. [MCTS based](https://arxiv.org/abs/1710.07859)
##### Fuzzing Based
1. [FuzzTesting](https://www.comp.nus.edu.sg/~abhik/pdf/ICSE20_Sensei.pdf)
2. [TensorFuzz](http://proceedings.mlr.press/v97/odena19a/odena19a.pdf)
3. [DLFuzz](https://arxiv.org/pdf/1808.09413.pdf)
4. [NeuFuzz](https://wcventure.github.io/FuzzingPaper/Paper/Access19_NeuFuzz%20.pdf)
# Testing for NLP Deep Models
1. [checklist](https://homes.cs.washington.edu/~marcotcr/acl20_checklist.pdf)
2. [errudite](https://homes.cs.washington.edu/~marcotcr/acl19_errudite.pdf)
3. [Semantically Equivalent Adversarial Rules for Debugging NLP Models](https://homes.cs.washington.edu/~marcotcr/acl18.pdf)
4. [Are Red Roses Red?Evaluating Consistency of Question-Answering Models](https://homes.cs.washington.edu/~marcotcr/acl19_implication.pdf)
5. [Robustness Verification for Transformers](https://arxiv.org/pdf/2002.06622.pdf)
6. [Towards a Robust Deep Neural Network in Texts: A Survey](https://arxiv.org/pdf/1902.07285.pdf)
7. [Certified Robustness to Adversarial Word Substitutions](https://arxiv.org/abs/1909.00986)