https://github.com/julesbelveze/nhelper
🧪 Behavioral testing of NLP models 🧪
https://github.com/julesbelveze/nhelper
behavioral deep-learning machine-learning natural-language-processing nlp testing
Last synced: 10 months ago
JSON representation
🧪 Behavioral testing of NLP models 🧪
- Host: GitHub
- URL: https://github.com/julesbelveze/nhelper
- Owner: JulesBelveze
- License: mit
- Created: 2022-04-15T09:52:22.000Z (about 4 years ago)
- Default Branch: main
- Last Pushed: 2023-04-30T19:55:12.000Z (about 3 years ago)
- Last Synced: 2025-04-16T05:36:24.489Z (about 1 year ago)
- Topics: behavioral, deep-learning, machine-learning, natural-language-processing, nlp, testing
- Language: Python
- Homepage:
- Size: 371 KB
- Stars: 7
- Watchers: 1
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# NheLPer
**NheLPer** is Python package designed to ease *behavioral testing* of Natural Language Processing models to identify
possible capability failures.
## 1. About the project
Behavioral tests are intended to test a model against some input data while treating as a black box. The aim is to
observe the model's reaction against some perturbations that might occur once the model is productionized. For a more
detailed explanation on behavioral testing of NLP models I encourage you to read the insightful
paper: [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://arxiv.org/abs/2005.04118)
**NLPtest** provides helper objects for three different aspects:
- easily generate text samples
- test some specific behaviors of your model
- aggregate the tests outcomes of your model
## 2. Getting started
### 2.1. Installation
You can directly install **NheLPer** using [pypi](https://pypi.org/project/nhelper/):
```
pip3 install nhelper
```
### 2.2. Usage
To help you get the hang of the library we provide three different Notebooks to the user, accessible from
the `examples/` folder:
1. `Samples_generation.ipynb`: shows you how to easily generate texts using the `Generator` object.
2. `Please_Behave.ipynb`: getting familiar with the `Behavior` object.
3. `End2End_tests.ipynb`: how to run tests and get an overview of your model behavior.
# References
Below, you can find resources that were used for the creation of **NLPtest** as well as relevant resources about
behavioral testing.
* [MadeWithML](https://madewithml.com/courses/mlops/testing/#behavioral-testing)
* [CheckList](https://github.com/marcotcr/checklist)
* [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://arxiv.org/abs/2005.04118)