https://github.com/julesbelveze/nhelper

🧪 Behavioral testing of NLP models 🧪
https://github.com/julesbelveze/nhelper

behavioral deep-learning machine-learning natural-language-processing nlp testing

Last synced: 12 days ago
JSON representation

🧪 Behavioral testing of NLP models 🧪

• Host: GitHub
• URL: https://github.com/julesbelveze/nhelper
• Owner: JulesBelveze
• License: mit
• Created: 2022-04-15T09:52:22.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2023-04-30T19:55:12.000Z (over 1 year ago)
• Last Synced: 2024-05-14T00:15:46.991Z (6 months ago)
• Topics: behavioral, deep-learning, machine-learning, natural-language-processing, nlp, testing
• Language: Python
• Homepage:
• Size: 371 KB
• Stars: 7
• Watchers: 2
• Forks: 2
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        


# NheLPer

**NheLPer** is Python package designed to ease *behavioral testing* of Natural Language Processing models to identify

possible capability failures.

## 1. About the project

Behavioral tests are intended to test a model against some input data while treating as a black box. The aim is to

observe the model's reaction against some perturbations that might occur once the model is productionized. For a more

detailed explanation on behavioral testing of NLP models I encourage you to read the insightful

paper: [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://arxiv.org/abs/2005.04118)

**NLPtest** provides helper objects for three different aspects:

- easily generate text samples

- test some specific behaviors of your model

- aggregate the tests outcomes of your model

## 2. Getting started

### 2.1. Installation

You can directly install **NheLPer** using [pypi](https://pypi.org/project/nhelper/):

```

pip3 install nhelper

```

### 2.2. Usage

To help you get the hang of the library we provide three different Notebooks to the user, accessible from

the `examples/` folder:

1. `Samples_generation.ipynb`: shows you how to easily generate texts using the `Generator` object.

2. `Please_Behave.ipynb`: getting familiar with the `Behavior` object.

3. `End2End_tests.ipynb`: how to run tests and get an overview of your model behavior.

# References

Below, you can find resources that were used for the creation of **NLPtest** as well as relevant resources about

behavioral testing.

* [MadeWithML](https://madewithml.com/courses/mlops/testing/#behavioral-testing)

* [CheckList](https://github.com/marcotcr/checklist)

* [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://arxiv.org/abs/2005.04118)