Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/MarcelRobeer/explabox
Explore/examine/explain/expose your model with the explabox!
https://github.com/MarcelRobeer/explabox
explainable-ai explainable-ml fairness interpretable-machine-learning machine-learning python responsible-ai robustness
Last synced: 3 months ago
JSON representation
Explore/examine/explain/expose your model with the explabox!
- Host: GitHub
- URL: https://github.com/MarcelRobeer/explabox
- Owner: MarcelRobeer
- License: lgpl-3.0
- Created: 2022-05-10T11:20:18.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-03-18T12:35:14.000Z (11 months ago)
- Last Synced: 2024-05-20T18:14:06.488Z (9 months ago)
- Topics: explainable-ai, explainable-ml, fairness, interpretable-machine-learning, machine-learning, python, responsible-ai, robustness
- Language: Python
- Homepage: https://explabox.readthedocs.io
- Size: 2.66 MB
- Stars: 13
- Watchers: 3
- Forks: 0
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
- Awesome-explainable-AI - https://github.com/MarcelRobeer/explabox - learn, Pytorch, Keras, Tensorflow, Huggingface  (Python Libraries(sort in alphabeta order) / Evaluation methods)
- awesome-interpretability - Explabox
README
*
*
**
"{`Explore` | `Examine` | `Expose` | `Explain`} your model with the *explabox*!"**
---
| Status | |
|:-----------------|:------------------
| _Latest release_ | [](https://pypi.org/project/explabox/) [](https://pepy.tech/project/explabox) [](https://pypi.org/project/explabox/) [](https://www.gnu.org/licenses/lgpl-3.0.en.html)
| _Development_ | [](https://github.com/MarcelRobeer/explabox/actions/workflows/check.yml) [](https://codecov.io/gh/MarcelRobeer/explabox) [](https://explabox.readthedocs.io/en/latest/?badge=latest) [](https://github.com/psf/black)
---
The `explabox` aims to support data scientists and machine learning (ML) engineers in explaining, testing and documenting AI/ML models, developed in-house or acquired externally. The `explabox` turns your **ingestibles** (AI/ML model and/or dataset) into **digestibles** (statistics, explanations or sensitivity insights)!
*
*
The `explabox` can be used to:
- __Explore__: describe aspects of the model and data.
- __Examine__: calculate quantitative metrics on how the model performs.
- __Expose__: see model sensitivity to random inputs (_safety_), test model generalizability (e.g. sensitivity to typos; _robustness_), and see the effect of adjustments of attributes in the inputs (e.g. swapping male pronouns for female pronouns; _fairness_), for the dataset as a whole (_global_) as well as for individual instances (_local_).
- __Explain__: use XAI methods for explaining the whole dataset (_global_), model behavior on the dataset (_global_), and specific predictions/decisions (_local_).
A number of experiments in the `explabox` can also be used to provide transparency and explanations to stakeholders, such as end-users or clients.
> :information_source: The `explabox` currently only supports natural language text as a modality. In the future, we intend to extend to other modalities.
© National Police Lab AI (NPAI), 2022
## Quick tour
The `explabox` is distributed on [PyPI](https://pypi.org/project/explabox/). To use the package with Python, install it (`pip install explabox`), import your `data` and `model` and wrap them in the `Explabox`:
```python
>>> from explabox import import_data, import_model
>>> data = import_data('./drugsCom.zip', data_cols='review', label_cols='rating')
>>> model = import_model('model.onnx', label_map={0: 'negative', 1: 'neutral', 2: 'positive'})
>>> from explabox import Explabox
>>> box = Explabox(data=data,
... model=model,
... splits={'train': 'drugsComTrain.tsv', 'test': 'drugsComTest.tsv'})
```
Then `.explore`, `.examine`, `.expose` and `.explain` your model:
```python
>>> # Explore the descriptive statistics for each split
>>> box.explore()
```
```python
>>> # Show wrongly classified instances
>>> box.examine.wrongly_classified()
```
```python
>>> # Compare the performance on the test split before and after adding typos to the text
>>> box.expose.compare_metrics(split='test', perturbation='add_typos')
```
```python
>>> # Get a local explanation (uses LIME by default)
>>> box.explain.box.explain_prediction('Hate this medicine so much!')
```
For more information, visit the [explabox documentation](https://explabox.rtfd.io).
# Contents
- [Quick tour](#quick-tour)
- [Installation](#installation)
- [Documentation](#documentation)
- [Example usage](#example-usage)
- [Releases](#releases)
- [Contributing](#contributing)
- [Citation](#citation)
## Installation
The easiest way to install the latest release of the `explabox` is through `pip`:
```console
user@terminal:~$ pip install explabox
Collecting explabox
...
Installing collected packages: explabox
Successfully installed explabox
```
> :information_source: The `explabox` requires _Python 3.8_ or above.
See the [full installation guide](INSTALLATION.md) for troubleshooting the installation and other installation methods.
## Documentation
Documentation for the `explabox` is hosted externally on [explabox.rtfd.io](https://explabox.rtfd.io).
The `explabox` consists of three layers:
1. __Ingestibles__ provide a unified interface for importing models and data, which abstracts away how they are accessed and allows for optimized processing.
2. __Analyses__ are used to turn opaque ingestibles into transparent digestibles. The four types of analyses are _explore_, _examine_, _explain_ and _expose_.
3. __Digestibles__ provide insights into model behavior and data, assisting stakeholders in increasing the explainability, fairness, auditability and safety of their AI systems. Depending on their needs, these can be accessed interactively (e.g. via the Jupyter Notebook UI or embedded via the API) or through static reporting.
## Example usage
The [example usage guide](EXAMPLE_USAGE.md) showcases the `explabox` for a black-box model performing multi-class classification of the [UCI Drug Reviews](https://archive.ics.uci.edu/ml/datasets/Drug+Review+Dataset+%28Drugs.com%29#) dataset.
Without requiring any local installations, the notebook is provided on [](https://colab.research.google.com/drive/14lXvXV01DaSruSAhD1RLbILRl2mPQ4nS?usp=sharing).
If you want to follow along on your own device, simply `pip install explabox-demo-drugreview` and run the lines in the [Jupyter notebook](https://git.science.uu.nl/m.j.robeer/explabox-demo-drugreview/-/blob/main/explabox_demo_drugreview.ipynb) we have prepared for you!
## Releases
The `explabox` is officially released through [PyPI](https://pypi.org/project/explabox/). The [changelog](CHANGELOG.md) includes a full overview of the changes for each version.
## Contributing
The `explabox` is an open-source project developed and maintained primarily by the Netherlands *National Police Lab AI* (NPAI). However, your contributions and improvements are still required! See [contributing](CONTRIBUTING.md) for a full contribution guide.
```bibtex
```