https://github.com/sanjaradylov/scikit-fallback
Machine Learning with a Reject Option
https://github.com/sanjaradylov/scikit-fallback
fallback-classification machine-learning scikit-learn selective-classification
Last synced: 8 months ago
JSON representation
Machine Learning with a Reject Option
- Host: GitHub
- URL: https://github.com/sanjaradylov/scikit-fallback
- Owner: sanjaradylov
- License: bsd-3-clause
- Created: 2024-06-18T12:05:35.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2025-01-22T13:21:09.000Z (over 1 year ago)
- Last Synced: 2025-01-31T19:11:12.584Z (over 1 year ago)
- Topics: fallback-classification, machine-learning, scikit-learn, selective-classification
- Language: Python
- Homepage: https://scikit-fallback.rtfd.io
- Size: 504 KB
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://pepy.tech/project/scikit-fallback)
[](https://www.codefactor.io/repository/github/sanjaradylov/scikit-fallback)
[](https://www.python.org/downloads/release/python-3913/)
[](https://github.com/psf/black)
[](https://github.com/PyCQA/pylint)
[](https://x.com/intent/tweet?text=Wow:%20https%3A%2F%2Fgithub.com%2Fsanjaradylov%2Fscikit-fallback%20@sanjaradylov)
[](https://twitter.com/sanjaradylov)
**scikit-fallback** is a scikit-learn-compatible Python package for machine learning
with a reject option.
### 👩💻 Usage
To allow your probabilistic pipeline to *fallback*—i.e., abstain from predictions—you can
wrap it with a `skfb` *rejector*. Training a rejector means both fitting your model and
learning to accept or reject predictions. Evaluation of a rejector depends
on *fallback mode* (inference with or without *fallback labels*) and measures the ability
of the rejector to both accept correct predictions and reject ambiguous ones.
For example, `skfb.estimators.ThresholdFallbackClassifierCV` fits the base estimator and then
finds the best confidence threshold via cross-validation. If `fallback_mode == "store"`, then the
rejector returns `skfb.core.array.FBNDArray` of predictions and a sparse fallback-mask property,
which lets us summarize the accuracy of both predictions and rejections.
```python
from skfb.estimators import ThresholdFallbackClassifierCV
from sklearn.linear_model import LogisticRegressionCV
rejector = ThresholdFallbackClassifierCV(
LogisticRegressionCV(cv=4, random_state=0),
thresholds=10,
ambiguity_threshold=0.05,
cv=5,
fallback_label=-1,
fallback_mode="store",
)
rejector.fit(X_train, y_train) # Train base estimator and learn best threshold
rejector.score(X_test, y_test) # Compute acceptance-correctness accuracy score
```
And for more motivation and information on usage, please visit our
[documentation](https://scikit-fallback.readthedocs.io/en/latest/index.html)
and refer to the
[Medium series](https://medium.com/@sshadylov) on machine learning with a reject option.
### 🏗 Installation
`scikit-fallback` requires:
* Python (>=3.9,<3.13)
* scikit-learn (>=1.0)
* numpy
* scipy
* matplotlib (>=3.0) (optional)
and along with the requirements can be installed via `pip` :
```bash
pip install -U scikit-fallback
```
### 📚 Examples
See the [`examples/`]([examples/](https://github.com/sanjaradylov/scikit-fallback/tree/main/examples))
directory for various applications of fallback estimators
and scorers to scikit-learn-compatible pipelines.
### 🔗 References
1. Hendrickx, K., Perini, L., Van der Plas, D. et al. Machine learning with a reject option: a survey. Mach Learn 113, 3073–3110 (2024). https://doi.org/10.1007/s10994-024-06534-x