https://github.com/younader/dnnr

The Python package of differential nearest neighbors regression (DNNR): Raising KNN-regression to levels of gradient boosting method. Build on-top of Numpy, Scikit-Learn, and Annoy.
https://github.com/younader/dnnr

annoy knn machine-learning machine-learning-algorithms numpy python3 scikit-learn tabular-data

Last synced: about 3 hours ago
JSON representation

The Python package of differential nearest neighbors regression (DNNR): Raising KNN-regression to levels of gradient boosting method. Build on-top of Numpy, Scikit-Learn, and Annoy.

• Host: GitHub
• URL: https://github.com/younader/dnnr
• Owner: younader
• License: mit
• Created: 2022-05-17T12:33:51.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2022-08-04T10:03:22.000Z (over 2 years ago)
• Last Synced: 2024-11-08T02:11:19.381Z (about 11 hours ago)
• Topics: annoy, knn, machine-learning, machine-learning-algorithms, numpy, python3, scikit-learn, tabular-data
• Language: Python
• Homepage: https://younader.github.io/dnnr/
• Size: 1.11 MB
• Stars: 14
• Watchers: 3
• Forks: 1
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# DNNR: Differential Nearest Neighbors Regression

[![Build Status](https://github.com/younader/dnnr/actions/workflows/dev.yml/badge.svg)](https://github.com/younader/dnnr/actions/workflows/dev.yml)

[[Paper](https://proceedings.mlr.press/v162/nader22a.html)]

[[Documentation](https://younader.github.io/dnnr/)]

The Python package of [differential nearest neighbors regression (DNNR)](https://proceedings.mlr.press/v162/nader22a.html): **Raising KNN-regression to levels of gradient boosting methods.**

Whereas KNN regression only uses the averaged value, DNNR also uses the gradient or even higher-order derivatives:

![KNN and DNNR Overview Image](knn_dnnr_overview.png)

Our implementation uses `numpy`, `sklearn`, and the [`annoy`](https://github.com/spotify/annoy) approximate nearest neighbor index. Using `annoy` is optional, as you can also use `sklearn`'s KDTree as index. We support Python 3.7 to 3.10.

# 🚀 Quickstart

To install this project, run:

```bash

pip install dnnr

```

# 🎉 Example

```python

import numpy as np

from dnnr import DNNR

X = np.array([[0], [1], [2], [3]])

y = np.array([0, 0, 1, 1])

model = DNNR(n_neighbors=1, n_derivative_neighbors=3)

model.fit(X, y)

model.predict([[1.5]])

# Will output: 0.25

```

Also check out our [Jupiter Notebook](./examples/dnnr_tutorial.ipynb) on how to use DNNR. [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/younader/dnnr/blob/main/examples/dnnr_tutorial.ipynb)

# 📊 Hyperparameters

DNNR has three main hyperparameters:

* `n_neighbors`: number of nearest neighbors to use. The default value of

      `3` is usually a good choice.

* `n_derivative_neighbors`: number of neighbors used in approximating the

      derivatives. As a default value, we choose `3 * dim`, where `dim` is

      the input dimension.

* `order`: Taylor approximation order, one of `1`, `2`, `2diag`, `3diag`.

      The preferable option here is `1`. Sometimes `2diag` can deliver

      small improvements. `2` and `3diag` are implemented but usually do

      not yield significant improvements.

We recommend a hyperparameter search over the `n_derivative_neighbors` variable to archive the best performance.

For all options, see the [documentation of the DNNR class](https://younader.github.io/dnnr/api/#dnnr.dnnr.DNNR).

#  🛠 Development Installation

```bash

python3 -m venv venv      # create a virtual environment

source venv/bin/activate  # and load it

git clone https://github.com/younader/dnnr.git

cd dnnr

pip install -U pip wheel poetry

poetry install

make test                 # to run the tests

```

# 📄 Citation

If you use this library for a scientific publication, please use the following BibTex entry to cite our work:

```bibtex

@InProceedings{pmlr-v162-nader22a,

  title = 	 {{DNNR}: Differential Nearest Neighbors Regression},

  author =       {Nader, Youssef and Sixt, Leon and Landgraf, Tim},

  booktitle = 	 {Proceedings of the 39th International Conference on Machine Learning},

  pages = 	 {16296--16317},

  year = 	 {2022},

  editor = 	 {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan},

  volume = 	 {162},

  series = 	 {Proceedings of Machine Learning Research},

  month = 	 {17--23 Jul},

  publisher =    {PMLR},

  pdf = 	 {https://proceedings.mlr.press/v162/nader22a/nader22a.pdf},

  url = 	 {https://proceedings.mlr.press/v162/nader22a.html},

}

```