https://github.com/daniel-lima-lopez/ada-knn-python

A python implementation of the classifier Ada-kNN
https://github.com/daniel-lima-lopez/ada-knn-python

ada-knn adaptive-knn knn knn-algorithm knn-classification machine-learning neural-networks

Last synced: 20 days ago
JSON representation

A python implementation of the classifier Ada-kNN

• Host: GitHub
• URL: https://github.com/daniel-lima-lopez/ada-knn-python
• Owner: daniel-lima-lopez
• Created: 2024-08-01T18:02:09.000Z (5 months ago)
• Default Branch: main
• Last Pushed: 2024-08-13T19:42:11.000Z (5 months ago)
• Last Synced: 2024-08-14T04:12:26.065Z (5 months ago)
• Topics: ada-knn, adaptive-knn, knn, knn-algorithm, knn-classification, machine-learning, neural-networks
• Language: Python
• Homepage:
• Size: 532 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Ada-kNN-Python

This is a python implementation of the Ada-kNN classifier, proposed by [Mullick et al.](https://ieeexplore.ieee.org/abstract/document/8326728)

## Classifier description

The classifier is an extension of kNN, in which a strategy based on Multi-layer perceptron (MLP) is proposed to automate the choice of the parameter k, for each instance to be classified.

The operation of Ada-kNN is described below:

- First, for each instance $x_i$ in the training set, a series of experiments are performed to identify the $k$ values which correctly classify $x_i$ with kNN.

- With this information, an MLP architecture is trained to predict the most appropriate $k$ value for classifying a given instance based on their attribute values.

- Once the neural network has been trained, for each instance to be classified, the most appropriate value of $k$ is predicted with this network, then a conventional kNN classifier performs a prediction with this value.

## Installation

Clone this repository:

```bash

git clone [email protected]:daniel-lima-lopez/Ada-kNN-Python.git

```

move to installation directory:

```bash

cd Ada-kNN-Python

```

## Basic usage

The classifier can be used without specifying any value:

```python

import Ada_kNN as ada

classifier = ada.Ada_kNN()

```

However, the training parameters can be modified:

- `alpha`: number of experiments consdiered on $X_{test}$

- `lr` (0.01): learning rate for MLP traning

- `batch_size` (4): batch size for MLP traning

- `epochs` (100): number of epochs for MLP traning

Once instantiated the classifier, we can perform a simple test:

```python

import pandas as pd

from sklearn.model_selection import train_test_split

from sklearn.metrics import accuracy_score

# data preapration

dataset = pd.read_csv('Datasets/wine.csv')

Xs = dataset.drop('class', axis=1).values

ys = dataset['class'].values

# data split

X_train, X_test, y_train, y_test = train_test_split(Xs, ys, test_size=0.2, random_state=2) #12

# fit the classifier

classifier.fit(X_train, y_train)

# predictions on test set

preds = classifier.predict(X_test)

print(f'accuracy: {accuracy_score(y_true=y_test, y_pred=preds)}')

```

## Experiments

Experiments were conducted to compare the performance of Ada-kNN with kNN. On each experiment, a 10-fold cross-validation was performed. For kNN, the `k` values considered are 1, 3, 5, 7, and 9. The accuracy on appendicitis, balance-scale, bands, climate, music and spectf datasets are presented on the following figures, where it is observed that the dynamic decision of 

k-value enhances accuracy in certain cases, but in others, it does not: