https://github.com/bits-bytes-nn/gauss-rank-scaler
Scikit-learn compatible implementation of the Gauss Rank scaling method
https://github.com/bits-bytes-nn/gauss-rank-scaler
gauss-rank scikit-learn transformer
Last synced: about 1 year ago
JSON representation
Scikit-learn compatible implementation of the Gauss Rank scaling method
- Host: GitHub
- URL: https://github.com/bits-bytes-nn/gauss-rank-scaler
- Owner: bits-bytes-nn
- License: bsd-3-clause
- Created: 2019-08-17T15:21:18.000Z (almost 7 years ago)
- Default Branch: master
- Last Pushed: 2023-10-12T16:03:24.000Z (over 2 years ago)
- Last Synced: 2025-02-22T09:32:20.279Z (over 1 year ago)
- Topics: gauss-rank, scikit-learn, transformer
- Language: Python
- Homepage:
- Size: 84 KB
- Stars: 73
- Watchers: 1
- Forks: 16
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Gauss Rank Scaler
A scikit-learn style transformer that scales numeric variables to normal distributions.
Input normalization for neural networks is very important. Gauss Rank is an effective algorithm for converting numeric variable distributions to normals. It is based on rank transformation. The first step is to assign a spacing between -1 and 1 to the sorted features, then apply the inverse of error function `erfinv` to make it look like a Gaussian.
This generally works much better than Standard or Min Max Scaler.
## Important Links
* [Interview of the Kaggle competition winner (Michael Jahrer)](https://www.kaggle.com/c/porto-seguro-safe-driver-prediction/discussion/44629#250927)
* [Blog post introducing Gauss Rank's concept and simple implementation (Zygmunt Zając)](http://fastml.com/preparing-continuous-features-for-neural-networks-with-rankgauss)
## Usage
Gauss Rank Scaler is a fully compatible sklearn transformer that can be used in pipelines or existing scripts. Supported input formats include numpy arrays and pandas dataframes. All columns passed to the transformer are properly scaled.
## Example
```python
from gauss_rank_scaler.gauss_rank_scaler import GaussRankScaler
import pandas as pd
from sklearn.datasets import load_boston
%matplotlib inline
# prepare some data
bunch = load_boston()
df_X_train = pd.DataFrame(bunch.data[:250], columns=bunch.feature_names)
df_X_test = pd.DataFrame(bunch.data[250:], columns=bunch.feature_names)
# plot histograms of two numeric variables
_ = df_X_train[['CRIM', 'DIS']].hist()
```
```python
# scale the numeric variables with Gauss Rank Scaler
scaler = GaussRankScaler()
df_X_new_train = scaler.fit_transform(df_X_train[['CRIM', 'DIS']])
# plot histograms of the scaled variables
_ = pd.DataFrame(df_X_new_train, columns=['CRIM', 'DIS']).hist()
```
```python
# scale test dataset with the fitted scaler
df_X_new_test = scaler.transform(df_X_test[['CRIM', 'DIS']])
```