https://github.com/jszitas/categoryencodings
Multiple methods to (quickly) encode factor variables, using data.table
https://github.com/jszitas/categoryencodings
categorical-variables feature-encoding feature-engineering r r-package
Last synced: 9 months ago
JSON representation
Multiple methods to (quickly) encode factor variables, using data.table
- Host: GitHub
- URL: https://github.com/jszitas/categoryencodings
- Owner: JSzitas
- Created: 2019-12-21T23:51:33.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2021-09-25T22:24:17.000Z (over 4 years ago)
- Last Synced: 2025-03-17T19:06:29.184Z (about 1 year ago)
- Topics: categorical-variables, feature-encoding, feature-engineering, r, r-package
- Language: R
- Homepage:
- Size: 161 KB
- Stars: 3
- Watchers: 1
- Forks: 1
- Open Issues: 3
-
Metadata Files:
- Readme: README.Rmd
Awesome Lists containing this project
README
---
output: github_document
---
```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "man/figures/README-",
out.width = "100%"
)
```
# categoryEncodings
[](https://github.com/JSzitas/categoryEncodings/actions)
[](https://ci.appveyor.com/project/JSzitas/categoryEncodings)
[](https://codecov.io/gh/JSzitas/categoryEncodings?branch=master)
[](https://www.tidyverse.org/lifecycle/#stable)
[](https://www.gnu.org/licenses/gpl-3.0)
[](https://CRAN.R-project.org/package=categoryEncodings)
**categoryEncodings** intends to provide a fast way to encode 'factor' or qualitative variables through various methods. The packages uses **data.table** as the backend for speed, with as few other dependencies as possible. Most of the methods are based on the paper of Johannemann et al.(2019) - Sufficient Representations for Categorical Variables (arXiv:1908.09874).
The current version features automatic inference of factors and uses a very simple heuristic for encoding, as well as allowing manual controls.
## Installation
You can install the latest version of **categoryEncodings** from [github](https://github.com/JSzitas/categoryEncodings) using the *devtools* package
``` {r, eval = FALSE}
devtools::install_github("JSzitas/categoryEncodings")
```
**NOTE:** The latest stable version available from **CRAN** contains features deprecated in the current development version - I hope to resolve this soon, and publish the development version.
## Example
Here we want to encode all of the factors in a given data.frame.
````{r, message = FALSE, eval = TRUE}
library(categoryEncodings)
# create some example data
data_fm <- cbind( data.frame(
matrix( rnorm(5*100),ncol = 5)),
sample(sample(letters, 10), 100, replace = TRUE),
sample(sample(letters, 20), 100, replace = TRUE),
sample(sample(1:10, 5), 100, replace = TRUE),
sample(sample(1:50, 35), 100, replace = TRUE ),
sample(1:2, 100, replace = TRUE ))
colnames(data_fm)[6:10] <- c( "few_letters", "many_letters",
"some_numbers", "many_numbers",
"binary" )
# it does not matter how many factor variables there are, whether they are encoded as factors
# and whether you supply a method to encode them by - some simple inference of factors is done
# based on the number of distinct values in every variable - over a certain threshold
# a variable is deemed as essentially a factor, and treated as such for conversion
# you will be notified of which variables are being converted via a warning
result <- encoder(data_fm)
# note that due to the data.table back-end, the result has to be saved to an object to be
# visible: otherwise printing is suppressed.
head(result$encoded)
````
We also recover a **function closure** which we can reuse to fit new data, as long as it conforms to the same format:
```{r}
# to fit to any dataset you can either call it directly - it is a single argument function
data_fm_encoded <- result$fitted_encoder(data_fm)
# or rename it, and stash it away for later use
encoding_function <- result$fitted_encoder
```
You also get a **"de-encoding"** function -
```{r}
deencoder <- result$fitted_deencoder
```
This undoes the "encoding", effectively returning the original data. This can be quite useful for interpretability methods, where the
interpretation becomes easier for un-encoded data.
Note that this sadly does not maintain the order of the data from the original - and some attributes may be lost. Nonetheless, the recovered data is almost the same, and equivalent for all practical purposes:
```{r}
original <- data.table::data.table(data_fm)
deencoded <- deencoder( result$encoded )
all.equal( data.table::setorder(original),
data.table::setorder(deencoded),
check.attributes = FALSE )
```
## Contributing
Please do contribute to the projects, all contributions are welcome, as long as people keep things civil - there is no need for negativity, hatred, and rudeness. Also, please do refrain from adding unnecessary dependencies (*Ex:* pipe) to the package (such pull requests as would add an unnecessary dependency will be denied/ suspended until the code can be made dependency free). This package wants to be as lightweight as possible - even if this means the code is a bit harder to write and maintain.