{"id":26038878,"url":"https://github.com/habedi/feature-factory","last_synced_at":"2025-08-01T21:08:56.472Z","repository":{"id":281084639,"uuid":"943551050","full_name":"habedi/feature-factory","owner":"habedi","description":"A high-performance feature engineering library for Rust powered by Apache DataFusion 🦀","archived":false,"fork":false,"pushed_at":"2025-03-29T19:47:35.000Z","size":90,"stargazers_count":16,"open_issues_count":3,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-03T01:17:12.792Z","etag":null,"topics":["data-preprocessing","data-science","feature-engineering","feature-selection","machine-learning","rust-lang","rust-library"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/habedi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE-APACHE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-05T22:20:55.000Z","updated_at":"2025-06-20T09:53:41.000Z","dependencies_parsed_at":null,"dependency_job_id":"62df208c-725e-406a-a13b-937af9a1d13c","html_url":"https://github.com/habedi/feature-factory","commit_stats":null,"previous_names":["habedi/feature-factory"],"tags_count":2,"template":false,"template_full_name":"habedi/template-rust-project","purl":"pkg:github/habedi/feature-factory","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/habedi%2Ffeature-factory","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/habedi%2Ffeature-factory/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/habedi%2Ffeature-factory/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/habedi%2Ffeature-factory/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/habedi","download_url":"https://codeload.github.com/habedi/feature-factory/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/habedi%2Ffeature-factory/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268297337,"owners_count":24228123,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-01T02:00:08.611Z","response_time":67,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-preprocessing","data-science","feature-engineering","feature-selection","machine-learning","rust-lang","rust-library"],"created_at":"2025-03-07T10:38:01.624Z","updated_at":"2025-08-01T21:08:56.460Z","avatar_url":"https://github.com/habedi.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003cpicture\u003e\n    \u003cimg alt=\"Weldon the Penguin\" src=\"assets/logo.png\" height=\"35%\" width=\"35%\"\u003e\n  \u003c/picture\u003e\n\u003c/div\u003e\n\n## Feature Factory\n\n[![Tests](https://img.shields.io/github/actions/workflow/status/habedi/feature-factory/tests.yml?label=tests\u0026style=flat\u0026labelColor=282c34\u0026color=4caf50\u0026logo=github)](https://github.com/habedi/feature-factory/actions/workflows/tests.yml)\n[![Lints](https://img.shields.io/github/actions/workflow/status/habedi/feature-factory/lints.yml?label=lints\u0026style=flat\u0026labelColor=282c34\u0026color=4caf50\u0026logo=github)](https://github.com/habedi/feature-factory/actions/workflows/lints.yml)\n[![Code Coverage](https://img.shields.io/codecov/c/github/habedi/feature-factory?style=flat\u0026labelColor=282c34\u0026color=ffca28\u0026logo=codecov)](https://codecov.io/gh/habedi/feature-factory)\n[![CodeFactor](https://img.shields.io/codefactor/grade/github/habedi/feature-factory?style=flat\u0026labelColor=282c34\u0026color=4caf50\u0026logo=codefactor)](https://www.codefactor.io/repository/github/habedi/feature-factory)\n[![Crates.io](https://img.shields.io/crates/v/feature-factory.svg?style=flat\u0026labelColor=282c34\u0026color=f46623\u0026logo=rust)](https://crates.io/crates/feature-factory)\n[![Docs.rs](https://img.shields.io/badge/docs.rs-feature--factory-66c2a5?style=flat\u0026labelColor=282c34\u0026logo=docs.rs)](https://docs.rs/feature-factory)\n[![Downloads](https://img.shields.io/crates/d/feature-factory?style=flat\u0026labelColor=282c34\u0026color=4caf50\u0026logo=rust)](https://crates.io/crates/feature-factory)\n[![MSRV](https://img.shields.io/badge/MSRV-1.83.0-007ec6?label=msrv\u0026style=flat\u0026labelColor=282c34\u0026logo=rust)](https://github.com/rust-lang/rust/releases/tag/1.83.0)\n[![License](https://img.shields.io/badge/license-MIT%2FApache--2.0-007ec6?style=flat\u0026labelColor=282c34\u0026logo=open-source-initiative)](https://github.com/habedi/feature-factory)\n[![Status: Alpha](https://img.shields.io/badge/status-alpha-ec407a.svg?style=flat\u0026labelColor=282c34)](https://github.com/habedi/feature-factory)\n\nFeature Factory is a feature engineering library for Rust built on top\nof [Apache DataFusion](https://datafusion.apache.org/).\nIt uses DataFusion internally for fast, in-memory data processing.\nIt is inspired by the [Feature-engine](https://feature-engine.readthedocs.io/en/latest/) Python library and\nprovides a wide range of components (referred to as transformers) for common feature engineering tasks like imputation,\nencoding, discretization, and feature selection.\n\nFeature Factory aims to be feature-rich and provide an API similar to [Scikit-learn](https://scikit-learn.org/stable/),\nwith the performance benefits of Rust and Apache DataFusion. Feature Factory transformers follow\na [fit-transform paradigm](https://scikit-learn.org/stable/data_transforms.html), where each transformer provides a\nconstructor, a `fit` method, and a `transform` method. Given an input dataframe, a transformer applies a\ntransformation to the data and returns a new dataframe.\nThe library also provides a pipeline API that allows users to chain multiple transformers together to create data\ntransformation pipelines for feature engineering.\n\n\u003e [!IMPORTANT]\n\u003e Feature Factory is currently in the early stage of development. APIs are unstable and may change without notice.\n\u003e Inconsistencies in documentation are expected, and not all features have been implemented yet.\n\u003e It has not yet been thoroughly tested, benchmarked, or optimized for performance.\n\u003e Bug reports, feature requests, and contributions are welcome!\n\n### Features\n\n- **High Performance**: Feature Factory uses Apache DataFusion as the backend data processing engine.\n- **Scikit-learn API**: It provides a Scikit-learn-like API which is familiar to most data scientists.\n- **Pipeline API**: Users can chain multiple transformers together to build a feature engineering pipeline.\n- **Large Set of Transformers**: Currently, Feature Factory includes the following transformers:\n\n| **Task**                                                          | **Transformers**                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         | Status |\n|-------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------|\n| [**Imputation**](src/transformers/imputation.rs)                  | - `MeanMedianImputer`: Replace missing values with the mean (or median).  \u003cbr\u003e- `ArbitraryNumberImputer`: Replace missing values with an arbitrary number.  \u003cbr\u003e- `EndTailImputer`: Replace missing values with values at distribution tails.  \u003cbr\u003e- `CategoricalImputer`: Replace missing values with an arbitrary string or most frequent category.  \u003cbr\u003e- `AddMissingIndicator`: Add a binary indicator for missing values.  \u003cbr\u003e- `DropMissingData`: Remove rows with missing values.                                                                                                                                                                                                                                                                | Tested |\n| [**Categorical Encoding**](src/transformers/categorical.rs)       | - `OneHotEncoder`: Perform one-hot encoding.  \u003cbr\u003e- `CountFrequencyEncoder`: Replace categories with their frequencies.  \u003cbr\u003e- `OrdinalEncoder`: Replace categories with ordered numbers.  \u003cbr\u003e- `MeanEncoder`: Replace categories with target mean.  \u003cbr\u003e- `WoEEncoder`: Replace categories with the weight of evidence.  \u003cbr\u003e- `RareLabelEncoder`: Group infrequent categories.                                                                                                                                                                                                                                                                                                                                                                        | Tested |\n| [**Variable Discretization**](src/transformers/discretization.rs) | - `ArbitraryDiscretizer`: Discretize based on user-defined intervals.  \u003cbr\u003e- `EqualFrequencyDiscretizer`: Discretize into equal-frequency bins.  \u003cbr\u003e- `EqualWidthDiscretizer`: Discretize into equal-width bins.  \u003cbr\u003e- `GeometricWidthDiscretizer`: Discretize into geometric intervals.                                                                                                                                                                                                                                                                                                                                                                                                                                                               | Tested |\n| [**Outlier Handling**](src/transformers/outliers.rs)              | - `ArbitraryOutlierCapper`: Cap outliers at user-defined bounds.  \u003cbr\u003e- `Winsorizer`: Cap outliers using percentile thresholds.  \u003cbr\u003e- `OutlierTrimmer`: Remove outliers from the dataset.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | Tested |\n| [**Numerical Transformations**](src/transformers/numerical.rs)    | - `LogTransformer`: Apply logarithmic transformation.  \u003cbr\u003e- `LogCpTransformer`: Apply log transformation with a constant.  \u003cbr\u003e- `ReciprocalTransformer`: Apply reciprocal transformation.  \u003cbr\u003e- `PowerTransformer`: Apply power transformation.  \u003cbr\u003e- `BoxCoxTransformer`: Apply Box-Cox transformation.  \u003cbr\u003e- `YeoJohnsonTransformer`: Apply Yeo-Johnson transformation.  \u003cbr\u003e- `ArcsinTransformer`: Apply arcsin transformation.                                                                                                                                                                                                                                                                                                                  | Tested |\n| [**Feature Creation**](src/transformers/feature_creation.rs)      | - `MathFeatures`: Create new features with mathematical operations.  \u003cbr\u003e- `RelativeFeatures`: Combine features with reference features.  \u003cbr\u003e- `CyclicalFeatures`: Encode cyclical features using sine or cosine.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       | Tested |\n| [**Datetime Features**](src/transformers/datetime.rs)             | - `DatetimeFeatures`: Extract features from datetime values.  \u003cbr\u003e- `DatetimeSubtraction`: Compute time differences between datetime values.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             | Tested |\n| [**Feature Selection**](src/transformers/feature_selection.rs)    | - `DropFeatures`: Drop specific features.\u003cbr\u003e- `DropConstantFeatures`: Remove constant and quasi-constant features.\u003cbr\u003e- `DropDuplicateFeatures`: Remove duplicate features.\u003cbr\u003e- `DropCorrelatedFeatures`: Remove highly correlated features.\u003cbr\u003e- `SmartCorrelatedSelection`: Select the best features from correlated groups.\u003cbr\u003e-`DropHighPSIFeatures`: Drop features based on Population Stability Index (PSI).\u003cbr\u003e- `SelectByInformationValue`: Select features based on information value.\u003cbr\u003e- `SelectBySingleFeaturePerformance`: Select features based on univariate estimators.\u003cbr\u003e- `SelectByTargetMeanPerformance`: Select features based on target mean encoding.\u003cbr\u003e- `MRMR`: Select features using Maximum Relevance Minimum Redundancy. | Tested |\n\n\u003e [!NOTE]\n\u003e Status shows whether the module is `Tested` (unit, integration, and documentation tests) and `Benchmarked`.\n\u003e Empty status means the module has not yet been tested and benchmarked.\n\n### Installation\n\n```shell\ncargo add feature-factory\n```\n\nOr add this to your `Cargo.toml`:\n\n```toml\n[dependencies]\nfeature-factory = \"0.1\"\n```\n\n*Feature Factory requires Rust 1.83 or later.*\n\n### Documentation\n\nYou can find the latest API documentation at [docs.rs/feature-factory](https://docs.rs/feature-factory).\n\n### Architecture\n\nThe main building blocks of Feature Factory are *transformers* and *pipelines*.\n\n#### Transformers\n\nA transformer takes one or more columns from an input DataFrame and creates new columns based on a transformation.\nTransformers can be *stateful* or *stateless*:\n\n- A stateful transformer needs to learn one or more parameters from the data during training (via calling `fit`) before\n  it can transform the data. A stateful transformer with learned parameters is referred to as a *fitted* transformer.\n- A Stateless transformer can directly transform the data without needing to learn any parameters.\n\nAll transformers implement the [`Transformer`](src/pipeline.rs) trait, which includes:\n\n| **Method**    | **Description**                                                                                     |\n|---------------|-----------------------------------------------------------------------------------------------------|\n| `new`         | Creates a new transformer instance. Can accept hyperparameters and column names as input arguments. |\n| `fit`         | Learns parameters from data. For stateless transformers this is a no-op.                            |\n| `transform`   | Applies the transformation to data. Stateful transformers require calling `fit` first.              |\n| `is_stateful` | Returns `true` if the transformer is stateful, otherwise `false`.                                   |\n\nThe figure below shows a high-level overview of how a single Feature Factory transformer works:\n\n![Feature Factory Transformer](assets/transformer_architecture.svg)\n\n\u003e [!IMPORTANT]\n\u003e In most cases, to avoid data leakage, the data used for training a transformer must not be the same as the data that\n\u003e is going to be transformed.\n\n#### Pipelines\n\nA pipeline chains multiple transformers together. Pipelines are created using the [`make_pipeline`](src/pipeline.rs)\nmacro, which accepts a list of `(name, transformer)` tuples.\nStateful transformers must be fitted before they're used in a pipeline.\n\nThe figure below shows a high-level overview of how a Feature Factory pipeline works:\n\n![Feature Factory Pipeline](assets/pipeline_architecture.svg)\n\n\u003e [!IMPORTANT]\n\u003e Currently, to use a stateful transformer in a pipeline, it must be already fitted.\n\n### Examples\n\nCheck out the [examples](examples) and [tests](tests) directories for examples of how to use Feature Factory.\n\n### Contributing\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) for details on how to make a contribution.\n\n### Logo\n\nThe mascot of this project is named \"Weldon the Penguin\".\nHe is a Rustacean penguin who loves to swim in the sea and play video games—and is always ready to help you with your\ndata.\n\nThe logo was created using Gimp, ComfyUI, and a Flux Schnell v2 model.\n\n### Licensing\n\nFeature Factory is available under the terms of either of these licenses:\n\n* MIT License ([LICENSE-MIT](LICENSE-MIT))\n* Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE))\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhabedi%2Ffeature-factory","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhabedi%2Ffeature-factory","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhabedi%2Ffeature-factory/lists"}