{"id":102827,"url":"https://github.com/mlpapers/feature-selection","name":"feature-selection","description":"Awesome papers on Feature Selection","projects_count":85,"last_synced_at":"2026-06-18T19:00:18.240Z","repository":{"id":301067491,"uuid":"253096195","full_name":"mlpapers/feature-selection","owner":"mlpapers","description":"Awesome papers on Feature Selection","archived":false,"fork":false,"pushed_at":"2026-02-14T21:37:57.000Z","size":32,"stargazers_count":39,"open_issues_count":1,"forks_count":3,"subscribers_count":4,"default_branch":"master","last_synced_at":"2026-06-02T02:03:09.893Z","etag":null,"topics":["awesome","awesome-list","correlation","dimensionality-reduction","feature-importance","feature-selection","mutual-information","variable-selection"],"latest_commit_sha":null,"homepage":"https://mlpapers.org/feature-selection/","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mlpapers.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2020-04-04T20:50:02.000Z","updated_at":"2026-04-11T21:46:19.000Z","dependencies_parsed_at":"2026-03-02T12:01:08.064Z","dependency_job_id":null,"html_url":"https://github.com/mlpapers/feature-selection","commit_stats":null,"previous_names":["mlpapers/feature-selection"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mlpapers/feature-selection","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlpapers%2Ffeature-selection","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlpapers%2Ffeature-selection/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlpapers%2Ffeature-selection/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlpapers%2Ffeature-selection/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mlpapers","download_url":"https://codeload.github.com/mlpapers/feature-selection/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlpapers%2Ffeature-selection/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34503511,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-18T02:00:06.871Z","response_time":128,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"created_at":"2026-01-02T00:00:41.133Z","updated_at":"2026-06-18T19:00:18.240Z","primary_language":null,"list_of_lists":false,"displayable":true,"categories":["Software","Uncategorized","Related Topics"],"sub_categories":["Uncategorized"],"readme":"# Feature selection (variable selection)\n\u003e Feature selection is the process of selecting a subset of relevant features (variables, predictors) for use in model construction ([Wikipedia](https://en.wikipedia.org/wiki/Feature_selection))\n\nWhy feature selection?\n1. Data exploration\n2. Curse of dimensionality\n3. Less features - faster models\n4. Better metrics\n\n- **Overview**\n  - [An Introduction to Variable and Feature Selection](http://jmlr.csail.mit.edu/papers/volume3/guyon03a/guyon03a.pdf) (2003) *Isabelle Guyon, Andre Elisseeff*\n  - [A Survey on Feature Selection](https://www.sciencedirect.com/science/article/pii/S1877050916313047) (2016) *Jianyu Miaoac, Lingfeng Niu*\n  - [Feature Selection: A Data Perspective](https://arxiv.org/pdf/1601.07996.pdf) (2016) *Jundong Li, Kewei Cheng, Suhang Wang, Fred Morstatter, Robert P. Trevino, Jiliang Tang, Huan Liu*\n  - [Feature Selection and Feature Extraction in Pattern Analysis: A Literature Review](https://arxiv.org/pdf/1905.02845.pdf) (2019) *Benyamin Ghojogh, Maria N. Samad, Sayema Asif Mashhadi,Tania Kapoor, Wahab Ali, Fakhri Karray, Mark Crowle*\n- **All-relevant vs minimal-optimal feature selection**\n  - [Consistent Feature Selection for Pattern Recognition in Polynomial Time](http://jmlr.csail.mit.edu/papers/volume8/nilsson07a/nilsson07a.pdf) (2007) *R. Nilsson, J. M. Peña, J. Björkegren, J. Tegnér*\n\n### Filter methods\nFilter methods use model-free ranking to filter less relevant features\n- **Missing Values Ratio**\n  - Removing features with a ratio of missing values greater than some threshold\n- **Low Variance Filter** ([sklearn](https://scikit-learn.org/stable/modules/feature_selection.html#removing-features-with-low-variance))\n  - Removing features with a variance lower than some threshold\n- **Correlation** ([Wiki](https://en.wikipedia.org/wiki/Correlation_and_dependence))\n- **χ²** Chi-squared statistic for categorical features ([Wiki](https://en.wikipedia.org/wiki/Chi-squared_test), [sklearn](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.chi2.html))\n- **ANOVA** F-value for quantitative features([Wiki](https://en.wikipedia.org/wiki/Analysis_of_variance), [sklearn](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.f_classif.html))\n- **Mutual information** ([Wiki](https://en.wikipedia.org/wiki/Mutual_information))\n  - [Conditional Likelihood Maximisation: A Unifying Framework for Information Theoretic Feature Selection](http://jmlr.csail.mit.edu/papers/volume13/brown12a/brown12a.pdf) (2012) *Gavin Brown, Adam Pocock, Ming-Jie Zhao, Mikel Lujan*\n  - [Feature Selection Based on Joint Mutual Information](https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.41.4424) (1999) *Howard Hua Yang, John Moody*\n  - [Estimating mutual information](https://arxiv.org/pdf/cond-mat/0305641.pdf) (2003) *Alexander Kraskov, Harald Stoegbauer, Peter Grassberger*\n- **mRMR** Minimum redundancy, maximal relevancy ([Link](http://home.penglab.com/proj/mRMR/), [Wiki](https://en.wikipedia.org/wiki/Minimum_redundancy_feature_selection))\n  - [Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy](http://home.penglab.com/papersall/docpdf/2005_TPAMI_FeaSel.pdf) (2005) *Hanchuan Peng, Fuhui Long, Chris Ding*\n- **Relief** ([Wiki](https://en.wikipedia.org/wiki/Relief_(feature_selection)))\n  - [The Feature Selection Problem: Traditional Methods and a New Algorithm](https://www.aaai.org/Papers/AAAI/1992/AAAI92-020.pdf) (1992) *Kira Kenji, Larry Rendell*\n  - [Relief-Based Feature Selection: Introduction and Review](https://arxiv.org/pdf/1711.08421.pdf) (2018) *Ryan J. Urbanowicz, Melissa Meeker, William LaCava, Randal S. Olson, Jason H.Moore*\n- **Markov Blanket** ([Wiki](https://en.wikipedia.org/wiki/Markov_blanket))\n  - [Markov Blanket based Feature Selection: A Review of Past Decade](http://www.iaeng.org/publication/WCE2010/WCE2010_pp321-328.pdf) (2010) *Shunkai Fu, Michel C. Desmarais*\n  - Incremental Association Markov Blanket: [Algorithms for Large Scale Markov Blanket Discovery](https://www.aaai.org/Papers/FLAIRS/2003/Flairs03-073.pdf) (2003) *Ioannis Tsamardinos, Constantin F. Aliferis, Alexander Statnikov*\n  - Grow-Shrink algorithm: [Bayesian Network Induction via Local Neighborhoods](http://robots.stanford.edu/papers/Margaritis99a.pdf) (2000) *Dimitris Margaritis, Sebastian Thrun*\n  - Koller-Sahami method: [Toward Optimal Feature Selection](http://ilpubs.stanford.edu:8090/208/1/1996-77.pdf) (1996) *Daphne Koller and Mehran Sahami*\n  - Max-Min Markov Blanket: [Time and Sample Efficient Discovery of Markov Blankets and Direct Causal Relations](https://dl.acm.org/doi/10.1145/956750.956838) (2003) *Ioannis Tsamardinos, Constantin F. Aliferis, Alexander Statnikov*\n- **Fast Correlation-based Filter**\n  - [Feature Selection for High-Dimensional Data: A Fast Correlation-Based Filter Solution](https://www.public.asu.edu/~huanliu/papers/icml03.pdf) (2003) *Lei Yu, Huan Liu*\n- **CBF** Consistency-Based Filters\n  - [Consistency-based search in feature selection](https://www.public.asu.edu/~huanliu/papers/aij03.pdf) (2003) *Manoranjan Dasha, Huan Liu*\n- **Interact**\n  - [Searching for Interacting Features](https://www.public.asu.edu/~huanliu/papers/ijcai07.pdf) (2007) *Zheng Zhao, Huan Liu*\n\n### Wrapper methods\nWrapper methods use a model and its performance to find the best feature subset\n- **SFS** Sequential Feature Selection\n- **SFFS** Sequential Floating Forward Selection\n  - [Floating search methods in feature selection](https://www.academia.edu/15425286/Floating_search_methods_in_feature_selection) (1994) *Pavel Pudil, Josef Kittler, Jana Novovicová*\n  - [Adaptive floating search methods in feature selection](https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.11.5032) (1999) *P. Somol , Pavel Pudil , Jana Novovicova , P. Paclik*\n- **Genertic algorithm** ([Wiki](https://en.wikipedia.org/wiki/Genetic_algorithm))\n- **PSO** Particle Swarm Optimization ([Wiki](https://en.wikipedia.org/wiki/Particle_swarm_optimization))\n  - [Particle Swarm Optimization](https://www.cs.tufts.edu/comp/150GA/homeworks/hw3/_reading6%201995%20particle%20swarming.pdf) (1995) *James Kennedy, Russell Eberhar*\n  - [Feature Selection using PSO-SVM](http://www.iaeng.org/IJCS/issues_v33/issue_1/IJCS_33_1_18.pdf) (2007) *Chung-Jui Tu, Li-Yeh Chuang, Jun-Yang Chang, Cheng-Hong Yang*\n- **Boruta** All-relevant feature selection ([CRAN](https://cran.r-project.org/web/packages/Boruta/), [PyPI](https://pypi.org/project/Boruta/))\n  - [Boruta – A System for Feature Selection](https://www.mimuw.edu.pl/~ajank/papers/Kursa2010.pdf) (2010) *Miron B. Kursa,  Aleksander Jankowski,  Witold R. Rudnick*\n  - BoostARoota - Boruta with XGBoost as a base model ([Code](https://github.com/chasedehan/BoostARoota))\n- **MUVR** ([GitLab](https://gitlab.com/CarlBrunius/MUVR))\n  - [Variable selection and validation in multivariate modelling](https://academic.oup.com/bioinformatics/article/35/6/972/5085367) (2018) *Lin Shi, Johan A Westerhuis, Johan Rosén, Rikard Landberg, Carl Brunius*\n- Wrappers methods and overfitting:\n  - [Wrappers for feature subset selection](http://machine-learning.martinsewell.com/feature-selection/KohaviJohn1997.pdf) (1996) *Ron Kohavi, George H. John*\n\n### Embedded methods\n- **LASSO**\n  - [Regression Shrinkage and Selection via the lasso](https://statweb.stanford.edu/~tibs/lasso/lasso.pdf) (1996) *Robert Tibshirani*\n- **Elastic net**\n  - [Regularization and variable selection via the elastic net](https://web.stanford.edu/~hastie/Papers/B67.2%20(2005)%20301-320%20Zou%20\u0026%20Hastie.pdf) (2005) *Hui Zou, Trevor Hastie*\n- **Spike and Slab regression** ([Wiki](https://en.wikipedia.org/wiki/Spike-and-slab_regression))\n  - [Bayesian variable selection in linear regression](1988) *T.J. Mitchell, J.J. Beuchamp*\n  - [Approaches for Bayesian variable selection](http://www-stat.wharton.upenn.edu/~edgeorge/Research_papers/GeorgeMcCulloch97.pdf) (1997) *Edward I. George, Robert E. McCulloch*\n- **Decision Tree** ([Wiki](https://en.wikipedia.org/wiki/Decision_tree))\n- **Random Forest** ([Wiki](https://en.wikipedia.org/wiki/Random_forest))\n  - [Random Forests](https://link.springer.com/article/10.1023/A:1010933404324) (2001) *Leo Breiman*\n  - [Overview of Random Forest Methodology and Practical Guidance with Emphasis on Computational Biology and Bioinformatic](https://epub.ub.uni-muenchen.de/13766/1/TR.pdf) (2012) *Anne-Laure Boulesteix, Silke Janitza, Jochen Kruppa, Inke R. Konig*\n  - [Variable selection using random forests](https://hal.archives-ouvertes.fr/hal-00755489/file/PRLv4.pdf) (2010) *Robin Genuer, Jean-Michel Poggi, Christine Tuleau-Malot*\n  - [Bias in random forest variable importance measures: Illustrations, sources and a solution](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1796903/pdf/1471-2105-8-25.pdf) (2007) *Carolin Strobl, Anne-Laure Boulesteix, Achim Zeileis, Torsten Hothorn*\n  - [Conditional Variable Importance for Random Forests](https://epub.ub.uni-muenchen.de/2821/1/deck.pdf) (2008) *Carolin Strobl, Anne-Laure Boulesteix, Thomas Kneib, Thomas Augustin, Achim Zeileis*\n  - [Correlation and variable importance in random forests](https://arxiv.org/pdf/1310.5726.pdf) (2016) *Baptiste Gregorutti, Bertrand Michel, Philippe Saint-Pierre*\n- **Gradient Boosting** ([Wiki](https://en.wikipedia.org/wiki/Gradient_boosting))\n  - [Greedy Function Approximation: A Gradient Boosting Machine](https://statweb.stanford.edu/~jhf/ftp/trebst.pdf) (1999) *Jerome H Friedman*\n  - [Boosting Algorithms as Gradient Descent](http://papers.nips.cc/paper/1766-boosting-algorithms-as-gradient-descent.pdf) (1999) *Llew Mason, Jonathan Baxter, Peter Bartlett, Marcus Frean*\n\n### Unsupervised and semi-supervised feature selection\n- **FSSEM** Feature Subset Selection using Expectation-Maximization\n  - [Feature Selection for Unsupervised Learning](http://www.jmlr.org/papers/volume5/dy04a/dy04a.pdf) (2004) *Jennifer G. Dy, Carla E. Brodley*\n- **Laplacian Score**\n  - Choosing features using a nearest neighbor graph\n  - [Laplacian Score for Feature Selection](https://papers.nips.cc/paper/2909-laplacian-score-for-feature-selection.pdf) (2005) *Xiaofei  He, Deng  Cai, Deng Cai, Partha  Niyogi, Partha Niyogi*\n- **Principal Feature Analysis**\n  - [Feature Selection Using Principal Feature Analysis](http://venom.cs.utsa.edu/dmz/techrep/2007/CS-TR-2007-011.pdf) (2007) *Yijuan Lu, Ira Cohen, Xiang Sean Zhou, Qi Tian*\n- **Spectral Feature Selection**\n  - Separates samples into clusters using a spectrum of pairwise similarity graph\n  - [Spectral Feature Selection forSupervised and Unsupervised Learning](https://www.public.asu.edu/~huanliu/papers/icml07.pdf) (2007) *Zheng Zhao, Huan Liu*\n- **MCFS** Multi-cluster Feature Selection\n  - [Unsupervised Feature Selection for Multi-Cluster Data](https://wwwx.cs.unc.edu/Courses/comp790-090-s11/Presentations/p333-cai.pdf) (2010) *Deng Cai, Chiyuan Zhang, Xiaofei He*\n- **Autoencoders** ([Wiki](https://en.wikipedia.org/wiki/Autoencoder))\n  - [Autoencoders, Unsupervised Learning, and Deep Architectures](http://proceedings.mlr.press/v27/baldi12a/baldi12a.pdf) (2012) *Pierre Baldi*\n  - [An Introduction to Variational Autoencoders](https://arxiv.org/pdf/1906.02691.pdf) (2019) *Diederik P. Kingma, Max Welling*\n  - [Concrete Autoencoders for Differentiable Feature Selection and Reconstruction](https://arxiv.org/pdf/1901.09346.pdf) (2019) *Abubakar Abid, Muhammad Fatih Balin, James Zou*\n\n### Stable feature selection\n- [Stability of Feature Selection Algorithms: a study on high dimensional spaces](http://cui.unige.ch/~kalousis/papers/stability/KalousisPradosHilarioKIS2007.pdf) (2007) *Alexandros Kalousis, Julien Prados, Melanie Hilario*\n- [Robust Feature Selection Using Ensemble Feature Selection Techniques](http://bioinformatics.psb.ugent.be/pdf/publications/978-3-540-87481-2.pdf) (2008) *Yvan Saeys, Thomas Abeel, Yves Van de Pee*\n- [Stability Selection](https://stat.ethz.ch/~nicolai/stability.pdf) (2009) *Nicolai Meinshausen, Peter Buhlmann*\n- [A Novel Weighted Combination Method for Feature Selection using Fuzzy Sets](https://arxiv.org/pdf/2005.05003.pdf) (2020) *Zixiao Shen, Xin Chen, Jonathan M. Garibald*\n- Stability of MDA, LIME and SHAP: [The best way to select features](https://arxiv.org/pdf/2005.12483.pdf) (2020) *Xin Man, Ernest P. Chan*\n\n### Domain-specific\n- **Uplift models**\n  - [Feature Selection Methods for Uplift Modeling](https://arxiv.org/pdf/2005.03447.pdf) (2020) *Zhenyu Zhao, Yumin Zhang, Totte Harinen, Mike Yung*\n\n### Meta feature selection\n- [A Feature Subset Selection Algorithm AutomaticRecommendation Method](https://arxiv.org/pdf/1402.0570.pdf) (2013) *Guangtao Wang, Qinbao Song, Heli Sun, Xueying Zhang, Baowen Xu, Yuming Zhou*\n- [Metalearning for Choosing Feature Selection Algorithms in Data Mining: Proposal of a New Framework](https://www.researchgate.net/profile/Antonio_Parmezan/publication/312482443_Metalearning_for_Choosing_Feature_Selection_Algorithms_in_Data_Mining_Proposal_of_a_New_Framework/links/5c3f0e7f92851c22a378a5a6/Metalearning-for-Choosing-Feature-Selection-Algorithms-in-Data-Mining-Proposal-of-a-New-Framework.pdf) (2017) *Antonio Rafael Sabino Parmezan, Huei Diana Lee*\n- [A Novel Meta Learning Framework for Feature Selection using Data Synthesis and Fuzzy Similarity](https://arxiv.org/pdf/2005.09856.pdf) (2020) *Zixiao Shen, Xin Chen, Jonathan M. Garibald*\n\n## Software\n- **R**\n  - Package: fscaret ([CRAN](https://cran.r-project.org/web/packages/fscaret/)) *Jakub Szlek*\n  - Package: praznik ([Code](https://gitlab.com/mbq/praznik)) *Miron Kursa*\n  - Package: FSinR ([CRAN](https://cran.r-project.org/web/packages/FSinR/), [Paper](https://arxiv.org/pdf/2002.10330.pdf)) *Francisco Aragón-Royón, Alfonso Jiménez-Vílchez, Antonio Arauzo-Azofra, José Manuel Benítez*\n  - Package: VSURF ([CRAN](https://cran.r-project.org/web/packages/VSURF/), [Paper](https://journal.r-project.org/archive/2015-2/genuer-poggi-tuleaumalot.pdf))\n  - Package: spikeSlabGAM ([Code](https://github.com/fabian-s/spikeSlabGAM), [CRAN](https://cran.r-project.org/web/packages/spikeSlabGAM/), [Paper](https://www.jstatsoft.org/article/view/v043i14))\n  - Package: copent ([CRAN](https://cran.r-project.org/web/packages/copent/), [Code](https://github.com/majianthu/copent), [Paper](https://arxiv.org/pdf/2005.14025.pdf))\n- **Python**\n  - Package: sklearn.feature_selection ([Homepage](https://scikit-learn.org/stable/), [Code](https://github.com/scikit-learn/scikit-learn), [PyPI](https://pypi.org/project/scikit-learn/))\n  - Package: scikit-feature ([Homepage](http://featureselection.asu.edu/), [Code](https://github.com/jundongl/scikit-feature))\n  - Package: feature-selector ([Code](https://github.com/WillKoehrsen/feature-selector), [PyPI](https://pypi.org/project/feature-selector/))\n- **Julia**\n  - the main packages for ML in Julia are [MLJ](https://github.com/alan-turing-institute/MLJ.jl) and [Flow](https://fluxml.ai/Flux.jl/stable/)\n\n## Related Topics\n- [Feature Extraction](https://mlpapers.org/feature-extraction/)\n- [Ensemble Learning](https://mlpapers.org/ensemble-learning/)\n- [AutoML](https://mlpapers.org/automl/)\n","projects_url":"https://awesome.ecosyste.ms/api/v1/lists/mlpapers%2Ffeature-selection/projects"}