{"id":15882651,"url":"https://github.com/chris-santiago/steps","last_synced_at":"2025-06-28T17:36:51.010Z","repository":{"id":48117907,"uuid":"391431103","full_name":"chris-santiago/steps","owner":"chris-santiago","description":"A SciKit-Learn style feature selector using best subsets and stepwise regression.","archived":false,"fork":false,"pushed_at":"2024-03-27T13:50:15.000Z","size":800,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-27T19:13:25.783Z","etag":null,"topics":["best-subset-selection","data-science","python","scikit-learn","stepwise-selection"],"latest_commit_sha":null,"homepage":"https://chris-santiago.github.io/steps/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chris-santiago.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-07-31T18:10:06.000Z","updated_at":"2025-03-14T02:03:01.000Z","dependencies_parsed_at":"2024-11-15T10:38:07.578Z","dependency_job_id":"cbad21f1-89eb-4dde-a2f4-55d8790abd02","html_url":"https://github.com/chris-santiago/steps","commit_stats":{"total_commits":42,"total_committers":4,"mean_commits":10.5,"dds":"0.26190476190476186","last_synced_commit":"e7f06a4aad76c7a6038324e5b4b75155f324116a"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/chris-santiago/steps","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chris-santiago%2Fsteps","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chris-santiago%2Fsteps/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chris-santiago%2Fsteps/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chris-santiago%2Fsteps/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chris-santiago","download_url":"https://codeload.github.com/chris-santiago/steps/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chris-santiago%2Fsteps/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262470641,"owners_count":23316536,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["best-subset-selection","data-science","python","scikit-learn","stepwise-selection"],"created_at":"2024-10-06T04:06:01.939Z","updated_at":"2025-06-28T17:36:50.986Z","avatar_url":"https://github.com/chris-santiago.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# step-select\n[![image](https://img.shields.io/badge/python-3.7--3.10-blue.svg)](https://www.python.org)\n[![Build Status](https://app.travis-ci.com/chris-santiago/steps.svg?branch=master)](https://app.travis-ci.com/chris-santiago/steps)\n[![codecov](https://codecov.io/gh/chris-santiago/steps/branch/master/graph/badge.svg?token=RIB2YFGWFX)](https://codecov.io/gh/chris-santiago/steps)\n\nA SciKit-Learn style feature selector using best subsets and stepwise regression.\n\n## Install\n\nCreate a virtual environment with Python 3.8 and install from PyPi:\n\n```bash\npip install step-select\n```\n\n## Use\n\n### Preliminaries\n\n*Note: this example requires two additional packages*: `pandas` and `statsmodels`.\n\nIn this example we'll show how the `ForwardSelector` and `SubsetSelector` classes can be used on their own or in conjuction with a Scikit-Learn `Pipeline` object.\n\n\n```python\nimport pandas as pd\nfrom sklearn.pipeline import Pipeline\nfrom sklearn.linear_model import LinearRegression\nimport statsmodels.datasets\nfrom statsmodels.api import OLS\nfrom statsmodels.tools import add_constant\n\nfrom steps.forward import ForwardSelector\nfrom steps.subset import SubsetSelector\n```\n\nWe'll download the `auto` dataset via `Statsmodels`; we'll use `mpg` as the endogenous variable and the remaining variables as exongenous.  We won't use `make`, as that will create several dummies and increase the number of paramters to 12+, which is too many for the `SubsetSelector` class; we'll also drop `price`.\n\n\n```python\ndata = statsmodels.datasets.webuse('auto')\ndata['foreign'] = pd.Series([x == 'Foreign' for x in data['foreign']]).astype(int)\ndata.fillna(0, inplace=True)\ndata.head()\n```\n\n\n\n\n\u003cdiv\u003e\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003emake\u003c/th\u003e\n      \u003cth\u003eprice\u003c/th\u003e\n      \u003cth\u003empg\u003c/th\u003e\n      \u003cth\u003erep78\u003c/th\u003e\n      \u003cth\u003eheadroom\u003c/th\u003e\n      \u003cth\u003etrunk\u003c/th\u003e\n      \u003cth\u003eweight\u003c/th\u003e\n      \u003cth\u003elength\u003c/th\u003e\n      \u003cth\u003eturn\u003c/th\u003e\n      \u003cth\u003edisplacement\u003c/th\u003e\n      \u003cth\u003egear_ratio\u003c/th\u003e\n      \u003cth\u003eforeign\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e0\u003c/th\u003e\n      \u003ctd\u003eAMC Concord\u003c/td\u003e\n      \u003ctd\u003e4099\u003c/td\u003e\n      \u003ctd\u003e22\u003c/td\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e2.5\u003c/td\u003e\n      \u003ctd\u003e11\u003c/td\u003e\n      \u003ctd\u003e2930\u003c/td\u003e\n      \u003ctd\u003e186\u003c/td\u003e\n      \u003ctd\u003e40\u003c/td\u003e\n      \u003ctd\u003e121\u003c/td\u003e\n      \u003ctd\u003e3.58\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e1\u003c/th\u003e\n      \u003ctd\u003eAMC Pacer\u003c/td\u003e\n      \u003ctd\u003e4749\u003c/td\u003e\n      \u003ctd\u003e17\u003c/td\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e11\u003c/td\u003e\n      \u003ctd\u003e3350\u003c/td\u003e\n      \u003ctd\u003e173\u003c/td\u003e\n      \u003ctd\u003e40\u003c/td\u003e\n      \u003ctd\u003e258\u003c/td\u003e\n      \u003ctd\u003e2.53\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e2\u003c/th\u003e\n      \u003ctd\u003eAMC Spirit\u003c/td\u003e\n      \u003ctd\u003e3799\u003c/td\u003e\n      \u003ctd\u003e22\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e12\u003c/td\u003e\n      \u003ctd\u003e2640\u003c/td\u003e\n      \u003ctd\u003e168\u003c/td\u003e\n      \u003ctd\u003e35\u003c/td\u003e\n      \u003ctd\u003e121\u003c/td\u003e\n      \u003ctd\u003e3.08\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e3\u003c/th\u003e\n      \u003ctd\u003eBuick Century\u003c/td\u003e\n      \u003ctd\u003e4816\u003c/td\u003e\n      \u003ctd\u003e20\u003c/td\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e4.5\u003c/td\u003e\n      \u003ctd\u003e16\u003c/td\u003e\n      \u003ctd\u003e3250\u003c/td\u003e\n      \u003ctd\u003e196\u003c/td\u003e\n      \u003ctd\u003e40\u003c/td\u003e\n      \u003ctd\u003e196\u003c/td\u003e\n      \u003ctd\u003e2.93\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e4\u003c/th\u003e\n      \u003ctd\u003eBuick Electra\u003c/td\u003e\n      \u003ctd\u003e7827\u003c/td\u003e\n      \u003ctd\u003e15\u003c/td\u003e\n      \u003ctd\u003e4.0\u003c/td\u003e\n      \u003ctd\u003e4.0\u003c/td\u003e\n      \u003ctd\u003e20\u003c/td\u003e\n      \u003ctd\u003e4080\u003c/td\u003e\n      \u003ctd\u003e222\u003c/td\u003e\n      \u003ctd\u003e43\u003c/td\u003e\n      \u003ctd\u003e350\u003c/td\u003e\n      \u003ctd\u003e2.41\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\n\n```python\nX = data.iloc[:, 3:]\ny = data['mpg']\n```\n\n### Forward Stepwise Selection\n\nThe `ForwardSelector` follows the standard stepwise regression algorithm: begin with a null model, iteratively test each variable and select the one that gives the most statistically significant improvement of the fit, and repeat. This greedy algorithm continues until the fit no longer improves.\n\nThe `ForwardSelector` is instantiated with two parameters: `normalize` and `metric`. `Normalize` defaults to `False`, assuming that this class is part of a larger pipeline; `metric` defaults to AIC.\n\n|Parameter|Type|Description|\n|---------|----|-----------|\n|normalize|bool|Whether to normalize features; default `False`|\n|metric|str|Optimization metric to use; must be one of `aic` or `bic`; default `aic`|\n\nThe `ForwardSelector` class follows the Scikit-Learn API.  After fitting the selector using the `.fit()` method, the selected features can be accessed using the boolean mask under the `.best_support_` attribute.\n\n\n```python\nselector = ForwardSelector(normalize=True, metric='aic')\nselector.fit(X, y)\n```\n\n\n\n\n    ForwardSelector(normalize=True)\n\n\n\n\n```python\nX.loc[:, selector.best_support_]\n```\n\n\n\n\n\u003cdiv\u003e\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003erep78\u003c/th\u003e\n      \u003cth\u003eweight\u003c/th\u003e\n      \u003cth\u003elength\u003c/th\u003e\n      \u003cth\u003egear_ratio\u003c/th\u003e\n      \u003cth\u003eforeign\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e0\u003c/th\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e2930\u003c/td\u003e\n      \u003ctd\u003e186\u003c/td\u003e\n      \u003ctd\u003e3.58\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e1\u003c/th\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e3350\u003c/td\u003e\n      \u003ctd\u003e173\u003c/td\u003e\n      \u003ctd\u003e2.53\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e2\u003c/th\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e2640\u003c/td\u003e\n      \u003ctd\u003e168\u003c/td\u003e\n      \u003ctd\u003e3.08\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e3\u003c/th\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e3250\u003c/td\u003e\n      \u003ctd\u003e196\u003c/td\u003e\n      \u003ctd\u003e2.93\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e4\u003c/th\u003e\n      \u003ctd\u003e4.0\u003c/td\u003e\n      \u003ctd\u003e4080\u003c/td\u003e\n      \u003ctd\u003e222\u003c/td\u003e\n      \u003ctd\u003e2.41\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e...\u003c/th\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e69\u003c/th\u003e\n      \u003ctd\u003e4.0\u003c/td\u003e\n      \u003ctd\u003e2160\u003c/td\u003e\n      \u003ctd\u003e172\u003c/td\u003e\n      \u003ctd\u003e3.74\u003c/td\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e70\u003c/th\u003e\n      \u003ctd\u003e5.0\u003c/td\u003e\n      \u003ctd\u003e2040\u003c/td\u003e\n      \u003ctd\u003e155\u003c/td\u003e\n      \u003ctd\u003e3.78\u003c/td\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e71\u003c/th\u003e\n      \u003ctd\u003e4.0\u003c/td\u003e\n      \u003ctd\u003e1930\u003c/td\u003e\n      \u003ctd\u003e155\u003c/td\u003e\n      \u003ctd\u003e3.78\u003c/td\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e72\u003c/th\u003e\n      \u003ctd\u003e4.0\u003c/td\u003e\n      \u003ctd\u003e1990\u003c/td\u003e\n      \u003ctd\u003e156\u003c/td\u003e\n      \u003ctd\u003e3.78\u003c/td\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e73\u003c/th\u003e\n      \u003ctd\u003e5.0\u003c/td\u003e\n      \u003ctd\u003e3170\u003c/td\u003e\n      \u003ctd\u003e193\u003c/td\u003e\n      \u003ctd\u003e2.98\u003c/td\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e74 rows × 5 columns\u003c/p\u003e\n\u003c/div\u003e\n\n\n\n### Best Subset Selection\n\nThe `SubsetSelector` follows a very simple algorithm: compare all possible models with $k$ predictors, and select the model that minimizes our selection criteria. This algorithm is only appropriate for $k\u003c=12$ features, as it becomes computationally expensive: there are $\\frac{k!}{(p-k)!}$possible models, where $p$ is the total number of paramters and $k$ is the number of features included in the model.\n\nThe `SubsetSelector` is instantiated with two parameters: `normalize` and `metric`. `Normalize` defaults to `False`, assuming that this class is part of a larger pipeline; `metric` defaults to AIC.\n\n|Parameter|Type|Description|\n|---------|----|-----------|\n|normalize|bool|Whether to normalize features; default `False`|\n|metric|str|Optimization metric to use; must be one of `aic` or `bic`; default `aic`|\n\nThe `SubsetSelector` class follows the Scikit-Learn API.  After fitting the selector using the `.fit()` method, the selected features can be accessed using the boolean mask under the `.best_support_` attribute.\n\n\n```python\nselector = SubsetSelector(normalize=True, metric='aic')\nselector.fit(X, y)\n```\n\n\n\n\n    SubsetSelector(normalize=True)\n\n\n\n\n```python\nX.loc[:, selector.get_support()]\n```\n\n\n\n\n\u003cdiv\u003e\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003erep78\u003c/th\u003e\n      \u003cth\u003eweight\u003c/th\u003e\n      \u003cth\u003elength\u003c/th\u003e\n      \u003cth\u003egear_ratio\u003c/th\u003e\n      \u003cth\u003eforeign\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e0\u003c/th\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e2930\u003c/td\u003e\n      \u003ctd\u003e186\u003c/td\u003e\n      \u003ctd\u003e3.58\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e1\u003c/th\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e3350\u003c/td\u003e\n      \u003ctd\u003e173\u003c/td\u003e\n      \u003ctd\u003e2.53\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e2\u003c/th\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e2640\u003c/td\u003e\n      \u003ctd\u003e168\u003c/td\u003e\n      \u003ctd\u003e3.08\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e3\u003c/th\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e3250\u003c/td\u003e\n      \u003ctd\u003e196\u003c/td\u003e\n      \u003ctd\u003e2.93\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e4\u003c/th\u003e\n      \u003ctd\u003e4.0\u003c/td\u003e\n      \u003ctd\u003e4080\u003c/td\u003e\n      \u003ctd\u003e222\u003c/td\u003e\n      \u003ctd\u003e2.41\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e...\u003c/th\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e69\u003c/th\u003e\n      \u003ctd\u003e4.0\u003c/td\u003e\n      \u003ctd\u003e2160\u003c/td\u003e\n      \u003ctd\u003e172\u003c/td\u003e\n      \u003ctd\u003e3.74\u003c/td\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e70\u003c/th\u003e\n      \u003ctd\u003e5.0\u003c/td\u003e\n      \u003ctd\u003e2040\u003c/td\u003e\n      \u003ctd\u003e155\u003c/td\u003e\n      \u003ctd\u003e3.78\u003c/td\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e71\u003c/th\u003e\n      \u003ctd\u003e4.0\u003c/td\u003e\n      \u003ctd\u003e1930\u003c/td\u003e\n      \u003ctd\u003e155\u003c/td\u003e\n      \u003ctd\u003e3.78\u003c/td\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e72\u003c/th\u003e\n      \u003ctd\u003e4.0\u003c/td\u003e\n      \u003ctd\u003e1990\u003c/td\u003e\n      \u003ctd\u003e156\u003c/td\u003e\n      \u003ctd\u003e3.78\u003c/td\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e73\u003c/th\u003e\n      \u003ctd\u003e5.0\u003c/td\u003e\n      \u003ctd\u003e3170\u003c/td\u003e\n      \u003ctd\u003e193\u003c/td\u003e\n      \u003ctd\u003e2.98\u003c/td\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e74 rows × 5 columns\u003c/p\u003e\n\u003c/div\u003e\n\n\n\n### Comparing the full model \n\nUsing the `SubsetSelector` selected features yields a model with 4 fewer parameters and slightly improved AIC and BIC metrics. The summaries indicate possible multicollinearity in both models, likely caused by `weight`, `length`, `displacement` and other features that are all related to the weight of a vehicle. \n\n*Note: Selection using BIC as the optimization metric yields a model where `weight` is the only selected feature. Bayesian information criteria penalizes additional parameters more then AIC.*\n\n\n```python\nmod = OLS(endog=y, exog=add_constant(X)).fit()\nmod.summary()\n```\n\n\n\n\n\n\u003ctable class=\"simpletable\"\u003e\n\u003ccaption\u003eOLS Regression Results\u003c/caption\u003e\n\u003ctr\u003e\n  \u003cth\u003eDep. Variable:\u003c/th\u003e           \u003ctd\u003empg\u003c/td\u003e       \u003cth\u003e  R-squared:         \u003c/th\u003e \u003ctd\u003e   0.720\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eModel:\u003c/th\u003e                   \u003ctd\u003eOLS\u003c/td\u003e       \u003cth\u003e  Adj. R-squared:    \u003c/th\u003e \u003ctd\u003e   0.681\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eMethod:\u003c/th\u003e             \u003ctd\u003eLeast Squares\u003c/td\u003e  \u003cth\u003e  F-statistic:       \u003c/th\u003e \u003ctd\u003e   18.33\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eDate:\u003c/th\u003e             \u003ctd\u003eSat, 07 Aug 2021\u003c/td\u003e \u003cth\u003e  Prob (F-statistic):\u003c/th\u003e \u003ctd\u003e1.29e-14\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eTime:\u003c/th\u003e                 \u003ctd\u003e15:37:36\u003c/td\u003e     \u003cth\u003e  Log-Likelihood:    \u003c/th\u003e \u003ctd\u003e -187.23\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eNo. Observations:\u003c/th\u003e      \u003ctd\u003e    74\u003c/td\u003e      \u003cth\u003e  AIC:               \u003c/th\u003e \u003ctd\u003e   394.5\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eDf Residuals:\u003c/th\u003e          \u003ctd\u003e    64\u003c/td\u003e      \u003cth\u003e  BIC:               \u003c/th\u003e \u003ctd\u003e   417.5\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eDf Model:\u003c/th\u003e              \u003ctd\u003e     9\u003c/td\u003e      \u003cth\u003e                     \u003c/th\u003e     \u003ctd\u003e \u003c/td\u003e   \n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eCovariance Type:\u003c/th\u003e      \u003ctd\u003enonrobust\u003c/td\u003e    \u003cth\u003e                     \u003c/th\u003e     \u003ctd\u003e \u003c/td\u003e   \n\u003c/tr\u003e\n\u003c/table\u003e\n\u003ctable class=\"simpletable\"\u003e\n\u003ctr\u003e\n        \u003ctd\u003e\u003c/td\u003e          \u003cth\u003ecoef\u003c/th\u003e     \u003cth\u003estd err\u003c/th\u003e      \u003cth\u003et\u003c/th\u003e      \u003cth\u003eP\u003e|t|\u003c/th\u003e  \u003cth\u003e[0.025\u003c/th\u003e    \u003cth\u003e0.975]\u003c/th\u003e  \n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003econst\u003c/th\u003e        \u003ctd\u003e   39.0871\u003c/td\u003e \u003ctd\u003e    9.100\u003c/td\u003e \u003ctd\u003e    4.295\u003c/td\u003e \u003ctd\u003e 0.000\u003c/td\u003e \u003ctd\u003e   20.907\u003c/td\u003e \u003ctd\u003e   57.267\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003erep78\u003c/th\u003e        \u003ctd\u003e    1.0021\u003c/td\u003e \u003ctd\u003e    0.357\u003c/td\u003e \u003ctd\u003e    2.809\u003c/td\u003e \u003ctd\u003e 0.007\u003c/td\u003e \u003ctd\u003e    0.290\u003c/td\u003e \u003ctd\u003e    1.715\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eheadroom\u003c/th\u003e     \u003ctd\u003e   -0.0167\u003c/td\u003e \u003ctd\u003e    0.611\u003c/td\u003e \u003ctd\u003e   -0.027\u003c/td\u003e \u003ctd\u003e 0.978\u003c/td\u003e \u003ctd\u003e   -1.237\u003c/td\u003e \u003ctd\u003e    1.204\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003etrunk\u003c/th\u003e        \u003ctd\u003e   -0.0772\u003c/td\u003e \u003ctd\u003e    0.154\u003c/td\u003e \u003ctd\u003e   -0.503\u003c/td\u003e \u003ctd\u003e 0.617\u003c/td\u003e \u003ctd\u003e   -0.384\u003c/td\u003e \u003ctd\u003e    0.230\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eweight\u003c/th\u003e       \u003ctd\u003e   -0.0037\u003c/td\u003e \u003ctd\u003e    0.002\u003c/td\u003e \u003ctd\u003e   -1.928\u003c/td\u003e \u003ctd\u003e 0.058\u003c/td\u003e \u003ctd\u003e   -0.008\u003c/td\u003e \u003ctd\u003e    0.000\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003elength\u003c/th\u003e       \u003ctd\u003e   -0.0752\u003c/td\u003e \u003ctd\u003e    0.061\u003c/td\u003e \u003ctd\u003e   -1.229\u003c/td\u003e \u003ctd\u003e 0.223\u003c/td\u003e \u003ctd\u003e   -0.197\u003c/td\u003e \u003ctd\u003e    0.047\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eturn\u003c/th\u003e         \u003ctd\u003e   -0.1762\u003c/td\u003e \u003ctd\u003e    0.187\u003c/td\u003e \u003ctd\u003e   -0.941\u003c/td\u003e \u003ctd\u003e 0.350\u003c/td\u003e \u003ctd\u003e   -0.550\u003c/td\u003e \u003ctd\u003e    0.198\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003edisplacement\u003c/th\u003e \u003ctd\u003e    0.0131\u003c/td\u003e \u003ctd\u003e    0.011\u003c/td\u003e \u003ctd\u003e    1.180\u003c/td\u003e \u003ctd\u003e 0.243\u003c/td\u003e \u003ctd\u003e   -0.009\u003c/td\u003e \u003ctd\u003e    0.035\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003egear_ratio\u003c/th\u003e   \u003ctd\u003e    3.7067\u003c/td\u003e \u003ctd\u003e    1.751\u003c/td\u003e \u003ctd\u003e    2.116\u003c/td\u003e \u003ctd\u003e 0.038\u003c/td\u003e \u003ctd\u003e    0.208\u003c/td\u003e \u003ctd\u003e    7.206\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eforeign\u003c/th\u003e      \u003ctd\u003e   -4.4633\u003c/td\u003e \u003ctd\u003e    1.385\u003c/td\u003e \u003ctd\u003e   -3.222\u003c/td\u003e \u003ctd\u003e 0.002\u003c/td\u003e \u003ctd\u003e   -7.230\u003c/td\u003e \u003ctd\u003e   -1.696\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\u003ctable class=\"simpletable\"\u003e\n\u003ctr\u003e\n  \u003cth\u003eOmnibus:\u003c/th\u003e       \u003ctd\u003e28.364\u003c/td\u003e \u003cth\u003e  Durbin-Watson:     \u003c/th\u003e \u003ctd\u003e   2.523\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eProb(Omnibus):\u003c/th\u003e \u003ctd\u003e 0.000\u003c/td\u003e \u003cth\u003e  Jarque-Bera (JB):  \u003c/th\u003e \u003ctd\u003e  52.945\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eSkew:\u003c/th\u003e          \u003ctd\u003e 1.389\u003c/td\u003e \u003cth\u003e  Prob(JB):          \u003c/th\u003e \u003ctd\u003e3.18e-12\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eKurtosis:\u003c/th\u003e      \u003ctd\u003e 6.074\u003c/td\u003e \u003cth\u003e  Cond. No.          \u003c/th\u003e \u003ctd\u003e7.55e+04\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\u003cbr/\u003e\u003cbr/\u003eNotes:\u003cbr/\u003e[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\u003cbr/\u003e[2] The condition number is large, 7.55e+04. This might indicate that there are\u003cbr/\u003estrong multicollinearity or other numerical problems.\n\n\n\n\n```python\nmod = OLS(endog=y, exog=add_constant(X.loc[:, selector.best_support_])).fit()\nmod.summary()\n```\n\n\n\n\n\n\u003ctable class=\"simpletable\"\u003e\n\u003ccaption\u003eOLS Regression Results\u003c/caption\u003e\n\u003ctr\u003e\n  \u003cth\u003eDep. Variable:\u003c/th\u003e           \u003ctd\u003empg\u003c/td\u003e       \u003cth\u003e  R-squared:         \u003c/th\u003e \u003ctd\u003e   0.710\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eModel:\u003c/th\u003e                   \u003ctd\u003eOLS\u003c/td\u003e       \u003cth\u003e  Adj. R-squared:    \u003c/th\u003e \u003ctd\u003e   0.688\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eMethod:\u003c/th\u003e             \u003ctd\u003eLeast Squares\u003c/td\u003e  \u003cth\u003e  F-statistic:       \u003c/th\u003e \u003ctd\u003e   33.25\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eDate:\u003c/th\u003e             \u003ctd\u003eSat, 07 Aug 2021\u003c/td\u003e \u003cth\u003e  Prob (F-statistic):\u003c/th\u003e \u003ctd\u003e5.22e-17\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eTime:\u003c/th\u003e                 \u003ctd\u003e15:37:40\u003c/td\u003e     \u003cth\u003e  Log-Likelihood:    \u003c/th\u003e \u003ctd\u003e -188.63\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eNo. Observations:\u003c/th\u003e      \u003ctd\u003e    74\u003c/td\u003e      \u003cth\u003e  AIC:               \u003c/th\u003e \u003ctd\u003e   389.3\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eDf Residuals:\u003c/th\u003e          \u003ctd\u003e    68\u003c/td\u003e      \u003cth\u003e  BIC:               \u003c/th\u003e \u003ctd\u003e   403.1\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eDf Model:\u003c/th\u003e              \u003ctd\u003e     5\u003c/td\u003e      \u003cth\u003e                     \u003c/th\u003e     \u003ctd\u003e \u003c/td\u003e   \n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eCovariance Type:\u003c/th\u003e      \u003ctd\u003enonrobust\u003c/td\u003e    \u003cth\u003e                     \u003c/th\u003e     \u003ctd\u003e \u003c/td\u003e   \n\u003c/tr\u003e\n\u003c/table\u003e\n\u003ctable class=\"simpletable\"\u003e\n\u003ctr\u003e\n       \u003ctd\u003e\u003c/td\u003e         \u003cth\u003ecoef\u003c/th\u003e     \u003cth\u003estd err\u003c/th\u003e      \u003cth\u003et\u003c/th\u003e      \u003cth\u003eP\u003e|t|\u003c/th\u003e  \u003cth\u003e[0.025\u003c/th\u003e    \u003cth\u003e0.975]\u003c/th\u003e  \n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003econst\u003c/th\u003e      \u003ctd\u003e   40.3703\u003c/td\u003e \u003ctd\u003e    7.860\u003c/td\u003e \u003ctd\u003e    5.136\u003c/td\u003e \u003ctd\u003e 0.000\u003c/td\u003e \u003ctd\u003e   24.687\u003c/td\u003e \u003ctd\u003e   56.054\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003erep78\u003c/th\u003e      \u003ctd\u003e    0.9040\u003c/td\u003e \u003ctd\u003e    0.342\u003c/td\u003e \u003ctd\u003e    2.647\u003c/td\u003e \u003ctd\u003e 0.010\u003c/td\u003e \u003ctd\u003e    0.223\u003c/td\u003e \u003ctd\u003e    1.586\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eweight\u003c/th\u003e     \u003ctd\u003e   -0.0030\u003c/td\u003e \u003ctd\u003e    0.002\u003c/td\u003e \u003ctd\u003e   -1.770\u003c/td\u003e \u003ctd\u003e 0.081\u003c/td\u003e \u003ctd\u003e   -0.006\u003c/td\u003e \u003ctd\u003e    0.000\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003elength\u003c/th\u003e     \u003ctd\u003e   -0.1058\u003c/td\u003e \u003ctd\u003e    0.053\u003c/td\u003e \u003ctd\u003e   -1.990\u003c/td\u003e \u003ctd\u003e 0.051\u003c/td\u003e \u003ctd\u003e   -0.212\u003c/td\u003e \u003ctd\u003e    0.000\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003egear_ratio\u003c/th\u003e \u003ctd\u003e    2.6905\u003c/td\u003e \u003ctd\u003e    1.511\u003c/td\u003e \u003ctd\u003e    1.780\u003c/td\u003e \u003ctd\u003e 0.079\u003c/td\u003e \u003ctd\u003e   -0.325\u003c/td\u003e \u003ctd\u003e    5.706\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eforeign\u003c/th\u003e    \u003ctd\u003e   -4.0123\u003c/td\u003e \u003ctd\u003e    1.320\u003c/td\u003e \u003ctd\u003e   -3.040\u003c/td\u003e \u003ctd\u003e 0.003\u003c/td\u003e \u003ctd\u003e   -6.646\u003c/td\u003e \u003ctd\u003e   -1.379\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\u003ctable class=\"simpletable\"\u003e\n\u003ctr\u003e\n  \u003cth\u003eOmnibus:\u003c/th\u003e       \u003ctd\u003e24.257\u003c/td\u003e \u003cth\u003e  Durbin-Watson:     \u003c/th\u003e \u003ctd\u003e   2.442\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eProb(Omnibus):\u003c/th\u003e \u003ctd\u003e 0.000\u003c/td\u003e \u003cth\u003e  Jarque-Bera (JB):  \u003c/th\u003e \u003ctd\u003e  39.774\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eSkew:\u003c/th\u003e          \u003ctd\u003e 1.252\u003c/td\u003e \u003cth\u003e  Prob(JB):          \u003c/th\u003e \u003ctd\u003e2.31e-09\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eKurtosis:\u003c/th\u003e      \u003ctd\u003e 5.576\u003c/td\u003e \u003cth\u003e  Cond. No.          \u003c/th\u003e \u003ctd\u003e6.59e+04\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\u003cbr/\u003e\u003cbr/\u003eNotes:\u003cbr/\u003e[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\u003cbr/\u003e[2] The condition number is large, 6.59e+04. This might indicate that there are\u003cbr/\u003estrong multicollinearity or other numerical problems.\n\n\n\n### Use in Scikit-Learn Pipeline\n\nBoth `ForwardSelector` and `SubsetSelector` objects are compatible with Scikit-Learn `Pipeline` objects, and can be used as feature selection steps:\n\n\n```python\npl = Pipeline([\n    ('feature_selection', SubsetSelector(normalize=True)),\n    ('regression', LinearRegression())\n])\npl.fit(X, y)\n```\n\n\n\n\n    Pipeline(steps=[('feature_selection', SubsetSelector(normalize=True)),\n                    ('regression', LinearRegression())])\n\n\n\n\n```python\npl.score(X, y)\n```\n\n\n\n\n    0.7097132531085899\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchris-santiago%2Fsteps","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchris-santiago%2Fsteps","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchris-santiago%2Fsteps/lists"}