{"id":13454101,"url":"https://github.com/scikit-learn-contrib/sklearn-pandas","last_synced_at":"2025-05-14T01:07:49.938Z","repository":{"id":8184139,"uuid":"9610653","full_name":"scikit-learn-contrib/sklearn-pandas","owner":"scikit-learn-contrib","description":"Pandas integration with sklearn","archived":false,"fork":false,"pushed_at":"2023-06-08T06:48:29.000Z","size":430,"stargazers_count":2827,"open_issues_count":39,"forks_count":416,"subscribers_count":92,"default_branch":"master","last_synced_at":"2025-04-03T01:40:00.863Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/scikit-learn-contrib.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2013-04-22T22:55:20.000Z","updated_at":"2025-04-01T04:06:21.000Z","dependencies_parsed_at":"2022-07-12T15:03:14.746Z","dependency_job_id":"9688d86a-b7fa-466f-b3f6-55afd2ef2307","html_url":"https://github.com/scikit-learn-contrib/sklearn-pandas","commit_stats":{"total_commits":232,"total_committers":40,"mean_commits":5.8,"dds":0.8017241379310345,"last_synced_commit":"c9db2d6dcbf515eade751073f43318e43cae5177"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scikit-learn-contrib%2Fsklearn-pandas","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scikit-learn-contrib%2Fsklearn-pandas/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scikit-learn-contrib%2Fsklearn-pandas/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scikit-learn-contrib%2Fsklearn-pandas/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/scikit-learn-contrib","download_url":"https://codeload.github.com/scikit-learn-contrib/sklearn-pandas/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248154572,"owners_count":21056540,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T08:00:50.900Z","updated_at":"2025-04-10T03:39:55.738Z","avatar_url":"https://github.com/scikit-learn-contrib.png","language":"Python","funding_links":[],"categories":["Python","Data Manipulation","Machine Learning Framework","Libraries","数据容器和结构","Uncategorized","Feature Extraction","🐍 Python"],"sub_categories":["Pipelines","General Purpose Framework","Statistical Method","Uncategorized","General Feature Extraction","Useful Python Tools for Data Analysis"],"readme":"\nSklearn-pandas\n==============\n\n.. image:: https://circleci.com/gh/scikit-learn-contrib/sklearn-pandas.svg?style=svg\n    :target: https://circleci.com/gh/scikit-learn-contrib/sklearn-pandas\n.. image:: https://img.shields.io/pypi/v/sklearn-pandas.svg\n   :target: https://pypi.python.org/pypi/sklearn-pandas/\n.. image:: https://anaconda.org/conda-forge/sklearn-pandas/badges/version.svg\n   :target: https://anaconda.org/conda-forge/sklearn-pandas/\n\n.. highlight:: python\n\nThis module provides a bridge between `Scikit-Learn \u003chttp://scikit-learn.org/stable\u003e`__'s machine learning methods and `pandas \u003chttps://pandas.pydata.org\u003e`__-style Data Frames.\nIn particular, it provides a way to map ``DataFrame`` columns to transformations, which are later recombined into features.\n\nInstallation\n------------\n\nYou can install ``sklearn-pandas`` with ``pip``::\n\n    # pip install sklearn-pandas\n\nor conda-forge::\n\n    # conda install -c conda-forge sklearn-pandas\n\nTests\n-----\n\nThe examples in this file double as basic sanity tests. To run them, use ``doctest``, which is included with python::\n\n    # python -m doctest README.rst\n\n\nUsage\n-----\n\n\nImport\n******\n\nImport what you need from the ``sklearn_pandas`` package. The choices are:\n\n* ``DataFrameMapper``, a class for mapping pandas data frame columns to different sklearn transformations\n\n\nFor this demonstration, we will import both::\n\n    \u003e\u003e\u003e from sklearn_pandas import DataFrameMapper\n\nFor these examples, we'll also use pandas, numpy, and sklearn::\n\n    \u003e\u003e\u003e import pandas as pd\n    \u003e\u003e\u003e import numpy as np\n    \u003e\u003e\u003e import sklearn.preprocessing, sklearn.decomposition, \\\n    ...     sklearn.linear_model, sklearn.pipeline, sklearn.metrics, \\\n    ...     sklearn.compose\n    \u003e\u003e\u003e from sklearn.feature_extraction.text import CountVectorizer\n\n\nLoad some Data\n**************\n\n\nNormally you'll read the data from a file, but for demonstration purposes we'll create a data frame from a Python dict::\n\n    \u003e\u003e\u003e data = pd.DataFrame({'pet':      ['cat', 'dog', 'dog', 'fish', 'cat', 'dog', 'cat', 'fish'],\n    ...                      'children': [4., 6, 3, 3, 2, 3, 5, 4],\n    ...                      'salary':   [90., 24, 44, 27, 32, 59, 36, 27]})\n\n\nTransformation Mapping\n----------------------\n\n\nMap the Columns to Transformations\n**********************************\n\nThe mapper takes a list of tuples. Each tuple has three elements:\n  1. column name(s): The first element is a column name from the pandas DataFrame, or a list containing one or multiple columns (we will see an example with multiple columns later) or an instance of a callable function such as `make_column_selector \u003chttps://scikit-learn.org/stable/modules/generated/sklearn.compose.make_column_selector.html\u003e`__. \n  2. transformer(s): The second element is an object which will perform the transformation which will be applied to that column. \n  3. attributes: The third one is optional and is a dictionary containing the transformation options, if applicable (see \"custom column names for transformed features\" below).\n\nLet's see an example::\n\n    \u003e\u003e\u003e mapper = DataFrameMapper([\n    ...     ('pet', sklearn.preprocessing.LabelBinarizer()),\n    ...     (['children'], sklearn.preprocessing.StandardScaler())\n    ... ])\n\nThe difference between specifying the column selector as ``'column'`` (as a simple string) and ``['column']`` (as a list with one element) is the shape of the array that is passed to the transformer. In the first case, a one dimensional array will be passed, while in the second case it will be a 2-dimensional array with one column, i.e. a column vector. \n\nThis behaviour mimics the same pattern as pandas' dataframes ``__getitem__``  indexing::\n\n    \u003e\u003e\u003e data['children'].shape\n    (8,)\n    \u003e\u003e\u003e data[['children']].shape\n    (8, 1)\n\nBe aware that some transformers expect a 1-dimensional input (the label-oriented ones) while some others, like ``OneHotEncoder`` or ``Imputer``, expect 2-dimensional input, with the shape ``[n_samples, n_features]``.\n\n\nTest the Transformation\n***********************\n\nWe can use the ``fit_transform`` shortcut to both fit the model and see what transformed data looks like. In this and the other examples, output is rounded to two digits with ``np.round`` to account for rounding errors on different hardware::\n\n    \u003e\u003e\u003e np.round(mapper.fit_transform(data.copy()), 2)\n    array([[ 1.  ,  0.  ,  0.  ,  0.21],\n           [ 0.  ,  1.  ,  0.  ,  1.88],\n           [ 0.  ,  1.  ,  0.  , -0.63],\n           [ 0.  ,  0.  ,  1.  , -0.63],\n           [ 1.  ,  0.  ,  0.  , -1.46],\n           [ 0.  ,  1.  ,  0.  , -0.63],\n           [ 1.  ,  0.  ,  0.  ,  1.04],\n           [ 0.  ,  0.  ,  1.  ,  0.21]])\n\nNote that the first three columns are the output of the ``LabelBinarizer`` (corresponding to ``cat``, ``dog``, and ``fish`` respectively) and the fourth column is the standardized value for the number of children. In general, the columns are ordered according to the order given when the ``DataFrameMapper`` is constructed.\n\nNow that the transformation is trained, we confirm that it works on new data::\n\n    \u003e\u003e\u003e sample = pd.DataFrame({'pet': ['cat'], 'children': [5.]})\n    \u003e\u003e\u003e np.round(mapper.transform(sample), 2)\n    array([[1.  , 0.  , 0.  , 1.04]])\n\n\nOutput features names\n*********************\n\nIn certain cases, like when studying the feature importances for some model,\nwe want to be able to associate the original features to the ones generated by\nthe dataframe mapper. We can do so by inspecting the automatically generated ``transformed_names_`` attribute of the mapper after transformation::\n\n    \u003e\u003e\u003e mapper.transformed_names_\n    ['pet_cat', 'pet_dog', 'pet_fish', 'children']\n\n\nCustom column names for transformed features\n********************************************\n\nWe can provide a custom name for the transformed features, to be used instead\nof the automatically generated one, by specifying it as the third argument\nof the feature definition::\n\n\n  \u003e\u003e\u003e mapper_alias = DataFrameMapper([\n  ...     (['children'], sklearn.preprocessing.StandardScaler(),\n  ...      {'alias': 'children_scaled'})\n  ... ])\n  \u003e\u003e\u003e _ = mapper_alias.fit_transform(data.copy())\n  \u003e\u003e\u003e mapper_alias.transformed_names_\n  ['children_scaled']\n\nAlternatively, you can also specify prefix and/or suffix to add to the column name. For example::\n\n\n  \u003e\u003e\u003e mapper_alias = DataFrameMapper([\n  ...     (['children'], sklearn.preprocessing.StandardScaler(), {'prefix': 'standard_scaled_'}),\n  ...     (['children'], sklearn.preprocessing.StandardScaler(), {'suffix': '_raw'})\n  ... ])\n  \u003e\u003e\u003e _ = mapper_alias.fit_transform(data.copy())\n  \u003e\u003e\u003e mapper_alias.transformed_names_\n  ['standard_scaled_children', 'children_raw']\n\n\nDynamic Columns\n***********************\nIn some situations the columns are not known before hand and we would like to dynamically select them during the fit operation. As shown below, in such situations you can provide either a custom callable or use `make_column_selector \u003chttps://scikit-learn.org/stable/modules/generated/sklearn.compose.make_column_selector.html\u003e`__.\n\n::\n\n    \u003e\u003e\u003e class GetColumnsStartingWith:\n    ...   def __init__(self, start_str):\n    ...     self.pattern = start_str\n    ...\n    ...   def __call__(self, X:pd.DataFrame=None):\n    ...     return [c for c in X.columns if c.startswith(self.pattern)]\n    ...\n    \u003e\u003e\u003e df = pd.DataFrame({\n    ...    'sepal length (cm)': [1.0, 2.0, 3.0],\n    ...    'sepal width (cm)': [1.0, 2.0, 3.0],\n    ...    'petal length (cm)': [1.0, 2.0, 3.0],\n    ...    'petal width (cm)': [1.0, 2.0, 3.0]\n    ... })\n    \u003e\u003e\u003e t = DataFrameMapper([\n    ...     (\n    ...       sklearn.compose.make_column_selector(dtype_include=float),\n    ...       sklearn.preprocessing.StandardScaler(),\n    ...       {'alias': 'x'}\n    ...     ),\n    ...     (\n    ...       GetColumnsStartingWith('petal'),\n    ...       None,\n    ...       {'alias': 'petal'}\n    ...     )], df_out=True, default=False)\n    \u003e\u003e\u003e t.fit(df).transform(df).shape\n    (3, 6)\n    \u003e\u003e\u003e t.transformed_names_\n    ['x_0', 'x_1', 'x_2', 'x_3', 'petal_0', 'petal_1']\n\n\n\nAbove we use `make_column_selector` to select all columns that are of type float and also use a custom callable function to select columns that start with the word 'petal'.\n\n\nPassing Series/DataFrames to the transformers\n*********************************************\n\nBy default the transformers are passed a numpy array of the selected columns\nas input. This is because ``sklearn`` transformers are historically designed to\nwork with numpy arrays, not with pandas dataframes, even though their basic\nindexing interfaces are similar.\n\nHowever we can pass a dataframe/series to the transformers to handle custom\ncases initializing the dataframe mapper with ``input_df=True``::\n\n    \u003e\u003e\u003e from sklearn.base import TransformerMixin\n    \u003e\u003e\u003e class DateEncoder(TransformerMixin):\n    ...    def fit(self, X, y=None):\n    ...        return self\n    ...\n    ...    def transform(self, X):\n    ...        dt = X.dt\n    ...        return pd.concat([dt.year, dt.month, dt.day], axis=1)\n    \u003e\u003e\u003e dates_df = pd.DataFrame(\n    ...     {'dates': pd.date_range('2015-10-30', '2015-11-02')})\n    \u003e\u003e\u003e mapper_dates = DataFrameMapper([\n    ...     ('dates', DateEncoder())\n    ... ], input_df=True)\n    \u003e\u003e\u003e mapper_dates.fit_transform(dates_df)\n    array([[2015,   10,   30],\n           [2015,   10,   31],\n           [2015,   11,    1],\n           [2015,   11,    2]])\n\nWe can also specify this option per group of columns instead of for the\nwhole mapper::\n\n  \u003e\u003e\u003e mapper_dates = DataFrameMapper([\n  ...     ('dates', DateEncoder(), {'input_df': True})\n  ... ])\n  \u003e\u003e\u003e mapper_dates.fit_transform(dates_df)\n  array([[2015,   10,   30],\n         [2015,   10,   31],\n         [2015,   11,    1],\n         [2015,   11,    2]])\n\nOutputting a dataframe\n**********************\n\nBy default the output of the dataframe mapper is a numpy array. This is so because most sklearn estimators expect a numpy array as input. If however we want the output of the mapper to be a dataframe, we can do so using the parameter ``df_out`` when creating the mapper::\n\n    \u003e\u003e\u003e mapper_df = DataFrameMapper([\n    ...     ('pet', sklearn.preprocessing.LabelBinarizer()),\n    ...     (['children'], sklearn.preprocessing.StandardScaler())\n    ... ], df_out=True)\n    \u003e\u003e\u003e np.round(mapper_df.fit_transform(data.copy()), 2)\n       pet_cat  pet_dog  pet_fish  children\n    0        1        0         0      0.21\n    1        0        1         0      1.88\n    2        0        1         0     -0.63\n    3        0        0         1     -0.63\n    4        1        0         0     -1.46\n    5        0        1         0     -0.63\n    6        1        0         0      1.04\n    7        0        0         1      0.21\n\nThe names for the columns are the same ones present in the ``transformed_names_``\nattribute.\n\nNote this does not work together with the ``default=True`` or ``sparse=True`` arguments to the mapper.\n\nDropping columns explictly\n*******************************\n\nSometimes it is required to drop a specific column/ list of columns.\nFor this purpose, ``drop_cols``  argument for ``DataFrameMapper`` can be used.\nDefault value is ``None``::\n\n    \u003e\u003e\u003e mapper_df = DataFrameMapper([\n    ...     ('pet', sklearn.preprocessing.LabelBinarizer()),\n    ...     (['children'], sklearn.preprocessing.StandardScaler())\n    ... ], drop_cols=['salary'])\n\nNow running ``fit_transform`` will run transformations on 'pet' and 'children' and drop 'salary' column::\n\n   \u003e\u003e\u003e np.round(mapper_df.fit_transform(data.copy()), 1)\n   array([[ 1. ,  0. ,  0. ,  0.2],\n          [ 0. ,  1. ,  0. ,  1.9],\n          [ 0. ,  1. ,  0. , -0.6],\n          [ 0. ,  0. ,  1. , -0.6],\n          [ 1. ,  0. ,  0. , -1.5],\n          [ 0. ,  1. ,  0. , -0.6],\n          [ 1. ,  0. ,  0. ,  1. ],\n          [ 0. ,  0. ,  1. ,  0.2]])\n\nTransformations may require multiple input columns. In these\n\nTransform Multiple Columns\n**************************\n\nTransformations may require multiple input columns. In these cases, the column names can be specified in a list::\n\n    \u003e\u003e\u003e mapper2 = DataFrameMapper([\n    ...     (['children', 'salary'], sklearn.decomposition.PCA(1))\n    ... ])\n\nNow running ``fit_transform`` will run PCA on the ``children`` and ``salary`` columns and return the first principal component::\n\n    \u003e\u003e\u003e np.round(mapper2.fit_transform(data.copy()), 1)\n    array([[ 47.6],\n           [-18.4],\n           [  1.6],\n           [-15.4],\n           [-10.4],\n           [ 16.6],\n           [ -6.4],\n           [-15.4]])\n\nMultiple transformers for the same column\n*****************************************\n\nMultiple transformers can be applied to the same column specifying them\nin a list::\n\n    \u003e\u003e\u003e from sklearn.impute import SimpleImputer\n    \u003e\u003e\u003e mapper3 = DataFrameMapper([\n    ...     (['age'], [SimpleImputer(),\n    ...                sklearn.preprocessing.StandardScaler()])])\n    \u003e\u003e\u003e data_3 = pd.DataFrame({'age': [1, np.nan, 3]})\n    \u003e\u003e\u003e mapper3.fit_transform(data_3)\n    array([[-1.22474487],\n           [ 0.        ],\n           [ 1.22474487]])\n\n\nColumns that don't need any transformation\n******************************************\n\nOnly columns that are listed in the DataFrameMapper are kept. To keep a column but don't apply any transformation to it, use `None` as transformer::\n\n    \u003e\u003e\u003e mapper3 = DataFrameMapper([\n    ...     ('pet', sklearn.preprocessing.LabelBinarizer()),\n    ...     ('children', None)\n    ... ])\n    \u003e\u003e\u003e np.round(mapper3.fit_transform(data.copy()))\n    array([[1., 0., 0., 4.],\n           [0., 1., 0., 6.],\n           [0., 1., 0., 3.],\n           [0., 0., 1., 3.],\n           [1., 0., 0., 2.],\n           [0., 1., 0., 3.],\n           [1., 0., 0., 5.],\n           [0., 0., 1., 4.]])\n\nApplying a default transformer\n******************************\n\nA default transformer can be applied to columns not explicitly selected\npassing it as the ``default`` argument to the mapper::\n\n    \u003e\u003e\u003e mapper4 = DataFrameMapper([\n    ...     ('pet', sklearn.preprocessing.LabelBinarizer()),\n    ...     ('children', None)\n    ... ], default=sklearn.preprocessing.StandardScaler())\n    \u003e\u003e\u003e np.round(mapper4.fit_transform(data.copy()), 1)\n    array([[ 1. ,  0. ,  0. ,  4. ,  2.3],\n           [ 0. ,  1. ,  0. ,  6. , -0.9],\n           [ 0. ,  1. ,  0. ,  3. ,  0.1],\n           [ 0. ,  0. ,  1. ,  3. , -0.7],\n           [ 1. ,  0. ,  0. ,  2. , -0.5],\n           [ 0. ,  1. ,  0. ,  3. ,  0.8],\n           [ 1. ,  0. ,  0. ,  5. , -0.3],\n           [ 0. ,  0. ,  1. ,  4. , -0.7]])\n\nUsing ``default=False`` (the default) drops unselected columns. Using\n``default=None`` pass the unselected columns unchanged.\n\n\nSame transformer for the multiple columns\n*****************************************\n\nSometimes it is required to apply the same transformation to several dataframe columns.\nTo simplify this process, the package provides ``gen_features`` function which accepts a list\nof columns and feature transformer class (or list of classes), and generates a feature definition,\nacceptable by ``DataFrameMapper``.\n\nFor example, consider a dataset with three categorical columns, 'col1', 'col2', and 'col3',\nTo binarize each of them, one could pass column names and ``LabelBinarizer`` transformer class\ninto generator, and then use returned definition as ``features`` argument for ``DataFrameMapper``::\n\n    \u003e\u003e\u003e from sklearn_pandas import gen_features\n    \u003e\u003e\u003e feature_def = gen_features(\n    ...     columns=['col1', 'col2', 'col3'],\n    ...     classes=[sklearn.preprocessing.LabelEncoder]\n    ... )\n    \u003e\u003e\u003e feature_def\n    [('col1', [LabelEncoder()], {}), ('col2', [LabelEncoder()], {}), ('col3', [LabelEncoder()], {})]\n    \u003e\u003e\u003e mapper5 = DataFrameMapper(feature_def)\n    \u003e\u003e\u003e data5 = pd.DataFrame({\n    ...     'col1': ['yes', 'no', 'yes'],\n    ...     'col2': [True, False, False],\n    ...     'col3': ['one', 'two', 'three']\n    ... })\n    \u003e\u003e\u003e mapper5.fit_transform(data5)\n    array([[1, 1, 0],\n           [0, 0, 2],\n           [1, 0, 1]])\n\nIf it is required to override some of transformer parameters, then a dict with 'class' key and\ntransformer parameters should be provided. For example, consider a dataset with missing values.\nThen the following code could be used to override default imputing strategy::\n\n    \u003e\u003e\u003e from sklearn.impute import SimpleImputer\n    \u003e\u003e\u003e import numpy as np\n    \u003e\u003e\u003e feature_def = gen_features(\n    ...     columns=[['col1'], ['col2'], ['col3']],\n    ...     classes=[{'class': SimpleImputer, 'strategy':'most_frequent'}]\n    ... )\n    \u003e\u003e\u003e mapper6 = DataFrameMapper(feature_def)\n    \u003e\u003e\u003e data6 = pd.DataFrame({\n    ...     'col1': [np.nan, 1, 1, 2, 3],\n    ...     'col2': [True, False, np.nan, np.nan, True],\n    ...     'col3': [0, 0, 0, np.nan, np.nan]\n    ... })\n    \u003e\u003e\u003e mapper6.fit_transform(data6)\n    array([[1.0, True, 0.0],\n           [1.0, False, 0.0],\n           [1.0, True, 0.0],\n           [2.0, True, 0.0],\n           [3.0, True, 0.0]], dtype=object)\n\nYou can also specify global prefix or suffix for the generated transformed column names using the prefix and suffix\nparameters::\n\n    \u003e\u003e\u003e feature_def = gen_features(\n    ...     columns=['col1', 'col2', 'col3'],\n    ...     classes=[sklearn.preprocessing.LabelEncoder],\n    ...     prefix=\"lblencoder_\"\n    ... )\n    \u003e\u003e\u003e mapper5 = DataFrameMapper(feature_def)\n    \u003e\u003e\u003e data5 = pd.DataFrame({\n    ...     'col1': ['yes', 'no', 'yes'],\n    ...     'col2': [True, False, False],\n    ...     'col3': ['one', 'two', 'three']\n    ... })\n    \u003e\u003e\u003e _ = mapper5.fit_transform(data5)\n    \u003e\u003e\u003e mapper5.transformed_names_\n    ['lblencoder_col1', 'lblencoder_col2', 'lblencoder_col3']\n\nFeature selection and other supervised transformations\n******************************************************\n\n``DataFrameMapper`` supports transformers that require both X and y arguments. An example of this is feature selection. Treating the 'pet' column as the target, we will select the column that best predicts it.\n\n::\n\n    \u003e\u003e\u003e from sklearn.feature_selection import SelectKBest, chi2\n    \u003e\u003e\u003e mapper_fs = DataFrameMapper([(['children','salary'], SelectKBest(chi2, k=1))])\n    \u003e\u003e\u003e mapper_fs.fit_transform(data[['children','salary']], data['pet'])\n    array([[90.],\n           [24.],\n           [44.],\n           [27.],\n           [32.],\n           [59.],\n           [36.],\n           [27.]])\n\nWorking with sparse features\n****************************\n\nA ``DataFrameMapper`` will return a dense feature array by default. Setting ``sparse=True`` in the mapper will return\na sparse array whenever any of the extracted features is sparse. Example::\n\n    \u003e\u003e\u003e mapper5 = DataFrameMapper([\n    ...     ('pet', CountVectorizer()),\n    ... ], sparse=True)\n    \u003e\u003e\u003e type(mapper5.fit_transform(data))\n    \u003cclass 'scipy.sparse.csr.csr_matrix'\u003e\n\nThe stacking of the sparse features is done without ever densifying them.\n\n\nUsing ``NumericalTransformer``\n***********************************\n\nWhile you can use ``FunctionTransformation`` to generate arbitrary transformers, it can present serialization issues\nwhen pickling. Use ``NumericalTransformer`` instead, which takes the function name as a string parameter and hence\ncan be easily serialized.\n\n::\n\n    \u003e\u003e\u003e from sklearn_pandas import NumericalTransformer\n    \u003e\u003e\u003e mapper5 = DataFrameMapper([\n    ...     ('children', NumericalTransformer('log')),\n    ... ])\n    \u003e\u003e\u003e mapper5.fit_transform(data)\n    array([[1.38629436],\n           [1.79175947],\n           [1.09861229],\n           [1.09861229],\n           [0.69314718],\n           [1.09861229],\n           [1.60943791],\n           [1.38629436]])\n\nChanging Logging level\n***********************************\n\nYou can change log level to info to print time take to fit/transform features. Setting it to higher level will stop printing elapsed time.\nBelow example shows how to change logging level.\n\n::\n\n    \u003e\u003e\u003e import logging\n    \u003e\u003e\u003e logging.getLogger('sklearn_pandas').setLevel(logging.INFO)\n    \n\nChangelog\n---------\n\n\n2.2.0 (2021-05-07)\n******************\n* Added an ability to provide callable functions instead of static column list.\n\n\n2.1.0 (2021-02-26)\n******************\n* Removed test for Python 3.6 and added Python 3.9\n* Added deprecation warning for NumericalTransformer\n* Fixed pickling issue causing integration issues with Baikal.  \n* Started publishing package to conda repo\n\n\n2.0.4 (2020-11-06)\n******************\n\n* Explicitly handling serialization (#224)\n* document fixes\n* Making transform function thread safe (#194)\n* Switched to nox for unit testing (#226)\n\n\n2.0.3 (2020-11-06)\n******************\n\n* Added elapsed time information for each feature.\n\n\n2.0.2 (2020-10-01)\n******************\n\n* Fix `DataFrameMapper` drop_cols attribute naming consistency with scikit-learn and initialization.\n\n\n2.0.1 (2020-09-07)\n******************\n\n* Added an option to explicitly drop columns.\n\n\n2.0.0 (2020-08-01)\n******************\n\n* Deprecated support for Python \u003c 3.6.\n* Deprecated support for old versions of scikit-learn, pandas and numpy. Please check setup.py for minimum requirement.\n* Removed CategoricalImputer, cross_val_score and GridSearchCV. All these functionality now exists as part of\n  scikit-learn. Please use SimpleImputer instead of CategoricalImputer. Also\n  Cross validation from sklearn now supports dataframe so we don't need to use cross validation wrapper provided over\n  here.\n* Added ``NumericalTransformer`` for common numerical transformations. Currently it implements log and log1p\n  transformation.\n* Added prefix and suffix options. See examples above. These are usually helpful when using gen_features.\n* Added ``drop_cols`` argument to DataframeMapper. This can be used to explicitly drop columns\n\n\n1.8.0 (2018-12-01)\n******************\n\n* Add ``FunctionTransformer`` class (#117).\n* Fix column names derivation for dataframes with multi-index or non-string\n  columns (#166).\n* Change behaviour of DataFrameMapper's fit_transform method to invoke each underlying transformers'\n  native fit_transform if implemented (#150).\n\n\n1.7.0 (2018-08-15)\n******************\n\n* Fix issues with unicode names in ``get_names`` (#160).\n* Update to build using ``numpy==1.14`` and ``python==3.6`` (#154).\n* Add ``strategy`` and ``fill_value`` parameters to ``CategoricalImputer`` to allow imputing\n  with values other than the mode (#144),(#161).\n* Preserve input data types when no transform is supplied (#138).\n\n\n1.6.0 (2017-10-28)\n******************\n\n* Add column name to exception during fit/transform (#110).\n* Add ``gen_feature`` helper function to help generating the same transformation for multiple columns (#126).\n\n\n1.5.0 (2017-06-24)\n******************\n\n* Allow inputting a dataframe/series per group of columns.\n* Get feature names also from ``estimator.get_feature_names()`` if present.\n* Attempt to derive feature names from individual transformers when applying a\n  list of transformers.\n* Do not mutate features in ``__init__`` to be compatible with\n  ``sklearn\u003e=0.20`` (#76).\n\n\n1.4.0 (2017-05-13)\n******************\n\n* Allow specifying a custom name (alias) for transformed columns (#83).\n* Capture output columns generated names in ``transformed_names_`` attribute (#78).\n* Add ``CategoricalImputer`` that replaces null-like values with the mode\n  for string-like columns.\n* Add ``input_df`` init argument to allow inputting a dataframe/series to the\n  transformers instead of a numpy array (#60).\n\n\n1.3.0 (2017-01-21)\n******************\n\n* Make the mapper return dataframes when ``df_out=True`` (#70, #74).\n* Update imports to avoid deprecation warnings in sklearn 0.18 (#68).\n\n\n1.2.0 (2016-10-02)\n******************\n\n* Deprecate custom cross-validation shim classes.\n* Require ``scikit-learn\u003e=0.15.0``. Resolves #49.\n* Allow applying a default transformer to columns not selected explicitly in\n  the mapper. Resolves #55.\n* Allow specifying an optional ``y`` argument during transform for\n  supervised transformations. Resolves #58.\n\n\n1.1.0 (2015-12-06)\n*******************\n\n* Delete obsolete ``PassThroughTransformer``. If no transformation is desired for a given column, use ``None`` as transformer.\n* Factor out code in several modules, to avoid having everything in ``__init__.py``.\n* Use custom ``TransformerPipeline`` class to allow transformation steps accepting only a X argument. Fixes #46.\n* Add compatibility shim for unpickling mappers with list of transformers created before 1.0.0. Fixes #45.\n\n\n1.0.0 (2015-11-28)\n*******************\n\n* Change version numbering scheme to SemVer.\n* Use ``sklearn.pipeline.Pipeline`` instead of copying its code. Resolves #43.\n* Raise ``KeyError`` when selecting unexistent columns in the dataframe. Fixes #30.\n* Return sparse feature array if any of the features is sparse and ``sparse`` argument is ``True``. Defaults to ``False`` to avoid potential breaking of existing code. Resolves #34.\n* Return model and prediction in custom CV classes. Fixes #27.\n\n\n0.0.12 (2015-11-07)\n********************\n\n* Allow specifying a list of transformers to use sequentially on the same column.\n\n\nCredits\n-------\n\nThe code for ``DataFrameMapper`` is based on code originally written by `Ben Hamner \u003chttps://github.com/benhamner\u003e`__.\n\nOther contributors:\n\n* Ariel Rossanigo (@arielrossanigo)\n* Arnau Gil Amat (@arnau126)\n* Assaf Ben-David (@AssafBenDavid)\n* Brendan Herger (@bjherger)\n* Cal Paterson (@calpaterson)\n* @defvorfu\n* Floris Hoogenboom (@FlorisHoogenboom)\n* Gustavo Sena Mafra (@gsmafra)\n* Israel Saeta Pérez (@dukebody)\n* Jeremy Howard (@jph00)\n* Jimmy Wan (@jimmywan)\n* Kristof Van Engeland (@kristofve91)\n* Olivier Grisel (@ogrisel)\n* Paul Butler (@paulgb)\n* Richard Miller (@rwjmiller)\n* Ritesh Agrawal (@ragrawal)\n* @SandroCasagrande\n* Timothy Sweetser (@hacktuarial)\n* Vitaley Zaretskey (@vzaretsk)\n* Zac Stewart (@zacstewart)\n* Parul Singh (@paro1234)\n* Vincent Heusinkveld (@VHeusinkveld)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscikit-learn-contrib%2Fsklearn-pandas","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscikit-learn-contrib%2Fsklearn-pandas","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscikit-learn-contrib%2Fsklearn-pandas/lists"}