{"id":13465653,"url":"https://github.com/jmcarpenter2/swifter","last_synced_at":"2025-05-14T01:03:02.120Z","repository":{"id":37335664,"uuid":"128570698","full_name":"jmcarpenter2/swifter","owner":"jmcarpenter2","description":"A package which efficiently applies any function to a pandas dataframe or series in the fastest available manner","archived":false,"fork":false,"pushed_at":"2024-03-20T17:02:58.000Z","size":2256,"stargazers_count":2528,"open_issues_count":22,"forks_count":102,"subscribers_count":30,"default_branch":"master","last_synced_at":"2024-10-29T15:05:10.593Z","etag":null,"topics":["dask","modin","pandas","pandas-dataframe","parallel-computing","parallelization"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jmcarpenter2.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-04-07T21:37:19.000Z","updated_at":"2024-10-28T13:06:48.000Z","dependencies_parsed_at":"2022-07-12T12:04:54.833Z","dependency_job_id":"f85b7a3b-c8da-4c26-a093-d730715a31e0","html_url":"https://github.com/jmcarpenter2/swifter","commit_stats":{"total_commits":401,"total_committers":16,"mean_commits":25.0625,"dds":0.3092269326683291,"last_synced_commit":"83c63f844d02e13fa175eb3ecf864ce017c1c9e4"},"previous_names":[],"tags_count":83,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmcarpenter2%2Fswifter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmcarpenter2%2Fswifter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmcarpenter2%2Fswifter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmcarpenter2%2Fswifter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jmcarpenter2","download_url":"https://codeload.github.com/jmcarpenter2/swifter/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248154400,"owners_count":21056538,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dask","modin","pandas","pandas-dataframe","parallel-computing","parallelization"],"created_at":"2024-07-31T15:00:33.340Z","updated_at":"2025-04-10T03:39:31.369Z","avatar_url":"https://github.com/jmcarpenter2.png","language":"Python","readme":"# swifter\nA package which efficiently applies any function to a pandas dataframe or series in the fastest available manner.\n\n[![PyPI version](https://badge.fury.io/py/swifter.svg)](https://badge.fury.io/py/swifter)\n[![CircleCI](https://circleci.com/gh/jmcarpenter2/swifter.svg?style=shield)](https://circleci.com/gh/jmcarpenter2/swifter)\n[![codecov](https://img.shields.io/codecov/c/github/jmcarpenter2/swifter?label=codecov\u0026logo=codecov\u0026style=flat)](https://codecov.io/gh/jmcarpenter2/swifter)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/ambv/black)\n![GitHub stars](https://img.shields.io/github/stars/jmcarpenter2/swifter.svg?style=popout)\n![PyPI - Downloads](https://img.shields.io/pypi/dm/swifter.svg)\n\n## Blog posts\n* [Release 1.0.0](https://medium.com/@jmcarpenter2/swifter-1-0-0-automatically-efficient-pandas-and-modin-dataframe-applies-cfbd9555e7c8)\n* [First release](https://medium.com/@jmcarpenter2/swiftapply-automatically-efficient-pandas-apply-operations-50e1058909f9)\n\n## Documentation\nTo know about latest improvements, please check the [changelog](docs/changelog.md).\n\nFurther documentations on swifter is available [here](docs/documentation.md).\n\nCheck out the [examples notebook](examples/swifter_apply_examples.ipynb), along with the [speed benchmark notebook](examples/swifter_speed_comparison.ipynb). The benchmarks are created using the library [perfplot](https://github.com/unutbu/perfplot).\n\n## Installation:\n```\n$ pip install -U pandas # upgrade pandas\n$ pip install swifter # first time installation\n$ pip install swifter[notebook] # first time installation including dependency for rich progress bar in jupyter notebooks\n$ pip install swifter[groupby] # first time installation including dependency for groupby.apply functionality\n\n$ pip install -U swifter # upgrade to latest version if already installed\n$ pip install -U swifter[notebook] # upgrade to latest version to include dependency for rich progress bar in jupyter notebooks\n$ pip install -U swifter[groupby] # upgrade to latest version to include dependency for groupby.apply functionality\n```\n\nalternatively, to install on [Anaconda](https://anaconda.org/conda-forge/swifter):\n```\nconda install -c conda-forge swifter  # Install swifter\nconda install -c conda-forge swifter\u003e=1.3.2 ray\u003e=1.0.0  # Install swifter with dependency for groupby.apply\n```\n\n...after installing, import `swifter` into your code along with `pandas` using:\n```python\nimport pandas as pd\nimport swifter\n```\n\n...alternatively, `swifter` can be used with `modin` dataframes in the same manner:\n```python\nimport modin.pandas as pd\nimport swifter\n```\nNOTE: if you import swifter before modin, you will have to additionally register modin: `swifter.register_modin()`\n\n\n## Easy to use\n```python\ndf = pd.DataFrame({'x': [1, 2, 3, 4], 'y': [5, 6, 7, 8]})\n\n# runs on single core\ndf['x2'] = df['x'].apply(lambda x: x**2)\n# runs on multiple cores\ndf['x2'] = df['x'].swifter.apply(lambda x: x**2)\n\n# use swifter apply on whole dataframe\ndf['agg'] = df.swifter.apply(lambda x: x.sum() - x.min())\n\n# use swifter apply on specific columns\ndf['outCol'] = df[['inCol1', 'inCol2']].swifter.apply(my_func)\ndf['outCol'] = df[['inCol1', 'inCol2', 'inCol3']].swifter.apply(my_func,\n             positional_arg, keyword_arg=keyword_argval)\n```\n\n## Vectorizes your function, when possible\n![Alt text](/assets/vectorizes_when_possible_compatible.png?raw=true)\n\n## When vectorization is not possible, automatically decides which is faster: to use dask parallel processing or a simple pandas apply\n![Alt text](/assets/multiprocessing_vs_single_compatible.png?raw=true)\n\n## Highly performant, even for groupby applies\n![Alt text](/assets/groupby_parallel_vs_single_compatible.png?raw=true)\n\nSee the [speed benchmark notebook](examples/swifter_speed_comparison.ipynb) for source of the above performance plots.\n\n## Notes\n1. The function is documented in the .py file. In Jupyter Notebooks, you can see the docs by pressing Shift+Tab(x3). Also, check out the complete documentation [here](docs/documentation.md) along with the [changelog](docs/changelog.md).\n\n2. Please upgrade your version of pandas, as the pandas extension api used in this module is a recent addition to pandas.\n\n3. Import modin before importing swifter, if you wish to use modin with swifter. Otherwise, use `swifter.register_modin()` to access it.\n\n4. Do not use swifter to apply a function that modifies external variables. Under the hood, swifter does sample applies to optimize performance. These sample applies will modify the external variable in addition to the final apply. Thus, you will end up with an erroneously modified external variable.\n\n5. It is advised to disable the progress bar if calling swifter from a forked process as the progress bar may get confused between various multiprocessing modules.\n\n6. If swifter return is different than pandas try explicitly casting type e.g.: `df.swifter.apply(lambda x: float(np.angle(x)))`\n","funding_links":[],"categories":["Python","Data Manipulation","Data Processing","Data Analytics","数据容器和结构","Data Containers \u0026 Dataframes"],"sub_categories":["Data Frames","Data Representation"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjmcarpenter2%2Fswifter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjmcarpenter2%2Fswifter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjmcarpenter2%2Fswifter/lists"}