{"id":25932438,"url":"https://github.com/dashdeckers/polars_list_utils","last_synced_at":"2026-05-28T13:31:46.929Z","repository":{"id":279822193,"uuid":"938415405","full_name":"dashdeckers/polars_list_utils","owner":"dashdeckers","description":"Polars Plugin for List-type Columns","archived":false,"fork":false,"pushed_at":"2025-09-25T10:59:42.000Z","size":907,"stargazers_count":10,"open_issues_count":3,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-25T12:29:55.747Z","etag":null,"topics":["fft","polars","polars-dataframe","polars-plugins","py-polars","py03","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dashdeckers.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-02-24T23:20:08.000Z","updated_at":"2025-09-25T10:59:45.000Z","dependencies_parsed_at":"2025-02-27T22:57:37.557Z","dependency_job_id":"fbb25b88-891e-4264-a0f0-a4c3aebd8a61","html_url":"https://github.com/dashdeckers/polars_list_utils","commit_stats":null,"previous_names":["dashdeckers/polars_list_utils"],"tags_count":19,"template":false,"template_full_name":null,"purl":"pkg:github/dashdeckers/polars_list_utils","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dashdeckers%2Fpolars_list_utils","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dashdeckers%2Fpolars_list_utils/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dashdeckers%2Fpolars_list_utils/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dashdeckers%2Fpolars_list_utils/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dashdeckers","download_url":"https://codeload.github.com/dashdeckers/polars_list_utils/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dashdeckers%2Fpolars_list_utils/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33611248,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-28T02:00:06.440Z","response_time":99,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fft","polars","polars-dataframe","polars-plugins","py-polars","py03","rust"],"created_at":"2025-03-04T00:37:37.971Z","updated_at":"2026-05-28T13:31:46.917Z","avatar_url":"https://github.com/dashdeckers.png","language":"Rust","funding_links":[],"categories":["Libraries/Packages/Scripts"],"sub_categories":["Polars plugins"],"readme":"# Polars List Utils (`polist`)\n\n[`polist`](https://github.com/dashdeckers/polars_list_utils) is a Python package that provides a set of utilities for working with List-type columns in Polars DataFrames, especially for signal processing and feature extraction.\n\nSo far these utilities comprise those that I found to be missing or lacking from\nthe List namespace within the Polars library while I was working on a project at\nwork that required extensive handling of signal data which I was storing in Polars\nDataFrames.\n\nBy providing these utilities as a Polars plugin, and thus not having to leave\nthe Polars DataFrame for these operations, I was able to significantly speed up\nthe processing of my data by benefiting from Polars query optimization and\nparallelization. So while the operations themselves are not necessarily faster\nthan their Numpy counterparts (although they might be in some cases), the\nintegration with Polars gave my larger processing pipeline a significant speed\nboost.\n\nStatus: Work-in-Progress!\n\n## Features\n\n- `polist.apply_fft`\n    - Applies a Fast Fourier Transform (FFT) to a List-type column of signal data.\n    - Can pre-process the signals with a windowing function (e.g. Hann, Hamming, Blackman).\n    - Can pre-process the signals with a Butterworth filter (low-pass, high-pass, band-pass).\n    - Can normalize the FFT amplitudes by the signal length or the window sum.\n\n- `polist.operate_scalar_on_list`\n    - Applies a scalar operation of a Float-type column to each element of a List-type column.\n    - This is currently not supported in Polars, see [this issue][list_eval_named].\n    - Can apply the operations `add`, `sub`, `mul`, and `div`.\n\n- `polist.interpolate_columns`\n    - Interpolates a new List-type column from 3 specified List-type columns.\n    - Behaviour as expected from the `numpy.interp` function, but for Polars DataFrames.\n    - Supply the `x_data`, `y_data`, and `x_interp` columns to obtain the `y_interp` column.\n\n- `polist.aggregate_list_col_elementwise`\n    - Applies element-wise list-aggregations to a List-type column in a GroupBy context.\n    - Currently supports `sum`, `mean`, and `count`.\n    - This is possible using the Polars API e.g. using `list.get(n)` (see my SO question [here][elementwise_agg]), but it does not scale well as for large lists and complicated queries it can lead to a stack overflow (see [this issue][stack_overflow] and many others).\n\n- `polist.mean_of_range`\n    - Computes the mean of a range of y-values defined by some x-values for List-type columns.\n    - This is useful for feature extraction from signals, e.g. to compute the mean of a signal in a certain time range or a spectrum in a certain frequency range.\n    - This is somewhat possible using the Polars API (e.g. using `list.slice` and `list.mean`), but can get very complicated for the simple case of wanting to specify certain y-values based on a custom x-axis.\n\n- `polist.fft_freqs`\n    - Computes the frequencies of the FFT bins for a given sampling rate and number of samples.\n    - This function does not operate dynamically on List-type columns, use it with `pl.lit`.\n    - This is useful for plotting the FFT spectrum in the frequency domain. Similar to the `numpy.fft.fftfreq` function, but in my opinion much simpler.\n\n- `polist.fft_freqs_linspace`\n    - Basically a thin wrapper around something like `numpy.linspace` to create a linearly spaced List of values.\n    - This function does not operate dynamically on List-type columns, use it with `pl.lit`.\n    - This is useful when used together with `polist.interpolate_columns`.\n\n[list_eval_named]: https://github.com/pola-rs/polars/issues/7210\n[elementwise_agg]: https://stackoverflow.com/questions/73776179/element-wise-aggregation-of-a-column-of-type-listf64-in-polars\n[stack_overflow]: https://github.com/pola-rs/polars/issues/5455\n\n\n### Example: (signal) -- (hann window) -- (FFT) -- (Freq. Normalization)\n\n![DSP Example](examples/showcase_dsp.png)\n\n\n## Installation (user)\n\n```bash\nuv pip install polars-list-utils\n```\n\n## Installation (developer)\n\n1) Setup Rust (i.e. install rustup)\n2) Setup Python (i.e. install uv)\n3) Setup environment and compile plugin:\n\n```bash\nuv sync --extra dev\nuv run maturin develop --release\n```\n\n4) (Maybe) configure Cargo to find uv's Python installs. For example:\n\n```\n# .cargo/config.toml\n[env]\nPYO3_PYTHON = \"C:\\\\Users\\\\travis.hammond\\\\AppData\\\\Roaming\\\\uv\\\\python\\\\cpython-3.12.0-windows-x86_64-none\\\\python.exe\"\n```\n\n5) Run:\n\n```bash\nuv run ./examples/showcase_dsp.py\n```\n\n6) Lint\n\n```bash\nuvx ruff check\ncargo fmt\n```\n\n## Todo\n\n- Add more features\n- Add more tests","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdashdeckers%2Fpolars_list_utils","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdashdeckers%2Fpolars_list_utils","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdashdeckers%2Fpolars_list_utils/lists"}