{"id":19644823,"url":"https://github.com/uscbiostats/partition","last_synced_at":"2025-04-28T13:31:49.016Z","repository":{"id":46579633,"uuid":"178615892","full_name":"USCbiostats/partition","owner":"USCbiostats","description":"A fast and flexible framework for data reduction in R","archived":false,"fork":false,"pushed_at":"2024-11-10T18:37:13.000Z","size":15794,"stargazers_count":36,"open_issues_count":2,"forks_count":4,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-22T08:03:37.648Z","etag":null,"topics":["data-reduction","dimensionality-reduction","partitional-clustering","r"],"latest_commit_sha":null,"homepage":"https://uscbiostats.github.io/partition/","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/USCbiostats.png","metadata":{"files":{"readme":"README.Rmd","changelog":"NEWS.md","contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":".github/CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-03-30T22:02:50.000Z","updated_at":"2025-02-01T18:56:11.000Z","dependencies_parsed_at":"2025-04-14T16:56:19.922Z","dependency_job_id":"ffdc94f7-036a-4ed1-a61f-8a4481481a13","html_url":"https://github.com/USCbiostats/partition","commit_stats":{"total_commits":171,"total_committers":3,"mean_commits":57.0,"dds":0.01754385964912286,"last_synced_commit":"a25cf12c9cb091f1f03ff1d7af062ec278faeb8e"},"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/USCbiostats%2Fpartition","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/USCbiostats%2Fpartition/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/USCbiostats%2Fpartition/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/USCbiostats%2Fpartition/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/USCbiostats","download_url":"https://codeload.github.com/USCbiostats/partition/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251319757,"owners_count":21570451,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-reduction","dimensionality-reduction","partitional-clustering","r"],"created_at":"2024-11-11T14:30:10.672Z","updated_at":"2025-04-28T13:31:43.985Z","avatar_url":"https://github.com/USCbiostats.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"---\noutput: github_document\nreferences:\n- id: R-partition\n  type: article-journal\n  author:\n  - family: Millstein\n    given: Joshua\n  - family: Battaglin\n    given: Francesca\n  - family: Barrett\n    given: Malcolm\n  - family: Cao\n    given: Shu\n  - family: Zhang\n    given: Wu\n  - family: Stintzing\n    given: Sebastian\n  - family: Heinemann\n    given: Volker\n  - family: Lenz\n    given: Heinz-Josef\n  issued:\n  - year: 2020\n  title: 'Partition: A surjective mapping approach for dimensionality reduction'\n  title-short: Partition\n  container-title: Bioinformatics\n  page: 676-681\n  volume: '36'\n  issue: '3'\n  URL: 'https://doi.org/10.1093/bioinformatics/btz661'\nparams:\n  invalidate_cache: false\n---\n\n\u003c!-- README.md is generated from README.Rmd. Please edit that file --\u003e\n\n```{r setup, include = FALSE}\nknitr::opts_chunk$set(\n  collapse = TRUE,\n  comment = \"#\u003e\",\n  fig.path = \"man/figures/README-\",\n  out.width = \"100%\",\n  dpi = 320\n)\n```\n\n\u003c!-- badges: start --\u003e\n[![R-CMD-check](https://github.com/USCbiostats/partition/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/USCbiostats/partition/actions/workflows/R-CMD-check.yaml)\n[![Coverage status](https://codecov.io/gh/USCbiostats/partition/branch/master/graph/badge.svg)](https://app.codecov.io/github/USCbiostats/partition?branch=master)\n[![CRAN status](https://www.r-pkg.org/badges/version-ago/partition)](https://cran.r-project.org/package=partition)\n[![JOSS](https://joss.theoj.org/papers/10.21105/joss.01991/status.svg)](https://doi.org/10.21105/joss.01991)\n[![DOI](https://zenodo.org/badge/178615892.svg)](https://zenodo.org/badge/latestdoi/178615892)\n[![USC IMAGE](https://raw.githubusercontent.com/USCbiostats/badges/master/tommy-image-badge.svg)](https://image.usc.edu)\n\u003c!-- badges: end --\u003e\n \n# partition\n\npartition is a fast and flexible framework for agglomerative partitioning. partition uses an approach  called Direct-Measure-Reduce to create new variables that maintain the user-specified minimum level of information. Each reduced variable is also interpretable: the original variables map to one and only one variable in the reduced data set. partition is flexible, as well: how variables are selected to reduce, how information loss is measured, and the way data is reduced can all be customized.  \n\n## Installation\n\nYou can install the partition from CRAN with:\n\n``` r\ninstall.packages(\"partition\")\n```\n\nOr you can install the development version of partition GitHub with:\n\n``` r\n# install.packages(\"remotes\")\nremotes::install_github(\"USCbiostats/partition\")\n```\n\n## Example\n\n```{r example}\nlibrary(partition)\nset.seed(1234)\ndf \u003c- simulate_block_data(c(3, 4, 5), lower_corr = .4, upper_corr = .6, n = 100)\n\n#  don't accept reductions where information \u003c .6\nprt \u003c- partition(df, threshold = .6)\nprt\n\n# return reduced data\npartition_scores(prt)\n\n# access mapping keys\nmapping_key(prt)\n\nunnest_mappings(prt)\n\n# use a lower threshold of information loss\npartition(df, threshold = .5, partitioner = part_kmeans())\n\n# use a custom partitioner\npart_icc_rowmeans \u003c- replace_partitioner(\n  part_icc, \n  reduce = as_reducer(rowMeans)\n)\npartition(df, threshold = .6, partitioner = part_icc_rowmeans) \n```\n\npartition also supports a number of ways to visualize partitions and permutation tests; these functions all start with `plot_*()`. These functions all return ggplots and can thus be extended using ggplot2.\n\n```{r stacked_area_chart, dpi = 320}\nplot_stacked_area_clusters(df) +\n  ggplot2::theme_minimal(14)\n```\n\n## Performance\n\npartition has been meticulously benchmarked and profiled to improve performance, and key sections are written in C++ or use C++-based packages. Using a data frame with 1 million rows on a 2017 MacBook Pro with 16 GB RAM, here's how each of the built-in partitioners perform: \n\n```{r benchmarks1, eval = FALSE}\nlarge_df \u003c- simulate_block_data(c(3, 4, 5), lower_corr = .4, upper_corr = .6, n = 1e6)\n\nbasic_benchmarks \u003c- microbenchmark::microbenchmark(\n  icc = partition(large_df, .3),\n  kmeans = partition(large_df, .3, partitioner = part_kmeans()),\n  minr2 = partition(large_df, .3, partitioner = part_minr2()),\n  pc1 = partition(large_df, .3, partitioner = part_pc1()),\n  stdmi = partition(large_df, .3, partitioner = part_stdmi())\n)\n```\n\n```{r secret_benchmarks1, echo = FALSE, warning=FALSE, message=FALSE}\nlibrary(microbenchmark)\nlibrary(ggplot2)\nif (params$invalidate_cache) {\n  large_df \u003c- simulate_block_data(c(3, 4, 5), lower_corr = .4, upper_corr = .6, n = 1e6)\n  \n  basic_benchmarks \u003c- microbenchmark::microbenchmark(\n    icc = partition(large_df, .3),\n    kmeans = partition(large_df, .3, partitioner = part_kmeans()),\n    minr2 = partition(large_df, .3, partitioner = part_minr2()),\n    pc1 = partition(large_df, .3, partitioner = part_pc1()),\n    stdmi = partition(large_df, .3, partitioner = part_stdmi())\n  )\n  \n  readr::write_rds(basic_benchmarks, \"basic_benchmarks.rds\")\n} else {\n  basic_benchmarks \u003c- readr::read_rds(\"basic_benchmarks.rds\")\n}\n\nbasic_benchmarks$expr \u003c- forcats::fct_reorder(basic_benchmarks$expr, basic_benchmarks$time)\nggplot2::autoplot(basic_benchmarks) %+% \n  ggplot2::stat_ydensity(color = \"#0072B2\", fill = \"#0072B2BF\") +\n  ggplot2::theme_minimal()\n```\n\n## ICC vs K-Means\n\nAs the features (columns) in the data set become greater than the number of observations (rows), the default ICC method scales more linearly than K-Means-based methods. While K-Means is often faster at lower dimensions, it becomes slower as the features outnumber the observations. For example, using three data sets with increasing numbers of columns, K-Means starts as the fastest and gets increasingly slower, although in this case it is still comparable to ICC:\n\n```{r benchmarks2, eval = FALSE}\nnarrow_df \u003c- simulate_block_data(3:5, lower_corr = .4, upper_corr = .6, n = 100)\nwide_df \u003c- simulate_block_data(rep(3:10, 2), lower_corr = .4, upper_corr = .6, n = 100)\nwider_df \u003c- simulate_block_data(rep(3:20, 4), lower_corr = .4, upper_corr = .6, n = 100)\n\nicc_kmeans_benchmarks \u003c- microbenchmark::microbenchmark(\n  icc_narrow = partition(narrow_df, .3),\n  icc_wide = partition(wide_df, .3),\n  icc_wider = partition(wider_df, .3),\n  kmeans_narrow = partition(narrow_df, .3, partitioner = part_kmeans()),\n  kmeans_wide = partition(wide_df, .3, partitioner = part_kmeans()),\n  kmeans_wider  = partition(wider_df, .3, partitioner = part_kmeans())\n)\n```\n\n```{r secret_benchmarks2, echo = FALSE, warning=FALSE, message=FALSE}\nif (params$invalidate_cache) {\n  narrow_df \u003c- simulate_block_data(3:5, lower_corr = .4, upper_corr = .6, n = 100)\n  wide_df \u003c- simulate_block_data(rep(3:10, 2), lower_corr = .4, upper_corr = .6, n = 100)\n  wider_df \u003c- simulate_block_data(rep(3:20, 4), lower_corr = .4, upper_corr = .6, n = 100)\n  \n  icc_kmeans_benchmarks \u003c- microbenchmark::microbenchmark(\n    icc_narrow = partition(narrow_df, .3),\n    icc_wide = partition(wide_df, .3),\n    icc_wider = partition(wider_df, .3),\n    kmeans_narrow = partition(narrow_df, .3, partitioner = part_kmeans()),\n    kmeans_wide = partition(wide_df, .3, partitioner = part_kmeans()),\n    kmeans_wider  = partition(wider_df, .3, partitioner = part_kmeans())\n  )\n  \n  readr::write_rds(icc_kmeans_benchmarks, \"icc_kmeans_benchmarks.rds\")\n} else {\n  icc_kmeans_benchmarks \u003c- readr::read_rds(\"icc_kmeans_benchmarks.rds\")\n}\n\nicc_kmeans_benchmarks$type \u003c- stringr::str_extract(icc_kmeans_benchmarks$expr, \"icc|kmeans\")\n\nggplot2::autoplot(icc_kmeans_benchmarks) %+% \n  ggplot2::stat_ydensity(color = \"#0072B2\", fill = \"#0072B2BF\") +\n  ggplot2::facet_wrap(~type, ncol = 1, scales = \"free_y\") + \n  ggplot2::theme_minimal()\n```\n\nFor more information, see [our paper in Bioinformatics](https://doi.org/10.1093/bioinformatics/btz661), which discusses these issues in more depth [@R-partition].\n\n## Contributing \n\nPlease read the [Contributor Guidelines](https://github.com/USCbiostats/partition/blob/master/.github/CONTRIBUTING.md) prior to submitting a pull request to partition. Also note that this project is released with a [Contributor Code of Conduct](https://github.com/USCbiostats/partition/blob/master/.github/CODE_OF_CONDUCT.md). By participating in this project you agree to abide by its terms.\n\n## References\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuscbiostats%2Fpartition","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fuscbiostats%2Fpartition","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuscbiostats%2Fpartition/lists"}