{"id":33187129,"url":"https://github.com/paulvanderlaken/ppsr","last_synced_at":"2026-03-11T17:35:24.071Z","repository":{"id":51324618,"uuid":"324570501","full_name":"paulvanderlaken/ppsr","owner":"paulvanderlaken","description":"R implementation of Predictive Power Score","archived":false,"fork":false,"pushed_at":"2024-02-18T12:45:17.000Z","size":1561,"stargazers_count":75,"open_issues_count":9,"forks_count":9,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-06-02T02:49:19.148Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/paulvanderlaken.png","metadata":{"files":{"readme":"README.Rmd","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-12-26T14:21:31.000Z","updated_at":"2025-03-25T02:17:11.000Z","dependencies_parsed_at":"2022-09-24T19:50:44.621Z","dependency_job_id":null,"html_url":"https://github.com/paulvanderlaken/ppsr","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/paulvanderlaken/ppsr","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paulvanderlaken%2Fppsr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paulvanderlaken%2Fppsr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paulvanderlaken%2Fppsr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paulvanderlaken%2Fppsr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/paulvanderlaken","download_url":"https://codeload.github.com/paulvanderlaken/ppsr/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paulvanderlaken%2Fppsr/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286079811,"owners_count":27282121,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-11-25T02:00:05.816Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-11-16T05:00:30.374Z","updated_at":"2025-11-25T18:00:41.262Z","avatar_url":"https://github.com/paulvanderlaken.png","language":"R","funding_links":[],"categories":["Data and models"],"sub_categories":[],"readme":"---\noutput: github_document\n---\n\n\u003c!-- README.md is generated from README.Rmd. Please edit that file --\u003e\n\n```{r, include = FALSE}\nknitr::opts_chunk$set(\n  collapse = TRUE,\n  comment = \"#\u003e\",\n  fig.path = \"man/figures/README-\",\n  fig.width = 7,\n  fig.height = 5, \n  dpi = 300)\n```\n\n# `ppsr` - Predictive Power Score\n\n\u003c!-- badges: start --\u003e\n[![R-CMD-check](https://github.com/paulvanderlaken/ppsr/workflows/R-CMD-check/badge.svg)](https://github.com/paulvanderlaken/ppsr/actions)\n[![CRAN status](https://www.r-pkg.org/badges/version/ppsr)](https://cran.r-project.org/package=ppsr)\n[![CRAN_Downloads_Total](http://cranlogs.r-pkg.org/badges/grand-total/ppsr)](https://cran.r-project.org/package=ppsr)\n\n \u003c!-- badges: end --\u003e\n\n`ppsr` is the R implementation of the **Predictive Power Score** (PPS). \n\nThe PPS is an asymmetric, data-type-agnostic score that can detect linear or \nnon-linear relationships between two variables. \nThe score ranges from 0 (no predictive power) to 1 (perfect predictive power).\n\nThe general concept of PPS is useful for data exploration purposes,\nin the same way correlation analysis is. \nYou can read more about the (dis)advantages of using PPS in [this blog post](https://towardsdatascience.com/rip-correlation-introducing-the-predictive-power-score-3d90808b9598).\n\n\n\n## Installation\n\n\nYou can install the latest stable version of `ppsr` from CRAN:\n\n```{r, eval = TRUE, echo = TRUE}\ninstall.packages('ppsr')\n```\n\nNot all recent features and bugfixes may be included in the CRAN release.\n\nInstead, you might want to download the most recent developmental version of `ppsr` from Github:\n\n```{r, eval = FALSE, echo = TRUE}\n# install.packages('devtools') # Install devtools if needed\ndevtools::install_github('https://github.com/paulvanderlaken/ppsr')\n```\n\n\n\n## Computing PPS\n\n**PPS** represents a **framework for evaluating predictive validity**.\n\nThere is _not one single way_ of computing a predictive power score, but rather there are _many different ways_.\n\nYou can select different machine learning algorithms, their associated parameters, \ncross-validation schemes, and/or model evaluation metrics. \nEach of these design decisions will affect your model's predictive performance and, \nin turn, affect the resulting predictive power score you compute.\n\nHence, you can compute many different PPS for any given predictor and target variable.\n\nFor example, the PPS computed with a _decision tree_ regression model...\n\n```{r}\nppsr::score(iris, x = 'Sepal.Length', y = 'Petal.Length', algorithm = 'tree')[['pps']]\n```\n\n...will differ from the PPS computed with a _simple linear regression_ model.\n\n```{r}\nppsr::score(iris, x = 'Sepal.Length', y = 'Petal.Length', algorithm = 'glm')[['pps']]\n```\n\n\n\n## Usage\n\nThe `ppsr` package has four main functions to compute PPS:\n\n* `score()` computes an x-y PPS\n* `score_predictors()` computes all X-y PPS\n* `score_df()` computes all X-Y PPS\n* `score_matrix()` computes all X-Y PPS, and shows them in a matrix\n\nwhere `x` and `y` represent an individual predictor/target, \nand `X` and `Y` represent all predictors/targets in a given dataset.\n\n### Examples:\n\n`score()` computes the PPS for a single target and predictor\n```{r}\nppsr::score(iris, x = 'Sepal.Length', y = 'Petal.Length')\n```\n\n`score_predictors()` computes all PPSs for a single target using all predictors in a dataframe\n```{r}\nppsr::score_predictors(df = iris, y = 'Species')\n```\n\n`score_df()` computes all PPSs for every target-predictor combination in a dataframe\n```{r}\nppsr::score_df(df = iris)\n```\n\n`score_df()` computes all PPSs for every target-predictor combination in a dataframe, \nbut returns only the scores arranged in a neat matrix, like the familiar correlation matrix\n```{r}\nppsr::score_matrix(df = iris)\n```\n\n\nCurrently, the `ppsr` package computes PPS by default using...\n\n* the default decision tree implementation of the `rpart` package, wrapped by `parsnip`\n* *weighted F1* scores to evaluate classification models, and _MAE_ to evaluate regression models\n* 5 cross-validations\n\nYou can call the `available_algorithms()` and `available_evaluation_metrics()` functions\nto see what alternative settings are supported.\n\nNote that the calculated PPS reflects the **out-of-sample** predictive validity\nwhen more than a single cross-validation is used. \nIf you prefer to look at in-sample scores, you can set `cv_folds = 1`.\nNote that in such cases overfitting can become an issue, particularly with the more flexible algorithms.\n\n\n\n## Visualizing PPS\nSubsequently, there are three main functions that wrap around these computational \nfunctions to help you visualize your PPS using `ggplot2`:\n\n* `visualize_pps()` produces a barplot of all X-y PPS, or a heatmap of all X-Y PPS\n* `visualize_correlations()` produces a heatmap of all X-Y correlations\n* `visualize_both()` produces the two heatmaps of all X-Y PPS and correlations side-by-side\n\n### Examples:\n\nIf you specify a target variable (`y`) in `visualize_pps()`, you get a barplot of its predictors.\n```{r, PPS-barplot}\nppsr::visualize_pps(df = iris, y = 'Species')\n```\n\nIf you do not specify a target variable in `visualize_pps()`, you get the PPS matrix visualized as a heatmap.\n```{r, PPS-heatmap}\nppsr::visualize_pps(df = iris)\n```\n\nSome users might find it useful to look at a correlation matrix for comparison.\n```{r, correlation-heatmap}\nppsr::visualize_correlations(df = iris)\n```\n\nWith `visualize_both` you generate the PPS and correlation matrices side-by-side, for easy comparison.\n```{r, sbs-heatmap, fig.width=14}\nppsr::visualize_both(df = iris)\n```\n\nYou can change the colors of the visualizations using the functions arguments.\nThere are also arguments to change the color of the text scores.\n\nFurthermore, the functions return `ggplot2` objects, so that you can easily change the theme and other settings.\n\n```{r, custom-plot}\nppsr::visualize_pps(df = iris,\n                    color_value_high = 'red', \n                    color_value_low = 'yellow',\n                    color_text = 'black') +\n  ggplot2::theme_classic() +\n  ggplot2::theme(plot.background = ggplot2::element_rect(fill = \"lightgrey\")) +\n  ggplot2::theme(title = ggplot2::element_text(size = 15)) +\n  ggplot2::labs(title = 'Add your own title', \n                subtitle = 'Maybe an informative subtitle',\n                caption = 'Did you know ggplot2 includes captions?',\n                x = 'You could call this\\nthe independent variable\\nas well')\n```\n\n\n## Parallelization\n\nThe number of predictive models that one needs to build in order to fill \nthe PPS matrix belonging to a dataframe increases exponentially \nwith every new column in that dataframe. \n\nFor traditional correlation analyses, this is not a problem. \nYet, with more computation-intensive algorithms, with many train-test splits, \nand with large or high-dimensional datasets, it can take a decent amount of time\nto build all the predictive models and derive their PPSs.\n\nOne way to speed matters up is to use the `ppsr::score_predictors()` function and \nfocus on predicting only the target/dependent variable you are most interested in.\n\nYet, since version `0.0.1`, all `ppsr::score_*` and `pssr::visualize_*` functions \nnow take in two arguments that facilitate parallel computing.\nYou can parallelize `ppsr`'s computations by setting the `do_parallel` argument to `TRUE`. \nIf done so, a cluster will be created using the `parallel` package. \nBy default, this cluster will use the maximum number of cores (see `parallel::detectCores()`) minus 1.\n\nHowever, with the second argument -- `n_cores` -- you can manually specify the number of cores you want `ppsr` to use.\n\nExamples:\n```{r, eval = FALSE}\nppsr::score_df(df = mtcars, do_parallel = TRUE)\n```\n\n```{r, eval = FALSE}\nppsr::visualize_pps(df = iris, do_parallel = TRUE, n_cores = 2)\n```\n\n\n## Interpreting PPS\n\nThe PPS is a **normalized score** that ranges from 0 (no predictive power) \nto 1 (perfect predictive power).\n\nThe normalization occurs by comparing how well we are able to predict the values of \na _target_ variable (`y`) using the values of a _predictor_ variable (`x`), \nrespective to two **benchmarks**: a perfect prediction, and a naive prediction \n\nThe **perfect prediction** can be theoretically derived. \nA perfect regression model produces no error (=0.0), \nwhereas a perfect classification model results in 100% accuracy, recall, et cetera (=1.0).\n\nThe **naive prediction** is derived empirically. \nA naive _regression_ model is simulated by predicting the mean `y` value for all observations. \nThis is similar to how R-squared is calculated. \nA naive _classification_ model is simulated by taking the best among two models:\none predicting the modal `y` class, and one predicting random `y` classes for all observations.\n\nWhenever we train an \"informed\" model to predict `y` using `x`, \nwe can assess how well it performs by comparing it to these two benchmarks.\n\nSuppose we train a regression model, and its mean average error (MAE) is 0.10.\nSuppose the naive model resulted in an MAE of 0.40.\nWe know the perfect model would produce no error, which means an MAE of 0.0.\n\nWith these three scores, we can normalize the performance of our informed regression model \nby interpolating its score between the perfect and the naive benchmarks. \nIn this case, our model's performance lies about 1/4\u003csup\u003eth\u003c/sup\u003e of the way from the perfect model, \nand 3/4\u003csup\u003eths\u003c/sup\u003e of the way from the naive model.\nIn other words, our model's predictive power score is 75%: it produced 75% less error than the naive baseline, and was only 25% short of perfect predictions. \n\nUsing such normalized scores for model performance allows us to easily interpret\nhow much better our models are as compared to a naive baseline. \nMoreover, such normalized scores allow us to compare and contrast different \nmodeling approaches, in terms of the algorithms, the target's data type, \nthe evaluation metrics, and any other settings used.\n\n\n\n\n## Considerations\n\nThe main use of PPS is as a tool for data exploration.\nIt trains out-of-the-box machine learning models to assess the predictive relations in your dataset.\n\nHowever, this PPS is quite a \"quick and dirty\" approach.\nThe trained models are not at all tailored to your specific regression/classification problem.\nFor example, it could be that you get many PPSs of 0 with the default settings.\nA known issue is that the default decision tree often does not find valuable splits\nand reverts to predicting the mean `y` value found at its root.\nHere, it could help to try calculating PPS with different settings (e.g., `algorithm = 'glm'`).\n\nAt other times, predictive relationships may rely on a combination of variables \n(i.e. interaction/moderation). These are not captured by the PPS calculations, \nwhich consider only univariate relations. \nPPS is simply not suited for capturing such complexities. \nIn these cases, it might be more interesting to train models on all your features simultaneously \nand turn to concepts like [feature/variable importance](https://topepo.github.io/caret/variable-importance.html),\n[partial dependency](https://christophm.github.io/interpretable-ml-book/pdp.html), [conditional expectations](https://christophm.github.io/interpretable-ml-book/ice.html), [accumulated local effects](https://christophm.github.io/interpretable-ml-book/ale.html), and others.\n\nIn general, the PPS should not be considered more than a fast and easy tool to \nfinding starting points for further, in-depth analysis. \nKeep in mind that you can build much better predictive models than the default \nPPS functions if you tailor your modeling efforts to your specific data context.\n\n\n\n## Open issues \u0026 development\n\nPPS is a relatively young concept, and likewise the `ppsr` package is still under development. \nIf you spot any bugs or potential improvements, please raise an issue or submit a pull request. \n\nOn the developmental agenda are currently:\n\n* Support for different modeling techniques/algorithms\n* Support for generalized linear models for multinomial classification\n* Passing/setting of parameters for models\n* Different model evaluation metrics\n* Support for user-defined model evaluation metrics\n* Downsampling for large datasets\n\n\n## Attribution\n\nThis R package was inspired by 8080labs' Python package [`ppscore`](https://github.com/8080labs/ppscore).\n\nThe same 8080labs also developed an earlier, unfinished [R implementation of PPS](https://github.com/8080labs/ppscoreR).\n\nRead more about the big ideas behind PPS in [this blog post](https://towardsdatascience.com/rip-correlation-introducing-the-predictive-power-score-3d90808b9598).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpaulvanderlaken%2Fppsr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpaulvanderlaken%2Fppsr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpaulvanderlaken%2Fppsr/lists"}