{"id":13857737,"url":"https://github.com/tidymodels/yardstick","last_synced_at":"2025-05-14T03:10:56.617Z","repository":{"id":37336698,"uuid":"108898402","full_name":"tidymodels/yardstick","owner":"tidymodels","description":"Tidy methods for measuring model performance","archived":false,"fork":false,"pushed_at":"2025-04-24T22:49:04.000Z","size":29571,"stargazers_count":388,"open_issues_count":51,"forks_count":53,"subscribers_count":15,"default_branch":"main","last_synced_at":"2025-05-14T01:54:56.181Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://yardstick.tidymodels.org/","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tidymodels.png","metadata":{"files":{"readme":"README.Rmd","changelog":"NEWS.md","contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":".github/CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-10-30T19:26:54.000Z","updated_at":"2025-04-24T22:44:33.000Z","dependencies_parsed_at":"2023-12-20T10:55:45.135Z","dependency_job_id":"9650880f-26a7-4866-a4a2-de9e409281a9","html_url":"https://github.com/tidymodels/yardstick","commit_stats":{"total_commits":1007,"total_committers":33,"mean_commits":"30.515151515151516","dds":0.5680238331678252,"last_synced_commit":"e5c36f206fb737fc54b1a6161c09bc0d63b79beb"},"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tidymodels%2Fyardstick","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tidymodels%2Fyardstick/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tidymodels%2Fyardstick/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tidymodels%2Fyardstick/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tidymodels","download_url":"https://codeload.github.com/tidymodels/yardstick/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254059513,"owners_count":22007769,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-05T03:01:45.487Z","updated_at":"2025-05-14T03:10:51.592Z","avatar_url":"https://github.com/tidymodels.png","language":"R","funding_links":[],"categories":["R","Tools"],"sub_categories":["Performance (\u0026 Automated ML)"],"readme":"---\noutput: github_document\n---\n\n```{r, echo = FALSE}\nknitr::opts_chunk$set(\n  collapse = TRUE,\n  comment = \"#\u003e\",\n  fig.path = \"man/figures/README-\"\n)\n```\n\n```{r load, include = FALSE, message = FALSE, warning = FALSE}\nlibrary(yardstick)\nlibrary(dplyr)\noptions(width = 100, digits = 3)\n```\n\n# yardstick \u003cimg src=\"man/figures/logo.png\" align=\"right\"/\u003e \n\n\u003c!-- badges: start --\u003e\n[![CRAN_Status_Badge](https://www.r-pkg.org/badges/version/yardstick)](https://CRAN.R-project.org/package=yardstick)\n[![Downloads](https://cranlogs.r-pkg.org/badges/yardstick)](https://CRAN.R-project.org/package=yardstick)\n[![R-CMD-check](https://github.com/tidymodels/yardstick/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/tidymodels/yardstick/actions/workflows/R-CMD-check.yaml)\n[![Codecov test coverage](https://codecov.io/gh/tidymodels/yardstick/graph/badge.svg)](https://app.codecov.io/gh/tidymodels/yardstick)\n\u003c!-- badges: end --\u003e\n\n## Overview\n\n`yardstick` is a package to estimate how well models are working using [tidy data](https://doi.org/10.18637/jss.v059.i10) principles. See the [package webpage](https://yardstick.tidymodels.org/) for more information.\n\n## Installation\n\nTo install the package:\n\n```{r install, eval = FALSE}\ninstall.packages(\"yardstick\")\n\n# Development version:\n# install.packages(\"pak\")\npak::pak(\"tidymodels/yardstick\")\n```\n\n## Two class metric\n\nFor example, suppose you create a classification model and predict on a new data set. You might have data that looks like this:\n\n```{r class-data}\nlibrary(yardstick)\nlibrary(dplyr)\n\nhead(two_class_example)\n```\n\nYou can use a `dplyr`-like syntax to compute common performance characteristics of the model and get them back in a data frame:\n\n```{r class-metrics}\nmetrics(two_class_example, truth, predicted)\n\n# or\n\ntwo_class_example %\u003e%\n  roc_auc(truth, Class1)\n```\n\n## Multiclass metrics\n\nAll classification metrics have at least one multiclass extension, with many\nof them having multiple ways to calculate multiclass metrics.\n\n```{r}\ndata(\"hpc_cv\")\nhpc_cv \u003c- as_tibble(hpc_cv)\nhpc_cv\n```\n\n```{r}\n# Macro averaged multiclass precision\nprecision(hpc_cv, obs, pred)\n\n# Micro averaged multiclass precision\nprecision(hpc_cv, obs, pred, estimator = \"micro\")\n```\n\n## Calculating metrics on resamples\n\nIf you have multiple resamples of a model, you can use a metric on a grouped\ndata frame to calculate the metric across all resamples at once.\n\nThis calculates multiclass ROC AUC using the method described in Hand, Till (2001),\nand does it across all 10 resamples at once.\n\n```{r}\nhpc_cv %\u003e%\n  group_by(Resample) %\u003e%\n  roc_auc(obs, VF:L)\n```\n\n## Autoplot methods for easy visualization\n\nCurve based methods such as `roc_curve()`, `pr_curve()` and `gain_curve()` all\nhave `ggplot2::autoplot()` methods that allow for powerful and easy visualization.\n\n```{r roc-curves}\n#| fig-alt: \"Faceted ROC curve. 1-specificity along the x-axis, sensitivity along the y-axis. Facets include the classes F, L, M, and VF. Each facet shows 10 lines colored to correspond to a resample. All the lines are quite overlapping. With VF having the tightest and highest values.\"\nlibrary(ggplot2)\n\nhpc_cv %\u003e%\n  group_by(Resample) %\u003e%\n  roc_curve(obs, VF:L) %\u003e%\n  autoplot()\n```\n\n## Contributing\n\nThis project is released with a [Contributor Code of Conduct](https://contributor-covenant.org/version/2/0/CODE_OF_CONDUCT.html). By contributing to this project, you agree to abide by its terms.\n\n- For questions and discussions about tidymodels packages, modeling, and machine learning, please [post on RStudio Community](https://forum.posit.co/new-topic?category_id=15\u0026tags=tidymodels,question).\n\n- If you think you have encountered a bug, please [submit an issue](https://github.com/tidymodels/yardstick/issues).\n\n- Either way, learn how to create and share a [reprex](https://reprex.tidyverse.org/articles/articles/learn-reprex.html) (a minimal, reproducible example), to clearly communicate about your code.\n\n- Check out further details on [contributing guidelines for tidymodels packages](https://www.tidymodels.org/contribute/) and [how to get help](https://www.tidymodels.org/help/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftidymodels%2Fyardstick","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftidymodels%2Fyardstick","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftidymodels%2Fyardstick/lists"}