{"id":13736763,"url":"https://github.com/Appsilon/shiny.benchmark","last_synced_at":"2025-05-08T13:31:17.323Z","repository":{"id":62646400,"uuid":"482827507","full_name":"Appsilon/shiny.benchmark","owner":"Appsilon","description":"Tools to measure performance improvements in shiny apps","archived":false,"fork":false,"pushed_at":"2024-04-18T20:50:36.000Z","size":1336,"stargazers_count":31,"open_issues_count":20,"forks_count":3,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-04-30T06:06:44.388Z","etag":null,"topics":["performance-testing","r","rhinoverse","shiny"],"latest_commit_sha":null,"homepage":"https://appsilon.github.io/shiny.benchmark/","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Appsilon.png","metadata":{"files":{"readme":"README.md","changelog":"NEWS.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-04-18T11:49:25.000Z","updated_at":"2025-03-25T15:26:01.000Z","dependencies_parsed_at":"2023-11-22T08:16:06.130Z","dependency_job_id":"4c589777-f3de-498b-8fca-fcf22986d49f","html_url":"https://github.com/Appsilon/shiny.benchmark","commit_stats":{"total_commits":235,"total_committers":5,"mean_commits":47.0,"dds":"0.31063829787234043","last_synced_commit":"5100ce2ffe5ee772822e85bb3d58e4f2bc7ea2a2"},"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Appsilon%2Fshiny.benchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Appsilon%2Fshiny.benchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Appsilon%2Fshiny.benchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Appsilon%2Fshiny.benchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Appsilon","download_url":"https://codeload.github.com/Appsilon/shiny.benchmark/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253077246,"owners_count":21850286,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["performance-testing","r","rhinoverse","shiny"],"created_at":"2024-08-03T03:01:28.136Z","updated_at":"2025-05-08T13:31:16.866Z","avatar_url":"https://github.com/Appsilon.png","language":"R","funding_links":[],"categories":["Developer Tools","R"],"sub_categories":["Profiling"],"readme":"# shiny.benchmark \u003ca href=\"https://appsilon.github.io/shiny.benchmark/\"\u003e\u003cimg src=\"man/figures/shiny_benchmark.png\" align=\"right\" alt=\"shiny.benchmark logo\" style=\"height: 140px;\"\u003e\u003c/a\u003e\n\n\u003e _Tools to measure performance improvements in shiny apps._\n\n\u003c!-- badges: start --\u003e\n[![CRAN status](https://www.r-pkg.org/badges/version/shiny.benchmark)](https://cran.r-project.org/package=shiny.benchmark)\n[![R-CMD-check](https://github.com/Appsilon/shiny.benchmark/workflows/R-CMD-check/badge.svg)](https://github.com/Appsilon/shiny.benchmark/actions?workflow=R-CMD-check)\n\u003c!-- badges: end --\u003e\n\n`shiny.benchmark` is a tool aimed to measure and compare the performance of different versions of a `shiny` application. Based on a list of different application versions, accessible by a git repo by its refs (commit hash or branch name), the user can write instructions to be executed using Cypress or `shinytest2`. These instructions are then evaluated by the different versions of your `shiny` application and therefore the performance's improvement/deterioration (time elapsed) are be recorded.\n\nThe package is flexible enough to allow different sets of tests for the different refs as well as different package versions (via `renv`). Also, the user can replicate the tests to have more accurate measures of performance.\n\n## How to install?\n\nStable version:\n\n```r\ninstall.packages(\"shiny.benchmark\")\n```\n\nDevelopment version:\n\n```r\nremotes::install_github(\"Appsilon/shiny.benchmark\")\n```\n\n## Dependencies\n\n`shiny.benchmark` can use two different engines to test the change in the performance of your application: [shinytest2](https://rstudio.github.io/shinytest2/) and [Cypress](https://www.cypress.io/).\nThe latter requires `Node` (version 12 or higher) and `yarn` (version 1.22.17 or higher) to be available.\nTo install them on your computer, follow the guidelines on the documentation pages:\n\n- [Node](https://nodejs.org/en/download/)\n- [yarn](https://yarnpkg.com/getting-started/install)\n\nBesides that, on Linux, it might be required to install other `Cypress` dependencies.\nCheck the [documentation](https://docs.cypress.io/guides/getting-started/installing-cypress#Linux-Prerequisites) to find out more.\n\n## How to use it?\n\nThe best way to start using `shiny.benchmark` is through an example. If you want a start point, you can use the `load_example` function. In order to use this, create a new folder in your computer and use the following code to generate an application to serve us as example for our performance checks:\n\n```r\nlibrary(shiny.benchmark)\n\nload_example(path = \"path/to/new/project\")\n```\n\nIt will create some useful files under `path/to/new/project`. The most important one is the `run_tests.R` which provides several instructions at the very top.\n\nAs we are comparing versions of the same application, we need different app versions in different branches/commits in `git`. Start using `cd app; git init` to initiate git inside `app/` folder.\n\nGet familiar with `app/server.R` file in order to generate more interesting scenarios. The basic idea is to use the `Sys.sleep` function to simulate some app's functionalities. Remember that, when running the benchmark, that is the amount of time it will take to measure the performance.\n\nWhen you are ready, commit your changes in master/main using `git add .; git commit -m \"your commit message\"`. Make some editions and commit these new changes into a new branch or in the same branch your are testing (it will have a different commit hash). Repeat the process adding as many new modifications as you want. E.g. add renv, add more tests, change the names of the tests/test files and so on.\n\nHere is a complete example on how to setup your `git`:\n\n```git\n# starting\ngit init\necho .Rproj.user \u003e\u003e .gitignore\necho *.Rproj \u003e\u003e .gitignore\necho .Rprofile \u003e\u003e .gitignore\necho renv \u003e\u003e .gitignore\necho .Rprofile \u003e\u003e .gitignore\n\n# master\ngit add .\ngit commit -m \"first commit\"\n\n# develop (decrease Sys.sleep times in server.R)\ngit checkout -b develop\ngit add .\ngit commit -m \"improving performance\"\n\n## Using renv\ngit branch renv_shiny1 develop\ngit checkout renv_shiny1\nR -e 'renv::init()'\ngit add .\ngit commit -m \"renv active\"\n\n## Downgrading shiny\ngit checkout -b renv_shiny2\nR -e 'renv::install(\"shiny@1.7.0\")'\nR -e 'renv::snapshot()'\ngit add .\ngit commit -m \"downgrading shiny\"\n\n## Switching back to develop\ngit checkout develop\n```\n\nNow you are ready to go. The `benchmark` function provides several arguments to make your life easier when running your performance checks. The mandatory arguments are:\n\n- `commit_list`: A vector with commits, branches or anything else you can use in `git checkout`\n- `cypress_dir` or `shinytest2_dir`: Folder containing the tests we want to check the performance. In our case it is `tests/cypress` and `tests` respectively.\n\nThe default behavior is to try to use `renv` in your project. If you do not have the renv structure, you can turn `renv` off using `use_renv = FALSE`\n\n```r\nlibrary(shiny.benchmark)\n\n# commits to compare\ncommit_list \u003c- c(\"develop\", \"renv_shiny1\", \"renv_shiny2\")\n\n# run performance check using Cypress\nbenchmark(\n  commit_list = commit_list,\n  cypress_dir = \"tests/cypress\"\n)\n```\n\nThat is all you need to run your `Cypress` tests. If you don't use `Cypress`, you may want to use `shinytest2` instead:\n\n```r\nbenchmark(\n  commit_list = commit_list,\n  shinytest2_dir = \"tests\"\n)\n```\n\nTo run just specific tests, you can take advantage of the `tests_pattern` argument. It will filter the test file's names based on regular expression:\n\n```r\nbenchmark(\n  commit_list = commit_list,\n  shinytest2_dir = \"tests\",\n  tests_pattern = \"use_this_one_[0-9]\"\n)\n```\n\nIf your project has `renv` structure, you can set `use_renv` to `TRUE` to guarantee that, for each application version your are using the correct packages. If you want to approve/reprove `renv::restore()`, you can set `renv_prompt = TRUE`.\n\n```r\nbenchmark(\n  commit_list = commit_list,\n  shinytest2_dir = \"tests\",\n  tests_pattern = \"use_this_one_[0-9]\",\n  use_renv = TRUE, # default\n  renv_prompt = TRUE\n)\n```\n\nTo have more accurate information about the time your application takes to perform some actions, you may need to replicate the tests. In this case, you can use the `n_rep` argument:\n\n```r\nout \u003c- benchmark(\n  commit_list = commit_list,\n  cypress_dir = \"tests/cypress\",\n  tests_pattern = \"use_this_one_[0-9]\",\n  use_renv = FALSE,\n  n_rep = 15\n)\n\nout\n```\n\nFor fast information about the tests results, you can use the `summary` and also the `plot` methods:\n\n```r\nsummary(out)\nplot(out)\n```\n\n## How to contribute?\n\nIf you want to contribute to this project please submit a regular PR, once you're done with new feature or bug fix.\n\nReporting a bug is also helpful - please use [GitHub issues](https://github.com/Appsilon/shiny.benchmark/issues) and describe your problem as detailed as possible.\n\n## Appsilon\n\n\u003cimg src=\"https://avatars0.githubusercontent.com/u/6096772\" align=\"right\" alt=\"\" width=\"6%\" /\u003e\n\nAppsilon is a **Posit (formerly RStudio) Full Service Certified Partner**.\u003cbr/\u003e\nLearn more at [appsilon.com](https://appsilon.com).\n\nGet in touch [opensource@appsilon.com](mailto:opensource@appsilon.com)\n\nExplore the [Rhinoverse](https://rhinoverse.dev) - a family of R packages built around [Rhino](https://appsilon.github.io/rhino/)!\n\n\u003ca href = \"https://appsilon.com/careers/\" target=\"_blank\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/Appsilon/website-cdn/gh-pages/WeAreHiring1.png\" alt=\"We are hiring!\"/\u003e\u003c/a\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAppsilon%2Fshiny.benchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FAppsilon%2Fshiny.benchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAppsilon%2Fshiny.benchmark/lists"}