{"id":23586714,"url":"https://github.com/pythonhealthdatascience/stars_wp1_summary","last_synced_at":"2026-02-09T08:03:26.456Z","repository":{"id":253095212,"uuid":"842422919","full_name":"pythonhealthdatascience/stars_wp1_summary","owner":"pythonhealthdatascience","description":"Summary of the eight computational reproducibility assessments conducted as part of STARS Work Package 1. These assessed discrete-event simulation papers with models in Python and R.","archived":false,"fork":false,"pushed_at":"2025-01-22T09:34:44.000Z","size":19514,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-17T04:07:23.229Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://pythonhealthdatascience.github.io/stars_wp1_summary/","language":"TeX","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc-by-4.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pythonhealthdatascience.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-08-14T10:16:54.000Z","updated_at":"2025-01-21T16:41:25.000Z","dependencies_parsed_at":"2024-08-19T09:45:55.966Z","dependency_job_id":"1a3c10cc-17ab-49b2-ae95-c480033e0939","html_url":"https://github.com/pythonhealthdatascience/stars_wp1_summary","commit_stats":null,"previous_names":["pythonhealthdatascience/stars_wp1_summary"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/pythonhealthdatascience/stars_wp1_summary","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pythonhealthdatascience%2Fstars_wp1_summary","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pythonhealthdatascience%2Fstars_wp1_summary/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pythonhealthdatascience%2Fstars_wp1_summary/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pythonhealthdatascience%2Fstars_wp1_summary/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pythonhealthdatascience","download_url":"https://codeload.github.com/pythonhealthdatascience/stars_wp1_summary/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pythonhealthdatascience%2Fstars_wp1_summary/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269833039,"owners_count":24482362,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-11T02:00:10.019Z","response_time":75,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-27T04:13:46.925Z","updated_at":"2026-02-09T08:03:17.011Z","avatar_url":"https://github.com/pythonhealthdatascience.png","language":"TeX","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\n  \u003ca href=\"https://github.com/pythonhealthdatascience\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/pythonhealthdatascience/stars_wp1_summary/main/images/stars_banner.png\" alt=\"Markdownify\"\u003e\u003c/a\u003e\n  \u003cbr\u003e\n  Computational Reproducibility Assessments: Summary\n  \u003cbr\u003e\n\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ci align=\"center\"\u003eSummary of the eight computational reproducibility assessments conducted as part of STARS Work Package 1.\u003c/i\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ca target=\"_blank\" href=\"https://doi.org/10.5281/zenodo.14267268\"\u003e\u003cimg src=\"https://zenodo.org/badge/DOI/10.5281/zenodo.14267268.svg\" alt=\"DOI 10.5281/zenodo.14267268\"/\u003e\u003c/a\u003e\n    \u003c!--\u003ca href=\"#\"\u003e\u003cimg src=\"https://img.shields.io/github/v/release/pythonhealthdatascience/stars_wp1_summary\" alt=\"GitHub release\" /\u003e\u003c/a\u003e\n    \u003ca href=\"#\"\u003e\u003cimg src=\"https://img.shields.io/github/release-date/pythonhealthdatascience/stars_wp1_summary\" alt=\"GitHub release date\" /\u003e\u003c/a\u003e--\u003e\n    \u003ca href=\"#\"\u003e\u003cimg src=\"https://img.shields.io/github/last-commit/pythonhealthdatascience/stars_wp1_summary\" alt=\"GitHub last commit\" /\u003e\u003c/a\u003e\n    \u003ca target=\"_blank\" href=\"https://github.com/pythonhealthdatascience/stars_wp1_summary/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/badge/license-CC--BY--4.0-blue.svg\" alt=\"MIT licence\"/\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n## Table of contents\n\n* [👋 About the repository](#-about-the-repository)\n* [📍 Locating tables and figures from the article](#-locating-tables-and-figures-from-the-article)\n* [📖 View book locally](#-view-book-locally)\n* [📝 Citation](#-citation)\n* [💰 Funding](#-funding)\n\n\u003cbr\u003e\n\u003cbr\u003e\n\n## 👋 About the repository\n\nIn work package 1, we assessed the computational reproducibility of eight discrete-event simulation papers with models in Python and R. The reproductions and findings are summarised at: \u003chttps://pythonhealthdatascience.github.io/stars_wp1_summary/\u003e.\n\n[![Python](https://img.shields.io/badge/-python-black?style=for-the-badge\u0026logoColor=white\u0026logo=python\u0026color=3776AB)](https://www.python.org/)\n[![R](https://img.shields.io/badge/-r-black?style=for-the-badge\u0026logoColor=white\u0026logo=r\u0026color=276DC3)](https://www.r-project.org/)\n\nRelevant GitHub repositories:\n\n| Repository | Description |\n| --- | --- |\n| [stars-reproduction-protocol](https://github.com/pythonhealthdatascience/stars_reproduction_protocol) | Latex files for reproduction protocol |\n| [stars-reproduce-allen-2020](https://github.com/pythonhealthdatascience/stars-reproduce-allen-2020) |Test run of reproducibility protocol on Allen et al. 2020 |\n| [stars-reproduction-template](https://github.com/pythonhealthdatascience/stars_reproduction_template) | Template for assessment of computational reproducibility |\n| [stars-reproduce-shoaib-2022](https://github.com/pythonhealthdatascience/stars-reproduce-shoaib-2022) | Reproduction study 1: Shoaib and Ramamohan 2022 (Python) |\n| [stars-reproduce-huang-2019](https://github.com/pythonhealthdatascience/stars-reproduce-huang-2019) | Reproduction study 2: Huang et al. 2019 (R) |\n| [stars-reproduce-lim-2020](https://github.com/pythonhealthdatascience/stars-reproduce-lim-2020) | Reproduction study 3: Lim et al. 2020 (Python) |\n| [stars-reproduce-kim-2021](https://github.com/pythonhealthdatascience/stars-reproduce-kim-2021) | Reproduction study 4: Kim et al. 2021 (R) |\n| [stars-reproduce-anagnostou-2022](https://github.com/pythonhealthdatascience/stars-reproduce-anagnostou-2022) | Reproduction study 5: Anagnostou et al. 2022 (Python) |\n| [stars-reproduce-johnson-2021](https://github.com/pythonhealthdatascience/stars-reproduce-johnson-2021) | Reproduction study 6: Johnson et al. 2021 (R) |\n| [stars-reproduce-hernandez-2015](https://github.com/pythonhealthdatascience/stars-reproduce-hernandez-2015) | Reproduction study 7: Hernandez et al. 2015 (Python + R) |\n| [stars-reproduce-wood-2021](https://github.com/pythonhealthdatascience/stars-reproduce-wood-2021) | Reproduction study 8: Wood et al. 2021 (R) |\n\nProcess followed for each study:\n\n![Workflow](./images/stars_wp1_workflow.png)\n\n\u003cbr\u003e\n\u003cbr\u003e\n\n## 📍 Locating tables and figures from the article\n\n| Figure/Table | Method | Location |\n| - | - | - |\n| **Figure 1.** Five standards that scientific code should strive to achieve, and the benefits of doing so | Inkscape | `images/5rs.svg` |\n| **Figure 2.** Time to complete items in scope for each reproduction, inspired by figure in Krafczyk et al. 2021 | Matplotlib | Created within `pages/reproduction.qmd`, saved as `images/article_times.png` |\n| **Figure 3.** Recommendations to support reproduction, with stars to highlight five recommendations considered to have the greatest potential impact. | Inkscape | `images/reproduction_wheel.svg`|\n| **Figure 4.** Recommendations to support troubleshooting and reuse | Inkscape | `images/troubleshooting_wheel.svg` |\n| **Figure 5.** Of the eight healthcare DES studies evaluated, proportion that met each recommendation in the current STARS framework. | Plotly express | Created within `pages/repo_evaluation.qmd`, saved as `images/stars_criteria.png` |\n| **Figure 6.** Of the eight healthcare DES studies evaluated, proportion that met each item in the current STRESS-DES criteria. | Plotly express | Created within `pages/paper_evaluation.qmd`, saved as `stress_criteria.png` |\n| **Figure 7.** Of the eight healthcare DES studies evaluated, proportion that met each criteria in the general reporting checklist for DES | Plotly express | Created within `pages/paper_evaluation.qmd`, saved as `ispor_criteria.png` |\n| **Table 2.** Evaluation of repositories against ACM badge criteria. | - | Created within `pages/repo_evaluation.qmd`, saved as `data/badges_table.csv` (and Table 2 is an extract from that table) |\n| **Table 3.** Proportion of applicable criteria that were fully met, from evaluation of repository or article, alongside the proportion of items reproduced from each study. | - | Combination of two tables: (1) `data/applicable_stars.csv` created within `pages/repo_evaluation.qmd`, and (2) `data/applicable_report.csv` created within `pages/paper_evaluation.qmd` |\n| **Table D1.** Evaluation of studies against badge criteria - grouped into three themes, as defined by NISO. | - | Created within `pages/repo_evaluation.qmd`, saved as `data/badges_table.csv` |\n\nThe remaining tables were created directly in the Latex article, rather than in this repository, as they are not describing results from reproduction:\n\n* **Table 1.** Description of the included studies.\n* **Table 4.** Simple checklists to assist reviewers in assessing the openness, longevity, and reproducibility of DES models during peer review.\n* **Table B1.** Links for reproduction and evaluation.\n* **Table B2.** Links to original study repositories.\n\n\u003cbr\u003e\n\u003cbr\u003e\n\n## 📖 View book locally\n\nThe [website](https://pythonhealthdatascience.github.io/stars_wp1_summary/) is a quarto book hosted with GitHub pages. If you want to view the book locally on your own machine you will need to:\n\n1. Clone GitHub repository\n\n```\ngit clone https://github.com/pythonhealthdatascience/stars_wp1_summary.git\n```\n\n2. Create the virtual environment\n\n```\nvirtualenv stars_wp1_summary\nsource stars_wp1_summary/bin/activate\npip install -r requirements.txt\n```\n\n3. Create the book\n\n```\nquarto render\n```\n\n4. Open the book in your browser (open the `_book/index.html` file).\n\n\u003cbr\u003e\n\u003cbr\u003e\n\n## 📝 Citation\n\nThis repository has been archived on Zenodo and can be cited as:\n\n\u003e Heather, A., Monks, T., \u0026 Harper, A. (2025). Computational Reproducibility Assessments: Summary. Zenodo. \u003chttps://doi.org/10.5281/zenodo.14267268\u003e.\n\nIf you wish to cite this repository on GitHub, please refer to the citation file `CITATION.cff`, and the auto-generated alternatives `citation_apalike.apa` and `citation_bibtex.bib`. Authors:\n\n| Member | ORCID | GitHub |\n| --- | --- | --- |\n| Amy Heather | [![ORCID: Heather](https://img.shields.io/badge/ORCID-0000--0002--6596--3479-brightgreen)](https://orcid.org/0000-0002-6596-3479) | https://github.com/amyheather |\n| Thomas Monks | [![ORCID: Monks](https://img.shields.io/badge/ORCID-0000--0003--2631--4481-brightgreen)](https://orcid.org/0000-0003-2631-4481) | https://github.com/TomMonks |\n| Alison Harper | [![ORCID: Harper](https://img.shields.io/badge/ORCID-0000--0001--5274--5037-brightgreen)](https://orcid.org/0000-0001-5274-5037) | https://github.com/AliHarp |\n\n\u003cbr\u003e\n\u003cbr\u003e\n\n## 💰 Funding\n\nThis project is supported by the Medical Research Council [grant number [MR/Z503915/1](https://gtr.ukri.org/projects?ref=MR%2FZ503915%2F1)].\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpythonhealthdatascience%2Fstars_wp1_summary","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpythonhealthdatascience%2Fstars_wp1_summary","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpythonhealthdatascience%2Fstars_wp1_summary/lists"}