{"id":22151670,"url":"https://github.com/epiverse-trace/cleanepi","last_synced_at":"2025-07-26T05:31:40.429Z","repository":{"id":119521035,"uuid":"607159823","full_name":"epiverse-trace/cleanepi","owner":"epiverse-trace","description":"R package to clean and standardize epidemiological data","archived":false,"fork":false,"pushed_at":"2024-10-24T14:37:55.000Z","size":9890,"stargazers_count":8,"open_issues_count":28,"forks_count":3,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-10-25T08:57:39.102Z","etag":null,"topics":["data-cleaning","epidemiology","epiverse","r","r-package"],"latest_commit_sha":null,"homepage":"https://epiverse-trace.github.io/cleanepi/","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/epiverse-trace.png","metadata":{"files":{"readme":"README.Rmd","changelog":"NEWS.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-27T12:46:37.000Z","updated_at":"2024-10-23T09:12:34.000Z","dependencies_parsed_at":null,"dependency_job_id":"24299c6a-902d-435f-90b4-437cdd46fd77","html_url":"https://github.com/epiverse-trace/cleanepi","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epiverse-trace%2Fcleanepi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epiverse-trace%2Fcleanepi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epiverse-trace%2Fcleanepi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epiverse-trace%2Fcleanepi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/epiverse-trace","download_url":"https://codeload.github.com/epiverse-trace/cleanepi/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":227652601,"owners_count":17799229,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-cleaning","epidemiology","epiverse","r","r-package"],"created_at":"2024-12-02T00:35:07.288Z","updated_at":"2024-12-02T00:35:08.151Z","avatar_url":"https://github.com/epiverse-trace.png","language":"R","funding_links":[],"categories":[],"sub_categories":[],"readme":"---\noutput: github_document\nalways_allow_html: true\neditor_options: \n  markdown: \n    wrap: 72\n---\n\n\u003c!-- The code to render this README is stored in .github/workflows/render-readme.yaml --\u003e\n\n\u003c!-- Variables marked with double curly braces will be transformed beforehand: --\u003e\n\n\u003c!-- `packagename` is extracted from the DESCRIPTION file --\u003e\n\n\u003c!-- `gh_repo` is extracted via a special environment variable in GitHub Actions --\u003e\n\n\u003c!-- README.md is generated from README.Rmd. Please edit that file --\u003e\n\n```{r, include = FALSE}\nknitr::opts_chunk$set(\n  collapse  = TRUE,\n  comment   = \"#\u003e\",\n  fig.path  = file.path(\"man\", \"figures\", \"README-\"),\n  out.width = \"100%\"\n)\n```\n\n# {{ packagename }}: Clean and standardize epidemiological data \u003cimg src=\"man/figures/logo.svg\" align=\"right\" width=\"130\"/\u003e\n\n\u003c!-- badges: start --\u003e\n\n[![License:\nMIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![R-CMD-check](https://github.com/epiverse-trace/cleanepi/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/epiverse-trace/cleanepi/actions/workflows/R-CMD-check.yaml)\n[![Codecov test\ncoverage](https://codecov.io/gh/epiverse-trace/cleanepi/branch/main/graph/badge.svg)](https://app.codecov.io/gh/epiverse-trace/cleanepi?branch=main)\n[![lifecycle-experimental](https://raw.githubusercontent.com/reconverse/reconverse.github.io/master/images/badge-experimental.svg)](https://www.reconverse.org/lifecycle.html#experimental)\n[![DOI](https://zenodo.org/badge/607159823.svg)](https://zenodo.org/doi/10.5281/zenodo.11473984)\n\n\u003c!-- badges: end --\u003e\n\n**{{ packagename }}** is an  R package designed for cleaning, curating, and standardizing epidemiological data. It streamlines various data cleaning tasks that are typically expected when working with datasets in epidemiology.\n\nKey functionalities of **{{ packagename }}** include:\n\n1. **Removing irregularities**: It removes duplicated and empty rows and columns, as well as columns with constant values.\n\n2. **Handling missing values**: It replaces missing values with the standard `NA` format, ensuring consistency and ease of analysis.\n\n3. **Ensuring data integrity**: It ensures the uniqueness of uniquely identified columns, thus maintaining data integrity and preventing duplicates.\n\n4. **Date conversion**: It offers functionality to convert character columns to Date format under specific conditions, enhancing data uniformity and facilitating temporal analysis. It also offers conversion of numeric values written in letters into numbers.\n\n5. **Standardizing entries**: It can standardize column entries into specified formats, promoting consistency across the dataset.\n\n6. **Time span calculation**: It calculates the time span between two elements of type `Date`, providing valuable demographic insights for epidemiological analysis.\n\n**{{ packagename }}** operates on data frames or similar structures like tibbles, as well as linelist objects commonly used in epidemiological research. It returns the processed data in the same format, ensuring seamless integration into existing workflows. Additionally, it generates a comprehensive report detailing the outcomes of each cleaning task.\n\n\n**{{ packagename }}** is developed by  the [Epiverse-TRACE](https://data.org/initiatives/epiverse/) team at the [Medical Research Council The Gambia unit at the London School of Hygiene and Tropical Medicine](https://www.lshtm.ac.uk/research/units/mrc-gambia).\n\n\n## Installation\n\n**{{ packagename }}** can be installed from CRAN using\n\n```{r message=FALSE, eval=FALSE}\ninstall.packages(\"cleanepi\")\n```\n\nThe latest development version of **{{ packagename }}** can be installed from  [GitHub](https://epiverse-trace.github.io/cleanepi/).\n\n```{r message=FALSE}\nif (!require(\"pak\")) install.packages(\"pak\")\npak::pak(\"{{ gh_repo }}\")\nlibrary(cleanepi)\n```\n\n## Quick start\n\nThe main function in **{{ packagename }}** is `clean_data(),` which internally makes call of almost all standard data cleaning functions, such as removal of empty and duplicated rows and columns, replacement of missing values, etc. However, each function can also be called independently to perform a specific task. This mechanism is explained in details in the **vignette**. Below is typical example of how to use the `clean_data()` function. \n\n\n```{r eval=TRUE}\n# READING IN THE TEST DATASET\ntest_data \u003c- readRDS(\n  system.file(\"extdata\", \"test_df.RDS\", package = \"cleanepi\")\n)\n```\n\n```{r echo=FALSE, eval=TRUE}\ntest_data %\u003e%\n  kableExtra::kbl() %\u003e%\n  kableExtra::kable_paper(\"striped\", font_size = 14, full_width = TRUE) %\u003e%\n  kableExtra::scroll_box(height = \"200px\", width = \"100%\",\n                         box_css = \"border: 1px solid #ddd; padding: 5px; \",\n                         extra_css = NULL,\n                         fixed_thead = TRUE)\n```\n\n```{r eval=TRUE}\n# READING IN THE DATA DICTIONARY\ntest_dictionary \u003c- readRDS(\n  system.file(\"extdata\", \"test_dictionary.RDS\", package = \"cleanepi\")\n)\n```\n\n```{r echo=FALSE, eval=TRUE}\ntest_dictionary %\u003e%\n  kableExtra::kbl() %\u003e%\n  kableExtra::kable_paper(\"striped\", font_size = 14, full_width = TRUE)\n```\n\n```{r eval=TRUE}\n# DEFINING THE CLEANING PARAMETERS\nreplace_missing_values \u003c- list(target_columns = NULL, na_strings = \"-99\")\nremove_duplicates \u003c- list(target_columns = NULL)\nstandardize_dates \u003c- list(\n  target_columns = NULL,\n  error_tolerance = 0.4,\n  format = NULL,\n  timeframe = as.Date(c(\"1973-05-29\", \"2023-05-29\")),\n  orders = list(\n    world_named_months = c(\"Ybd\", \"dby\"),\n    world_digit_months = c(\"dmy\", \"Ymd\"),\n    US_formats = c(\"Omdy\", \"YOmd\")\n  )\n)\nstandardize_subject_ids \u003c- list(\n  target_columns = \"study_id\",\n  prefix = \"PS\",\n  suffix = \"P2\",\n  range = c(1, 100),\n  nchar = 7\n)\nremove_constants \u003c- list(cutoff = 1)\nstandardize_column_names \u003c- list(\n  keep = \"date.of.admission\",\n  rename = c(DOB = \"dateOfBirth\")\n)\nto_numeric \u003c- list(target_columns = \"sex\", lang = \"en\")\n```\n\n```{r eval=TRUE}\n# PERFORMING THE DATA CLEANING\ncleaned_data \u003c- clean_data(\n  data = test_data,\n  standardize_column_names = standardize_column_names,\n  remove_constants = remove_constants,\n  replace_missing_values = replace_missing_values,\n  remove_duplicates = remove_duplicates,\n  standardize_dates = standardize_dates,\n  standardize_subject_ids = standardize_subject_ids,\n  to_numeric = to_numeric,\n  dictionary = test_dictionary,\n  check_date_sequence = NULL\n)\n```\n\n```{r echo=FALSE, eval=TRUE}\n# VISUALISE THE CLEANED DATASET\ncleaned_data %\u003e%\n  kableExtra::kbl() %\u003e%\n  kableExtra::kable_paper(\"striped\", font_size = 14, full_width = FALSE) %\u003e%\n  kableExtra::scroll_box(height = \"200px\", width = \"100%\",\n                         box_css = \"border: 1px solid #ddd; padding: 5px; \",\n                         extra_css = NULL,\n                         fixed_thead = TRUE)\n```\n\n```{r eval=TRUE}\n# EXTRACT THE DATA CLEANING REPORT\nreport \u003c- attr(cleaned_data, \"report\")\n```\n\n```{r eval=FALSE}\n# DISPLAY THE DATA CLEANING REPORT\nprint_report(report)\n```\n\n## Vignette\n\n```{r eval=FALSE}\nbrowseVignettes(\"cleanepi\")\n```\n\n### Lifecycle\n\nThis package is currently an *experimental*, as defined by the [RECON software\nlifecycle](https://www.reconverse.org/lifecycle.html). This means that it is\nfunctional, but interfaces and functionalities may change over time, testing and documentation may be lacking.\n\n### Contributions\n\nContributions are welcome via [pull\nrequests](https://github.com/{{ gh_repo }}/pulls).\n\n### Code of Conduct\n\nPlease note that the {{ packagename }} project is released with a [Contributor\nCode of\nConduct](https://github.com/epiverse-trace/.github/blob/main/CODE_OF_CONDUCT.md).\nBy contributing to this project, you agree to abide by its terms.\n\n## Citing this package\n\n```{r message=FALSE, warning=FALSE}\ncitation(\"cleanepi\")\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepiverse-trace%2Fcleanepi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fepiverse-trace%2Fcleanepi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepiverse-trace%2Fcleanepi/lists"}