Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/easystats/datawizard

Magic potions to clean and transform your data 🧙
https://github.com/easystats/datawizard

data dplyr hacktoberfest janitor manipulation r-package reshape rstats tidyr wrangling

Last synced: 3 months ago
JSON representation

Magic potions to clean and transform your data 🧙

Awesome Lists containing this project

README 

        
---

output: github_document

---



# `datawizard`: Easy Data Wrangling and Statistical Transformations 



```{r, echo=FALSE, warning=FALSE, message=FALSE}

knitr::opts_chunk$set(

  collapse = TRUE,

  dpi = 300,

  out.width = "100%",

  fig.path = "man/figures/",

  comment = "#>"

)



set.seed(333)

library(datawizard)

```



[![DOI](https://joss.theoj.org/papers/10.21105/joss.04684/status.svg)](https://doi.org/10.21105/joss.04684)

[![downloads](http://cranlogs.r-pkg.org/badges/datawizard)](https://cran.r-project.org/package=datawizard)

[![total](https://cranlogs.r-pkg.org/badges/grand-total/datawizard)](https://cranlogs.r-pkg.org/) [![lifecycle](https://img.shields.io/badge/lifecycle-maturing-blue.svg)](https://lifecycle.r-lib.org/articles/stages.html)



`{datawizard}` is a lightweight package to easily manipulate, clean, transform, and prepare your data for analysis. It is part of the [easystats ecosystem](https://easystats.github.io/easystats/), a suite of R packages to deal with your entire statistical analysis, from cleaning the data to reporting the results. 



It covers two aspects of data preparation: 



- **Data manipulation**: `{datawizard}` offers a very similar set of functions to that of the *tidyverse* packages, such as a `{dplyr}` and `{tidyr}`, to select, filter and reshape data, with a few key differences. 1) All data manipulation functions start with the prefix `data_*` (which makes them easy to identify). 2) Although most functions can be used exactly as their *tidyverse* equivalents, they are also string-friendly (which makes them easy to program with and use inside functions). Finally, `{datawizard}` is super lightweight (no dependencies, similar to [poorman](https://github.com/nathaneastwood/poorman)), which makes it awesome for developers to use in their packages.



- **Statistical transformations**: `{datawizard}` also has powerful functions to easily apply common data [transformations](https://easystats.github.io/datawizard/reference/index.html#statistical-transformations), including standardization, normalization, rescaling, rank-transformation, scale reversing, recoding, binning, etc. 






# Installation



[![CRAN_Status_Badge](https://www.r-pkg.org/badges/version/datawizard)](https://cran.r-project.org/package=datawizard) [![insight status badge](https://easystats.r-universe.dev/badges/datawizard)](https://easystats.r-universe.dev) [![R-CMD-check](https://github.com/easystats/datawizard/workflows/R-CMD-check/badge.svg?branch=main)](https://github.com/easystats/datawizard/actions)



Type | Source | Command

---|---|---

Release | CRAN | `install.packages("datawizard")`

Development | r-universe | `install.packages("datawizard", repos = "https://easystats.r-universe.dev")`

Development | GitHub | `remotes::install_github("easystats/datawizard")`



> **Tip**

>

> **Instead of `library(datawizard)`, use `library(easystats)`.**

> **This will make all features of the  easystats-ecosystem available.**

>

> **To stay updated, use `easystats::install_latest()`.**



# Citation



To cite the package, run the following command:



```{r, comment=""}

citation("datawizard")

```



# Features



Most courses and tutorials about statistical modeling assume that you are working with a clean and tidy dataset. In practice, however, a major part of doing statistical modeling is preparing your data--cleaning up values, creating new columns, reshaping the dataset, or transforming some variables. `{datawizard}` provides easy to use tools to perform these common, critical, and sometimes tedious data preparation tasks. 



## Data wrangling 



### Select, filter and remove variables



The package provides helpers to filter rows meeting certain conditions...



```{r}

data_match(mtcars, data.frame(vs = 0, am = 1))

```



... or logical expressions:



```{r}

data_filter(mtcars, vs == 0 & am == 1)

```



Finding columns in a data frame, or retrieving the data of selected columns, can be  achieved using `extract_column_names()` or `data_select()`:



```{r}

# find column names matching a pattern

extract_column_names(iris, starts_with("Sepal"))



# return data columns matching a pattern

data_select(iris, starts_with("Sepal")) |> head()

```



It is also possible to extract one or more variables:



```{r}

# single variable

data_extract(mtcars, "gear")



# more variables

head(data_extract(iris, ends_with("Width")))

```



Due to the consistent API, removing variables is just as simple:



```{r}

head(data_remove(iris, starts_with("Sepal")))

```



### Reorder or rename



```{r}

head(data_relocate(iris, select = "Species", before = "Sepal.Length"))

```



```{r}

head(data_rename(iris, c("Sepal.Length", "Sepal.Width"), c("length", "width")))

```



### Merge



```{r}

x <- data.frame(a = 1:3, b = c("a", "b", "c"), c = 5:7, id = 1:3)

y <- data.frame(c = 6:8, d = c("f", "g", "h"), e = 100:102, id = 2:4)



x

y



data_merge(x, y, join = "full")



data_merge(x, y, join = "left")



data_merge(x, y, join = "right")



data_merge(x, y, join = "semi", by = "c")



data_merge(x, y, join = "anti", by = "c")



data_merge(x, y, join = "inner")



data_merge(x, y, join = "bind")

```



### Reshape



A common data wrangling task is to reshape data.



Either to go from wide/Cartesian to long/tidy format



```{r}

wide_data <- data.frame(replicate(5, rnorm(10)))



head(data_to_long(wide_data))

```



or the other way



```{r}

long_data <- data_to_long(wide_data, rows_to = "Row_ID") # Save row number



data_to_wide(long_data,

  names_from = "name",

  values_from = "value",

  id_cols = "Row_ID"

)

```



### Empty rows and columns



```{r}

tmp <- data.frame(

  a = c(1, 2, 3, NA, 5),

  b = c(1, NA, 3, NA, 5),

  c = c(NA, NA, NA, NA, NA),

  d = c(1, NA, 3, NA, 5)

)



tmp



# indices of empty columns or rows

empty_columns(tmp)

empty_rows(tmp)



# remove empty columns or rows

remove_empty_columns(tmp)

remove_empty_rows(tmp)



# remove empty columns and rows

remove_empty(tmp)

```



### Recode or cut dataframe



```{r}

set.seed(123)

x <- sample(1:10, size = 50, replace = TRUE)



table(x)



# cut into 3 groups, based on distribution (quantiles)

table(categorize(x, split = "quantile", n_groups = 3))

```



## Data Transformations



The packages also contains multiple functions to help transform data.



### Standardize



For example, to standardize (*z*-score) data:



```{r}

# before

summary(swiss)



# after

summary(standardize(swiss))

```



### Winsorize



To winsorize data:



```{r}

# before

anscombe



# after

winsorize(anscombe)

```



### Center



To grand-mean center data



```{r}

center(anscombe)

```



### Ranktransform



To rank-transform data:



```{r}

# before

head(trees)



# after

head(ranktransform(trees))

```



### Rescale



To rescale a numeric variable to a new range:



```{r}

change_scale(c(0, 1, 5, -5, -2))

```



### Rotate or transpose



```{r}

x <- mtcars[1:3, 1:4]



x



data_rotate(x)

```



## Data properties



`datawizard` provides a way to provide comprehensive descriptive summary for all variables in a dataframe:



```{r}

data(iris)

describe_distribution(iris)

```



Or even just a variable



```{r}

describe_distribution(mtcars$wt)

```



There are also some additional data properties that can be computed using this package.



```{r}

x <- (-10:10)^3 + rnorm(21, 0, 100)

smoothness(x, method = "diff")

```



## Function design and pipe-workflow



The design of the `{datawizard}` functions follows a design principle that makes it easy for user to understand and remember how functions work:



1. the first argument is the data

2. for methods that work on data frames, two arguments are following to `select` and `exclude` variables

3. the following arguments are arguments related to the specific tasks of the functions



Most important, functions that accept data frames usually have this as their first argument, and also return a (modified) data frame again. Thus, `{datawizard}` integrates smoothly into a "pipe-workflow".



```{r}

iris |>

  # all rows where Species is "versicolor" or "virginica"

  data_filter(Species %in% c("versicolor", "virginica")) |>

  # select only columns with "." in names (i.e. drop Species)

  data_select(contains("\\.")) |>

  # move columns that ends with "Length" to start of data frame

  data_relocate(ends_with("Length")) |>

  # remove fourth column

  data_remove(4) |>

  head()

```

  

# Contributing and Support



In case you want to file an issue or contribute in another way to the package, please follow [this guide](https://easystats.github.io/datawizard/CONTRIBUTING.html). For questions about the functionality, you may either contact us via email or also file an issue.



# Code of Conduct



Please note that this project is released with a 

[Contributor Code of Conduct](https://easystats.github.io/datawizard/CODE_OF_CONDUCT.html). By participating in this project you agree to abide by its terms.