Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/gdemin/expss
expss: Tables and Labels in R
https://github.com/gdemin/expss
excel labels labels-support msexcel pivot-tables r recode spss spss-statistics tables variable-labels vlookup
Last synced: 29 days ago
JSON representation
expss: Tables and Labels in R
- Host: GitHub
- URL: https://github.com/gdemin/expss
- Owner: gdemin
- Created: 2015-02-24T17:16:42.000Z (almost 10 years ago)
- Default Branch: master
- Last Pushed: 2024-04-09T15:38:46.000Z (8 months ago)
- Last Synced: 2024-11-07T03:11:44.299Z (about 1 month ago)
- Topics: excel, labels, labels-support, msexcel, pivot-tables, r, recode, spss, spss-statistics, tables, variable-labels, vlookup
- Language: R
- Homepage: https://cran.r-project.org/web/packages/expss/
- Size: 16 MB
- Stars: 83
- Watchers: 9
- Forks: 16
- Open Issues: 7
-
Metadata Files:
- Readme: README.MD
- Funding: .github/FUNDING.yml
Awesome Lists containing this project
- jimsghstars - gdemin/expss - expss: Tables and Labels in R (R)
README
# expss
[](https://cran.r-project.org/package=expss)
[](https://cran.rstudio.com/web/packages/expss/index.html)
[](https://cran.rstudio.com/web/packages/expss/index.html)
[](https://codecov.io/github/gdemin/expss?branch=master)
## Introduction
`expss` computes and displays tables with support for 'SPSS'-style labels, multiple / nested banners, weights, multiple-response variables and significance testing. There are facilities for nice output of tables in 'knitr', R notebooks, 'Shiny' and 'Jupyter' notebooks. Proper methods for labelled variables add value labels support to base R functions and to some functions from other packages. Additionally, the package offers useful functions for data processing in marketing research / social surveys - popular data transformation functions from 'SPSS' Statistics and 'Excel' ('RECODE', 'COUNT', 'COUNTIF', 'VLOOKUP', etc.). Package is intended to help people to move data processing from 'Excel'/'SPSS' to R. See examples below. You can get help about any function by typing `?function_name` in the R console.
### Links
- [Online introduction](http://gdemin.github.io/expss/)
- [expss on CRAN](https://cran.r-project.org/package=expss)
- [expss on Github](https://github.com/gdemin/expss)
- [expss on Stackoverflow](https://stackoverflow.com/questions/tagged/expss)
- [Issues](https://github.com/gdemin/expss/issues)
## Installation
`expss` is on CRAN, so for installation you can print in the console
`install.packages("expss")`.
## Cross-tablulation examples
We will use for demonstartion well-known `mtcars` dataset. Let's start with adding labels to the dataset. Then we can continue with tables creation.
```R
library(expss)
data(mtcars)
mtcars = apply_labels(mtcars,
mpg = "Miles/(US) gallon",
cyl = "Number of cylinders",
disp = "Displacement (cu.in.)",
hp = "Gross horsepower",
drat = "Rear axle ratio",
wt = "Weight (1000 lbs)",
qsec = "1/4 mile time",
vs = "Engine",
vs = c("V-engine" = 0,
"Straight engine" = 1),
am = "Transmission",
am = c("Automatic" = 0,
"Manual"=1),
gear = "Number of forward gears",
carb = "Number of carburetors"
)
```
For quick cross-tabulation there are `fre` and `cross` family of function. For simplicity we demonstrate here only `cross_cpct` which calculates column percent. Documentation for other functions, such as `cross_cases` for counts, `cross_rpct` for row percent, `cross_tpct` for table percent and `cross_fun` for custom summary functions can be seen by typing `?cross_cpct` and `?cross_fun` in the console.
```R
# 'cross_*' examples
# just simple crosstabulation, similar to base R 'table' function
cross_cases(mtcars, am, vs)
# Table column % with multiple banners
cross_cpct(mtcars, cyl, list(total(), am, vs))
# magrittr pipe usage and nested banners
mtcars %>%
cross_cpct(cyl, list(total(), am %nest% vs))
```
We have more sophisticated interface for table construction with `magrittr` piping. Table construction consists of at least of three functions chained with pipe operator: `%>%`. At first we need to specify variables for which statistics will be computed with `tab_cells`. Secondary, we calculate statistics with one of the `tab_stat_*` functions. And last, we finalize table creation with `tab_pivot`, e. g.: `dataset %>% tab_cells(variable) %>% tab_stat_cases() %>% tab_pivot()`. After that we can optionally sort table with `tab_sort_asc`, drop empty rows/columns with `drop_rc` and transpose with `tab_transpose`. Resulting table is just a `data.frame` so we can use usual R operations on it. Detailed documentation for table creation can be seen via `?tables`. For significance testing see `?significance`.
Generally, tables automatically translated to HTML for output in knitr or Jupyter notebooks. However, if we want HTML output in the R notebooks or in the RStudio viewer we need to set options for that: `expss_output_rnotebook()` or `expss_output_viewer()`.
```R
# simple example
mtcars %>%
tab_cells(cyl) %>%
tab_cols(total(), am) %>%
tab_stat_cpct() %>%
tab_pivot()
# table with caption
mtcars %>%
tab_cells(mpg, disp, hp, wt, qsec) %>%
tab_cols(total(), am) %>%
tab_stat_mean_sd_n() %>%
tab_last_sig_means(subtable_marks = "both") %>%
tab_pivot() %>%
set_caption("Table with summary statistics and significance marks.")
# Table with the same summary statistics. Statistics labels in columns.
mtcars %>%
tab_cells(mpg, disp, hp, wt, qsec) %>%
tab_cols(total(label = "#Total| |"), am) %>%
tab_stat_fun(Mean = w_mean, "Std. dev." = w_sd, "Valid N" = w_n, method = list) %>%
tab_pivot()
# Different statistics for different variables.
mtcars %>%
tab_cols(total(), vs) %>%
tab_cells(mpg) %>%
tab_stat_mean() %>%
tab_stat_valid_n() %>%
tab_cells(am) %>%
tab_stat_cpct(total_row_position = "none", label = "col %") %>%
tab_stat_rpct(total_row_position = "none", label = "row %") %>%
tab_stat_tpct(total_row_position = "none", label = "table %") %>%
tab_pivot(stat_position = "inside_rows")
# Table with split by rows and with custom totals.
mtcars %>%
tab_cells(cyl) %>%
tab_cols(total(), vs) %>%
tab_rows(am) %>%
tab_stat_cpct(total_row_position = "above",
total_label = c("number of cases", "row %"),
total_statistic = c("u_cases", "u_rpct")) %>%
tab_pivot()
# Linear regression by groups.
mtcars %>%
tab_cells(sheet(mpg, disp, hp, wt, qsec)) %>%
tab_cols(total(label = "#Total| |"), am) %>%
tab_stat_fun_df(
function(x){
frm = reformulate(".", response = as.name(names(x)[1]))
model = lm(frm, data = x)
sheet('Coef.' = coef(model),
confint(model)
)
}
) %>%
tab_pivot()
```
## Example of data processing with multiple-response variables
Here we use truncated dataset with data from product test of two samples of
chocolate sweets. 150 respondents tested two kinds of sweets (codenames:
VSX123 and SDF546). Sample was divided into two groups (cells) of 75
respondents in each group. In cell 1 product VSX123 was presented first and
then SDF546. In cell 2 sweets were presented in reversed order. Questions
about respondent impressions about first product are in the block A (and
about second tested product in the block B). At the end of the questionnaire
there was a question about the preferences between sweets.
List of variables:
- `id` Respondent Id
- `cell` First tested product (cell number)
- `s2a` Age
- `a1_1-a1_6` What did you like in these sweets? Multiple response. First tested product
- `a22` Overall quality. First tested product
- `b1_1-b1_6` What did you like in these sweets? Multiple response. Second tested product
- `b22` Overall quality. Second tested product
- `c1` Preferences
```R
data(product_test)
w = product_test # shorter name to save some keystrokes
# here we recode variables from first/second tested product to separate variables for each product according to their cells
# 'h' variables - VSX123 sample, 'p' variables - 'SDF456' sample
# also we recode preferences from first/second product to true names
# for first cell there are no changes, for second cell we should exchange 1 and 2.
w = w %>%
let_if(cell == 1,
h1_1 %to% h1_6 := recode(a1_1 %to% a1_6, other ~ copy),
p1_1 %to% p1_6 := recode(b1_1 %to% b1_6, other ~ copy),
h22 := recode(a22, other ~ copy),
p22 := recode(b22, other ~ copy),
c1r = c1
) %>%
let_if(cell == 2,
p1_1 %to% p1_6 := recode(a1_1 %to% a1_6, other ~ copy),
h1_1 %to% h1_6 := recode(b1_1 %to% b1_6, other ~ copy),
p22 := recode(a22, other ~ copy),
h22 := recode(b22, other ~ copy),
c1r := recode(c1, 1 ~ 2, 2 ~ 1, other ~ copy)
) %>%
let(
# recode age by groups
age_cat = recode(s2a, lo %thru% 25 ~ 1, lo %thru% hi ~ 2),
# count number of likes
# codes 2 and 99 are ignored.
h_likes = count_row_if(1 | 3 %thru% 98, h1_1 %to% h1_6),
p_likes = count_row_if(1 | 3 %thru% 98, p1_1 %to% p1_6)
)
# here we prepare labels for future usage
codeframe_likes = num_lab("
1 Liked everything
2 Disliked everything
3 Chocolate
4 Appearance
5 Taste
6 Stuffing
7 Nuts
8 Consistency
98 Other
99 Hard to answer
")
overall_liking_scale = num_lab("
1 Extremely poor
2 Very poor
3 Quite poor
4 Neither good, nor poor
5 Quite good
6 Very good
7 Excellent
")
w = apply_labels(w,
c1r = "Preferences",
c1r = num_lab("
1 VSX123
2 SDF456
3 Hard to say
"),
age_cat = "Age",
age_cat = c("18 - 25" = 1, "26 - 35" = 2),
h1_1 = "Likes. VSX123",
p1_1 = "Likes. SDF456",
h1_1 = codeframe_likes,
p1_1 = codeframe_likes,
h_likes = "Number of likes. VSX123",
p_likes = "Number of likes. SDF456",
h22 = "Overall quality. VSX123",
p22 = "Overall quality. SDF456",
h22 = overall_liking_scale,
p22 = overall_liking_scale
)
```
Are there any significant differences between preferences? Yes, difference is significant.
```R
# 'tab_mis_val(3)' remove 'hard to say' from vector
w %>% tab_cols(total(), age_cat) %>%
tab_cells(c1r) %>%
tab_mis_val(3) %>%
tab_stat_cases() %>%
tab_last_sig_cases() %>%
tab_pivot()
```
Further we calculate distribution of answers in the survey questions.
```R
# lets specify repeated parts of table creation chains
banner = w %>% tab_cols(total(), age_cat, c1r)
# column percent with significance
tab_cpct_sig = . %>% tab_stat_cpct() %>%
tab_last_sig_cpct(sig_labels = paste0("",LETTERS, ""))
# means with siginifcance
tab_means_sig = . %>% tab_stat_mean_sd_n(labels = c("Mean", "sd", "N")) %>%
tab_last_sig_means(
sig_labels = paste0("",LETTERS, ""),
keep = "means")
# Preferences
banner %>%
tab_cells(c1r) %>%
tab_cpct_sig() %>%
tab_pivot()
# Overall liking
banner %>%
tab_cells(h22) %>%
tab_means_sig() %>%
tab_cpct_sig() %>%
tab_cells(p22) %>%
tab_means_sig() %>%
tab_cpct_sig() %>%
tab_pivot()
# Likes
banner %>%
tab_cells(h_likes) %>%
tab_means_sig() %>%
tab_cells(mrset(h1_1 %to% h1_6)) %>%
tab_cpct_sig() %>%
tab_cells(p_likes) %>%
tab_means_sig() %>%
tab_cells(mrset(p1_1 %to% p1_6)) %>%
tab_cpct_sig() %>%
tab_pivot()
# below more complicated table where we compare likes side by side
# Likes - side by side comparison
w %>%
tab_cols(total(label = "#Total| |"), c1r) %>%
tab_cells(list(unvr(mrset(h1_1 %to% h1_6)))) %>%
tab_stat_cpct(label = var_lab(h1_1)) %>%
tab_cells(list(unvr(mrset(p1_1 %to% p1_6)))) %>%
tab_stat_cpct(label = var_lab(p1_1)) %>%
tab_pivot(stat_position = "inside_columns")
```
We can save labelled dataset as *.csv file with accompanying R code for labelling.
```R
write_labelled_csv(w, file filename = "product_test.csv")
```
Or, we can save dataset as *.csv file with SPSS syntax to read data and apply labels.
```R
write_labelled_spss(w, file filename = "product_test.csv")
```
## Export to Microsoft Excel
To export `expss` tables to *.xlsx you need to install excellent `openxlsx` package. To install it just type in the console `install.packages("openxlsx")`.
### Examples
First we apply labels on the mtcars dataset and build simple table with caption.
```R
library(expss)
library(openxlsx)
data(mtcars)
mtcars = apply_labels(mtcars,
mpg = "Miles/(US) gallon",
cyl = "Number of cylinders",
disp = "Displacement (cu.in.)",
hp = "Gross horsepower",
drat = "Rear axle ratio",
wt = "Weight (lb/1000)",
qsec = "1/4 mile time",
vs = "Engine",
vs = c("V-engine" = 0,
"Straight engine" = 1),
am = "Transmission",
am = c("Automatic" = 0,
"Manual"=1),
gear = "Number of forward gears",
carb = "Number of carburetors"
)
mtcars_table = mtcars %>%
cross_cpct(
cell_vars = list(cyl, gear),
col_vars = list(total(), am, vs)
) %>%
set_caption("Table 1")
mtcars_table
```
Then we create workbook and add worksheet to it.
```R
wb = createWorkbook()
sh = addWorksheet(wb, "Tables")
```
Export - we should specify workbook and worksheet.
```R
xl_write(mtcars_table, wb, sh)
```
And, finally, we save workbook with table to the xlsx file.
```R
saveWorkbook(wb, "table1.xlsx", overwrite = TRUE)
```
Screenshot of the exported table:
### Automation of the report generation
First of all, we create banner which we will use for all our tables.
```R
banner = with(mtcars, list(total(), am, vs))
```
Then we generate list with all tables. If variables have small number of discrete values we create column percent table. In other cases we calculate table with means. For both types of tables we mark significant differencies between groups.
```R
list_of_tables = lapply(mtcars, function(variable) {
if(length(unique(variable))<7){
cro_cpct(variable, banner) %>% significance_cpct()
} else {
# if number of unique values greater than seven we calculate mean
cro_mean_sd_n(variable, banner) %>% significance_means()
}
})
```
Create workbook:
```R
wb = createWorkbook()
sh = addWorksheet(wb, "Tables")
```
Here we export our list with tables with additional formatting. We remove '#' sign from totals and mark total column with bold. You can read about formatting options in the manual fro `xl_write` (`?xl_write` in the console).
```R
xl_write(list_of_tables, wb, sh,
# remove '#' sign from totals
col_symbols_to_remove = "#",
row_symbols_to_remove = "#",
# format total column as bold
other_col_labels_formats = list("#" = createStyle(textDecoration = "bold")),
other_cols_formats = list("#" = createStyle(textDecoration = "bold")),
)
```
Save workbook:
```R
saveWorkbook(wb, "report.xlsx", overwrite = TRUE)
```
Screenshot of the generated report:
## Labels support for base R
Variable label is human readable description of the variable. R supports rather long variable names and these names can contain even spaces and punctuation but short variables names make coding easier. Variable label can give a nice, long description of variable. With this description it is easier to remember what those variable names refer to.
Value labels are similar to variable labels, but value labels are descriptions of the values a variable can take. Labeling values means we don’t have to remember if 1=Extremely poor and 7=Excellent or vice-versa. We can easily get dataset description and variables summary with `info` function.
The usual way to connect numeric data to labels in R is factor variables. However, factors miss important features which the value labels provide. Factors only allow for integers to be mapped to a text label, these integers have to be a count starting at 1 and every value need to be labelled. Also, we can’t calculate means or other numeric statistics on factors.
With labels we can manipulate short variable names and codes when we analyze our data but in the resulting tables and graphs we will see human-readable text.
It is easy to store labels as variable attributes in R but most R functions cannot use them or even drop them. `expss` package integrates value labels support into base R functions and into functions from other packages. Every function which internally converts variable to factor will utilize labels. Labels will be preserved during variables subsetting and concatenation. Additionally, there is a function (`use_labels`) which greatly simplify variable labels usage. See examples below.
### Getting and setting variable and value labels
First, apply value and variables labels to dataset:
```R
library(expss)
data(mtcars)
mtcars = apply_labels(mtcars,
mpg = "Miles/(US) gallon",
cyl = "Number of cylinders",
disp = "Displacement (cu.in.)",
hp = "Gross horsepower",
drat = "Rear axle ratio",
wt = "Weight (1000 lbs)",
qsec = "1/4 mile time",
vs = "Engine",
vs = c("V-engine" = 0,
"Straight engine" = 1),
am = "Transmission",
am = c("Automatic" = 0,
"Manual"=1),
gear = "Number of forward gears",
carb = "Number of carburetors"
)
```
In addition to `apply_labels` we have SPSS-style `var_lab` and `val_lab` functions:
```R
nps = c(-1, 0, 1, 1, 0, 1, 1, -1)
var_lab(nps) = "Net promoter score"
val_lab(nps) = num_lab("
-1 Detractors
0 Neutralists
1 Promoters
")
```
We can read, add or remove existing labels:
```R
var_lab(nps) # get variable label
val_lab(nps) # get value labels
# add new labels
add_val_lab(nps) = num_lab("
98 Other
99 Hard to say
")
# remove label by value
# %d% - diff, %n_d% - names diff
val_lab(nps) = val_lab(nps) %d% 98
# or, remove value by name
val_lab(nps) = val_lab(nps) %n_d% "Other"
```
Additionaly, there are some utility functions. They can applied on one variable as well as on the entire dataset.
```R
drop_val_labs(nps)
drop_var_labs(nps)
unlab(nps)
drop_unused_labels(nps)
prepend_values(nps)
```
There is also `prepend_names` function but it can be applied only to data.frame.
### Labels with base R and ggplot2 functions
Base `table` and plotting with value labels:
```R
with(mtcars, table(am, vs))
with(mtcars,
barplot(
table(am, vs),
beside = TRUE,
legend = TRUE)
)
```
There is a special function for variables labels support - `use_labels`. By now variables labels support available only for expression which will be evaluated inside data.frame.
```R
# table with dimension names
use_labels(mtcars, table(am, vs))
# linear regression
use_labels(mtcars, lm(mpg ~ wt + hp + qsec)) %>% summary
# boxplot with variable labels
use_labels(mtcars, boxplot(mpg ~ am))
```
And, finally, `ggplot2` graphics with variables and value labels. Note that with ggplot2 version 3.2.0 and higher you need to explicitly convert labelled variables to factors in the `facet_grid` formula:
```R
library(ggplot2, warn.conflicts = FALSE)
use_labels(mtcars, {
# '..data' is shortcut for all 'mtcars' data.frame inside expression
ggplot(..data) +
geom_point(aes(y = mpg, x = wt, color = qsec)) +
facet_grid(factor(am) ~ factor(vs))
})
```
### Extreme value labels support
We have an option for extreme values lables support: `expss_enable_value_labels_support_extreme()`. With this option `factor`/`as.factor` will take into account empty levels. However, `unique` will give weird result for labelled variables: labels without values will be added to unique values. That's why it is recommended to turn off this option immediately after usage. See examples.
We have label 'Hard to say' for which there are no values in `nps`:
```R
nps = c(-1, 0, 1, 1, 0, 1, 1, -1)
var_lab(nps) = "Net promoter score"
val_lab(nps) = num_lab("
-1 Detractors
0 Neutralists
1 Promoters
99 Hard to say
")
```
Here we disable labels support and get results without labels:
```R
expss_disable_value_labels_support()
table(nps) # there is no labels in the result
unique(nps)
```
Results with default value labels support - three labels are here but "Hard to say" is absent.
```R
expss_enable_value_labels_support()
# table with labels but there are no label "Hard to say"
table(nps)
unique(nps)
```
And now extreme value labels support - we see "Hard to say" with zero counts. Note the weird `unique` result.
```R
expss_enable_value_labels_support_extreme()
# now we see "Hard to say" with zero counts
table(nps)
# weird 'unique'! There is a value 99 which is absent in 'nps'
unique(nps)
```
Return immediately to defaults to avoid issues:
```R
expss_enable_value_labels_support()
```
### Labels are preserved during common operations on the data
There are special methods for subsetting and concatenating labelled variables. These methods preserve labels during common operations. We don't need to restore labels on subsetted or sorted data.frame.
`mtcars` with labels:
```R
str(mtcars)
```
Make subset of the data.frame:
```R
mtcars_subset = mtcars[1:10, ]
```
Labels are here, nothing is lost:
```R
str(mtcars_subset)
```
### Interaction with 'haven'
To use `expss` with `haven` you need to load `expss` strictly after `haven` (or other package with implemented 'labelled' class) to avoid conflicts. And it is better to use `read_spss` with explict package specification: `haven::read_spss`. See example below.
`haven` package doesn't set 'labelled' class for variables which have variable label but don't have value labels. It leads to labels losing during subsetting and other operations. We have a special function to fix this: `add_labelled_class`. Apply it to dataset loaded by `haven`.
```R
# we need to load packages strictly in this order to avoid conflicts
library(haven)
library(expss)
spss_data = haven::read_spss("spss_file.sav")
# add missing 'labelled' class
spss_data = add_labelled_class(spss_data)
```