Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/yjunechoe/jlmerclusterperm
Fast cluster-based permutation test for densely-sampled, multi-level time series data (ex: eyetracking, EEG)
https://github.com/yjunechoe/jlmerclusterperm
cluster-based-permutation-test eeg eyetracking mixed-effects-models timeseries
Last synced: 2 months ago
JSON representation
Fast cluster-based permutation test for densely-sampled, multi-level time series data (ex: eyetracking, EEG)
- Host: GitHub
- URL: https://github.com/yjunechoe/jlmerclusterperm
- Owner: yjunechoe
- License: other
- Created: 2023-03-19T19:49:15.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-11-10T17:48:41.000Z (2 months ago)
- Last Synced: 2024-11-10T18:35:13.206Z (2 months ago)
- Topics: cluster-based-permutation-test, eeg, eyetracking, mixed-effects-models, timeseries
- Language: R
- Homepage: https://yjunechoe.github.io/jlmerclusterperm/
- Size: 19.9 MB
- Stars: 11
- Watchers: 2
- Forks: 1
- Open Issues: 3
-
Metadata Files:
- Readme: README.Rmd
- Changelog: NEWS.md
- Contributing: .github/CONTRIBUTING.md
- License: LICENSE
- Code of conduct: .github/CODE_OF_CONDUCT.md
- Codemeta: codemeta.json
Awesome Lists containing this project
README
---
output: github_document
---
```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.align = "center",
fig.path = "man/figures/README-",
asciicast_theme = if (Sys.getenv("IN_PKGDOWN") == "true") "pkgdown" else "readme"
)
asciicast::init_knitr_engine(
echo = TRUE,
echo_input = FALSE,
timeout = 60,
record_env = c("asciicast_cols" = 80),
same_process = TRUE
)
```
# jlmerclusterperm 
[](https://CRAN.R-project.org/package=jlmerclusterperm)
[](https://yjunechoe.r-universe.dev/jlmerclusterperm)
[](https://github.com/yjunechoe/jlmerclusterperm/actions/workflows/R-CMD-check.yaml)
[](https://github.com/yjunechoe/jlmerclusterperm/actions?query=workflow%3Apkgcheck)
[](https://app.codecov.io/gh/yjunechoe/jlmerclusterperm?branch=main)
[](https://cranlogs.r-pkg.org/badges/grand-total/jlmerclusterperm)
Julia [GLM.jl](https://github.com/JuliaStats/GLM.jl) and [MixedModels.jl](https://github.com/JuliaStats/MixedModels.jl) based implementation of the cluster-based permutation test for time series data, powered by [JuliaConnectoR](https://github.com/stefan-m-lenz/JuliaConnectoR).
```{r, echo = FALSE}
knitr::include_graphics("man/figures/clusterpermute_animation.gif", error = FALSE)
```
## Installation and usage
Install the released version of jlmerclusterperm from CRAN:
```{r, eval = FALSE}
install.packages("jlmerclusterperm")
```
Or install the development version from [GitHub](https://github.com/yjunechoe/jlmerclusterperm) with:
```{r, eval = FALSE}
# install.packages("remotes")
remotes::install_github("yjunechoe/jlmerclusterperm")
```
Using `jlmerclusterperm` requires a prior installation of the Julia programming language, which can be downloaded from either the [official website](https://julialang.org/) or using the command line utility [juliaup](https://github.com/JuliaLang/juliaup). Julia version >=1.8 is required and [1.9](https://julialang.org/blog/2023/04/julia-1.9-highlights/#caching_of_native_code) or higher is preferred for the substantial speed improvements.
Before using functions from `jlmerclusterperm`, an initial setup is required via calling `jlmerclusterperm_setup()`. The very first call on a system will install necessary dependencies (this only happens once and takes around 10-15 minutes).
Subsequent calls to `jlmerclusterperm_setup()` incur a small overhead of around 30 seconds, plus slight delays for first-time function calls. You pay up front for start-up and warm-up costs and get blazingly-fast functions from the package.
```{r setup, include = FALSE}
library(jlmerclusterperm)
jlmerclusterperm_setup(cache_dir = tempdir())
```
```{asciicast setup-io}
# Both lines must be run at the start of each new session
library(jlmerclusterperm)
jlmerclusterperm_setup()
```
See the [Get Started](https://yjunechoe.github.io/jlmerclusterperm/articles/jlmerclusterperm.html) page on the [package website](https://yjunechoe.github.io/jlmerclusterperm/) for background and tutorials.
## Quick tour of package functionalities
### Wholesale CPA with `clusterpermute()`
A time series data:
```{r chickweight, out.width = "75%"}
chickweights <- ChickWeight
chickweights$Time <- as.integer(factor(chickweights$Time))
matplot(
tapply(chickweights$weight, chickweights[c("Time", "Diet")], mean),
type = "b", lwd = 3, ylab = "Weight", xlab = "Time"
)
```
```{asciicast chickweight-io, include = FALSE}
chickweights <- ChickWeight
chickweights$Time <- as.integer(factor(chickweights$Time))
```
Preparing a specification object with `make_jlmer_spec()`:
```{r spec, include = FALSE}
chickweights_spec <- make_jlmer_spec(
formula = weight ~ 1 + Diet,
data = chickweights,
subject = "Chick", time = "Time"
)
```
```{asciicast spec-io}
chickweights_spec <- make_jlmer_spec(
formula = weight ~ 1 + Diet,
data = chickweights,
subject = "Chick", time = "Time"
)
chickweights_spec
```
Cluster-based permutation test with `clusterpermute()`:
```{asciicast JIT, include = FALSE}
clusterpermute(chickweights_spec, threshold = 2.5, nsim = 2)
```
```{asciicast CPA-io}
set_rng_state(123L)
clusterpermute(
chickweights_spec,
threshold = 2.5,
nsim = 100
)
```
Including random effects:
```{asciicast reCPA-io}
chickweights_re_spec <- make_jlmer_spec(
formula = weight ~ 1 + Diet + (1 | Chick),
data = chickweights,
subject = "Chick", time = "Time"
)
set_rng_state(123L)
clusterpermute(
chickweights_re_spec,
threshold = 2.5,
nsim = 100
)$empirical_clusters
```
### Piecemeal approach to CPA
Computing time-wise statistics of the observed data:
```{r empirical_statistics, out.width = "75%"}
empirical_statistics <- compute_timewise_statistics(chickweights_spec)
matplot(t(empirical_statistics), type = "b", pch = 1, lwd = 3, ylab = "t-statistic")
abline(h = 2.5, lty = 3)
```
```{asciicast empirical_statistics-io, include = FALSE}
empirical_statistics <- compute_timewise_statistics(chickweights_spec)
```
Identifying empirical clusters:
```{asciicast empirical_clusters}
empirical_clusters <- extract_empirical_clusters(empirical_statistics, threshold = 2.5)
empirical_clusters
```
Simulating the null distribution:
```{asciicast null_statistics}
set_rng_state(123L)
null_statistics <- permute_timewise_statistics(chickweights_spec, nsim = 100)
null_cluster_dists <- extract_null_cluster_dists(null_statistics, threshold = 2.5)
null_cluster_dists
```
Significance testing the cluster-mass statistic:
```{asciicast calculate_clusters_pvalues}
calculate_clusters_pvalues(empirical_clusters, null_cluster_dists, add1 = TRUE)
```
Iterating over a range of threshold values:
```{asciicast walk_threshold_steps, message = FALSE}
walk_threshold_steps(empirical_statistics, null_statistics, steps = c(2, 2.5, 3))
```
## Acknowledgments
- The paper [Maris & Oostenveld (2007)](https://doi.org/10.1016/j.jneumeth.2007.03.024) which originally proposed the cluster-based permutation analysis.
- The [JuliaConnectoR](https://github.com/stefan-m-lenz/JuliaConnectoR) package for powering the R interface to Julia.
- The Julia packages [GLM.jl](https://github.com/JuliaStats/GLM.jl) and [MixedModels.jl](https://github.com/JuliaStats/MixedModels.jl) for fast implementations of (mixed effects) regression models.
- Existing implementations of CPA in R ([permuco](https://jaromilfrossard.github.io/permuco/), [permutes](https://cran.r-project.org/package=permutes), etc.) whose designs inspired the CPA interface in jlmerclusterperm.
## Citations
If you use jlmerclusterperm for cluster-based permutation test with mixed-effects models in your research, please cite one (or more) of the following as you see fit.
To cite jlmerclusterperm:
- Choe, J. (`r format(Sys.Date(), "%Y")`). jlmerclusterperm: Cluster-Based Permutation Analysis for Densely Sampled Time Data. R package version `r as.character(utils::packageVersion('jlmerclusterperm'))`. [10.32614/CRAN.package.jlmerclusterperm](https://doi.org/10.32614/CRAN.package.jlmerclusterperm).
To cite the cluster-based permutation test:
- Maris, E., & Oostenveld, R. (2007). Nonparametric statistical testing of EEG- and MEG-data. _Journal of Neuroscience Methods, 164_, 177–190. doi: 10.1016/j.jneumeth.2007.03.024.
To cite the Julia programming language:
- Bezanson, J., Edelman, A., Karpinski, S., & Shah, V. B. (2017). Julia: A Fresh Approach to Numerical Computing. _SIAM Review, 59_(1), 65–98. doi: 10.1137/141000671.
To cite the GLM.jl and MixedModels.jl Julia libraries, consult their Zenodo pages:
- GLM: https://doi.org/10.5281/zenodo.3376013
- MixedModels: https://zenodo.org/badge/latestdoi/9106942
```{asciicast close-io, include = FALSE}
JuliaConnectoR::stopJulia()
```
```{r close, include = FALSE}
JuliaConnectoR::stopJulia()
```
```{r srr, include = FALSE}
#' @srrstats {G1.0} References Maris & Oostenveld (2007) which originally proposed the cluster-based permutation analysis.
#' @srrstats {G1.1} Package is an improvement over existing implementations in R (mainly in speed and interpretability).
#' This is explained in the readme and the case study vignettes.
#' @srrstats {G1.2} Lifecycle is active and stable.
#' @srrstats {G1.3} The many moving parts are explained across the readme, the function documentation, and the topics/case study vignettes.
#' Users are assumed to already have familiarity with (genearlized, mixed-effects) regression to use this package.
#' @srrstats {G1.4} `roxygen2` is used throughout the package.
```