https://github.com/poissonconsulting/batchr
An R package to batch process files with a user-supplied R function
https://github.com/poissonconsulting/batchr
batch-processing rstats
Last synced: 10 months ago
JSON representation
An R package to batch process files with a user-supplied R function
- Host: GitHub
- URL: https://github.com/poissonconsulting/batchr
- Owner: poissonconsulting
- License: other
- Created: 2019-07-04T12:28:31.000Z (almost 7 years ago)
- Default Branch: main
- Last Pushed: 2025-06-02T21:46:18.000Z (12 months ago)
- Last Synced: 2025-06-03T12:31:53.164Z (12 months ago)
- Topics: batch-processing, rstats
- Language: R
- Homepage: https://poissonconsulting.github.io/batchr/
- Size: 2.08 MB
- Stars: 6
- Watchers: 5
- Forks: 0
- Open Issues: 2
-
Metadata Files:
- Readme: README.Rmd
- Changelog: NEWS.md
- Contributing: .github/CONTRIBUTING.md
- License: LICENSE
- Code of conduct: .github/CODE_OF_CONDUCT.md
- Support: .github/SUPPORT.md
Awesome Lists containing this project
README
---
output: github_document
---
```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "man/figures/README-",
out.width = "100%"
)
```
# batchr 
[](https://lifecycle.r-lib.org/articles/stages.html#experimental)
[](https://github.com/poissonconsulting/batchr/actions/workflows/R-CMD-check.yaml)
[](https://app.codecov.io/gh/poissonconsulting/batchr)
[](https://opensource.org/license/mit/)
[](https://CRAN.R-project.org/package=batchr)
`batchr` is an R package to batch process files using an R function.
The key design principle is that only files which were last modified *before*
the directory was configured are processed.
A hidden file stores the configuration time and function etc while
successfully processed files are automatically touched to update
their modification date.
As a result batch processing can be stopped and restarted and any files created (or modified or deleted) during processing are ignored.
To allow the user control over the reprocessing of problematic files,
all processing attempts (SUCCESS or FAILURE) are recorded in a hidden log file.
## Installation
You can install the released version of batchr from [CRAN](https://CRAN.R-project.org) with:
``` r
install.packages("batchr")
```
And the development version from [GitHub](https://github.com/poissonconsulting/batchr) with:
``` r
# install.packages("remotes")
remotes::install_github("poissonconsulting/batchr")
```
## Demonstration
Consider a directory with two .csv files
```{r}
path <- file.path(tempdir(), "example")
unlink(path, force = TRUE)
dir.create(path)
write.csv(data.frame(x = 1), file.path(path, "file1.csv"), row.names = FALSE)
write.csv(data.frame(x = 3), file.path(path, "file2.csv"), row.names = FALSE)
```
First define the function to process them.
```{r}
fun <- function(file) {
data <- read.csv(file)
data$x <- data$x * 2
write.csv(data, file, row.names = FALSE)
TRUE
}
```
Then simply call `batch_process()` to apply the function to all the files.
```{r}
library(batchr)
batch_process(fun, path, ask = FALSE)
```
The files have been updated as follows.
```{r}
read.csv(file.path(path, "file1.csv"))
read.csv(file.path(path, "file2.csv"))
```
For a more realistic demonstration with finer control over the batch processing see the [Batchr Demonstration](https://poissonconsulting.github.io/batchr/articles/batchr.html) vignette.
### Parallel Chains
To process the files in parallel simply set
```{r, eval=FALSE}
library(future)
plan(multisession)
```
## Contribution
Please report any [issues](https://github.com/poissonconsulting/batchr/issues).
[Pull requests](https://github.com/poissonconsulting/batchr/pulls) are always welcome.
## Code of Conduct
Please note that the batchr project is released with a [Contributor Code of Conduct](https://contributor-covenant.org/version/2/0/CODE_OF_CONDUCT.html). By contributing to this project, you agree to abide by its terms.