{"id":23442300,"url":"https://github.com/repronim/containers","last_synced_at":"2026-01-20T06:33:29.917Z","repository":{"id":39879609,"uuid":"182005926","full_name":"ReproNim/containers","owner":"ReproNim","description":"Containers \"distribution\" for reproducible neuroimaging","archived":false,"fork":false,"pushed_at":"2026-01-15T20:09:33.000Z","size":5406,"stargazers_count":30,"open_issues_count":45,"forks_count":16,"subscribers_count":6,"default_branch":"master","last_synced_at":"2026-01-15T22:32:42.969Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Roff","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ReproNim.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2019-04-18T02:58:38.000Z","updated_at":"2026-01-15T20:09:40.000Z","dependencies_parsed_at":"2025-11-28T00:07:52.044Z","dependency_job_id":null,"html_url":"https://github.com/ReproNim/containers","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/ReproNim/containers","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ReproNim%2Fcontainers","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ReproNim%2Fcontainers/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ReproNim%2Fcontainers/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ReproNim%2Fcontainers/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ReproNim","download_url":"https://codeload.github.com/ReproNim/containers/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ReproNim%2Fcontainers/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28597640,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-20T02:08:49.799Z","status":"ssl_error","status_checked_at":"2026-01-20T02:08:44.148Z","response_time":117,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-23T17:29:19.270Z","updated_at":"2026-01-20T06:33:29.910Z","avatar_url":"https://github.com/ReproNim.png","language":"Roff","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ReproNim/containers - containerized environments for reproducible neuroimaging\n\n[![CI Status](https://github.com/ReproNim/containers/actions/workflows/base-tests.yaml/badge.svg)](https://github.com/ReproNim/containers/actions/workflows/base-tests.yaml)\n[![Additional tests](https://github.com/ReproNim/containers/workflows/Additional%20tests/badge.svg)](https://github.com/ReproNim/containers/actions?query=workflow%3A%22Additional+tests%22)\n\nThis repository provides a [DataLad] dataset (git/git-annex\nrepository) with a collection of popular computational tools provided\nwithin ready to use containerized environments.  At the moment it\nprovides only [Singularity] images.  Versions of all images are tracked using\n[git-annex] with content of the images provided from a dedicated\n[Singularity Hub Collection] and http://datasets.datalad.org (AKA `///` of\nDataLad) or other original collections.\n\nThe aims for this project is\n\n- to be able to include this repository as a\n  subdataset within larger study (super)datasets to facilitate rapid and\n  reproducible computation, while\n  adhering to [YODA principles] and retaining clear and unambiguous\n  association between data, code, and computing environments using\n  git/git-annex/DataLad;\n- to assist with containers execution in \"sanitized\" environments:  no `$HOME` or\n  system-wide `/tmp` is bind-mounted inside the containers, no\n  environment variables from the host system made available inside;\n- make Singularity images transparently usable on non-Linux (OSX) systems via\n  [Docker].\n\n![ReproNim/containers as a YODA building block](https://raw.githubusercontent.com/ReproNim/containers-artwork/master/repronim-containers-yoda_30dpi.png)\n\nAll images are \"registered\" within the dataset for execution using\n[datalad containers-run], so it is trivial to list available\ncontainers:\n\n```shell\n$\u003e datalad containers-list\narg-test -\u003e scripts/tests/arg-test.simg\nbids-aa -\u003e images/bids/bids-aa--0.2.0.sing\nbids-afni-proc -\u003e images/bids/bids-afni-proc--0.0.2.sing\nbids-antscorticalthickness -\u003e images/bids/bids-antscorticalthickness--2.2.0-1.sing\nbids-baracus -\u003e images/bids/bids-baracus--1.1.2.sing\nbids-brainiak-srm -\u003e images/bids/bids-brainiak-srm--latest.sing\n...  many more to list them all ...\n```\n\nand execute either via `datalad containers-run` (which would also take care\nabout getting them first if not present):\n\n```shell\n$\u003e datalad containers-run -n bids-validator -- --help\n[INFO   ] Making sure inputs are available (this may take some time)\n[INFO   ] == Command start (output follows) =====\nUsage: bids-validator \u003cdataset_directory\u003e [options]\n\nOptions:\n  --help, -h            Show help                                      [boolean]\n  --version, -v         Show version number                            [boolean]\n  --ignoreWarnings      Disregard non-critical issues                  [boolean]\n  --ignoreNiftiHeaders  Disregard NIfTI header content during validation\n                                                                       [boolean]\n  --verbose             Log more extensive information about issues    [boolean]\n  --json                Output results as JSON                         [boolean]\n  --config, -c          Optional configuration file. See\n                        https://github.com/bids-standard/bids-validator for more\n                        info\n\nThis tool checks if a dataset in a given directory is compatible with the Brain\nImaging Data Structure specification. To learn more about Brain Imaging Data\nStructure visit http://bids.neuroimaging.io\n[INFO   ] == Command exit (modification check follows) =====\naction summary:\n  get (notneeded: 1)\n  save (notneeded: 1)\n```\n\nor first getting them using [datalad get] and then either using\n`singularity` `run` or `exec` directly, or (recommended) via\n[scripts/singularity_cmd](). That is the helper which is used by\n`containers-run` (see [.datalad/config]()).\n\n## [scripts/singularity_cmd]()\n\nSingularity execution by default is optimized for convenience and not for reproducibility.\nThis helper script assists in making singularity execution reproducible by\n\n- disabling passing environment variables inside your containerized environment\n- creating temporary `/tmp` directory for the environment, so there is no\n  interaction with file paths outside of the current directory (which should\n  ideally be a DataLad dataset)\n- using custom and nearly empty [binds/HOME]() HOME directory, so there is\n  no possible leakage of locally user-level installed Python and other modules\n  to affect your computation\n\nThe [binds/HOME]() also provides a custom minimalistic [.bashrc](binds/HOME/.bashrc) file\nwith e.g. a customized prompt to inform you about which image you are in ATM for use\nin interactive sessions:\n\n    $\u003e scripts/singularity_cmd exec images/repronim/repronim-reproin--0.5.4.sing bash\n    singularity:repronim-reproin--0.5.4 \u003e yoh@hopa:/home/yoh/proj/repronim/containers$ heudiconv --version\n    0.5.4\n\n### Singularity via Docker\n\nOn non-Linux systems, or if `REPRONIM_USE_DOCKER` environment variable is set to a non-empty value,\n[scripts/singularity_cmd]() will use [Docker] shim image (in privileged mode) to run\nsingularity within it.  All necessary paths will be bind mounted as with a regular direct execution using\nsingularity.\n\n### Resource Monitoring with Duct\n\n`repronim/containers` supports optional resource monitoring using [duct](https://github.com/con/duct), a lightweight wrapper that collects execution data including CPU usage, memory consumption, and runtime statistics.\n\n**Note:** Duct integration is currently only supported with direct Singularity execution, not with Docker mode.\n\nDuct integration requires the `con-duct` package:\n\n```bash\npip install con-duct\n```\n\nTo enable resource monitoring, set the `REPRONIM_USE_DUCT` environment variable:\n\n```bash\nexport REPRONIM_USE_DUCT=1\ndatalad containers-run -n bids-mriqc --input sourcedata/raw --output . \\\n    '{inputs}' '{outputs}' participant group -w workdir\n```\n\nThis will wrap your container execution with duct monitoring and automatically generate detailed resource usage logs.\n\nSee the [duct documentation](https://github.com/con/duct/blob/main/README.md) for configuration options and tools for working with duct logs.\n\n### Interactive sessions\n\nSee [WiP PR #9](https://github.com/ReproNim/containers/pull/9) to\nestablish \"reproducible interactive sessions\" with the help of that script.\n\n# Conventions\n\n## Container image files\n\nSingularity image files have [`.sif` extension](https://github.com/apptainer/sif)\n(older ones created using Singularity \u003c 3, will have `.sing`).\n\n# A typical YODA workflow\n\nLets summarize YODA principles as a possible workflow:\n\n- create a new dataset which would contain results and everything needed\n    to obtain them\n- install/add subdatasets(code, other datasets, containers)\n- perform the analysis using **only** materials available within the reach of this dataset.\n\n\nLet's assume that our goal is to do Quality Control of an MRI dataset\n(which is available as DataLad dataset ds000003). We will create a new\ndataset with the output of the QC results (as analyzed by mriqc\nBIDS-App). mriqc is provided by the ReproNim/containers dataset of\ncontainers. Below, we execute a simple analysis workflow which\nadheres to YODA principles and we **end up with a dataset that contains\nall components necessary a history of how it was achieved.**\n\nThis would help to guarantee reproducibility in the future because all the\nmaterials would be *reachable* within that dataset.\n\n\n## Runnable script\n\nFor advanced users who are comfortable with DataLad, the following\nscript may give you everything you need.\n\n\u003cdetails\u003e\n\u003csummary\u003eThe version of the script with all commands explained\u003c/summary\u003e\n\n```shell\n#!/bin/sh\n(  # so it could be just copy pasted or used as a script\nPS4='\u003e '; set -xeu  # to see what we are doing and exit upon error\n# Work in some temporary directory\ncd $(mktemp -d ${TMPDIR:-/tmp}/repro-XXXXXXX)\n# Create a dataset to contain mriqc output\ndatalad create -d ds000003-qc -c text2git\ncd ds000003-qc\n# Install our containers collection:\ndatalad install -d . -s ///repronim/containers code/containers\n# Optionally -- copy container of interest definition to the current (or desired)\n# version # to facilitate reproducibility while still being able to upgrade containers\n# subdataset if so desired to get access to newer versions.\n# We will also use 0.16.0 since newer ones require more memory and\n# would fail to run on CI.\ndatalad run -m \"Downgrade/Freeze mriqc container version\" \\\n  code/containers/scripts/freeze_versions --save-dataset=. bids-mriqc=0.16.0\n# That version of mriqc does not have an option  --no-datalad-get  we had to\n# hardcode for mriqc to workaround an issue. So let's remove it\ndatalad run -m \"Remove ad-hoc option for mriqc for older frozen version\" sed -i -e 's, --no-datalad-get,,g' .datalad/config\n# Install input data:\ndatalad install -d . -s https://github.com/ReproNim/ds000003-demo sourcedata/raw\n# Setup git to ignore workdir to be used by pipelines\necho \"workdir/\" \u003e .gitignore \u0026\u0026 datalad save -m \"Ignore workdir\" .gitignore\n# Execute desired preprocessing while creating a provenance record\n# in git history\ndatalad containers-run \\\n        -n bids-mriqc \\\n        --input sourcedata/raw \\\n        --output . \\\n        '{inputs}' '{outputs}' participant group -w workdir\n)\n\n```\n\u003c/details\u003e\n\n## Walkthrough\n\nFor users who are new to these components, we will walk through how\nthese components are used together in a typical YODA workflow.\nthe steps\n\n```bash\nmkdir ~/my-experiments\ncd ~/my-experiments\ndatalad create -d ds000003-qc -c text2git\ncd ds000003-qc\n```\n\nDataLad has created a new directory for our results, `ds000003-qc`.\nAccording to YODA principles, this dataset should also contain our input\ndata, code, and anything else we need to run the analysis.\n\nInstall the input dataset:\n\n```bash\ndatalad install -d . -s https://github.com/ReproNim/ds000003-demo sourcedata/raw\n```\n\nNext we install the `ReproNim/containers` collection.\n\n```bash\ndatalad install -d . -s ///repronim/containers code/containers\n```\n\nNow let's take a look at what we have.\n\n```\nds000003-qc/  # The root dataset contains everything\n |- sourcedata/\n |  \\- raw/  # we call it source, but it is actually ds000003-demo \"raw\" BIDS dataset\n \\- code/\n    \\- containers/  # repronim/containers, this is where our non-custom code lives\n```\n\n### Freezing Container Image Versions\n\n`freeze_versions` is an optional step that will record and \"freeze\" the\nversion of the container used. Even if the `///repronim/containers` dataset is\nupgraded with a newer version of our container, we are \"pinned\" to the\ncontainer we explicitly determined. Note: To switch version of the container\n(e.g., to upgrade to a new one), rerun `freeze_versions` script with the version\nspecified.\n\nThe container version can be \"frozen\" into the clone of the `///repronim/containers`\ndataset, **or** the top-level dataset.\n\n\n**Option 1: Top level dataset (recommended)**\n\n```bash\n# Run from ~/my-experiments/ds000003-qc\ndatalad run -m \"Downgrade/Freeze mriqc container version\" \\\n  code/containers/scripts/freeze_versions --save-dataset=. bids-mriqc=0.16.0\n```\n\n\n\n**Option 2: ///repronim/containers**\n\n```bash\n# Run from ~/my-experiments/ds000003-qc/\ndatalad run -m \"Downgrade/Freeze mriqc container version\" \\\n    code/containers/scripts/freeze_versions bids-mriqc=0.16.0\n```\n\nNote: It is recommended to freeze a container image version into the\ntop-level dataset to simplify reuse. If `///repronim/containers` is\nmodified in any way, the author must ensure that their altered fork of\n`///repronim/containers` is publicly available and that its URL\nspecified in the `.gitmodules`. By freezing into the top-level dataset\ninstead, authors do not need to host a modified version of\n`///reporonim/containers`.\n\n### Fixup datalad config\n\nThe version of mriqc we are using does not have an option  `--no-datalad-get` which is hardcoded\ninto mriqc config, so we should remove it.\n\n```bash\ndatalad run -m \"Remove ad-hoc option for mriqc for older frozen version\" sed -i -e 's, --no-datalad-get,,g' .datalad/config\n```\n\n### Running the Containers\n\nWhen we run the bids-mriqc container, it will need a working directory\nfor intermediate files. These are not helpful to commit, so we will\ntell `git` (and `datalad`) to ignore the whole directory.\n\n```bash\necho \"workdir/\" \u003e .gitignore \u0026\u0026 datalad save -m \"Ignore workdir\" .gitignore\n```\n\nNow we use `datalad containers-run` to perform the analysis.\n\n```bash\ndatalad containers-run \\\n        -n bids-mriqc \\\n        --input sourcedata/raw \\\n        --output . \\\n        '{inputs}' '{outputs}' participant group -w workdir\n```\n\nIf everything worked as expected, we will now see our new analysis, and\na commit message of how it was obtained! All of this is contained within\na single (nested) dataset with a complete record of how all the data was\nobtained.\n\n```shell\n(git) .../ds000003-qc[master] $ git show --quiet\nAuthor: Austin \u003caustin@dartmouth.edu\u003e\nDate:   Wed Jun 5 15:41:59 2024 -0400\n\n    [DATALAD RUNCMD] ./code/containers/scripts/singularity_cm...\n\n    === Do not change lines below ===\n    {\n     \"chain\": [],\n     \"cmd\": \"./code/containers/scripts/singularity_cmd run code/containers/images/bids/bids-mriqc--0.16.0.sing '{inputs}' '{outputs}' participant group -w workdir\",\n     \"dsid\": \"c9c96ab9-f803-43ba-83e2-2eaec7ab4725\",\n     \"exit\": 0,\n     \"extra_inputs\": [\n      \"code/containers/images/bids/bids-mriqc--0.16.0.sing\"\n     ],\n     \"inputs\": [\n      \"sourcedata/raw\"\n     ],\n     \"outputs\": [\n      \".\"\n     ],\n     \"pwd\": \".\"\n    }\n    ^^^ Do not change lines above ^^^\n```\n\nThis record could later be reused (by anyone) using [datalad rerun] to rerun\nthis computation using exactly the same version(s) of input data and the\nsingularity container. You can even now [datalad uninstall] sourcedata/raw and even containers\nsub-datasets to save space - they will be retrievable at those exact versions later\non if you need to extend or redo your analysis.\n\n#### Notes:\n\n- aforementioned example requires DataLad \u003e= 0.11.5 and datalad-containers \u003e= 0.4.0;\n- for more eleborate example with use of [reproman] to parallelize execution on\n  remote resources, see [ReproNim/reproman PR#438](https://github.com/ReproNim/reproman/pull/438);\n- a copy of the dataset is made available from [`///repronim/ds000003-qc`](http://datasets.datalad.org/?dir=/repronim/ds000003-qc)\n  and [https://github.com/ReproNim/ds000003-qc]().\n- if you would like to create `licenses/` folder in your project datasets\n  to e.g. contain license for freesurfer, then you better add them to git-annex.\n  Following commands provide one way to do it:\n\n```shell\nmkdir licenses\n# instruct git-annex to add license files to annex, but this added file with instructions to git\necho -e '* annex.largefiles=anything\\n.gitattributes annex.largefiles=nothing' \u003e licenses/.gitattributes\ndatalad save -m \"Add licenses must go into git-annex so I could avoid sharing them\" licenses/.gitattributes\ncp ~/.freesurfer-license licenses/freesurfer\ndatalad save -m 'added freesurfer license' licenses/freesurfer\n```\n\n\n\n# Installation\n\nIt is a DataLad dataset, so you can either just [git clone] or [datalad install] it.\nYou will need to have [git-annex] available to retrieve any images. And you will\nneed DataLad and [datalad-container] extension installed for [datalad containers-run].\nSince Singularity is Linux-only application, it will be \"functional\" only on\nLinux. On OSX (and possibly Windows), if you have Docker installed, singularity\nimages will be executed through the provided docker shim image.\n\n## Environment variables\n\nA few environment variables (in addition to those consulted by datalad\nand datalad-container) are considered in the scripts of this\nrepository:\n\n### `SINGULARITY_CMD`\n\nThe default command (as \"hardcoded\" in [.datalad/config]()) is `run`\nso running the container executes its default \"entry point\".  Setting\n`SINGULARITY_CMD=exec` makes it possible to run an alternative command\nin them (e.g. `bash` for interactive sessions)::\n\n    SINGULARITY_CMD=exec datalad containers-run --explicit -n repronim-reproin bash\n\nand then have `datalad` record any of the introduced changes.  Such\nruns will not be reproducible but at least clearly annotated in what\nenvironment corresponding actions were taken.\n\n# Acknowledgements\n\n## Grants\n\nDevelopment of this project and [datalad-container] extension was supported by the ReproNim project\n([NIH 1P41EB019936-01A1](https://projectreporter.nih.gov/project_info_description.cfm?projectnumber=1P41EB019936-01A1)).\nDataLad development was supported by a US-German collaboration in computational neuroscience (CRCNS) \"DataGit: converging catalogues, warehouses, and deployment logistics into a federated 'data distribution'\" (Halchenko/Hanke), co-funded by the US National Science Foundation ([NSF 1429999](https://www.nsf.gov/awardsearch/showAward?AWD_ID=1429999)) and the German Federal Ministry of Education and Research (BMBF 01GQ1411). Additional support is provided by the German federal state of Saxony-Anhalt and the European Regional Development Fund, Project: Center for Behavioral Brain Sciences, Imaging Platform.\n\n## Copyrighted works\n\nAll container images are collections of various projects governed by the\ncorresponding copyrights/licenses.  Some are not completely FOSS and might\nrequire additional license(s) to be obtained and provided (e.g. FreeSurfer\nlicense for `fmriprep`).\n\n### `artwork/repronim-containers-yoda_*`\n\nBased on the artwork Copyright 2018-2019 Michael Hanke, from\n[myyoda/poster](https://github.com/myyoda/poster), distributed under [CC BY](https://creativecommons.org/licenses/by/4.0/).\n\n# References\n\n## Related resources and articles\n\n- https://www.augmentedmind.de/2025/03/30/the-9-best-docker-registry-tools/\n- https://oci.dag.dev/ - explore docker registries interactively\n- https://github.com/oras-project/oras-py  - Python interface to interact with registries\n\n[git-annex]: http://git-annex.branchable.com\n[DataLad]: http://datalad.org\n[datalad-container]: http://docs.datalad.org/projects/container\n[datalad containers-run]: http://docs.datalad.org/projects/container/en/latest/generated/man/datalad-containers-run.html\n[datalad get]: http://docs.datalad.org/en/stable/generated/man/datalad-get.html\n[datalad run]: http://docs.datalad.org/en/stable/generated/man/datalad-run.html\n[datalad rerun]: http://docs.datalad.org/en/stable/generated/man/datalad-rerun.html\n[datalad install]: http://docs.datalad.org/en/stable/generated/man/datalad-install.html\n[datalad uninstall]: http://docs.datalad.org/en/stable/generated/man/datalad-uninstall.html\n\n[git clone]: https://git-scm.com/docs/git-clone\n\n[Docker]: http://docker.com\n[reproman]: http://reproman.repronim.org\n\n[YODA principles]: https://github.com/myyoda/poster/blob/master/ohbm2018.pdf\n\n[Singularity]: https://www.sylabs.io/singularity/\n[Singularity Hub]: https://singularity-hub.org\n[Singularity Hub Collection]: https://www.singularity-hub.org/collections/2761\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frepronim%2Fcontainers","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frepronim%2Fcontainers","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frepronim%2Fcontainers/lists"}