{"id":40802710,"url":"https://github.com/cvmfs/cvmfsexec","last_synced_at":"2026-01-21T20:45:45.665Z","repository":{"id":40790901,"uuid":"181965675","full_name":"cvmfs/cvmfsexec","owner":"cvmfs","description":"Mount cvmfs repositories as an unprivileged user","archived":false,"fork":false,"pushed_at":"2025-11-03T16:00:09.000Z","size":292,"stargazers_count":29,"open_issues_count":6,"forks_count":16,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-11-03T17:28:20.860Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cvmfs.png","metadata":{"files":{"readme":"README.md","changelog":"ChangeLog","contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2019-04-17T20:45:32.000Z","updated_at":"2025-11-03T15:58:58.000Z","dependencies_parsed_at":"2024-11-14T16:31:31.614Z","dependency_job_id":"f8085ed6-520a-4c07-bbbb-6994aa5156b9","html_url":"https://github.com/cvmfs/cvmfsexec","commit_stats":null,"previous_names":[],"tags_count":63,"template":false,"template_full_name":null,"purl":"pkg:github/cvmfs/cvmfsexec","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvmfs%2Fcvmfsexec","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvmfs%2Fcvmfsexec/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvmfs%2Fcvmfsexec/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvmfs%2Fcvmfsexec/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cvmfs","download_url":"https://codeload.github.com/cvmfs/cvmfsexec/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvmfs%2Fcvmfsexec/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28642301,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-21T18:04:35.752Z","status":"ssl_error","status_checked_at":"2026-01-21T18:03:55.054Z","response_time":86,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-21T20:45:45.525Z","updated_at":"2026-01-21T20:45:45.659Z","avatar_url":"https://github.com/cvmfs.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# cvmfsexec package\n\nWhenever possible it is best to install standard\n[cvmfs](https://cernvm.cern.ch/fs/) from native OS packages, but\nsometimes that is not an option.\nThis package is for mounting cvmfs as an unprivileged user, without the\ncvmfs package being installed by a system administrator.  The package can\ndo this in 4 different ways:\n\n1. On systems where only fusermount is available, the `mountrepo` and\n   `umountrepo` commands can be used to mount cvmfs repositories in the\n   user's own file space.  That path can then be bindmounted at /cvmfs\n   by a container manager such as\n   [apptainer](https://github.com/apptainer/apptainer)\n   (formerly known as singularity).\n   A big disadvantage compared to mode 3 below is that if the processes\n   are hard-killed (kill -9), mountpoints are left behind and difficult\n   to clean up.\n2. On systems where fusermount is available and unprivileged user\n   namespaces are enabled, but unprivileged namespace fuse mounts are not\n   available (in particular RHEL \u003c=7.7 with\n   `sysctl user.max_user_namespaces` \u003e 0),\n   the `cvmfsexec` command can mount cvmfs repositories, map them into\n   /cvmfs, and unmount them when it exits.  apptainer may even be\n   run unprivileged from cvmfs from within cvmfsexec (it has to run\n   unprivileged because setuid-root does not work inside a user\n   namespace).  \n   This mode shares the disadvantage that mode 1 has compared to mode 3\n   in that if the processes are hard-killed, mountpoints are left behind.\n3. On systems where unprivileged namespace fuse mounts are available\n   (newer kernels \u003e= 4.18 as on RHEL8 or \u003e= 3.10.0-1127 as on RHEL 7.8),\n   the `cvmfsexec` command can entirely manage mounting and unmounting\n   of cvmfs repositories in the namespace, so if they get killed\n   everything gets cleanly unmounted.  fusermount is not needed in this\n   case.\n4. On systems that have no fusermount nor unprivileged user namespace\n   fuse mounts but do have a setuid installation of singularity \u003e= 3.4\n   or apptainer,\n   an entirely separate command in this package `singcvmfs` can mount\n   cvmfs repositories inside a container using the `--fusemount` feature.\n   With singularity \u003e= 3.6 and RHEL \u003e= 7.8 and\n   unprivileged user namespaces enabled,\n   this can also be used with unprivileged singularity or apptainer.\n\n# Supported operating systems\n\nOperating systems currently supported by this package are Red Hat\nEnterprise Linux (versions 7, 8, and 9) and its derivatives (CentOS,\nScientific Linux, Rocky Linux, Alma Linux)\nand SUSE Linux Enterprise (version 15)\nand its derivatives (openSUSE Leap).  \nDebian (11 and 12) and Ubuntu (22.04 \u0026 24.04) are also supported\nwith modes 1 to 3,\nby using rhel8 and rhel9 binaries;\nthose binaries aren't compatible enough for mode 4, however.\nAll of those support the x86_64 architecture,\nand rhel8 also supports ppc64le and aarch64.\n\nEven though RHEL7 is now officially End of Life, cvmfsexec will still\nsupport it for a while because some people continue to use it with\nextended support.\n\n# Making the cvmfs distribution\n\nAll of the ways this package supports unprivileged cvmfs make use of a\ncopy of the cvmfs software.  The cvmfs software and configuration are\nexpected to be in a `dist` subdirectory under where the scripts are.  The\neasiest way to create the dist directory is to use `makedist`.  It takes a\nparameter of `osg`, `egi`, or `default` to download the latest cvmfs and\nconfiguration rpm from one of those three sources. Note: `egi` does not\ncurrently provide rpms for RHEL8 or 9. Additionally, specifying `none`\nwill download from the `default` source but exclude the\n`cvmfs-config-default` package.\n\nThe `makedist` command does require 3 tools that are not always\ninstalled on Linux distributions: `curl`, `rpm2cpio`, and `cpio`.\nIf those cannot be installed, you should be able to run the makedist\ncommand on a host that does have them and copy the distribution to\nthe system where you need it.\n\nBy default a distribution for `cvmfsexec` and `mountrepo/umountrepo` is\ncreated.  To instead make a distribution for `singcvmfs`, add the `-s`\nmakedist option.  By default the distribution made will match the \noperating system it runs on, but another distribution can be selected\nwith the `-m` option.  See the makedist usage for supported machine types.\n\nTo customize any cvmfs configuration settings, put them in\n`dist/etc/cvmfs/default.local`.  In particular you may want to set\nCVMFS_HTTP_PROXY, although the default is to use WLCG Web Proxy Auto\nDiscovery.  You may also want to set CVMFS_QUOTA_LIMIT, otherwise the\ndefault is 4000 MB.  The default CVMFS_CACHE_BASE for the cache\nshared between the cvmfs repository is under the dist directory,\n`dist/var/lib/cvmfs`, unless the `-m` option is used to add an e2fs\nfilesystem (details [below](#optionally-mount-a-scratch-filesystem)).\nMake sure that the cache does not get shared between multiple machines.\n\n## Self-contained distribution\n\nFor the cases where `cvmfsexec` or `singcvmfs` can be used, you can also\nmake a self-contained distribution in a single file that contains both\nthe command and all supporting files.  This makes it easy to share the\ndistribution with other users or distribute it to many machines.\n\nAfter running makedist and making any customizations you want, use\nthis to make a cvmfsexec distribution:\n```\nmakedist -o /tmp/cvmfsexec\n```\nor this to make a singcvmfs distribution:\n```\nmakedist -s -o /tmp/singcvmfs\n```\n\nExecuting a cvmfsexec file that is created in that way leaves behind a\n.cvmfsexec directory in the directory where it is run from, and running\na singcvmfs file leaves behind a .singcvmfs directory.\n\n# cvmfsexec command (modes 2 and 3)\n\nThe cvmfsexec command requires unprivileged user namespaces.  On RHEL8/9\nunprivileged user namepaces (and user namespace fuse mounts) are\navailable by default, but on RHEL7 they need to be enabled by setting a\nsysctl parameter as detailed in the\n[OSG unprivileged apptainer instructions](https://osg-htc.org/docs/worker-node/install-apptainer/#enabling-unprivileged-apptainer).\nIn addition cvmfsexec requires fusermount on kernels older than\nthose that come with RHEL7.8.\n\nTo execute a command in an environment where cvmfs repositories are\nmounted at /cvmfs and automatically unmounted upon exit, use\n`cvmfsexec repository.name ... -- [command]` where the default command\nis $SHELL.  It will automatically mount the configuration repository if\none is defined.\n\nUnless disabled with the `-N` option,\ninside the command you can mount additional repositories by using\n`$CVMFSMOUNT repository.name`.  Since the mounts have to happen outside\nthe user namespace, it actually sends a message to the original process\nto mount, and makes the current process wait until completion.\nRepositories that are already mounted are ignored.  You can also unmount\nrepositories from within the command with `$CVMFSUMOUNT repository.name`.\nIf you want to use this feature and also\ninvoke additional processes within the original process that are\nnot trustworthy, such as user payloads that are invoked with\n`--contain` option of singularity or apptainer,\nthen close the $CVMFSEXEC_CMDFD file descriptor\nfor those processes.  This can be done in bash with\n`exec {CVMFSEXEC_CMDFD}\u003e\u0026-`.\n\nNote that setuid-root programs do not work inside an unprivileged user\nnamepace, so if you use singularity or apptainer it has to be run unprivileged.\n\nCache considerations: by default cvmfsexec starts a cache manager\nprocess for all the cvmfs repositories it mounts, which means only one\ncvmfsexec process can share a cache on a single machine.  The cvmfs\nconfiguration could be set to use a different path for a cache for\ndifferent invocations (the default is `dist/var/lib/cvmfs`), or it could\nbe set to use cvmfs alien cache mode which doesn't use a cache manager,\nbut the best approach is to start cvmfsexec from a pilot process and run\nonly one pilot per machine.  If possible the cache should be on local\ndisk, because otherwise the many file accesses can overwhelm a shared\nfilesystem's metadata server.  Also, many network filesystems types\ncannot handle the locks that cvmfs creates.\nIf there is no local disk the next best\noption is to mount a filesystem separately for each worker\nnode from a big enough file on the shared filesystem, or alternatively\na RAM disk on the local node if there is sufficient RAM.\nThe `-m` option (described in the next section) can mount that separate\nscratch filesystem for you in the cvmfsexec namespace.\nOtherwise, if the cache directory needs to be changed that can be done\nby setting CVMFS_CACHE_BASE in `dist/etc/cvmfs/default.local`,\nor you can just install the whole distribution in the local filesystem.\n\n## Optionally mount a scratch filesystem\n\nIf there is no local disk available, cvmfsexec can mount a scratch\next2/3/4 filesystem with the `-m` option, using the fuse2fs command.\nThe filesystem will appear in the cvmfsexec namespace at `/e2fs`.\nIf you have not set CVMFS_CACHE_BASE in `dist/etc/cvmfs/default.local`\nthen this filesystem will automatically be used for cvmfs cache, and\nit can also be used as scratch workspace for jobs.\n\nMake sure that there is a unique file for each running copy of cvmfsexec.\nCreate the file with commands like this:\n```\ntruncate -s 600G scratch.img\nmkfs.ext4 -F -O ^has_journal scratch.img\n```\nChoose a count of the number of gigabytes you want.  The default cache\nsize is 4000 megabytes, and the recommendation is to reserve an\nadditional 1000 megabytes plus 20%, so make it at least 6 gigabytes\nand add any additional space you want to use as scratch workspace.\nIt's a sparse file so there's little penalty for creating it bigger if\nthe space is never used.\nThen start cvmfsexec with `-m`.  For example, to mount only the\ncvmfs configuration repository and run a shell do\n```\ncvmfsexec -m scratch.img --\n```\nThen check out `/e2fs`.\n\nNOTE: although this functions easily and well, fuse2fs performance is\nnot great so if a more performant option is available that is likely\nto be preferred.\n\n## Better cvmfsexec operation on newer kernels (mode 3)\n\nA caveat on older kernels (for example RHEL7.7 and older) is that a\nkill -9  of all the processes will not clean up the mounts, and they\nhave to be separately unmounted later with `umountrepo` or `fusermount -u`.\nOn kernels \u003e= 4.18 (for example RHEL8) or \u003e= 3.10.0-1127 (for example on\nRHEL7.8) the operation changes to do fuse mounts only inside of\nunprivileged user namespaces, which always completely cleans up mounts\neven with kill -9.  This also normally uses a pid namespace to ensure\nthat all fuse processes are always cleaned up when the command exits\n(the exception is when running under docker and kubernetes default\nconfigurations that \"mask\" /proc).\n\n$CVMFSMOUNT/$CVMFSUMOUNT still send a request to a parent process to\nmount/umount but it's not the original process, it's an intermediate\nprocess that has fakeroot access in the user namespace.\n\n## mountrepo/umountrepo without cvmfsexec (mode 1)\n\nWhen not using cvmfsexec, but with fusermount available use\n`mountrepo repository.name` to mount a repository.  Note that the osg\nconfiguration requires \"config-osg.opensciencegrid.org\" to be mounted\nfirst, and the egi configuration requires \"config-egi.egi.eu\".\n\nIf you are using a container system, bind mount $PWD/dist/cvmfs into the\ncontainer as /cvmfs.\n\nTo unmount all repositories, use `umountrepo -a`, or to unmount an\nindividual repository use `umountrepo repository.name`.  Make sure that\nall the processes do not get killed or the repositories will remain\nmounted but inaccessible.\n\nCache considerations for this mode are the same as with the cvmfsexec\ncommand.\n\n## Debugging\n\nSyslog messages from cvmfs go in the `log` subdirectory alongside\n`dist`.  A separate log file is created for each repository.  cvmfs\ndebugging logs can also be enabled in the usual way, by setting\nCVMFS_DEBUGLOG in the cvmfs configuration.\n\n## Running from docker\n\nDocker supports unprivileged user namespaces including unprivileged fuse\nmounts on the kernels that support it, without using the `--privileged`\noption or adding capabilities.  The following set of docker options is\nsufficient:\n\n```\n--security-opt seccomp=unconfined --security-opt systempaths=unconfined --device=/dev/fuse\n```\n\nIf you have no need for running any setuid executables in docker then\nyou can improve security further by adding:\n```\n--security-opt no-new-privileges\n```\nSingularity and apptainer always have the equivalent protection enabled for the\ncontainers they run.\n\n# singcvmfs command (mode 4)\n\nWhen a privileged setuid installation of singularity \u003e= 3.4 or\nany version of apptainer is\navailable, the `singcvmfs` command can be used to mount cvmfs\nrepositories inside a container.  With singularity \u003e= 3.6 and\nRHEL \u003e= 7.8 or a kernel \u003e= 4.18\nwith unprivileged user namespaces enabled\nthis can also be used with an\nunprivileged non-setuid singularity or apptainer installation.\nThe command line interface is different than cvmfsexec because it is\ndesigned for ease of use by end users on a laptop/desktop and as a\ndrop-in replacement for singularity when it executes containers.\n\nPut cvmfs repositories to mount comma-separated in a\n`SINGCVMFS_REPOSITORIES` environment variable.  If a configuration\nrepository is needed it will be automatically mounted.  Then you can use\nsingcvmfs exactly like singularity with one of its exec, instance, run,\nor shell commands (note: it cannot read an image from cvmfs).  For example,\nonce you have [made a singcvmfs distribution](#making-the-cvmfs-distribution)\nthe following should work (replace `centos:7` with a container matching\nthe operating system you're running on):\n\n```\n$ export SINGCVMFS_REPOSITORIES=\"grid.cern.ch,atlas.cern.ch\"\n$ singcvmfs -s exec -cip docker://centos:7 bash\nSingularity\u003e ls /cvmfs\natlas.cern.ch  config-osg.opensciencegrid.org  grid.cern.ch\nSingularity\u003e ls /cvmfs/atlas.cern.ch\nrepo\nSingularity\u003e exit\n\n# or using singularity instances:\n$ export SINGCVMFS_REPOSITORIES=\"grid.cern.ch,atlas.cern.ch\"\n$ singcvmfs -s instance start docker://centos:7 myexampleinstance\n$ singcvmfs -s run instance://myexampleinstance ls /cvmfs\natlas.cern.ch  config-osg.opensciencegrid.org  grid.cern.ch\n$ singcvmfs -s run instance://myexampleinstance ls /cvmfs/atlas.cern.ch\nrepo\n$ singcvmfs -s instance stop myexampleinstance\n```\n\nThe first time you run the above it will take a long time as singularity\ndownloads the image from dockerhub and cvmfs fills its cache, but\nrunning it again should be very fast.\n\nAlternatively, to make it easier to execute repeatedly interactively\nfrom the command line, you can put the singularity container path in a\n`SINGCVMFS_IMAGE` environment variable and leave out the singularity\ncommand.  The image cannot come from cvmfs, but it can come from docker,\nshub, a local image file, or a local \"sandbox\" unpacked image directory.\nThen the usage is `singcvmfs [command]` where the default command is\n$SHELL.  For example:\n\n```\n$ export SINGCVMFS_REPOSITORIES=\"grid.cern.ch,atlas.cern.ch\"\n$ export SINGCVMFS_IMAGE=\"docker://centos:7\"\n$ singcvmfs ls /cvmfs\natlas.cern.ch  config-osg.opensciencegrid.org  grid.cern.ch\n$ singcvmfs ls /cvmfs/atlas.cern.ch\nrepo\n```\n\nThere are other optional environment variables that may be set.\nRun `singcvmfs -h` for more details.\n\nCaveat: singcvmfs works by bind-mounting all of the files from the cvmfs\ndistribution into the container, including the fuse3 libraries.  It\nexpects other base system libraries to be available inside the\ncontainer, so if the container uses a different base OS distribution it\nwill likely not work unless compatible libraries are in the container.\n\nCache considerations: by default singcvmfs starts a cache manager for all\nthe cvmfs repositories it mounts, which means that caches cannot be shared\nbetween different invocations of singcvmfs on the same machine.\nThis tends to be more of a problem than with the cvmfsexec command\nbecause it is more common to run many payload jobs on a machine with\nsingularity than it is to run many pilots.  You can\nselect a different cache directory for each invocation by setting\nSINGCVMFS_CACHEDIR.  Alternatively it's possible to use the [cvmfs alien\ncache](https://cvmfs.readthedocs.io/en/stable/cpt-configure.html#alien-cache)\nfeature to share the cache, but you would have to make sure that the\ncache doesn't grow too big and that it gets cleaned up at some point.\nMake sure that caches are not on shared filesystems because they're\nlikely to do too many metadata operations.\n\nThere is also a SINGCVMFS_CACHEIMAGE variable which can be set to an\next3 filesystem image for the cache.  If set, it must point to a\nfile with such a filesystem including a directory that is writable by\nthe user or a `shared` directory within it that is writable by the user.\nThe directory within the filesystem can be changed by setting\nSINGCVMFS_CACHEDIR but defaults to the root directory, `/`.\nIf running as an unprivileged user, this can directory can be created\nusing the `mkfs.ext3 -d` option.  Unfortunately the version of\n`mkfs.ext3` on RHEL7 is too old, but it can be compiled from\n[source](https://github.com/tytso/e2fsprogs/blob/master/INSTALL).\nThen to make the image file these commands should work:\n\n```\n$ truncate -s 6G scratch.img\n$ mkdir -p tmp/shared\n$ mkfs.ext3 -F -O ^has_journal -d tmp scratch.img\n```\n\nBy default the cvmfs logs are written to a top-level `log` directory, alongside\nthe top-level `dist` directory. The variable `SINGCVMFS_LOGDIR` can be used to\nwrite them to a different directory, which will be created if it doesn't exist.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcvmfs%2Fcvmfsexec","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcvmfs%2Fcvmfsexec","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcvmfs%2Fcvmfsexec/lists"}