{"id":41591473,"url":"https://github.com/opensciencegrid/tarball-client","last_synced_at":"2026-01-24T09:31:42.877Z","repository":{"id":5710274,"uuid":"53746632","full_name":"opensciencegrid/tarball-client","owner":"opensciencegrid","description":"Tools for creating the tarball-based distributions of the OSG client software","archived":false,"fork":false,"pushed_at":"2025-10-24T19:50:39.000Z","size":293,"stargazers_count":0,"open_issues_count":0,"forks_count":8,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-10-24T21:25:39.996Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/opensciencegrid.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2016-03-12T18:24:20.000Z","updated_at":"2025-10-24T19:50:44.000Z","dependencies_parsed_at":"2023-01-11T16:52:02.299Z","dependency_job_id":"aac74d7a-e6ea-4b42-af42-aa51e6eb70c2","html_url":"https://github.com/opensciencegrid/tarball-client","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/opensciencegrid/tarball-client","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opensciencegrid%2Ftarball-client","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opensciencegrid%2Ftarball-client/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opensciencegrid%2Ftarball-client/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opensciencegrid%2Ftarball-client/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/opensciencegrid","download_url":"https://codeload.github.com/opensciencegrid/tarball-client/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opensciencegrid%2Ftarball-client/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28723233,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-24T08:27:05.734Z","status":"ssl_error","status_checked_at":"2026-01-24T08:27:01.197Z","response_time":89,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-24T09:31:42.230Z","updated_at":"2026-01-24T09:31:42.871Z","avatar_url":"https://github.com/opensciencegrid.png","language":"Python","readme":"tarball-client\n==============\n\n`tarball-client` contains tools and supporting files for creating tarballs\n(called \"bundles\") for client software, where the bundles also contain most of\nthe dependencies required for the software to run. The primary use for these\ntools is the OSG Worker Node Client (`osg-wn-client`).\n\n\nOverview\n--------\n\nBundles are based on RPM installations of software and are dependent on a base\nOS install (e.g. `libc`). This means that bundles created for EL6 are not\nexpected to run on EL7 hosts, or vice versa. However, bundles should contain all\ntheir dependencies that are not expected to be on a base OS install.\n\nPatches are used to replace hard-coded paths in config files and scripts to\ndepend on the environment variable `$OSG_LOCATION` instead. However, binary\nexecutables with paths baked into them cannot be fixed, since bundles are\ncreated from binary RPMs and no compilation is done.\n\n\nUsage\n-----\n\nBundle contents are defined in `bundles.ini`. The default bundles are the OSG\nworker node clients, based on the `--osgver` flag passed on the command line.\n\n    make-client-tarball --osgver \u003c3.5|3.6\u003e --all\n\nwill make the whole set of osg-wn-client (and osg-afs-client) tarballs.\n\n    make-client-tarball --osgver \u003c3.5|3.6\u003e --dver \u003cel7|el8\u003e\n\nwill make specific osg-wn-client tarballs.\n\nYou can also build a specific bundle by passing `--bundle=\u003cbundle name\u003e`.\nIn this case, don't pass `--osgver`.\n\nPass `--version` to set the version of the tarball, e.g. `--version 3.5.21`.\nIf not present, it will use the version of the `osg-version` RPM or \"unknown\".\nNote that `--version` is only used for the name of the tarball -- you cannot\nbuild an earlier build a tarball from an earlier release by passing that\nrelease to `--version`.\n\n\n### Building in a VM\n\n**Requirements:**\n\nTarball builds must be done on an X86\\_64 machine running an RPM/YUM-based distribution, e.g. CentOS.\nEL8 tarballs must be built on an EL8 or newer distribution to handle a change in RPM format.\nIn addition, the following utilities must be present:\n\n- find\n- patch\n- tar\n- yumdownloader (from yum-utils)\n- yum-plugin-priorities (EL7 hosts only)\n\n\n\n**Instructions:**\n\n1.  Clone this repo into `/tmp/tarball-client`.\n\n1.  `cd /tmp/tarball-client`.\n\n1.  Run `make-client-tarball` as above, e.g. with\n\n        ./make-client-tarball --osgver 3.5 --version 3.5.30 --all\n\n    The resulting tarballs will be in the current directory.\n\n\n\n### Building in a Container\n\n**Requirements:**\n\nYou must have Docker installed on your machine and be a member of the `docker` Unix group;\nalternatively, you can use Podman but you must be `root`.  This is because the container\nneeds to be run in privileged mode in order to create chroots and mount `/proc`.\n\n\n**Instructions:**\n\n1.  Clone this repo.\n\n1.  Follow above instructions for `make-client-tarball` except run `docker-make-client-tarball` instead.\n    For example:\n\n        ./docker-make-client-tarball --osgver 3.5 --version 3.5.30 --all\n\n    The resulting tarballs will be in the current directory.\n\n\nMode of Operation\n-----------------\n\n`tarball-client` works inside a chroot in two stages. In stage 1, base OS\npackages are \"fake-installed\" into the chroot. These are the packages that are\nassumed to already be on the system that the resulting bundle will be run on.\nFor `osg-wn-client`, the list of packages to install is in `osg-stage1.lst` but\n`bundles.ini` may specify a different one.\n\nOnce the stage 1 packages are installed into the chroot,\n_all of the files from the packages will be deleted_.\nHowever, the entries in the YUM and RPM databases will remain. That means that\nYUM will consider those dependencies to be satisfied when installing packages\nlater, even though the files are not there. This is what is meant by\n\"fake-install.\"\n\nMultiple things happen in stage 2:\n\n-   The packages we want to keep in the bundle are installed. The list of packages to install is defined in `bundles.ini`.\n-   Patches may be applied to the files in the bundle. The directories which will be searched for patches are defined in `bundles.ini`. Patches will be applied in base filename order, _ignoring their directories_.\n-   Various other fixes are applied. These fixes are hardcoded in `stage2.py`.\n-   The tarballs are created.\n\n\nFiles\n-----\n\n\n### bundles.ini\n\nThe definitions for each bundle in Python SafeConfigParser format. Each bundle is in its own section.\n\n1.  Attributes\n\n    -   paramsets:\n\n        Space-seperated \"dver,arch\" pairs that define what distro versions and\n        architectures using `--all` will build for. For example, `el6,x86_64\n        el7,x86_64` will build for EL6 and EL7 on X86\\_64 only.\n\n    -   patchdirs:\n\n        The directories to apply patches from. `dver` and `basearch` are\n        available for substitution, where `dver` is `el6` or `el7`, and\n        `basearch` is `i386` or `x86_64`.\n\n    -   dirname:\n\n        The name of the top-level directory inside the tarball.\n\n    -   tarballname:\n\n        The name pattern for the tarball. `version`, `relnum`, `dver`, and\n        `basearch` are available for substitution.\n\n    -   packages:\n\n        The RPM packages that should be installed in stage 2. These are passed\n        to the `yum install` command; they may be groups using the `@groupname`\n        syntax.\n\n    -   repofile:\n\n        The YUM repo definition file _templates_ that stage 1 and stage 2\n        packages will be installed from. These look like files in\n        `/etc/yum.repos.d` but `dver` and `basearch` are available for\n        substitution within these files.\n\n    -   versionrpm (optional):\n\n        The RPM whose version will be used for the `version` variable in the\n        tarball name. If missing, you must specify `--version` on the command\n        line.\n\n    -   stage1file:\n\n        File containing the list of packages to install for stage 1. Again, YUM\n        will consider these packages to be installed during the stage 2 install,\n        but the files within them will not exist.\n\n\n### envsetup.py\n\nModule to write the template `setup.sh` and `setup.csh` files.\n\n\n### make-client-tarball\n\nThe script to be executed by the user to create the tarball.\n\n\n### stage1.py\n\nCode to do the stage 1 installation.\n\n### stage2.py\n\nCode to do the stage 2 installation, subsequent patching and fixes, and creation of the tarball.\n\n### yumconf.py\n\nDefines the `YumInstaller` class which deals with setting up YUM repo definition\nfiles and running the YUM commands to do the installations.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopensciencegrid%2Ftarball-client","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopensciencegrid%2Ftarball-client","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopensciencegrid%2Ftarball-client/lists"}