{"id":30019832,"url":"https://github.com/veupathdb/vagrant-workflow-software","last_synced_at":"2026-02-09T00:31:11.820Z","repository":{"id":34607567,"uuid":"38555849","full_name":"VEuPathDB/vagrant-workflow-software","owner":"VEuPathDB","description":"simulation environment for deployment  of EuPathDB workflow software","archived":false,"fork":false,"pushed_at":"2020-10-12T19:02:13.000Z","size":49,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-08-06T09:50:43.374Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/VEuPathDB.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-07-05T03:47:03.000Z","updated_at":"2020-10-12T19:02:16.000Z","dependencies_parsed_at":"2022-08-17T20:50:59.249Z","dependency_job_id":null,"html_url":"https://github.com/VEuPathDB/vagrant-workflow-software","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/VEuPathDB/vagrant-workflow-software","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VEuPathDB%2Fvagrant-workflow-software","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VEuPathDB%2Fvagrant-workflow-software/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VEuPathDB%2Fvagrant-workflow-software/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VEuPathDB%2Fvagrant-workflow-software/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/VEuPathDB","download_url":"https://codeload.github.com/VEuPathDB/vagrant-workflow-software/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VEuPathDB%2Fvagrant-workflow-software/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29251464,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-08T22:49:53.206Z","status":"ssl_error","status_checked_at":"2026-02-08T22:49:51.384Z","response_time":57,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-06T01:21:42.181Z","updated_at":"2026-02-09T00:31:11.814Z","avatar_url":"https://github.com/VEuPathDB.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"## vagrant-workflow-software\n\nVagrant manifest to set up a VirtualBox instance for developing and\ntesting the deployment of workflow software for compute clusters and\nworkflow servers. See\n[https://wiki.apidb.org/index.php/PreparingClusters](https://wiki.apidb.org/index.php/PreparingClusters)\nfor specifications.\n\n### Requirements\n\n__Software__\n\n- Vagrant\n- VirtualBox\n\n\n### Usage\n\n    git clone git@github.com:EuPathDB/vagrant-workflow-software.git\n    cd vagrant-workflow-software\n    vagrant up\n\n### Vagrantfile\n\nThe `Vagrantfile` specifies a dictionary at the top.\n\n    WF_SERVERS = {\n      :el6 =\u003e {\n        :vagrant_box     =\u003e 'puppetlabs/centos-6.6-64-nocm',\n        :wf_hostname     =\u003e 'consign.vm',\n        :wf_eupath_dir   =\u003e '/project/eupathdblab',\n        :wf_user         =\u003e 'debbie',\n        :wf_shared_group =\u003e 'eupathdblab',\n      },\n      :el7 =\u003e {\n        :vagrant_box     =\u003e 'ebrc/centos-7-64-puppet',\n        :wf_hostname     =\u003e 'elm.vm',\n        :wf_eupath_dir   =\u003e '/eupath',\n        :wf_user         =\u003e 'debbie',\n        :wf_shared_group =\u003e 'eupa',\n      },\n    }\n\nThis allows defining properties for multiple Vagrant boxes. The above\nexample will set up CentOS 6 and CentOS 7 environments on two virtual\nmachines. These will be named according the top key values (`el6` and\n`el7` in this example). These names will be used with `vagrant ssh` to\nidentify which managed machine to connect to.\n\n`wf_hostname` can be given the name of an existing cluster. This permits\nusing an existing Puppet node manifest and leverages existing profile\nconfigurations (e.g. `$eupath_dir/etc/bashrc`) that use hostname\nconditionals. Note that different compute clusters use various OS\nversions (e.g. CentOS 6 and CentOS 7). Be sure the `vagrant_box` in the\n`WF_SERVERS` dictionary is configured for a Vagrant box with a matching\nOS for the chosen `wf_hostname`. For example, the physical sapelo in\nthe datacenter uses CentOS 6 so be sure the virtual machine simulation\nalso uses CentOS 6.\n\n`wf_user_path` should match `user_path` in the Puppet node manifest for\nthe host.\n\n`wf_shared_group` should match `shared_group` in the Puppet node\nmanifest for the host.\n\n`wf_user` can be any name. An account of this name will be created on\nthe virtual machine and its shell environment will be configured as a\ntypical workflow user. This account is just for testing on the virtual\nmachines, there is no corresponding value in the Puppet manifests.\n\n### Simulating a workflow user\n\nThe vagrant box includes the account `debbie` whose `.bashrc` matches\nthat of a typical workflow user. Use this account to test running the\nworkflow software and ensure that the environment is set up correctly.\nThe `.bashrc` is on the guest VM is provisioned by Ansible from\n`ansible/files/workflow_bashrc`. This file and the\n[WorkflowSoftware wiki instructions](https://wiki.apidb.org/index.php/WorkflowSoftware)\nshould be kept in sync if you need to make changes.\n\nTo log in as `debbie`, run\n\n    vagrant ssh el7 -- -l debbie\n\nwhere `el7` is Vagrant box you want to connect to (as defined in the\nVagrantfile's `WF_SERVER` dictionary).\n\n### Installing and Updating workflow software\n\nThe \"sysadmin\" of the workflow software infrastructure is the user\n`vagrant`. This user should be used for updating the workflow software\nand changing the deployment infrastructure. The `workpuppet` script is\nused for routine updates. See the wiki for details.\n\nThe `workpuppet` script (see the [PreparingClusters\nwiki](https://wiki.apidb.org/index.php/PreparingClusters)) is run on\neach invocation of `vagrant\nprovision`. In this case `workpuppet` is run by Ansible which eats\nstdout, so it can be difficult to troubleshoot problems. However, the\nstderr and stdout is recorded in `~vagrant/workpuppet.log`, so that may\nbe of some use.\n\nAlternatively you can ssh to the virtual machine and run the command\nmanually. This option allow you to monitor progress through stdout and\nmore closely simulates how cluster software is updated in the wild.\n\n    vagrant ssh el7 -c 'source $admin_path/bashrc; workpuppet'\n\nwhere `el7` is Vagrant box you want to connect to (as defined in the\nVagrantfile's `WF_SERVER` dictionary).\n\nInteractive changes and gefingerpoken can, of course, also be done from\nthe `vagrant` shell. To log in as `vagrant`, run\n\n    vagrant ssh el7\n\nwhere `el7` is Vagrant box you want to connect to (as defined in the\nVagrantfile's `WF_SERVER` dictionary).\n\n_Caution: When screwing around in terminal windows be very careful which\nwindow you are working in. Because the virtual machine may have the same\nname as a real server, and because you may have terminals open to both,\nyou want to be sure to keep track of the difference, lest you\naccidentally do development changes on the real server. The user shells\non the virtual machine are configured with a distinctive command prompt\nto help disambiguate virtual from real._\n\n__The Development Environment__\n\nGit cloning from git.apidb.org requires ssh key authentication. The\n`Vagrantfile` specifies `config.ssh.forward_agent = true` to use your\nhost ssh agent - so you need an agent running on your Vagrant host.\n\nThe `init.sh` script that does the initial provisioning assumes a VM\nthat mounts the Vagrant project directory from host to `/vagrant` on the\nguest. The `puppetlabs/centos-6.6-64-nocm` specified in the\n`Vagrantfile` satisfies this requirement.\n\nOne of the temptations of having a virtualized development environment\nis working off-campus or offline. Be aware that the provisioning and\nmany other operations require network access to resources behind UGA\nfirewalls and/or expect clients originating from trusted IP addresses.\nIf you don't meet expected network conditions, silent or cryptic\nfailures await you. If you work remotely, fire up UGA's VPN and you\nshould be OK.\n\n### Manual Package Deployment\n\nIf you have a newly made RPM to test before adding to the YUM\nrepository, you can copy the RPM to the virtual machine and install it\nmanually.\n\nA `wfpkginstall` script is included in the vagrant user's home `bin`\ndir. This script can be used to install a single package without going\nthrough Puppet and YUM repository. Manually place the desired rpm file\ninto `$admin_path/yum-workflow/\u003cos_major_version\u003e/x86_64/` and then run\nthe `wfpkginstall` script with the package name (not file name). This\nsimulates what Puppet will do, including processing the\n`MANIFEST.EUPATH` file bundled with the software package.\n\n*Limitation: You may encounter conflicts if you use `wfpkginstall` to\ninstall a newer version of a package that is already installed. In such\na case you should add the new package to `$active_software` in the\n`packages.pp` Puppet manifest (and the old version to\n`$inactive_software`), and then run the `workpuppet` script.*\n\n    wfpkginstall python27-2.7.10\n\n**Tip:** The `$admin_path/yum-workflow/\u003cos_major_version\u003e/x86_64/`\ndirectory is shared with the Vagrant host under the `scratch` directory\nof the Vagrant project. The\n[`vagrant-rpmbuild`](https://github.com/EuPathDB/vagrant-rpmbuild)\nproject - the recommended environment for generating workflow-software\nrpms - places the rpm files built there under the `scratch` directory of\nthat project. Therefore you can use host tools (e.g. drag-and-drop) to\nquickly copy the rpm file from the `rpmbuild` to the `workflow-software`\nproject and then run `wfpkginstall` on the guest.\n\n**Tip:** Tab completion is available for `wfpkginstall`.\n\n### Provisioning\n\nThe `init.sh` script initializes and bootstraps the\n`${BASE_DIR}/sysadmin` directory with required infrastructure. It's\nmostly copy/paste from the [PreparingClusters\nwiki](https://wiki.apidb.org/index.php/PreparingClusters) instructions\n(and the two should be kept in sync if you make changes) with a few\ntweaks to aid Vagrant VM work. The design is such that\n`${BASE_DIR}/sysadmin` is only built on the first `vagrant up`.\nSubsequent `vagrant provision` invocations will skip `init.sh` if\n`${BASE_DIR}/sysadmin` exists. So if if you need to re-provision\n`${BASE_DIR}/sysadmin` you might as well `vagrant destroy` and start\nover.\n\nInvocations of `vagrant provision` after the initial setup will invoke\nthe `workpuppet` script. This script manages the YUM repo updates and\nruns the `puppet apply`. You can skip this step by passing Ansible\n`--skip-tags` via the `ANSIBLE_ARGS` environment variable.\n\n    ANSIBLE_ARGS=\"--skip-tags=workpuppet\" vagrant provision el7\n\nThe git repo of Puppet manifests is checked out to the guest\n`/vagrant/scratch` mount point and symlinked in to the\n`${BASE_DIR}/sysadmin` directory. The `/vagrant` volume is provided by\nthe host so its contents persist across `vagrant destroy` (reducing the\nchanges of losing uncommitted changes). This also has the side-effect\nthat Puppet manifest editing and Git management can be done on you local\nhost.\n\nThe `yum-workflow` directory is also symlinked in `${BASE_DIR}/sysadmin`\nto the `/vagrant/scratch` mountpoint on the guest, shared with the host,\nso it is persistent across `vagrant destroy`. This is just to save yum\npackage download time on subsequent provisionings. The directory can be\ndeleted manually if you want to test a complete provision from scratch.\n\n### Re-provisioning\n\nYou can skip tagged tasks by passing Ansible `--skip-tags` via the\n`ANSIBLE_ARGS` environment variable.\n\n    ANSIBLE_ARGS=\"--skip-tags=workpuppet\" vagrant provision el7\n\n\n### Known Issues\n\n- The disconnect after 'change sysadmin user group' no longer works\nin modern Ansible. The 'meta: reset_connection' task seems to do\nnothing in 2.3.1.0.\n\n- workpuppet run at the end exits cleanly but doesn't install the\nbioinformatics software. Possibly due to\n`Dependency File[/project/eupathdblab/workflow-software/tmp] has failures: true`\nRunning workpuppet manually succeeds. Maybe related to\nfailure of 'change sysadmin user group' above. (this seems to be the case\nas waiting a couple of minutes after 'change sysadmin user group' fails\nbefore running provision again (giving time for multiplexing to timeout)\nseems to workaround this issue.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fveupathdb%2Fvagrant-workflow-software","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fveupathdb%2Fvagrant-workflow-software","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fveupathdb%2Fvagrant-workflow-software/lists"}