{"id":15759321,"url":"https://github.com/lsst-it/foreman_envsync","last_synced_at":"2025-08-27T01:54:40.496Z","repository":{"id":46277318,"uuid":"373916927","full_name":"lsst-it/foreman_envsync","owner":"lsst-it","description":"Sync pupperserver envs with foreman","archived":false,"fork":false,"pushed_at":"2024-04-30T15:28:48.000Z","size":55,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-15T13:34:40.237Z","etag":null,"topics":["foreman","foreman-proxy","hammer","puppet","puppetserver","ruby"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lsst-it.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-06-04T17:34:51.000Z","updated_at":"2024-04-30T15:28:51.000Z","dependencies_parsed_at":"2024-05-06T16:32:53.259Z","dependency_job_id":null,"html_url":"https://github.com/lsst-it/foreman_envsync","commit_stats":{"total_commits":68,"total_committers":2,"mean_commits":34.0,"dds":"0.014705882352941124","last_synced_commit":"58c52df35a70d26a5e498667a5ae53cc1a15ed94"},"previous_names":[],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lsst-it%2Fforeman_envsync","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lsst-it%2Fforeman_envsync/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lsst-it%2Fforeman_envsync/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lsst-it%2Fforeman_envsync/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lsst-it","download_url":"https://codeload.github.com/lsst-it/foreman_envsync/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246604612,"owners_count":20804100,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["foreman","foreman-proxy","hammer","puppet","puppetserver","ruby"],"created_at":"2024-10-04T10:04:44.331Z","updated_at":"2025-04-01T14:32:04.702Z","avatar_url":"https://github.com/lsst-it.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# foreman_envsync\n\nA highly opinionated utility to sync foreman puppet environments with the set\nof environments known to a `puppetserver` instance *without* also importing the\nclasses within those puppet environments.\n\n## OCI Image\n\nAn OCI image containing this gem is published to docker hub as\n[`ghcr.io/lsst-it/foreman_envsync`](ghcr.io/lsst-it/foreman_envsync).\n\n\n## The Problem\n\nWhen using [`r10k`](https://github.com/puppetlabs/r10k) with the [Dynamic\nEnvironments](https://github.com/puppetlabs/r10k/blob/main/doc/dynamic-environments.mkd)\npattern in concert with foreman as an ENC, it is highly desirable for new\nenvironments to automatically appear within foreman.  Foreman has a built in\n\"import environments\" feature which triggers a poll of a `puppetserver`'s known\nenvironments.  Foreman's `hammer` CLI has the ability to trigger an import of\nall environments (and classes) with the `proxy import-classes` sub-sub-command.\nE.g.:\n\n```bash\n/bin/hammer proxy import-classes --id=1\n```\n\n`r10k` provides a hook to run a shell commands after completing an environment\nbuild, which can be used to invoke `hammer`. E.g.:\n\n```yaml\n---\n:postrun: [\"/bin/hammer\", \"proxy\", \"import-classes\", \"--id=1\"]\n```\n\nThis automation setup is known to work reasonable well on a VM hosting both\n`foreman 2.4.0` + `puppetserver 6.15.3` provisioned with 16 x64 cores / 32GiB\nRAM / SSD storage under the following conditions:\n\n* The `puppetserver` environment class cache is **enabled**.\n* The number of puppet environments is relatively small. I.e. `\u003c= 10`\n* [Probably?] The environments are of moderate complexity. I.e. `\u003c ~100` modules\n* `r10k` is triggering an import via `hammer` 10s of times per day.\n\nHowever, it has been observed that all foreman puma workers (regardless of the\nnumber configured) will over time creep up to consume 100% of a core,\ninteractivity via the www console is abysmal, and environment import may fail\ncompletely when:\n\n* The `puppetserver` environment class cache is **disabled**.\n* The puppet environments have `\u003e 100` modules.\n* `r10k` is triggering an import via `hammer` 10s of times per day.\n* There are `\u003e 10` puppet environments.\n\nDistressingly, when the puppetserver environment cache is **disabled**, even a\nsmall number of puppet agent hosts (~10) will trigger foreman to consume all\navailable CPU cores.  Reducing the number of puppet environments to `\u003c 10` is\nhelpful but not a guarantee that an environment/class import will succeed.  It\nwas observed that above `40` environments, an import is virtually guaranteed to\nfail.\n\n\n##  What Is The Malfunction?\n\n*Disclaimer: This is essentially speculation based on observed behavior and\nisn't backed up by careful inspection of the code.*\n\nFrom external observation of foreman's behavior, it appears that there is a\nstrong built-in assumption that the `puppetserver` environment class cache is\nalways enabled and `ETag`s will turn most queries for the classes in an\nenvironment into essentially a no-op.\n\nWhen a foreman environment import is triggered, foreman not only enumerates\n`puppetserver`'s set of known environments via the `/puppet/v3/environments`\nAPI endpoint, it also queries for all the classes within an environment using\n`/puppet/v3/environment_classes`.  When the environment class cache is disabled\nthe time for this query to return seems to be highly variable for the test\nenvironment with response times ranging from a few seconds to over 30s *per\nenvironment*.  This means that an import cycle with many environments can take\n10s of minutes.  It appears that something in this process causes foreman's\npuma workers to consume 100% of a CPU.  It isn't known if this is busy waiting,\nparsing the class list, or something else all together.\n\nCompounding the extremely slow performance caused by bulk environment\nimportation, foreman also is calling `/puppet/v3/environment_classes` when it\nis invoked as an ENC.  This means that every puppet agent run results in a\npuppetserver instance compiling an environment to provide a class list to\nforeman and then again after ENC/fact data has been supplied to create a\ncatalog for the agent.\n\nTo add insult to injury, in this situation the foreman enc is used solely to\nsupply parameters and never for class inclusion.  All of the effort to produce\nenvironment class lists and parsing them is completely wasted.\n\n\n## How Does This Thing Help?\n\n`foreman_envsync` directly obtains the list of puppet environments from\n`puppetserver` without requesting class information.  It then removes any\nenvironments which are present in foreman but not within `puppetserver`.  If\nthere are any unknown-to-foreman environments, then are created as members of\nall foreman organizations and locations.  This completely avoids foreman\nhaving to wait on `puppetserver` to provide class lists and parsing them.\n\n\n## Narrow Use-case\n\nThis utility is opinionated and has many hard-coded assumptions. Including:\n\n* It is being installed on Centos 7.\n* Foreman and `puppetserver` are installed on the same host.\n* Foreman has been installed via `foreman-installer` / puppet code.\n* The `puppetserver` TLS related files are in the standard Puppetlabs locations.\n* `hammer` CLI has been installed and pre-configure with credentials.\n* \"new\" puppet environments should be visible to all foreman organizations and locations.\n\n\n## Obvious Improvements\n\n`foreman_envsync` is shelling out to invoke `hammer`. It should probably be\nconverted to be a `hammer` plugin.\n\nIt probably makes sense to use `foreman-proxy` rather than connecting to\n`puppetserver` directly as it avoid needing to deal with TLS authentication.\n\nIt would be fantastic if foreman's puppet integration was better able to handle\nthe environment class cache being disabled.  Including:\n\n* Not requesting an environment's class list any time the ENC functionality was invoked.\n* A configuration setting to completely disable class handling when it is not needed.\n* An option to import environments only (similar to `foreman_envsync`).\n* A configuration setting to disable the \"import environments\" feature which\n  also requests class lists.\n\n\n## Installation\n\nThis gem is currently intended to provide `foreman_envsync` CLI utility and not\nfor use as a library.\n\nIt is recommend that it is installed into the foreman `tfm` SCL and that all\ndependencies are ignore to avoid disrupting foreman.  It is expected to function\nwith foremans' gem deps at least as of foreman `2.4.0`.\n\n\n```bash\nscl enable tfm -- gem install --bindir /bin --ignore-dependencies --no-ri --no-rdoc --version 0.1.4 foreman_envsync\n```\n\n**If** dependencies need to be installed, the ruby devel package a compiler will be needed. E.g.:\n\n```bash\nyum install -y rh-ruby25-ruby-devel devtoolset-7\nscl enable devtoolset-7 tfm -- gem install --bindir /bin --ignore-dependencies --no-ri --no-rdoc --version 0.1.4 foreman_envsync\n```\n\n### `r10k` Config\n\nIt is is highly recommend that the `systemd-cat` utility to be used when configuring the `postrun` hook in `/etc/puppetlabs/r10k/r10k.yaml` so that status output is logged. E.g.:\n\n```yaml\n---\n:postrun: [\"/bin/scl\", \"enable\", \"tfm\", \"--\", \"systemd-cat\", \"-t\", \"foreman_envsync\", \"/bin/foreman_envsync\", \"--verbose\"]\n```\n\nExample of output in the journal when using `systemd-cat`:\n\n```\nJun 08 23:00:38 foreman.example.org foreman_envsync[10643]: found 13 puppetserver environment(s).\nJun 08 23:00:38 foreman.example.org foreman_envsync[10643]: ---\nJun 08 23:00:38 foreman.example.org foreman_envsync[10643]: - IT_2978_foreman_tuning_ruby\nJun 08 23:00:38 foreman.example.org foreman_envsync[10643]: - IT_2953_dds_debugging\nJun 08 23:00:38 foreman.example.org foreman_envsync[10643]: - IT_2907_tu_reip\nJun 08 23:00:38 foreman.example.org foreman_envsync[10643]: - production\nJun 08 23:00:38 foreman.example.org foreman_envsync[10643]: - coredev_production\nJun 08 23:00:38 foreman.example.org foreman_envsync[10643]: - IT_2978_foreman_tuning\nJun 08 23:00:38 foreman.example.org foreman_envsync[10643]: - IT_2373_velero\nJun 08 23:00:38 foreman.example.org foreman_envsync[10643]: - IT_2949_poc_encrypt\nJun 08 23:00:38 foreman.example.org foreman_envsync[10643]: - IT_2452_pagerduty\nJun 08 23:00:38 foreman.example.org foreman_envsync[10643]: - IT_2471_opsgenie\nJun 08 23:00:38 foreman.example.org foreman_envsync[10643]: - IT_2879_ipam\nJun 08 23:00:38 foreman.example.org foreman_envsync[10643]: - IT_2935_pathfinder_hcu01\nJun 08 23:00:38 foreman.example.org foreman_envsync[10643]: - IT_2987_rpi_puppet\nJun 08 23:00:39 foreman.example.org foreman_envsync[10643]: found 13 foreman environment(s).\nJun 08 23:00:39 foreman.example.org foreman_envsync[10643]: ---\nJun 08 23:00:39 foreman.example.org foreman_envsync[10643]: - coredev_production\nJun 08 23:00:39 foreman.example.org foreman_envsync[10643]: - IT_2373_velero\nJun 08 23:00:39 foreman.example.org foreman_envsync[10643]: - IT_2452_pagerduty\nJun 08 23:00:39 foreman.example.org foreman_envsync[10643]: - IT_2471_opsgenie\nJun 08 23:00:39 foreman.example.org foreman_envsync[10643]: - IT_2879_ipam\nJun 08 23:00:39 foreman.example.org foreman_envsync[10643]: - IT_2907_tu_reip\nJun 08 23:00:39 foreman.example.org foreman_envsync[10643]: - IT_2935_pathfinder_hcu01\nJun 08 23:00:39 foreman.example.org foreman_envsync[10643]: - IT_2949_poc_encrypt\nJun 08 23:00:39 foreman.example.org foreman_envsync[10643]: - IT_2953_dds_debugging\nJun 08 23:00:39 foreman.example.org foreman_envsync[10643]: - IT_2978_foreman_tuning\nJun 08 23:00:39 foreman.example.org foreman_envsync[10643]: - IT_2978_foreman_tuning_ruby\nJun 08 23:00:39 foreman.example.org foreman_envsync[10643]: - IT_2987_rpi_puppet\nJun 08 23:00:39 foreman.example.org foreman_envsync[10643]: - production\nJun 08 23:00:39 foreman.example.org foreman_envsync[10643]: found 0 foreman environment(s) unknown to puppetserver.\nJun 08 23:00:39 foreman.example.org foreman_envsync[10643]: found 0 puppetserver environment(s) unknown to foreman.\n```\n\n### `foreman-proxy` Config\n\nIf `foreman_envsync` is being considered then it is probably that foreman is also not being used for class inclusion. In that case, there is no reason for foreman to waste wallclock-time waiting for `puppetserver` to return class lists.\n\nThe wasted time may be minimized by setting\n\n```yaml\n:api_timeout: 1\n```\n\nin `/etc/foreman-proxy/settings.d/puppet_proxy_puppet_api.yml` (and restarting\n`foreman-proxy`).\n\n\n## Usage\n\n*Note that `foreman_envsync` requires permissions to read `puppetserver`'s TLS\nrelated files under `/etc/puppetlabs/puppet/ssl`. This may typically be\nachieved by membership in the `puppet` group or running as `root`.*\n\n`foreman_envsync` is silent, except for fatal errors, by default:\n\n```bash\nscl enable tfm -- foreman_envsync\n```\n\nThe `--verbose` enables detailed output.\n\n```bash\nscl enable tfm -- foreman_envsync --verbose\n```\n\nExample of verbose output:\n\n```bash\n# scl enable tfm -- foreman_envsync --verbose\nfound 22 puppetserver environment(s).\n---\n- master\n- IT_2953_dds_debugging\n- corecp_production\n- IT_2935_pathfinder_hcu01\n- coredev_production\n- IT_2949_poc_encrypt\n- production\n- IT_2907_tu_reip\n- IT_2373_velero\n- IT_2452_pagerduty\n- IT_2978_foreman_tuning\n- IT_2987_rpi_puppet\n- corels_production\n- ncsa_production\n- disable_IT_2569_tomcat_tls\n- tu_production\n- disable_IT_2483_letencrypt_renewal\n- disable_IT_2417_maddash\n- disable_jhoblitt_rspec\n- disable_jhoblitt_colina\n- IT_2879_ipam\n- IT_2471_opsgenie\n\nfound 36 foreman environment(s).\n---\n- corecp_production\n- coredev_production\n- corels_production\n- coretu_production\n- IT_2373_velero\n- IT_2417_maddash\n- IT_2441_dns3_cp\n- IT_2452_pagerduty\n- IT_2471_opsgenie\n- IT_2483_letencrypt_renewal\n- IT_2494_nfs\n- IT_2569_tomcat_tls\n- IT_2613_no_ccs_sal\n- IT_2655_net_audit_ls\n- IT_2657_dhcp\n- IT_2667_poc\n- IT_2693_arista\n- IT_2694_vlan_change\n- IT_2753_comcam_software_v2\n- IT_2820_auxtel_ccs\n- IT_2854_nfs_export\n- IT_2879_ipam\n- IT_2907_tu_reip\n- IT_2911_ntp\n- IT_2935_pathfinder_hcu01\n- IT_2949_poc_encrypt\n- IT_2953_dds_debugging\n- IT_2978_foreman_tuning\n- jhoblitt_colina\n- jhoblitt_rspec\n- master\n- ncsa_production\n- production\n- tickets_DM_25966\n- tickets_DM_27839\n- tu_production\n\nfound 20 foreman environment(s) unknown to puppetserver.\n---\n- coretu_production\n- IT_2417_maddash\n- IT_2441_dns3_cp\n- IT_2483_letencrypt_renewal\n- IT_2494_nfs\n- IT_2569_tomcat_tls\n- IT_2613_no_ccs_sal\n- IT_2655_net_audit_ls\n- IT_2657_dhcp\n- IT_2667_poc\n- IT_2693_arista\n- IT_2694_vlan_change\n- IT_2753_comcam_software_v2\n- IT_2820_auxtel_ccs\n- IT_2854_nfs_export\n- IT_2911_ntp\n- jhoblitt_colina\n- jhoblitt_rspec\n- tickets_DM_25966\n- tickets_DM_27839\n\ndeleted 20 foreman environment(s).\n---\n- message: Environment deleted.\n  id: 12\n  name: coretu_production\n- message: Environment deleted.\n  id: 4602\n  name: IT_2417_maddash\n- message: Environment deleted.\n  id: 4664\n  name: IT_2441_dns3_cp\n- message: Environment deleted.\n  id: 4637\n  name: IT_2483_letencrypt_renewal\n- message: Environment deleted.\n  id: 4649\n  name: IT_2494_nfs\n- message: Environment deleted.\n  id: 4658\n  name: IT_2569_tomcat_tls\n- message: Environment deleted.\n  id: 4676\n  name: IT_2613_no_ccs_sal\n- message: Environment deleted.\n  id: 4691\n  name: IT_2655_net_audit_ls\n- message: Environment deleted.\n  id: 4688\n  name: IT_2657_dhcp\n- message: Environment deleted.\n  id: 4736\n  name: IT_2667_poc\n- message: Environment deleted.\n  id: 4709\n  name: IT_2693_arista\n- message: Environment deleted.\n  id: 4710\n  name: IT_2694_vlan_change\n- message: Environment deleted.\n  id: 4771\n  name: IT_2753_comcam_software_v2\n- message: Environment deleted.\n  id: 4746\n  name: IT_2820_auxtel_ccs\n- message: Environment deleted.\n  id: 4792\n  name: IT_2854_nfs_export\n- message: Environment deleted.\n  id: 4762\n  name: IT_2911_ntp\n- message: Environment deleted.\n  id: 4735\n  name: jhoblitt_colina\n- message: Environment deleted.\n  id: 4619\n  name: jhoblitt_rspec\n- message: Environment deleted.\n  id: 4581\n  name: tickets_DM_25966\n- message: Environment deleted.\n  id: 4686\n  name: tickets_DM_27839\n\nfound 6 puppetserver environment(s) unknown to foreman.\n---\n- IT_2987_rpi_puppet\n- disable_IT_2569_tomcat_tls\n- disable_IT_2483_letencrypt_renewal\n- disable_IT_2417_maddash\n- disable_jhoblitt_rspec\n- disable_jhoblitt_colina\n\nfound 1 foreman location(s).\n---\n- 2\n\nfound 1 foreman organization(s).\n---\n- 1\n\ncreated 6 foreman environment(s).\n---\n- message: Environment created.\n  id: 4800\n  name: IT_2987_rpi_puppet\n- message: Environment created.\n  id: 4801\n  name: disable_IT_2569_tomcat_tls\n- message: Environment created.\n  id: 4802\n  name: disable_IT_2483_letencrypt_renewal\n- message: Environment created.\n  id: 4803\n  name: disable_IT_2417_maddash\n- message: Environment created.\n  id: 4804\n  name: disable_jhoblitt_rspec\n- message: Environment created.\n  id: 4805\n  name: disable_jhoblitt_colina\n```\n\n\n## Contributing\n\nBug reports and pull requests are welcome on GitHub at https://github.com/lsst-it/foreman_envsync.\n\n\n## License\n\nThe gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flsst-it%2Fforeman_envsync","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flsst-it%2Fforeman_envsync","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flsst-it%2Fforeman_envsync/lists"}