{"id":16288507,"url":"https://github.com/kiemlicz/envoy","last_synced_at":"2026-01-20T00:12:16.110Z","repository":{"id":118637743,"uuid":"86742509","full_name":"kiemlicz/envoy","owner":"kiemlicz","description":"Salt states for any machine provisioning","archived":false,"fork":false,"pushed_at":"2024-07-21T08:17:08.000Z","size":778,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-21T01:51:12.281Z","etag":null,"topics":["automation","configuration-management","extension","salt","saltstack"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kiemlicz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-03-30T19:50:26.000Z","updated_at":"2020-09-30T07:52:53.000Z","dependencies_parsed_at":null,"dependency_job_id":"67814d83-7328-4853-90c4-71a244534103","html_url":"https://github.com/kiemlicz/envoy","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiemlicz%2Fenvoy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiemlicz%2Fenvoy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiemlicz%2Fenvoy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiemlicz%2Fenvoy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kiemlicz","download_url":"https://codeload.github.com/kiemlicz/envoy/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247318215,"owners_count":20919454,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automation","configuration-management","extension","salt","saltstack"],"created_at":"2024-10-10T19:48:31.150Z","updated_at":"2026-01-20T00:12:16.069Z","avatar_url":"https://github.com/kiemlicz.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"THIS REPO IS NO LONGER ACTIVE\n\nAll of its contents are now part of the project [Ambassador](https://github.com/kiemlicz/ambassador)\n\nImages are published from the Ambassador CI\n\nOld readme:\n\n[![Build status](https://travis-ci.org/kiemlicz/envoy.svg?branch=master)](https://travis-ci.org/kiemlicz/envoy)\n# Basics \n[Salt](https://saltstack.com/) _states_ for provisioning machines in generic yet sensible way.  \nThe goal is to create _salt environments_ usable by developers as well as admins during the setup of either server or 'client' machines.\n\n## Setup  \nThere are multiple options to deploy **envoy**.  \nThey depend on how you want to provision machines:  \n 1. Separate `salt-master` process provisioning `salt-minions`:  \nRefer to SaltStack documentation of [gitfs](https://docs.saltstack.com/en/latest/topics/tutorials/gitfs.html) \n(if you prefer local filesystem then familiarize with [multienv](https://docs.saltstack.com/en/latest/ref/states/top.html)) or use \nfully automated setup of SaltStack via associated [project ambassador](https://github.com/kiemlicz/ambassador)\n\n 2. Master-less provisioning (machine provisions itself):  \n **Steps**  \n    1. `curl -o /tmp/bootstrap-salt.sh -L https://bootstrap.saltstack.com`, requires (`apt-get install curl python-pip python-pygit2`)\n    2. `sh /tmp/bootstrap-salt.sh -X`\n    3. Create masterless configs: `config/common.conf` and `config/gitfs.conf` (put under `/etc/salt/minion.d/`), use associated [project ambassador](https://github.com/kiemlicz/ambassador) for guidelines how to create such configs\n    4. `systemctl start salt-minion`  \n    5. Optionally run `salt-call --local saltutil.sync_all`\n \nIt is possible to use both methods, e.g., initially provision the machine using master-minion setup, \"unplug\" the minion and use master-less when needed.  \n \n### Using as Vagrant provisioner\nVagrant supports [_Salt_ provisioner](https://www.vagrantup.com/docs/provisioning/salt.html)\n\n  1. Add following sections to `Vagrantfile`.\n```\n    Vagrant.configure(\"2\") do |config|\n    ...\n        config.vm.synced_folder \"/srv/salt/\", \"/srv/salt/\"  # add states from host\n    \n        config.vm.provision \"init\", type: \"shell\" do |s|\n          s.path = \"https://gist.githubusercontent.com/kiemlicz/33e891dd78e985bd080b85afa24f5d0a/raw/b9aba40aa30f238a24fe4ecb4ddc1650d9d685af/init.sh\"\n        end\n    \n        config.vm.provision :salt do |salt|\n          salt.masterless = true\n          salt.minion_config = \"minion.conf\"\n          salt.run_highstate = true\n          salt.salt_args = [ \"saltenv=server\" ]\n        end\n    ...\n    end\n```\n\n`init.sh`: bash [script](https://gist.github.com/kiemlicz/33e891dd78e985bd080b85afa24f5d0a) that installs salt requisites, e.g., git, pip packages (jinja2) etc.  \n`minion.conf`: configure `file_client: local` and whatever you like (mutlienvs, gitfs, ext_pillar)\n  \n  2. `vagrant up`\n\n### Using Kubernetes\nDepending on use case, different deployment strategies exist.\n\n#### Using envoy to deploy Kubernetes\n_Salt Master_ installed on separate machine, _Salt Minion_ installed on each [Kubernetes node](https://kubernetes.io/docs/concepts/architecture/nodes/).\n\nThis way it is possible to automatically create Kubernetes master and worker nodes\n\nFor documentation refer to [Kubernetes states](https://github.com/kiemlicz/envoy/tree/master/salt/server/kubernetes#kubernetes)\n\n#### Using envoy to provision in-Kubernetes pods\nIn this strategy the _Salt Master_ is deployed within dedicated pod and _Salt Minions_ are deployed as [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/).  \nIn this approach, the _Salt Minion_ is **not** the provisioned entity. \nInstead the _Salt Minion_ registers [`docker_events` engine](https://docs.saltstack.com/en/latest/ref/engines/all/salt.engines.docker_events.html). The engine captures \ndocker host events and forwards them to _Salt Master Event Bus_. [_Salt Master's Reactor System_](https://github.com/kiemlicz/util/wiki/Salt-Events-and-Reactor) is then used to\nadd additional provisioning logic that is impossible (not in an easy way at least) to provide using Kubernetes tools only.  \nExample: creating and maintaining Redis Cluster.\n\nMind that _Salt Minion_ is **not** installed on every container and **not** used to fully configure that container. That would be possible but\nthis should be the responsibility of the tool that is used to create that container (of course it is possible to use _Salt_ as such tool)\n\nMore detailed description can be found in [POD provisioning section](https://github.com/kiemlicz/envoy/tree/master/salt/server/kubernetes#provisioning-pods) \n    \n## Components\nIn order to run _states_ against _minions_, _pillar_ must be configured.  \nRefer to `pillar.example.sls` files in states themselves for particular structure.  \n_States_ must be written with assumption that given pillar entry may not exist.\nFor detailed state description, refer to particular states' README file.\n \n# Structure\nStates are divided in environments:\n 1. `base` - the main one. Any other environment comprises of at least `base`. Contains core states responsible for operations like\n repositories configuration, core packages installation or user setup\n 2. `dev` - for developer machines. Includes `base`. Contains states that install tons of dev apps along with their configuration (like add entry to `PATH` variable)\n 3. `server` - installs/configured typical server tools, e.g., Kubernetes or LVS. Includes `base` and `dev`\n\n# Extensions\nIn order to keep _states_ readable and configuration of whole SaltStack as flexible as possible, some extensions and custom states were introduced.\n\nAll of the custom states can be found in default _Salt_ extensions' [directories](https://docs.saltstack.com/en/latest/ref/file_server/dynamic-modules.html) (`_pillar`, `_runner`, etc.)\n\n## Custom Pillars\n### privgit\nDynamically configured git pillar.  \nAllows users to configure their own _pillar_ data git repository in the runtime - using pillar entries.\nNormally `git_pillar` must be configured in the _Salt Master_ configuration beforehand.\n\n#### Usage\nAppend `privgit` to `ext_pillar` configuration option to enable this extension.  \nThe syntax:\n```\next_pillar:                # Salt option\n  - privgit:               # extension name\n    - name1:               # first entry identifier\n        param1:            # the parameters dict\n        param2:            # append in config only the options that most likely won't be changed by users\n```\nFully static configuration (use _git_pillar_ instead of such):\n```\next_pillar:\n  - privgit:\n    - name1:\n        url: git@github.com:someone/somerepo.git\n        branch: master  \n        env: custom\n        root: pillar\n        privkey: |\n        some\n        sensitive data\n        pubkey: and so on\n    - name2:\n        url: git@github.com:someone/somerepo.git\n        branch: develop\n        env: custom\n        privkey_location: /location/on/master\n        pubkey_location: /location/on/master\n```\nParameters are formed as a list, next entries override previous:  \n```\nprivgit:\n  - name1:\n        url: git@github.com:someone/somerepo.git\n        branch: master  \n        env: custom\n        root: pillar\n        privkey: |\n        some\n        sensitive data\n        pubkey: and so on\n  - name2:\n        url: git@github.com:someone/somerepo.git\n        branch: develop\n        env: custom\n        privkey_location: /location/on/master\n        pubkey_location: /location/on/master\n  - name2:\n        url: git@github.com:someone/somerepo.git\n        branch: notdevelop\n```\nEntries order does matter, last one is the most specific one. It doesn't affect further pillar merge strategies.\n\nDue to potential integration with systems like [foreman](https://theforeman.org/) that support string keys only, \nanother (unpleasant, flat) syntax exists:\n```\nprivgit_name1_url: git@github.com:someone/somerepo.git\nprivgit_name1_branch: master \nprivgit_name1_env: custom\nprivgit_name1_root: pillar\nprivgit_name1_privkey: |\n        some\n        sensitive data\nprivgit_name1_pubkey: and so on\nprivgit_name2_url: git@github.com:someone/somerepo.git\nprivgit_name2_branch: develop\nprivgit_name2_env: custom\nprivgit_name2_privkey_location: /location/on/master\nprivgit_name2_pubkey_location: /location/on/master\n```\n\n### kubectl\nPulls any Kubernetes information and adds them to pillar.  \nIt is possible to specify pillar key under which the Kubernetes data will be hooked up.  \nUnder the hood this extension executes:\n`kubectl get -o yaml -n \u003cnamespace or deafult\u003e \u003ckind\u003e \u003cname\u003e` or \n`kubectl get -o yaml -n \u003cnamespace or deafult\u003e \u003ckind\u003e -l \u003cselector\u003e` if name is not provided\n\nThere is no (not yet) per-minion filtering of Kubernetes pillar data, thus this data will be matched to all minions.  \nFor Kubernetes deployments (minions as daemon set) this should be acceptable.\n \n#### Usage\n```\next_pillar:\n  - kubectl:\n      config: \"/some/path/to/kubernetes/access.conf\"   # all queries will use this config\n      queries:\n        - kind: statefulsets\n          name: redis-cluster\n          key: \"redis:kubernetes\"                      # nest the results under `redis:kubernetes` \n``` \n## Custom States\n### dotfile\nCustom _state_ that manages [dotfiles](https://en.wikipedia.org/wiki/Dot-file).  \nClones them from passed repository and sets up according to following [technique](https://developer.atlassian.com/blog/2016/02/best-way-to-store-dotfiles-git-bare-repo/)\n\n### devtool\nMost dev tools setup comes down to downloading some kind of archive, unpacking it and possibly adding symlink to some generic location.  \nThis state does pretty much that.\n\n### envops\nEnvironment variables operations.\n\n## Custom Execution Modules\n \n# Tests\nTests are performed on different OSes (in docker) in _Salt_ masterless mode.  \nDifferent [pillar data](https://github.com/kiemlicz/envoy/tree/master/.travis/pillar) is mixed with different [saltenvs](https://github.com/kiemlicz/envoy/tree/master/salt).  \nThen the `salt-call --local state.show_sls \u003cstate name\u003e` is invoked and checked if renders properly\n\nMore complex tests that perform actual state application in different environments are performed in associated [ambassador project](https://github.com/kiemlicz/ambassador)\n \n# References\n1. SaltStack [quickstart](https://docs.saltstack.com/en/latest/topics/states/index.html)\n2. SaltStack [best practices](https://docs.saltstack.com/en/latest/topics/best_practices.html)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkiemlicz%2Fenvoy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkiemlicz%2Fenvoy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkiemlicz%2Fenvoy/lists"}