{"id":15438050,"url":"https://github.com/mudler/yip","last_synced_at":"2025-05-15T12:06:38.514Z","repository":{"id":41825129,"uuid":"280724472","full_name":"mudler/yip","owner":"mudler","description":" :pushpin: Yaml Instructions Processor - Simply applies a cloud-init style yaml file to the system","archived":false,"fork":false,"pushed_at":"2025-05-15T08:01:08.000Z","size":28666,"stargazers_count":88,"open_issues_count":26,"forks_count":21,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-05-15T09:21:31.619Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mudler.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-07-18T19:26:19.000Z","updated_at":"2025-05-15T08:00:19.000Z","dependencies_parsed_at":"2024-06-25T16:45:31.071Z","dependency_job_id":"ea1db53f-0008-473a-948d-75dda6a8cd23","html_url":"https://github.com/mudler/yip","commit_stats":{"total_commits":240,"total_committers":15,"mean_commits":16.0,"dds":0.5,"last_synced_commit":"4fd77a2709e0d98c25c14925530f74f55d704ac6"},"previous_names":[],"tags_count":92,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mudler%2Fyip","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mudler%2Fyip/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mudler%2Fyip/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mudler%2Fyip/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mudler","download_url":"https://codeload.github.com/mudler/yip/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254337613,"owners_count":22054253,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-01T19:00:24.767Z","updated_at":"2025-05-15T12:06:33.503Z","avatar_url":"https://github.com/mudler.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# :pushpin: yip\n\n\nSimply applies a configuration to the system described with yaml files.\n\n\n```yaml\nstages:\n   # \"test\" is the stage\n   test:\n     - systemd_firstboot:\n         keymap: us\n     - files:\n        - path: /tmp/bar\n          content: |\n                    test\n          permissions: 0777\n          owner: 1000\n          group: 100\n       if: \"[ ! -e /tmp/bar ]\"\n     - files:\n        - path: /tmp/foo\n          content: |\n                    test\n          permissions: 0777\n          owner: 1000\n          group: 100\n       commands:\n        - echo \"test\"\n       modules:\n       - nvidia\n       environment:\n         FOO: \"bar\"\n       systctl:\n         debug.exception-trace: \"0\"\n       hostname: \"foo\"\n       systemctl:\n         enable:\n         - foo\n         disable:\n         - bar\n         start:\n         - baz\n         mask:\n         - foobar\n       authorized_keys:\n          user:\n          - \"github:mudler\"\n          - \"ssh-rsa ....\"\n       dns:\n         path: /etc/resolv.conf\n         nameservers:\n         - 8.8.8.8\n       ensure_entities:\n       -  path: /etc/passwd\n          entity: |\n                  kind: \"user\"\n                  username: \"foo\"\n                  password: \"pass\"\n                  uid: 0\n                  gid: 0\n                  info: \"Foo!\"\n                  homedir: \"/home/foo\"\n                  shell: \"/bin/bash\"\n       delete_entities:\n       -  path: /etc/passwd\n          entity: |\n                  kind: \"user\"\n                  username: \"foo\"\n                  password: \"pass\"\n                  uid: 0\n                  gid: 0\n                  info: \"Foo!\"\n                  homedir: \"/home/foo\"\n                  shell: \"/bin/bash\"\n      datasource:\n        providers:\n          - \"digitalocean\"\n          - \"aws\"\n          - \"gcp\"\n        path: \"/usr/local/etc\"\n```\n\n- Simple\n- Small scope, pluggable, extensible\n\nYip uses a simple, yet powerful distro-agnostic cloud-init style format for the definition.\n\n```bash\n$\u003e yip -s test yip1.yaml yip2.yaml\n$\u003e yip -s test https://..\n```\n---\n\nThat's it! by default `yip` uses the default stage and the `default` executor, but you can customize its execution.\n\n\n```\nyip loads cloud-init style yamls and applies them in the system.\n\nFor example:\n\n        $\u003e yip -s initramfs https://\u003cyip.yaml\u003e /path/to/disk \u003cdefinition.yaml\u003e ...\n        $\u003e yip -s initramfs \u003cyip.yaml\u003e \u003cyip2.yaml\u003e ...\n        $\u003e cat def.yaml | yip -\n\nUsage:\n  yip [flags]\n\nFlags:\n  -e, --executor string   Executor which applies the config (default \"default\")\n  -h, --help              help for yip\n  -s, --stage string      Stage to apply (default \"default\")\n```\n\n\n## How it works\n\n\nYip works in *stages*. You can define *stages* that you can decide to run and apply in various ways and in a different enviroment (that's why *stages*).  \n\nA stage is just a list of steps, for example the following:\n\n```yaml\nstages:\n   default:\n     - files:\n        - path: /tmp/bar\n          content: |\n                    #!/bin/sh\n                    echo \"test\"\n          permissions: 0777\n          owner: 1000\n          group: 100\n       commands:\n        - /tmp/bar\n```\n\nwrites a `/tmp/bar` file during the `default` stage and will also run it afterwards. \n\nNow we can execute it:\n\n```bash\n$\u003e cat myfile.yaml | yip -s default -\n```\n\nAs `yip` by default runs the `default` stage we could have just run:\n\n```bash\n$\u003e cat myfile.yaml | yip -\n```\n\nA yaml file can define multiple stages, which can be run from the `cli` with `-s`. Each stage is defined under `stages`, and in each stage are defined a list of `steps` to execute.\n\n`Yip` will execute the steps and report failures. It will exit non-zero if one of the steps failed executing. It will, however, keep running all the detected `yipfiles` and stages.\n\n## Compatibility with Cloud Init format\n\nA subset of the official [cloud-config spec](http://cloudinit.readthedocs.org/en/latest/topics/format.html#cloud-config-data) is implemented by yip. \n\nIf a yaml file starts with `#cloud-config` it is parsed as a standard cloud-init, associated it to the yip `boot` stage. For example:\n\n```yaml\n#cloud-config\nusers:\n- name: \"bar\"\n  passwd: \"foo\"\n  groups: \"users\"\n  ssh_authorized_keys:\n  - faaapploo\nssh_authorized_keys:\n  - asdd\nruncmd:\n- foo\nhostname: \"bar\"\nwrite_files:\n- encoding: b64\n  content: CiMgVGhpcyBmaWxlIGNvbnRyb2xzIHRoZSBzdGF0ZSBvZiBTRUxpbnV4\n  path: /foo/bar\n  permissions: \"0644\"\n  owner: \"bar\"\n```\n\nTo execute it with yip, run `yip -s boot cloud-config.yaml`.\n\n\n## Node-data interpolation\n\n`yip` interpolates host data retrieved by [sysinfo](https://github.com/zcalusic/sysinfo#sample-output) and are templated in the commands, file and entities  fields.\n\nThis means that templating like the following is possible:\n\n```yaml\nstages:\n  foo:\n  - name: \"echo\"\n    commands:\n    - echo \"{{.Values.node.hostname}}\"\n\nname: \"Test yip!\"\n```\n\n## Filtering stages by node hostname\n\n`yip` can skip stages based on the node hostname:\n\n\n```yaml\nstages:\n  foo:\n  - name: \"echo\"\n    commands:\n    - echo hello\n    node: \"hostname\" # Node hostname\n\nname: \"Test yip!\"\n```\n\n## Filtering stages with if statement\n\n`yip` can skip stages based on if statements:\n\n\n```yaml\nstages:\n  foo:\n  - name: \"echo\"\n    commands:\n    - echo hello\n    if: \"cat /proc/cmdline | grep debug\"\n\nname: \"Test yip!\"\n```\n\nThe expression inside the if will be evaluated in bash and, if specified, the stage gets executed only if the condition returns successfully (exit 0).\n\n## Filtering stages with only_os and only_os_version statement\n\n`yip` can skip stages based on the OS and OS version:\n\nUse the `only_os` and `only_os_version` fields to specify the OS and OS version where the stage should be executed. They can be used together or separately and mixed with the normal `if` statement to provide several filtering criteria.\n\nNotice that both fields are compiled as a regex, so you can use regex patterns to match the OS and OS version.\n\n```yaml\nstages:\n  foo:\n  - name: \"echo ubuntu\"\n    commands:\n    - echo hello\n    only_os: \"ubuntu\"\n  - name: \"echo ubuntu and opensuse-leap\"\n    commands:\n       - echo hello\n    only_os: \"ubuntu|opensuse-leap\"\n  - name: \"echo everything but ubuntu\"\n    commands:\n       - echo hello\n    only_os: \"^(?!ubuntu).*\"\n  - name: \"echo ubuntu 20.04\"\n    commands:\n       - echo hello\n    only_os: \"ubuntu\"\n    only_os_version: \"20.04\"\n  - name: \"echo ubuntu 20.04 or 22.04\"\n    commands:\n       - echo hello\n    only_os: \"ubuntu\"\n    only_os_version: \"20.04|22.04\"\n\nname: \"Test yip!\"\n```\n\n\n\n## Configuration reference\n\nBelow is a reference of all keys available in the cloud-init style files.\n\n\n### `stages.\u003cstageID\u003e.[\u003cstepN\u003e].name`\n\nA description of the stage step. Used only when printing output to console.\n\n### `stages.\u003cstageID\u003e.[\u003cstepN\u003e].files`\n\nA list of files to write to disk.\n\n```yaml\nstages:\n   default:\n     - files:\n        - path: /tmp/bar\n          encoding: \"b64\" # \"base64\", \"gz\", \"gzip\", \"gz+base64\", \"gzip+base64\", \"gz+b64\", \"gzip+b64\"\n          content: |\n                    #!/bin/sh\n                    echo \"test\"\n          permissions: 0777\n          owner: 1000\n          group: 100\n          # or\n          # owner_string: \"user:group\", or \"user\"\n```\n\n### `stages.\u003cstageID\u003e.[\u003cstepN\u003e].downloads`\n\nA list of http urls to download and write to disk.\n\n```yaml\nstages:\n   default:\n     - downloads:\n        - path: /tmp/bar\n          url: \"\"\n          timeout: 0\n          permissions: 0777\n          owner: 1000\n          group: 100\n          # or\n          # owner_string: \"user:group\", or \"user\"\n```\n\n### `stages.\u003cstageID\u003e.[\u003cstepN\u003e].directories`\n\nA list of directories to be created on disk. Runs before `files`.\n\n```yaml\nstages:\n   default:\n     - name: \"Setup folders\"\n       directories: \n       - path: \"/etc/foo\"\n         permissions: 0600\n         owner: 0\n         group: 0\n```\n\n### `stages.\u003cstageID\u003e.[\u003cstepN\u003e].dns`\n\nA way to configure the `/etc/resolv.conf` file.\n\n```yaml\nstages:\n   default:\n     - name: \"Setup dns\"\n       dns: \n         nameservers:\n         - 8.8.8.8\n         - 1.1.1.1\n         search:\n         - foo.bar\n         options:\n         - ..\n         path: \"/etc/resolv.conf.bak\"\n```\n### `stages.\u003cstageID\u003e.[\u003cstepN\u003e].hostname`\n\nA string representing the machine hostname. It sets it in the running system, updates `/etc/hostname` and adds the new hostname to `/etc/hosts`.\n\n```yaml\nstages:\n   default:\n     - name: \"Setup hostname\"\n       hostname: \"foo\"\n```\n### `stages.\u003cstageID\u003e.[\u003cstepN\u003e].sysctl`\n\nKernel configuration. It sets `/proc/sys/\u003ckey\u003e` accordingly, similarly to `sysctl`.\n\n```yaml\nstages:\n   default:\n     - name: \"Setup exception trace\"\n       systctl:\n         debug.exception-trace: \"0\"\n```\n\n### `stages.\u003cstageID\u003e.[\u003cstepN\u003e].authorized_keys`\n\nA list of SSH authorized keys that should be added for each user. \nSSH keys can be obtained from GitHub user accounts by using the format github:${USERNAME},  similarly for Gitlab with gitlab:${USERNAME}.\n\n```yaml\nstages:\n   default:\n     - name: \"Setup exception trace\"\n       authorized_keys:\n         mudler:\n         - github:mudler\n         - ssh-rsa: ...\n```\n\n### `stages.\u003cstageID\u003e.[\u003cstepN\u003e].node`\n\nIf defined, the node hostname where this stage has to run, otherwise it skips the execution. The node can be also a regexp in the Golang format.\n\n```yaml\nstages:\n   default:\n     - name: \"Setup logging\"\n       node: \"bastion\"\n```\n\n### `stages.\u003cstageID\u003e.[\u003cstepN\u003e].users`\n\nA map of users and user info to set. Passwords can be also encrypted.\n\nThe `users` parameter adds or modifies the specified list of users. Each user is an object which consists of the following fields. Each field is optional and of type string unless otherwise noted.\nIn case the user is already existing, the password only will be overwritten.\n\n- **name**: Required. Login name of user\n- **gecos**: GECOS comment of user\n- **passwd**: Hash of the password to use for this user. Unencrypted strings supported too.\n- **homedir**: User's home directory. Defaults to /home/*name*\n- **no-create-home**: Boolean. Skip home directory creation.\n- **primary-group**: Default group for the user. Defaults to a new group created named after the user.\n- **groups**: Add user to these additional groups\n- **no-user-group**: Boolean. Skip default group creation.\n- **ssh-authorized-keys**: List of public SSH keys to authorize for this user\n- **system**: Create the user as a system user. No home directory will be created.\n- **no-log-init**: Boolean. Skip initialization of lastlog and faillog databases.\n- **shell**: User's login shell.\n\n```yaml\nstages:\n   default:\n     - name: \"Setup users\"\n       users: \n          bastion: \n            passwd: \"strongpassword\"\n            homedir: \"/home/foo\n```\n\n### `stages.\u003cstageID\u003e.[\u003cstepN\u003e].ensure_entities`\n\nA `user` or a `group` in the [entity](https://github.com/mudler/entities) format to be configured in the system\n\n```yaml\nstages:\n   default:\n     - name: \"Setup users\"\n       ensure_entities:\n       -  path: /etc/passwd\n          entity: |\n                  kind: \"user\"\n                  username: \"foo\"\n                  password: \"x\"\n                  uid: 0\n                  gid: 0\n                  info: \"Foo!\"\n                  homedir: \"/home/foo\"\n                  shell: \"/bin/bash\"\n```\n### `stages.\u003cstageID\u003e.[\u003cstepN\u003e].delete_entities`\n\nA `user` or a `group` in the [entity](https://github.com/mudler/entities) format to be pruned from the system\n\n```yaml\nstages:\n   default:\n     - name: \"Setup users\"\n       delete_entities:\n       -  path: /etc/passwd\n          entity: |\n                  kind: \"user\"\n                  username: \"foo\"\n                  password: \"x\"\n                  uid: 0\n                  gid: 0\n                  info: \"Foo!\"\n                  homedir: \"/home/foo\"\n                  shell: \"/bin/bash\"\n```\n### `stages.\u003cstageID\u003e.[\u003cstepN\u003e].modules`\n\nA list of kernel modules to load.\n\n```yaml\nstages:\n   default:\n     - name: \"Setup users\"\n       modules:\n       - nvidia\n```\n### `stages.\u003cstageID\u003e.[\u003cstepN\u003e].systemctl`\n\nA list of systemd services to `enable`, `disable`, `mask` or `start`.\n\nA list of overrides to apply to the service files witht he following fields:\n - `service`: The service name to add the override for. Required. `.service` extension is appended to the name if not provided.\n   - `name`: The name of the override file. Optional, if not provided it will be named `override-yip.conf`. `.conf` extension is appended to the name if not provided.\n - `content`: The content of the override file.\n\nOverrides don't check if the service is enabled or disabled, or even if it exists, it will apply the override anyway.\n\n```yaml\nstages:\n   default:\n     - name: \"Setup users\"\n       systemctl:\n         enable:\n          - systemd-timesyncd\n          - cronie\n         mask:\n          - purge-kernels\n         disable:\n          - crond\n         start:\n          - cronie\n         overrides:\n            - service: \"systemd-timesyncd\"\n              name: \"override-custom.conf\"\n              content: |\n                [Service]\n                ExecStart=\n                ExecStart=/usr/lib/systemd/systemd-timesyncd\n```\n### `stages.\u003cstageID\u003e.[\u003cstepN\u003e].environment`\n\nA map of variables to write in `/etc/environment`, or otherwise specified in `environment_file`\n\n```yaml\nstages:\n   default:\n     - name: \"Setup users\"\n       environment:\n         FOO: \"bar\"\n```\n### `stages.\u003cstageID\u003e.[\u003cstepN\u003e].environment_file`\n\nA string to specify where to set the environment file\n\n```yaml\nstages:\n   default:\n     - name: \"Setup users\"\n       environment_file: \"/home/user/.envrc\"\n       environment:\n         FOO: \"bar\"\n```\n### `stages.\u003cstageID\u003e.[\u003cstepN\u003e].timesyncd`\n\nSets the `systemd-timesyncd` daemon file (`/etc/system/timesyncd.conf`) file accordingly. The documentation for `timesyncd` and all the options can be found [here](https://www.freedesktop.org/software/systemd/man/timesyncd.conf.html).\n\n```yaml\nstages:\n   default:\n     - name: \"Setup NTP\"\n       systemctl:\n         enable:\n         - systemd-timesyncd\n       timesyncd: \n          NTP: \"0.pool.org foo.pool.org\"\n          FallbackNTP: \"\"\n          ...\n```\n\n### `stages.\u003cstageID\u003e.[\u003cstepN\u003e].systemd_firstboot`\n\nRuns `systemd-firstboot` with the given map\n\n```yaml\nstages:\n   default:\n     - name: \"Setup Locale\"\n       systemd_firstboot:\n         keymap: us\n```\n\n### `stages.\u003cstageID\u003e.[\u003cstepN\u003e].commands`\n\nA list of arbitrary commands to run after file writes and directory creation.\n\n```yaml\nstages:\n   default:\n     - name: \"Setup something\"\n       commands:\n         - echo 1 \u003e /bar\n```\n\n### `stages.\u003cstageID\u003e.[\u003cstepN\u003e].datasource`\n\nSets to fetch user data from the specified cloud providers. It iterates\nthrough the list of providers and the first one that succeeds to\nextract some user data is the one being used. It populates provider\nspecific data into `/run/config` folder and the custom user data is stored\ninto the provided path.\n\n\n```yaml\nstages:\n   default:\n     - name: \"Fetch cloud provider's user data\"\n       datasource:\n         providers:\n           - \"aws\"\n           - \"digitalocean\"\n         path: \"/etc/cloud-data\"\n```\n\n### `stages.\u003cstageID\u003e.[\u003cstepN\u003e].layout`\n\nSets additional partitions on disk free space, if any, and/or expands the last\npartition. All sizes are expressed in MiB only and default value of `size: 0`\nmeans all available free space in disk. This plugin is useful to be used in\noem images where the default partitions might not suit the actual disk geometry.\n\n\n```yaml\nstages:\n   default:\n     - name: \"Repart disk\"\n       layout:\n         device:\n           # It will partition a device including the given filesystem label\n           # or partition label (filesystem label matches first) or the device\n           # provided in 'path'. The label check has precedence over path when\n           # both are provided.\n           label: COS_RECOVERY\n           path: /dev/sda\n         # Only last partition can be expanded and it happens before any other\n         # partition is added.\n         expand_partition:\n           size: 4096 #  size: 0 means all available free space\n         add_partitions:\n           - fsLabel: COS_STATE\n             size: 8192\n             # No partition label is applied if omitted\n             pLabel: state\n           - fsLabel: COS_PERSISTENT\n             # default filesystem is ext2 if omitted\n             filesystem: ext4\n```\n\n### `stages.\u003cstageID\u003e.[\u003cstepN\u003e].unpack_images`\n\nUnpacks a list of OCI images to disk.\nAccepts a list of `source` and `target` paths.\nThe `source` is the image to unpack, and the `target` is the path where the image will be unpacked.\nOptially a `platform` can be specified to unpack the image for a specific platform. By default it will unpack the image for the current platform.\n\n```yaml\nstages:\n   default:\n     - name: \"Unpack images\"\n       unpack_images:\n         - source: \"quay.io/luet/base:latest\"\n           target: \"/usr/local/luet/\"\n         - source: \"rancher/k3s:latest\"\n           target: \"/usr/local/k3s-arm64\"\n           platform: \"linux/arm64\"\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmudler%2Fyip","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmudler%2Fyip","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmudler%2Fyip/lists"}