Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/jtyr/ansible-yaml_inventory
Ansible dynamic inventory reading the inventory from YAML file.
https://github.com/jtyr/ansible-yaml_inventory
ansible ansible-inventory
Last synced: 3 months ago
JSON representation
Ansible dynamic inventory reading the inventory from YAML file.
- Host: GitHub
- URL: https://github.com/jtyr/ansible-yaml_inventory
- Owner: jtyr
- License: mit
- Created: 2017-03-27T13:03:46.000Z (almost 8 years ago)
- Default Branch: master
- Last Pushed: 2020-01-06T14:30:17.000Z (about 5 years ago)
- Last Synced: 2024-10-04T13:26:57.697Z (4 months ago)
- Topics: ansible, ansible-inventory
- Language: Python
- Size: 24.4 KB
- Stars: 19
- Watchers: 4
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-starred - jtyr/ansible-yaml_inventory - Ansible dynamic inventory reading the inventory from YAML file. (ansible)
README
yaml_inventory
==============
Ansible dynamic inventory script which reads the inventory from a specially
formatted YAML file.
Description
-----------
Standard Ansible inventory suffers of several issues:
- It must be a single file or multiple files in a directory.
- It has a flat structure where all groups are on the same level and relate to
each other via `:children` definition leading to long group names when trying
to capture more complex relationships. That also affects the names of the
`group_vars`.
- When using Vault file, it requires the files the be either named like the
group or to be placed inside a directory of the group name or to be
explicitely included in the play.
- Hosts that belong to multiple groups must be defined in multiple places.
- Sharing the same `group_vars` across multiple groups is a challenging problem.
This Ansible dynamic inventory script is trying to address these issues
by allowing the following features:
- Possibility to split single inventory into multiple files.
- Self-generating group names based on the YAML structure.
- Truly hierarchical vars files.
- Automatic `.vault` files loading.
- Possibility to add hosts to an other group.
- Possibility to include hosts from an other groups via regexp.
- Using vars files as a common template.
See the usage bellow for more details.
Installation
------------
```
$ git clone https://github.com/jtyr/ansible-yaml_inventory yaml_inventory
$ ln -s yaml_inventory/yaml_inventory.py hosts
$ ansible-playbook -i hosts site.yaml
```
Usage
-----
### Inventory YAML file
Here is an example of the standard Ansible inventory file:
```
[aws:children]
aws-dev
aws-qa
aws-stg
aws-prd
[aws-dev]
aws-dev-host01 ansible_host=192.168.1.15
[aws-dev:children]
aws-dev-jenkins
[aws-dev-jenkins]
aws-dev-jenkins01 ansible_host=192.168.1.16
[aws-qa]
aws-qa-host01 ansible_host=192.168.2.15
[aws-stg]
aws-stg-host01 ansible_host=192.168.3.15
[aws-prd]
aws-prd-host01 ansible_host=192.168.4.15
[azure:children]
azure-dev
azure-qa
azure-stg
azure-prd
[azure-dev]
azure-dev-host01 ansible_host=10.0.1.15
[azure-dev:children]
azure-dev-jenkins
[azure-dev-jenkins]
azure-dev-jenkins01 ansible_host=10.0.1.16
[azure-qa]
azure-qa-host01 ansible_host=10.0.2.15
[azure-stg]
azure-stg-host01 ansible_host=10.0.3.15
[azure-prd]
azure-prd-host01 ansible_host=10.0.4.15
```
And here is the same but in the YAML format:
```
---
aws:
dev:
:hosts:
- aws-dev-host01: { ansible_host: 192.168.1.15 }
jenkins:
:hosts:
- aws-dev-jenkins01: { ansible_host: 192.168.1.16 }
qa:
:hosts:
- aws-qa-host01: { ansible_host: 192.168.2.15 }
stg:
:hosts:
- aws-stg-host01: { ansible_host: 192.168.3.15 }
prd:
:hosts:
- aws-prd-host01: { ansible_host: 192.168.4.15 }
azure:
dev:
:hosts:
- azure-dev-host01: { ansible_host: 10.0.1.15 }
jenkins:
:hosts:
- azure-dev-jenkins01: { ansible_host: 10.0.1.16 }
qa:
:hosts:
- azure-qa-host01: { ansible_host: 10.0.2.15 }
stg:
:hosts:
- azure-stg-host01: { ansible_host: 10.0.3.15 }
prd:
:hosts:
- azure-prd-host01: { ansible_host: 10.0.4.15 }
```
The main YAML inventory should be stored in the `main.yaml` file located
by default in the `inventory` directory. The location can be changed in
the config file (`inventory_path` - see the `yaml_inventory.conf` file)
or via environment variable (`YAML_INVENTORY_PATH`).
This is an example of a monolithic inventory YAML file:
```
---
aws:
dev:
elk:
elasticsearch:
# Hosts of the aws-dev-elk-elasticsearch group
:hosts:
# Hosts with variables
- elk01: { ansible_host: 192.168.1.11 }
- elk02: { ansible_host: 192.168.1.12 }
- elk03: { ansible_host: 192.168.1.13 }
kibana:
# Hosts of the aws-dev-elk-kibana group
:hosts:
# Host with no variables
- elk04
```
The same like above but with YAML reference:
```
---
# This is a subset of the main data structure referenced bellow
aws-dev: &aws-dev
elk:
elasticsearch:
:hosts:
- elk01: { ansible_host: 192.168.1.11 }
- elk02: { ansible_host: 192.168.1.12 }
- elk03: { ansible_host: 192.168.1.13 }
kibana:
:hosts:
- elk04
# This is the main data structure
aws:
dev:
# Reference to the above data structure
<<: *aws-dev
```
This is the same like above but with the referenced content in a separate
file:
Content of the `aws-dev.yaml` file:
```
---
# This can be still referenced from the main YAML file
aws-dev: &aws-dev
elk:
elasticsearch:
:hosts:
- elk01: { ansible_host: 192.168.1.11 }
- elk02: { ansible_host: 192.168.1.12 }
- elk03: { ansible_host: 192.168.1.13 }
kibana:
:hosts:
- elk04
```
Content of the `main.yaml` file:
```
---
# This is the main data structure
aws:
dev:
# Refference the above data structure
<<: *aws-dev
```
The inventory script reads all YAML files from the `inventory` directory
and merges them all together. The `main.yaml` portion is always inserted
at the end so that the YAML references can still be resolved. Group names
are composed from elements of the tree separated by `-` sign.
As visible above, the YAML structure contains special keyword `:hosts`.
That keyword indicates that content of that section is a list of hosts.
Other keywords are `:vars`, `:template`, `:groups` and `:add_hosts`.
The following is an example of usage of all the keywords:
```
---
g1:
:hosts:
- ahost1
- ahost2
:vars:
# Variable for the group g1
my_var: 123
g2:
:hosts:
- bhost1
- bhost2
:groups:
# Add all hosts from group g2 into the group g1
- g1
g3:
:hosts:
- chost1
- chost2
:templates:
# This creates g3@template-g3 group which has the g3 group as its
# child. That will allow to load the inventory vars from the file in
# inventory/vars/template/g3. As the g3 group is a child of the
# template group, variables from the g3 inventory vars file can
# still override the vars in the template file.
- template-g3
g4:
:add_hosts:
# Regular expression to add hosts ahost2 and bhost2 into this group
- ^[ab]host2
```
### Inventory vars
Classical `group_vars` files can be structured in two levels only - files
in the `group_vars` directory and file in the directories located in the
`group_vars` directory. This is quite restrictive and forces the user to
capture the inventory groups structure in the `group_vars` file name.
This Ansible dynamic inventory is trying to address this issue by
introducing the inventory vars. Inventory vars are variation of the
`group_vars` with the difference that it copies the hierarchical
structure of YAML inventory file. The inventory vars directory is by
default in the `inventory/vars` directory but can be changed in the
config file (`inventory_vars_path`) or via environment variable
(`YAML_INVENTORY_VARS_PATH`). This feature is enabled by default but can
be disabled by setting the `create_symlinks` config option or the
`YAML_INVENTORY_CREATE_SYMLINKS` environment variable to value `no`.
The inventory vars file for a specific group can be called either like
the last element of the group name or like `all` inside a directory
called like the last element of the group name. If the group contains
another groups, only the second option is available because there cannot
coexist file and directory of the same name.
If this is the inventory file:
```
$ cat inventory/main.yaml
---
# This is a group containing another group (vars in `all` file)
aws:
dev:
# This is the leaf group (vars in `jenkins` file)
jenkins:
:hosts:
- aws-dev-jenkins01
```
then the corresponding file structure of the inventory vars can look like
this:
```
$ tree -p inventory/vars
inventory/vars
└── [drwxr-xr-x] aws
├── [-rw-r--r--] all
└── [drwxr-xr-x] dev
└── [-rw-r--r--] jenkins
```
If enabled, the inventory vars are symlinked into the `group_vars`
directory during the execution of the inventory script. The `group_vars`
file names are based on the structure of the inventory vars directory.
From the example above, the path `invenotory/aws/all` is symlinked like
`group_vars/aws` and the path `invenotory/aws/dev/jenkins` is symlinked
like `group_vars/aws-dev-jenkins`.
```
$ ls -la ./group_vars
total 8
drwxr-xr-x 2 jtyr users 4096 Mar 28 17:20 .
drwxr-xr-x 9 jtyr users 4096 Mar 27 10:10 ..
lrwxrwxrwx 1 jtyr users 21 Mar 28 17:20 aws -> ../inventory/vars/aws/all
lrwxrwxrwx 1 jtyr users 29 Mar 28 17:20 aws-dev-jenkins -> ../inventory/vars/aws/dev/jenkins
```
The script also simplifies the use of Vault files by automatically
creating relationship between the group (e.g. `mygroup`) and the secured
content of that group (`mygroup.vault`). This convention makes sure that
the Vault file is always loaded if it exists.
### Inventory script
The inventory script can be used as any other Ansible dynamic inventory.
With the default settings (the `main.yaml` inventory file in the
`./inventory` directory and the inventory vars in the `./inventory/vars`
directory) the command can be as follows:
```
$ ansible-playbook -i ./yaml_inventory.py site.yaml
```
The inventory script implements the standard `--list` and `--host`
command line options and can be influenced by a config file (see
`yaml_inventory.conf` file) or environment variables:
```
usage: yaml_inventory.py [-h] [--list] [--host HOST]
Ansible dynamic inventory reading YAML file.
optional arguments:
-h, --help show this help message and exit
--list list all groups and hosts
--host HOST get vars for a specific host
environment variables:
YAML_INVENTORY_CONFIG_PATH
location of the config file (default locations:
./yaml_inventory.conf
~/.ansible/yaml_inventory.conf
/etc/ansible/yaml_inventory.conf)
YAML_INVENTORY_PATH
location of the inventory directory (./inventory by default)
YAML_INVENTORY_VARS_PATH
location of the inventory vars directory (YAML_INVENTORY_PATH/vars by default)
YAML_INVENTORY_GROUP_VARS_PATH
location of the vars directory (./group_vars by default)
YAML_INVENTORY_SUPPORT_VAULTS\n'
flag to take in account .vault files (yes by default)
YAML_INVENTORY_CREATE_SYMLINKS
flag to create group_vars symlinks (yes by default)
```
### Combining it with other inventory scripts
Put the YAML dynamic inventory script together with the other (e.g.
[AWS EC2](http://docs.ansible.com/ansible/latest/intro_dynamic_inventory.html#example-aws-ec2-external-inventory-script))
dynamic inventory script into the `inventory_scripts` directory and name them
to be in alphabetical order that the YAML inventory is first and the other
inventory is second:
```
$ ls inventory_scripts
01_yaml_inventory
02_aws_inventory
```
Use the YAML inventory to create the desired inventory structure and the
other inventory to fill in the hosts into the groups:
```
$ cat inventory/main.yaml
---
aws:
dev:
jenkins:
:hosts: []
```
Then run Ansible like this:
```
$ ansible-playbook -i inventory_scripts site.yaml
```
Issues
------
- When using `create_symlinks = yes` and the inventory structure has changed,
the YAML inventory must be run manually before the Ansible runs because the
`group_vars` symlinks get created after the `group_vars` files are normally
read by Ansible.
- No Vault support for `all` group due to the circular relationship between
the `all.vault` and the `all` group.
- No support for encrypted variables available in Ansible v2.3+.
TODO
----
- Implement hosts enumeration.
- Refactor it into an inventory plugin to facilitate support for encrypted
variables.
License
-------
MIT
Author
------
Jiri Tyr