{"id":13582192,"url":"https://github.com/NeonSludge/ansible-dns-inventory","last_synced_at":"2025-04-06T14:30:28.339Z","repository":{"id":47518047,"uuid":"220020211","full_name":"NeonSludge/ansible-dns-inventory","owner":"NeonSludge","description":"A tool that processes sets of host attributes stored as DNS TXT records or key/value pairs in etcd to create a tree-like inventory of your infrastructure that can be immediately consumed by Ansible or exported in several helpful formats.","archived":false,"fork":false,"pushed_at":"2024-12-12T00:12:06.000Z","size":223,"stargazers_count":16,"open_issues_count":2,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-24T03:43:58.275Z","etag":null,"topics":["ansible","ansible-dynamic-inventory","ansible-inventories","ansible-inventory","automation","configuration-management","dns-inventory","dynamic-inventories","dynamic-inventory","inventory"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NeonSludge.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-11-06T14:48:40.000Z","updated_at":"2024-11-08T02:46:15.000Z","dependencies_parsed_at":"2025-02-16T09:32:39.782Z","dependency_job_id":"105289ee-e7c8-43f8-9264-66b5f94f54f1","html_url":"https://github.com/NeonSludge/ansible-dns-inventory","commit_stats":{"total_commits":161,"total_committers":2,"mean_commits":80.5,"dds":0.006211180124223614,"last_synced_commit":"d388ae8fedf7eb676f4dfac2594d84c138cf42a4"},"previous_names":[],"tags_count":34,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NeonSludge%2Fansible-dns-inventory","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NeonSludge%2Fansible-dns-inventory/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NeonSludge%2Fansible-dns-inventory/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NeonSludge%2Fansible-dns-inventory/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NeonSludge","download_url":"https://codeload.github.com/NeonSludge/ansible-dns-inventory/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247495610,"owners_count":20948084,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ansible","ansible-dynamic-inventory","ansible-inventories","ansible-inventory","automation","configuration-management","dns-inventory","dynamic-inventories","dynamic-inventory","inventory"],"created_at":"2024-08-01T15:02:29.033Z","updated_at":"2025-04-06T14:30:28.007Z","avatar_url":"https://github.com/NeonSludge.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# ansible-dns-inventory\n\n[![Go Report Card](https://goreportcard.com/badge/github.com/NeonSludge/ansible-dns-inventory)](https://goreportcard.com/report/github.com/NeonSludge/ansible-dns-inventory)\n[![Go Reference](https://pkg.go.dev/badge/github.com/NeonSludge/ansible-dns-inventory.svg)](https://pkg.go.dev/github.com/NeonSludge/ansible-dns-inventory)\n\nA CLI tool (and a library) that processes sets of host attributes stored as DNS TXT records or key/value pairs in etcd to create a tree-like inventory of your infrastructure. It can be used directly as an Ansible [dynamic inventory script](https://docs.ansible.com/ansible/latest/user_guide/intro_dynamic_inventory.html) or export the inventory in several helpful formats.\n\n## Features\n\n- Files and environment variables are supported as configuration sources. \n- DNS and etcd are available as data sources.\n- **(DNS data source)** two modes of operation: zone transfers and regular DNS queries.\n- **(DNS data source)** TSIG support for zone transfers.\n- **(Etcd data source)** authentication and mTLS support.\n- **(Etcd data source)** importing host records from a YAML file.\n- Unlimited number and length of inventory tree branches.\n- Predictable and stable inventory structure.\n- Multiple records per host supported.\n- Optional custom Ansible variables in host records (see caveats in the 'Host variables' section).\n- Can be used as a library.\n\n## Usage\n\n```txt\nUsage of dns-inventory:\n  -attrs\n    \texport host attributes\n  -format string\n    \tselect export format, if available (default \"yaml\")\n  -groups\n    \texport groups\n  -host string\n    \tproduce a JSON dictionary of host variables for Ansible\n  -hosts\n    \texport hosts\n  -import string\n    \timport host records from file\n  -list\n    \tproduce a JSON inventory for Ansible\n  -tree\n    \texport raw inventory tree\n  -version\n    \tdisplay ansible-dns-inventory version and build info\n```\n\n## Prerequisites\n\n### DNS data source\n\n1. Allow zone transfers (AXFR) from your DNS server to the host that is going to be running the `dns-inventory` utility and setup TSIG parameters in the configuration file (if needed) or use the no-transfer mode (the `dns.notransfer.enabled` parameter).\n2. Add one or more properly formatted DNS TXT records either for the managed hosts themselves or for a special host (the `dns.notransfer.host` parameter) if you're using the no-transfer mode.\n3. Set other relevant parameters in the configuration file or via environment variables.\n\n### Etcd data source\n\n1. Add one or more properly formatted key/value pairs for all managed hosts.\n2. Set other relevant parameters in the configuration file or via environment variables.\n\n## Configuration file\n\n`ansible-dns-inventory` can use a YAML configuration file, a set of environment variables or both as its configuration source.\n\nIt will try to load the file specified in the `ADI_CONFIG_FILE` environment variable if it is defined.\nIf this variable is not defined or has an empty value, it looks for an `ansible-dns-inventory.yaml` file inside these directories (in this specific order):\n\n1. current working directory\n2. `\u003chome directory\u003e/.ansible/`\n3. `/etc/ansible/`\n\n`ansible-dns-inventory` will panic if a configuration file was found but there was a problem reading it.\nIf no configuration file was found, it will fall back to using default values and environment variables.\n\nEvery parameter can also be overriden by a corresponding environment variable.\nThere is a [template](config/ansible-dns-inventory.yaml) in this repository that lists descriptions, environment variable names and default values for all available parameters.\n\n### Example of a config file\n\n```yaml\ndatasource: dns\ndns:\n  server: \"10.100.100.1:53\"\n  timeout: \"120s\"\n  zones:\n    - server.local.\n    - infra.local.\netcd:\n  endpoints:\n    - 10.100.100.1:2379\n    - 10.100.100.2:2379\n    - 10.100.100.3:2379\n  tls:\n    insecure: true\ntxt:\n  kv:\n    separator: \"|\"\n  keys:\n    env: \"PRJ\"\n\n```\n\n## Host records\n\n### DNS data source\n\nThere are two ways to add a host to the inventory:\n\n1. Create a DNS TXT record for this host and format it properly, specifying host attributes as a set of key/value pairs. One host can have an unlimited number of TXT records: all of them will be parsed by `ansible-dns-inventory`.\n2. Enable the no-transfer mode, add a TXT record for the special host (`ansible-dns-inventory.your.domain` by default) and format it properly, referencing the host you want to add to your inventory and specifying its attributes as a set of key/value pairs. Again, one host can have any number of records here.\n\nHere is an example of using both of these ways:\n\n#### Example of a DNS TXT record (regular mode)\n\n| Host                  | TXT record                                                                       |\n| --------------------- | -------------------------------------------------------------------------------- |\n| `app01.infra.local`   | `OS=linux;ENV=dev;ROLE=app;SRV=tomcat_backend_auth;VARS=key1=value1,key2=value2` |\n\n#### Example of a DNS TXT record (no-transfer mode)\n\n| Host                                | TXT record                                                                                         |\n| ----------------------------------- | -------------------------------------------------------------------------------------------------- |\n| `ansible-dns-inventory.infra.local` | `app01.infra.local:OS=linux;ENV=dev;ROLE=app;SRV=tomcat_backend_auth;VARS=key1=value1,key2=value2` |\n\nThe separator between the hostname and the attribute string in the no-transfer mode is customizable (the `dns.notransfer.separator` parameter).\n\n### Etcd data source\n\nThere is only one way of adding host records to an etcd data source.\nYou create a key/value pair where the value is formatted the same way as with the DNS data source and the key name must be constructed by strictly following this scheme:\n\n`\u003cprefix\u003e/\u003czone\u003e/\u003chostname\u003e/\u003cindex\u003e`\n\n...where:\n\n- `\u003cprefix\u003e` is the same as the `etcd.prefix` configuration parameter value\n- `\u003czone\u003e` is one of the zones listed in the `etcd.zones` parameter\n- `\u003chostname\u003e` is the FQDN of a host\n- `\u003cindex\u003e` starts at 0 and is incremented for each additional record belonging to the same host.\n\n#### Example of a key/value pair\n\n| Key                                                  | Value                                                                            |\n| ---------------------------------------------------- | -------------------------------------------------------------------------------- |\n| `ANSIBLE_INVENTORY/infra.local./app01.infra.local/0` | `OS=linux;ENV=dev;ROLE=app;SRV=tomcat_backend_auth;VARS=key1=value1,key2=value2` |\n\n\n\n### Host attributes (default keys)\n\n| Key  | Description                                                                                                                                                 |\n| ---- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| OS   | Operating system identifier. Required.                                                                                                                      |\n| ENV  | Environment identifier. Required.                                                                                                                           |\n| ROLE | Host role identifier(s). Required. Can be a comma-delimited list.                                                                                           |\n| SRV  | Host service identifier(s). This will be split further using the `txt.keys.separator` to produce a hierarchy of groups. Required. Can also be a comma-delimited list. |\n| VARS | Optional host variables.                                                                                                                                    |\n\nAll keys and separators are customizable via `ansible-dns-inventory`'s config file.\nValues are validated and can only contain numbers and letters of the Latin alphabet, except for the service identifier(s) which can also contain the `txt.keys.separator` symbol.\n\nAll host attributes (except for `VARS`) can be referenced by their keys in Ansible code via the `inventory_attributes` group variable. Its availability doesn't depend on the host variables feature (see below).\n\n### Host variables\n\n`ansible-dns-inventory` supports passing additional host variables to Ansible via the `VARS` attribute. This feature is disabled by default, you can enable it by setting the `txt.vars.enabled` parameter to `true`.\nThis is meant to be used in cases where storing some Ansible host variables directly in TXT records could be a good idea. For example, you might want to put variables like `ansible_user` there.\n\nWARNING: This feature adds an additional DNS request for every host in your inventory so be careful when using it with large inventories.\nThe no-transfer mode may particularly suffer a perfomance hit if host variables are used.\n\n## Inventory structure\n\nIn general, if you have a single TXT record for a `HOST` and this record has all 4 required attributes set then this `HOST` will end up in this hierarchy of groups:\n\n```txt\n@all:\n  |--@all_\u003cROLE\u003e:\n  |  |--@all_\u003cROLE\u003e_\u003cSRV[1]\u003e:\n  |  |  |--@all_\u003cROLE\u003e_\u003cSRV[1]\u003e_\u003cSRV[2]\u003e:\n  |  |  |  |--@all_\u003cROLE\u003e_\u003cSRV[1]\u003e_\u003cSRV[2]\u003e_..._\u003cSRV[n]\u003e:\n  |  |  |  |  |--\u003cHOST\u003e\n  |--@all_host:\n  |  |--@all_host_\u003cOS\u003e:\n  |  |  |--\u003cHOST\u003e\n  |--@\u003cENV\u003e:\n  |  |--@\u003cENV\u003e_\u003cROLE\u003e:\n  |  |  |--@\u003cENV\u003e_\u003cROLE\u003e_\u003cSRV[1]\u003e:\n  |  |  |  |--@\u003cENV\u003e_\u003cROLE\u003e_\u003cSRV[1]\u003e_\u003cSRV[2]\u003e:\n  |  |  |  |  |--@\u003cENV\u003e_\u003cROLE\u003e_\u003cSRV[1]\u003e_\u003cSRV[2]\u003e_..._\u003cSRV[n]\u003e:\n  |  |  |  |  |  |--\u003cHOST\u003e\n  |  |--@\u003cENV\u003e_host:\n  |  |  |--@\u003cENV\u003e_host_\u003cOS\u003e:\n  |  |  |  |--\u003cHOST\u003e\n```\n\nLet's say you have these records in your DNS server:\n\n| Host                | TXT record                                            |\n| ------------------- | ----------------------------------------------------- |\n| `app01.infra.local` | `OS=linux;ENV=dev;ROLE=app;SRV=tomcat_backend_auth`   |\n| `app02.infra.local` | `OS=linux;ENV=dev;ROLE=app;SRV=tomcat_backend_auth`   |\n| `app03.infra.local` | `OS=linux;ENV=dev;ROLE=app;SRV=tomcat_backend_media`  |\n\nThese will produce the following Ansible inventory tree:\n\n```txt\n@all:\n  |--@all_app:\n  |  |--@all_app_tomcat:\n  |  |  |--@all_app_tomcat_backend:\n  |  |  |  |--@all_app_tomcat_backend_auth:\n  |  |  |  |  |--app01.infra.local\n  |  |  |  |  |--app02.infra.local\n  |  |  |  |--@all_app_tomcat_backend_media:\n  |  |  |  |  |--app03.infra.local\n  |--@all_host:\n  |  |--@all_host_linux:\n  |  |  |--app01.infra.local\n  |  |  |--app02.infra.local\n  |  |  |--app03.infra.local\n  |--@dev:\n  |  |--@dev_app:\n  |  |  |--@dev_app_tomcat:\n  |  |  |  |--@dev_app_tomcat_backend:\n  |  |  |  |  |--@dev_app_tomcat_backend_auth:\n  |  |  |  |  |  |--app01.infra.local\n  |  |  |  |  |  |--app02.infra.local\n  |  |  |  |  |--@dev_app_tomcat_backend_media:\n  |  |  |  |  |  |--app03.infra.local\n  |  |--@dev_host:\n  |  |  |--@dev_host_linux:\n  |  |  |  |--app01.infra.local\n  |  |  |  |--app02.infra.local\n  |  |  |  |--app03.infra.local\n  |--@ungrouped:\n```\n\n## Export mode\n\n`ansible-dns-inventory` can also export the inventory in several formats. This makes it possible to use your inventory in some third-party software.\nAn example of this use case would be using this output as a dictionary in a [Logstash translate filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-translate.html#plugins-filters-translate-dictionary_path) to populate a `groups` field during log processing to be able to filter events coming from a specific group of hosts.\n\nThere are several export modes, which support different export formats.\n\n| Flag      | Description                                                             | Formats                                 |\n| --------- | ----------------------------------------------------------------------- | --------------------------------------- |\n| `-hosts`  | Export hosts, mapping each one to a list of groups.                     | `json`, `yaml`, `yaml-list`, `yaml-csv` |\n| `-groups` | Export groups, mapping each one to a list of hosts.                     | `json`, `yaml`, `yaml-list`, `yaml-csv` |\n| `-attrs`  | Export hosts, mapping each one to a list of dictionaries of attributes. | `json`, `yaml`, `yaml-flow`             |\n| `-tree`   | Export the raw inventory tree.                                          | `json`, `yaml`                          |\n\nThe default format is always `yaml`.\n\nThe `-attrs` mode exports a list of dictionaries of attributes for each host. If a host has multiple TXT records or multiple elements in a comma-separated list in the `ROLE` or `SRV` attribute, the attribute list for this host in the `-attrs` output will contain multiple dictionaries: one for each detected attribute \"set\".\n\n### Examples\n\n```txt\n$ dns-inventory -hosts -format yaml-list\n...\n\"app01.infra.local\": [\"all\", \"all_app\", \"all_app_tomcat\", \"all_host\", ...]\n...\n\n$ dns-inventory -hosts -format yaml-csv\n...\n\"app01.infra.local\": \"all,all_app,all_app_tomcat,all_host,...\"\n...\n\n$ dns-inventory -attrs -format yaml-flow\n...\n\"app01.infra.local\": [{\"OS\": \"linux\", \"ENV\": \"dev\", \"ROLE\": \"app\", \"SRV\": \"tomcat_backend_auth\", \"VARS\": \"key1=value1,key2=value2\"}]\n...\n```\n\n## Import mode\n\nSome `ansible-dns-inventory` datasources support importing host records from a YAML file. These currently include:\n- etcd datasource\n\nTo populate one of these datasources with host records, first create a YAML file with the same structure as the `-attrs` export mode output:\n```\n# cat import.yaml\napp01.infra.local:\n- ENV: dev\n  OS: linux\n  ROLE: app\n  SRV: tomcat_backend_auth\n  VARS: ansible_host=10.0.0.1\napp02.infra.local:\n- ENV: dev\n  OS: linux\n  ROLE: app\n  SRV: tomcat_backend_auth\n  VARS: ansible_host=10.0.0.2\n```   \n\nCustom host attribute keys will be expected here if set in the configuration (`txt.keys`).\n\nThen run `ansible-dns-inventory` in the import mode:\n```\ndns-inventory -import ./import.yaml\n```\n\n## Roadmap\n\n- [x] Implement key-value stores support (etcd, Consul, etc.).\n- [x] Support using `ansible-dns-inventory` as a library.\n- [x] Implement import mode for some of the datasources. (implemented for the etcd datasource)\n- [ ] Support more datasource types.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FNeonSludge%2Fansible-dns-inventory","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FNeonSludge%2Fansible-dns-inventory","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FNeonSludge%2Fansible-dns-inventory/lists"}