{"id":15009891,"url":"https://github.com/shalomb/tofu","last_synced_at":"2025-04-09T17:52:34.432Z","repository":{"id":62584888,"uuid":"127344846","full_name":"shalomb/tofu","owner":"shalomb","description":"A terraform-on-openstack dynamic inventory for ansible.","archived":false,"fork":false,"pushed_at":"2019-01-16T03:03:25.000Z","size":77,"stargazers_count":3,"open_issues_count":1,"forks_count":3,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-17T17:07:18.717Z","etag":null,"topics":["ansible","ansible-inventory","ansible-openstack","dynamic-inventory","openstack-ansible","python2","terraform","terraform-ansible-openstack","terraform-openstack","terraform-state","tofu","wrapper-script"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shalomb.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-03-29T20:36:48.000Z","updated_at":"2024-03-06T11:28:50.000Z","dependencies_parsed_at":"2022-11-03T21:38:45.905Z","dependency_job_id":null,"html_url":"https://github.com/shalomb/tofu","commit_stats":null,"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shalomb%2Ftofu","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shalomb%2Ftofu/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shalomb%2Ftofu/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shalomb%2Ftofu/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shalomb","download_url":"https://codeload.github.com/shalomb/tofu/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248083352,"owners_count":21045079,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ansible","ansible-inventory","ansible-openstack","dynamic-inventory","openstack-ansible","python2","terraform","terraform-ansible-openstack","terraform-openstack","terraform-state","tofu","wrapper-script"],"created_at":"2024-09-24T19:29:02.569Z","updated_at":"2025-04-09T17:52:34.412Z","avatar_url":"https://github.com/shalomb.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# tofu\n\nA TerraForm OpenStack dynamic inventory script that parses the JSON object\nfrom `terraform state pull` to generate a YAML or JSON blurb that is suitable\nfor an ansible dynamic inventory.\n\nThis approach is slightly different to the official\n[OpenStack dynamic inventory script](http://docs.ansible.com/ansible/2.5/user_guide/intro_dynamic_inventory.html#example-openstack-external-inventory-script)\nthat queries the OpenStack APIs to compose the inventory. The advantage with\n`tofu` is it aims to use the TerraForm state independently of the OpenStack state\nto do the same and with no other python library dependencies\n(e.g. [Shade](https://pypi.python.org/pypi/shade))\n\n## Requirements\n\n* Python 2.x\n* TerraForm\n\n## Usage\n\n```\n$ tofu.py --help\nusage: tofu.py [-h] [--dump] [--dir DIR] [--example] [--groupby GROUPBY] [--hosts] [--file FILE] [--json] [--yaml]\n\nTerraform OpenStack Dynamic Inventory Script\n\noptional arguments:\n  -h, --help         show this help message and exit\n  --dump             Dump the raw terraform JSON\n  --dir DIR          Dir to use for terraform state/config\n  --example          Show an example JSON inventory\n  --accessip         Use the instance access IP address for the value of\n                     ansible_host\n  --groupby GROUPBY  Instance attribute to group hosts by (default=name)\n  --groups GROUPS    Instance attribute with comma-separated list of groups\n                     (default=none)\n  --list             Output entire inventory (default, implied)\n  --hosts            Print entries for /etc/hosts\n  --file FILE        Path to file containing terraform state as JSON i.e.\n                     `terraform state pull`\n  --json             Output inventory as JSON (faster)\n  --yaml             Output inventory as YAML (default, slower)\n\n```\n\nThe basic use-case is to have ansible execute `tofu` to return a YAML/JSON\nobject to use as a\n[dynamic inventory](http://docs.ansible.com/ansible/latest/user_guide/intro_dynamic_inventory.html)\n\n```\nchmod +x tofu.py\nansible -i tofu.py -u ubuntu AZ1-BER-NORTH -m ping\n```\n\nOr for a less in-your-face integration ..\n\n```\n$ cat ansible.cfg\n\n[defaults]\ninventory = bin/tofu.py\n ```\n\n```\nchmod +x bin/tofu.py\n\nterraform plan\nterraform apply\nansible-playbook ./site.yml -l webserver-0[1:6]\n```\n\n### Grouping\n\nAs is often the case, the above is no good as infrastructure is composed of\nlogical groups of instances where you need to run different plays against\ndifferent groups. To support this, you can use `--groupby` to collate\nterraform resource instances into groups grouped by some instance attribute\nthat terraform holds in its state.\n\ne.g. consider the typical object representation of an instance in\nterraform's state (See `terraform state pull` for a complete JSON object)\nand assume there are other instances sharing similar attributes that can\nbe used as the key to do the grouping.\n\n```\nnginx-01:\n  ...\nnginx-02:\n  access_ip_v4: 172.28.216.195\n  access_ip_v6: ''\n  all_metadata.%: '9'\n  all_metadata.availability_zone: AZ1\n  all_metadata.cluster: reverse-proxy\n  all_metadata.datacenter: AZ1-BER-NORTH\n  all_metadata.description: nginx-member\n  all_metadata.external_network: floating-ip-network\n  all_metadata.flavor_name: m1.medium\n  all_metadata.owner: apportune\n  all_metadata.tags: reverse-proxy,proxy,nginx\n  all_metadata.tier: app\n  availability_zone: AZ1\n  flavor_id: '3'\n  flavor_name: m1.medium\n  force_delete: 'false'\n  id: fbc114e3-2bf4-4948-9379-892991e9ac3c\n  image_id: 2d1e80fb-2207-48b1-9184-ca9a3645102b\n  image_name: xenial\n  key_pair: apportune-ansible-keypair\n  metadata.%: '9'\n  metadata.availability_zone: AZ1\n  metadata.cluster: reverse-proxy\n  metadata.datacenter: AZ1-BER-NORTH\n  metadata.description: reverse-proxy member\n  metadata.external_network: floating-ip-network\n  metadata.flavor_name: m1.medium\n  metadata.owner: apportune\n  metadata.tags: reverse-proxy,load-balancer,proxy,nginx\n  metadata.tier: app\n  name: nginx-02\n  network.#: '1'\n  network.0.access_network: 'false'\n  network.0.fixed_ip_v4: 172.28.216.195\n  network.0.fixed_ip_v6: ''\n  network.0.floating_ip: ''\n  network.0.mac: fa:16:3e:ad:21:fd\n  network.0.name: home-net\n  network.0.port: ''\n  network.0.uuid: 8697f9f1-5a24-4a86-aed5-d5214d8ebec7\n  region: RegionOne\n  scheduler_hints.#: '1'\n  scheduler_hints.3495812989.additional_properties.%: '0'\n  scheduler_hints.3495812989.build_near_host_ip: ''\n  scheduler_hints.3495812989.different_host.#: '0'\n  scheduler_hints.3495812989.group: 3cf5de1a-9976-42d6-8166-10eab28b2582\n  scheduler_hints.3495812989.query.#: '0'\n  scheduler_hints.3495812989.same_host.#: '0'\n  scheduler_hints.3495812989.target_cell: ''\n  security_groups.#: '6'\n  security_groups.1268873843: mgmt-ha-in\n  security_groups.1379193405: ssh-in\n  security_groups.2240927461: web-in\n  security_groups.3115836137: check-mk-in\n  security_groups.3682986736: nrpe-in\n  security_groups.3814588639: default\n  stop_before_destroy: 'false'\n  user_data: 326307df82495cebcc95f6cdc5abbea55c634a3c\nnginx-03\n      ...\n```\n\nthen\n\n```\ntofu.py --groupby metadata.cluster # uses the metadata.cluster attribute\n                                   # of the instances to do the grouping\n\ntofu.py --groupby network.0.name   # Will group instances by their placement\n                                   # on their default/home networks\n\ntofu.py --groupby availability_zone # group by AZ\n\ntofu.py --groupby image_id         # Group by isntances sharing the same\n                                   # deployment image?\n\ntofu.py --groupby name             # Useful? Creates groups of size 1\n\n...\n```\n\n\u003e NOTE: You will need a wrapper script that becomes the executable that\n\u003e ansible invokes to call `tofu` to do grouping this way.\n\nIf you use\n[Server Groups](https://www.terraform.io/docs/providers/openstack/r/compute_servergroup_v2.html)\nin terraform to describe groups or affinity/anti-affinity policies then `tofu`\nwill use the servergroup name as the grouping key. This is useful in that it is\nprobably what you need and is a logical fit for `tofu` and so avoids the need\nfor a wrapper script.\n\nGrouping can also be done in a\n[static fashion](http://docs.ansible.com/ansible/latest/user_guide/intro_dynamic_inventory.html#static-groups-of-dynamic-groups)\n\nTo do grouping similar to Ansible Openstack inventory plugin, i.e. use a\ncomma-separated list of groups from metadata.groups attribute, you would use\n`--groups=metadata.groups`.\n\n### Generating /etc/hosts entries\n\n    tofu.py --hosts\n\n### Working with multiple terraform projects/directories\n\n    tofu.py --dir terraform/AZ1-BER-NORTH/ \u003e AZ1-BER_NORTH.json\n    tofu.py --dir terraform/AZ1-MUN-SOUTH/ \u003e AZ1-MUN-SOUTH.json\n\n    /path/to/json-merge-tool AZ1-BER-NORTH.json AZ1-MUN-SOUTH.json\n\nWhere `json-merge-tool` is some tool that merges the 2/many JSON files and\nreturns that as the dynamic inventory to `ansible`.\n\n## Controlling the IP address used for ansible_host\n\nBy default, `tofu` will use the value of the instances' first floating IP\naddress in the inventory to allow direct ansible/SSH access into the instances.\n\nIf this is not desired or if ansible is able to connect directly via ssh into\nthe instances, the `--accessip` flag dictates that `tofu` use the instance\nprivate/access IP address instead.\n\n## Troubleshooting\n\n`tofu` runs `terraform state ...`, so you will need to ensure it is run in\nthe same directory that `terraform` runs.\n\n    terraform state pull \u003e tf-state.json # Ensure this file is populated first\n\n    tofu.py --file tf-state.json --json | python -m json.tool\n\n    tofu.py --file tf-state.json --dump\n\n## TODO\n\n* Implement caching and `--refresh` semantics to speed up the dev cycle.\n* Support using hostnames or FQDNs for the value of `ansible_host` in\n  the inventory where using IP addresses may not be desirable.\n* Honour the `--host` argument for the inventory of a single host.\n* Change `--hosts` to mean the plural of `--host`, consider a better\n  argument to replace `--hosts` as we have now.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshalomb%2Ftofu","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshalomb%2Ftofu","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshalomb%2Ftofu/lists"}