{"id":15681645,"url":"https://github.com/manicminer/ansible-vars-plugin-aws","last_synced_at":"2025-05-07T12:42:19.128Z","repository":{"id":150672800,"uuid":"126207262","full_name":"manicminer/ansible-vars-plugin-aws","owner":"manicminer","description":"Vars plugin for Ansible to retrieve AWS resource information","archived":false,"fork":false,"pushed_at":"2018-04-22T20:22:09.000Z","size":68,"stargazers_count":12,"open_issues_count":1,"forks_count":3,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-31T10:11:09.525Z","etag":null,"topics":["ansible","ansible-plugin","ansible-plugins"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/manicminer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-03-21T16:18:05.000Z","updated_at":"2025-03-07T21:04:41.000Z","dependencies_parsed_at":"2023-06-26T00:01:46.673Z","dependency_job_id":null,"html_url":"https://github.com/manicminer/ansible-vars-plugin-aws","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manicminer%2Fansible-vars-plugin-aws","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manicminer%2Fansible-vars-plugin-aws/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manicminer%2Fansible-vars-plugin-aws/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manicminer%2Fansible-vars-plugin-aws/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/manicminer","download_url":"https://codeload.github.com/manicminer/ansible-vars-plugin-aws/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252879642,"owners_count":21818832,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ansible","ansible-plugin","ansible-plugins"],"created_at":"2024-10-03T16:58:02.481Z","updated_at":"2025-05-07T12:42:19.118Z","avatar_url":"https://github.com/manicminer.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Ansible AWS Vars Plugin\n\nThis is a drop-in plugin for Ansible 2.5+ which provides the following:\n\n* Searches one or more AWS accounts for VPC, subnet, security group and ELB target group details,\n* Matches tags of all these resources with configured set of tag names, then\n* Builds a hierarchical dictionary of resources mapped by tag values,\n* All the above information is made available to all hosts managed by Ansible by means of native host variables\n* Bonus feature: brings native support for multiple AWS accounts with automatic account switching once per playbook based on extra vars passed at runtime.\n\nRead below for more detailed explanations.\n\n\n# How To Use\n\nThis module is shipped with a skeleton structure with the intention that you can test it right out of this repository. An example AWS config file is given with `aws.ini` and wrapper scripts `ansible.sh`/`playbook.sh` to set some useful environment variables for you.\n\nHowever, it's more likely that you already have an Ansible project, in which case all you need to do it copy the `vars_plugins/` directory into your project root (relative to your playbooks). The plugin should be automatically detected by Ansible.\n\nIf you have a different vars_plugin directory configured in `ansible.cfg`, just drop `aws.py` and `aws.yml` it into that directory instead.\n\nYou'll want to change the settings in `vars_plugin/aws.yml` to match your environment. These settings are:\n\n`regions:`  \nThis is a list of regions where the plugin will look for resources\n\n`use_cache: [yes|no]`  \nWhether or not to cache resource details after retrieving them. Recommended.\n\n`cache_max_age: 600`  \nHow long to cache resource details before retrieving them again from AWS. Defaults to 600 seconds (10 mins).\n\n`cache_env_vars:`  \nA list of environment variables to inspect and save the values of when caching resource details. Should the values of any of these environment variables change, the cache will be invalidated.\n\n`aws_profiles:`  \nCan be either a list of profile names, or a dictionary having profile names as keys, and each value being a dictionary of extra variables to inspect when selecting a default account for the current playbook (see below). When a list, no matching is performed and no credentials are set.\n\n`vpc_tags:`  \nA list of tag keys to match when building a dictionary of VPC IDs. When specified, the global host variable `vpc_ids` contains a nested dictionary of resource IDs denominated by tag values. If not specified, no dictionary is set.\n\n`subnet_tags:`  \nA list of tag keys to match when building a dictionary of subnet IDs. When specified, the global host variable `subnet_ids` contains a nested dictionary of resource IDs denominated by tag values. If not specified, no dictionary is set.\n\n`security_group_tags:`  \nA list of tag keys to match when building a dictionary of security group IDs. When specified, the global host variable `security_group_ids` contains a nested dictionary of resource IDs denominated by tag values. If not specified, no dictionary is set.\n\n`elb_target_group_tags:`  \nA list of tag keys to match when building a dictionary of ELB target groups IDs. When specified, the global host variable `elb_target_group_arns` contains a nested dictionary of resource IDs denominated by tag values. If not specified, no dictionary is set.\n\n\n# Global Variables\n\nThe following host variables are set by this plugin for every host Ansible attempts to manage, essentially these are global variables usable anywhere within your playbooks or roles.\n\n- `aws_account_ids` - a dictionary of AWS account IDs with profile name as keys and account ID as values.\n- `aws_profile` - the currently selected AWS profile when matched by extra vars\n- `elb_target_groups` - a dictionary of ELB target groups with resource ID as keys and dictionary of useful information as values\n- `elb_target_group_arns` - a complex nested dictionary of ELB target group ARNs denominated by matched tag values\n- `security_groups` - a dictionary of security groups with resource ID as keys and dictionary of useful information as values\n- `security_group_ids` - a complex nested dictionary of security group IDs denominated by matched tag values\n- `subnets` - a dictionary of subnets with resource ID as keys and dictionary of useful information as values\n- `subnet_ids` - a complex nested dictionary of subnet IDs denominated by matched tag values\n- `vpcs` - a dictionary of VPCs with resource ID as keys and dictionary of useful information as values\n- `vpc_ids` - a complex nested dictionary of VPC IDs denominated by matched tag values\n\n\n# AWS Resources\n\nIn the configuration file for this plugin, you can specify a list of tag keys for each type of supported resource. When the plugin runs (right before playbook execution) it fetches resource descriptions from AWS and organizes them into hierarchical dictionaries based on the values of the configured tag keys.\n\nFor example, given this configuration:\n\n```yaml\nsubnet_tags:\n  - project\n  - env\n  - tier\n```\n\nand having subnets in your AWS account(s) like:\n\nSubnet ID       | Tag: project | Tag: env | Tag: tier\n--------------- | ------------ | -------- | ---------\nsubnet-aabbcc12 | apollo       | prod     | app\nsubnet-aabbcc13 | apollo       | prod     | app\nsubnet-aabbcc14 | apollo       | prod     | data\nsubnet-aabbcc15 | apollo       | prod     | data\nsubnet-aabbcc16 | apollo       | prod     | lb\nsubnet-aabbcc17 | apollo       | prod     | lb\nsubnet-aabbcc18 | apollo       | staging  | app\nsubnet-aabbcc19 | apollo       | staging  | app\nsubnet-aabbcc20 | apollo       | staging  | data\nsubnet-aabbcc21 | apollo       | staging  | data\nsubnet-aabbcc22 | apollo       | staging  | lb\nsubnet-aabbcc23 | apollo       | staging  | lb\nsubnet-aabbcc24 | manhattan    | prod     | app\nsubnet-aabbcc25 | manhattan    | prod     | app\nsubnet-aabbcc26 | manhattan    | prod     | data\nsubnet-aabbcc27 | manhattan    | prod     | data\nsubnet-aabbcc28 | manhattan    | prod     | lb\nsubnet-aabbcc29 | manhattan    | prod     | lb\nsubnet-aabbcc30 | manhattan    | staging  | app\nsubnet-aabbcc31 | manhattan    | staging  | app\nsubnet-aabbcc32 | manhattan    | staging  | data\nsubnet-aabbcc33 | manhattan    | staging  | data\nsubnet-aabbcc34 | manhattan    | staging  | lb\nsubnet-aabbcc35 | manhattan    | staging  | lb\n\nYou'll end up with a global dictionary like:\n\n```yaml\nsubnet_ids:\n  us-east-1:\n    apollo:\n      prod:\n        app:\n          - subnet-aabbcc12\n          - subnet-aabbcc13\n        data:\n          - subnet-aabbcc14\n          - subnet-aabbcc15\n        lb:\n          - subnet-aabbcc16\n          - subnet-aabbcc17\n      staging\n        app:\n          - subnet-aabbcc18\n          - subnet-aabbcc19\n        data:\n          - subnet-aabbcc20\n          - subnet-aabbcc21\n        lb:\n          - subnet-aabbcc22\n          - subnet-aabbcc23\n    manhattan:\n      prod:\n        app:\n          - subnet-aabbcc24\n          - subnet-aabbcc25\n        data:\n          - subnet-aabbcc26\n          - subnet-aabbcc27\n        lb:\n          - subnet-aabbcc28\n          - subnet-aabbcc29\n      staging\n        app:\n          - subnet-aabbcc30\n          - subnet-aabbcc31\n        data:\n          - subnet-aabbcc32\n          - subnet-aabbcc33\n        lb:\n          - subnet-aabbcc34\n          - subnet-aabbcc35\n```\n\nWhich you can reference like this:\n\n```yaml\n- hosts: localhost\n  connection: local\n  tasks:\n    - ec2:\n        instance_type: t2.micro\n        vpc_subnet_id: \"{{ subnet_ids['us-east-1']['manhattan']['staging']['app'] | random }}\"\n        state: present\n```\n\nThe same pattern is implemented for VPCs, subnets, security groups and ELB target groups, alowing you to specify and target resources in your playbooks without hard coding resource IDs, without using `*_facts` modules everywhere, and without needing to know the exact names of every resource. It's important to note that these resources can reside in any number of AWS accounts, as long as they can be reached by your Ansible control host.\n\n\n# Multi Account Support\n\nThe multi account support provided by this plugin comes in two flavors.\n\n1. With multiple accounts configured as AWS profiles on your Ansible control host, the plugin will traverse all accounts to retrieve resource information.\n2. Additionally, with rules configured in the configuration file, it will inspect any extra vars you specify when running your playbook and *automatically select* one of the profiles to use for your playbook execution. This is accomplished by requesting temporary credentials from STS and exporting them within Ansible as a set of `AWS_*` environment variables. These exported env vars are consumed by Boto2 and Boto3-based modules alike.\n\nNote that resources are still retrieved from all accounts, even when auto-selection is configured.\n\nFor example, given this configuration:\n\n```yaml\naws_profiles:\n  staging:\n    env:\n      - development\n      - staging\n  production:\n    env: production\n```\n\nAnd the following playbook invocation:\n\n```\n$ ansible-playbook do-a-thing.yml -e env=staging\n```\n\nThe plugin would select the `staging` profile, obtain temporary credentials using that profile, then export those credentials to be automatically used by any AWS modules/tasks in your playbook.\n\nAny number of extra vars can be specified for each profile, and all must match for a profile to be selected. For example, you might have something like this, where each project resides in its own AWS account, and development happens in a default account (possibly the developers' own accounts):\n\n```yaml\naws_profiles:\n  default:\n    env: development\n  apollo-staging:\n    env: staging\n    project: apollo\n  apollo-production:\n    env: production\n    project: apollo\n  manhattan-staging:\n    env: staging\n    project: manhattan\n  manhattan-production:\n    env: production\n    project: manhattan\n  ops:\n    env: ops\n```\n\nThe primary limitation of this approach is that the AWS account is selected once, prior to playbook execution. However, it's possible to trivially use a different account for a given task by requesting credentials with the `sts_assume_role` module and specifying them explicitly for a task, which overrides the environment variables set by this plugin.\n\n# Putting It All Together\n\nBy passing the `env`, `project` and `service` extra vars to `ansible-playbook`, you can invoke the multi-account support to auto-select the correct AWS account to use, and use those same extra vars to pick the right resources. In the example below, the instance will be launched in the desired account, with the appropriate security group and subnet for the service type.\n\n```\n$ ansible-playbook launch.yml -e env=staging -e project=manhattan -e service=app\n```\n```yaml\n- hosts: localhost\n  connection: local\n\n  vars:\n    region: us-east-1\n\n  tasks:\n\n    - name: Launch instance\n      ec2:\n        region: \"{{ region }}\"\n        instance_tags:\n          env: \"{{ env }}\"\n          project: \"{{ project }}\"\n          service: \"{{ service }}\"\n        instance_type: t2.micro\n        group_id: \"{{ security_group_ids[region][project][env][service] }}\"\n        vpc_subnet_id: \"{{ subnet_ids[region][project][env][service] | random }}\"\n        wait: yes\n      register: result_ec2\n\n    - name: Register in inventory\n      add_host:\n        name: \"{{ item.private_ip }}\"\n        groups: launch\n        ec2_id: \"{{ item.id }}\"\n      when: item.state == 'running'\n      with_flattened:\n        - \"{{ result_ec2.results | map(attribute='instances') | list }}\"\n\n    - name: Wait for SSH\n      wait_for:\n        host: \"{{ item }}\"\n        port: 22\n        timeout: 420\n        state: started\n      with_items: \"{{ groups.launch }}\"\n\n\n- hosts: launch\n  roles:\n    - role: common\n```\n\n# Also Note\n\n## Regions\n\nFor maximum control, this plugin requires that regions be explicitly configured. The configuration setting `regions` is a list. All regions configured will be searched for resources, and the resulting dictionaries are nested by region.\n\n## Resource Caching\n\nJust like Ansible's EC2 inventory script, this plugin caches all the resource information it finds, in order to speed up subsequent playbook executions. The cache can be disabled by setting `use_cache: no` in the configuration file, and the cache timeout (which defaults to 10 minutes) can be specified [in seconds] with the `cache_max_age` setting.\n\nIt's possible that factors outside of Ansible could invalidate cached information, so it's also possible to configure one or more environment variables, the values of which will be saved and if they change between playbook runs, the cache will be automatically invalidated.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmanicminer%2Fansible-vars-plugin-aws","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmanicminer%2Fansible-vars-plugin-aws","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmanicminer%2Fansible-vars-plugin-aws/lists"}