{"id":16139416,"url":"https://github.com/networkop/netbox-awx-automation","last_synced_at":"2026-01-20T01:59:58.023Z","repository":{"id":47843283,"uuid":"484379682","full_name":"networkop/netbox-awx-automation","owner":"networkop","description":"fork of https://gitlab.com/nvidia-networking/systems-engineering/poc-support/netbox-awx-automation","archived":false,"fork":false,"pushed_at":"2022-04-22T09:47:54.000Z","size":1122,"stargazers_count":2,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-02-12T23:45:16.225Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jinja","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/networkop.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-04-22T09:47:37.000Z","updated_at":"2024-08-30T06:23:43.000Z","dependencies_parsed_at":"2022-08-22T21:30:33.806Z","dependency_job_id":null,"html_url":"https://github.com/networkop/netbox-awx-automation","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/networkop%2Fnetbox-awx-automation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/networkop%2Fnetbox-awx-automation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/networkop%2Fnetbox-awx-automation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/networkop%2Fnetbox-awx-automation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/networkop","download_url":"https://codeload.github.com/networkop/netbox-awx-automation/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247526675,"owners_count":20953141,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-09T23:49:01.191Z","updated_at":"2026-01-20T01:59:57.997Z","avatar_url":"https://github.com/networkop.png","language":"Jinja","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Network Automation with Netbox and Ansible AWX\n\nThe goal of this lab is to demonstrate how to use Ansible AWX (Tower) to configure Cumulus Linux devices based on information extracted from Netbox. This demo will walk you through the following steps:\n\n* Initial Netbox configuration — populating the base Netbox data model with device information and IP address details.\n* Configuring AWX — using Netbox as an inventory source for AWX and pulling device and IPAM details from Netbox.\n* Using Netbox as a configuration source of truth — populating Netbox with configuration context that will be used by Ansible playbooks to generate final device configs.\n\n![](https://gitlab.com/nvidia-networking/systems-engineering/poc-support/netbox-awx-automation/-/raw/main/diagram-logical.png)\n\n\n## Lab details\n\nBoth AWX and Netbox are deployed in the Kubernetes cluster running inside the `netq-ts` server. Netbox is deployed using the [`bootc/netbox-chart`](https://github.com/bootc/netbox-chart) helm chart and AWX is deployed using the [AWX operator](https://github.com/ansible/awx-operator). They both share the same Postgres database that is deployed as a part of Netbox helm chart. \n\n\u003e **NOTE**: For instructions on how to build the demo, install and configure both Netbox and AWX see the see the [`./air`](https://gitlab.com/nvidia-networking/systems-engineering/poc-support/netbox-awx-automation/-/tree/main/air) directory.\n\n![](https://gitlab.com/nvidia-networking/systems-engineering/poc-support/netbox-awx-automation/-/raw/main/deployment.png)\n\n\n| Device/Application | sw version | username | password | \n| -- | -- | -- | -- | \n| oob-mgmt-server | Ubuntu 18.04 | ubuntu | nvidia | \n| netq-ts | NetQ 4.0.0 | cumulus | cumulus | \n| leaf01, leaf02 | CL 5.0 | cumulus | CumulusLinux! | \n| netbox | v3.0.11 |  admin | admin | \n| AWX | 19.5.0 | admin | rncnVRf949WvvrZGxQKxSOE0g5bl9mFJ | \n\nIn order to connect to the web UI of Netbox and AWX, we'll use SSH port forwarding through `oob-mgmt-server`. Change to the \"Advanced\" lab view in Air and click \"Enable SSH Service\". Use the following command when connecting to the `oob-mgmt-server` (adjust SSH URL based on the generated host and port numbers):\n\n```\nssh -L 8080:192.168.200.250:30845 -L 8081:192.168.200.250:31768 ssh://ubuntu@worker07.air.nvidia.com:22708\n```\n\nThis should make Netbox available on [localhost:8080](http://localhost:8080) and AWX available on [localhost:8081](http://localhost:8081).\n\n## 1. Configuring Netbox\n\nThe default way of interacting with Netbox is via its web UI, however, in this demo instead of including dozens of screenshots, we'll be using the [netbox shell](https://netbox.readthedocs.io/en/stable/administration/netbox-shell/) in order to configure Netbox programmatically. \n\n\nConnect to the `netq-ts` node and start a shell inside the `citc-netbox` Pod:\n\n```bash\nubuntu@oob-mgmt-server:~$ ssh cumulus@netq-ts\ncumulus@netq-ts:~$ sudo -i\nroot@netq-ts:~# kubectl exec -it deploy/citc-netbox bash\nsource /opt/netbox/venv/bin/activate\n/opt/netbox/netbox/manage.py nbshell\n```\n\nStart by creating the basic netbox object model layout with [site](https://netbox.readthedocs.io/en/stable/core-functionality/sites-and-racks/#sites), [device role](https://netbox.readthedocs.io/en/stable/core-functionality/devices/#device-roles) and [manufacturer](https://netbox.readthedocs.io/en/stable/core-functionality/device-types/#manufacturers) details. \n\n```python\nSite(name=\"CITC\", status=\"active\").save()\nsite = Site.objects.get(name=\"CITC\")\n\nDeviceRole(name=\"leaf\").save()\nrole = DeviceRole.objects.get(name=\"leaf\")\n\nManufacturer(name=\"nvidia\").save()\nm = Manufacturer.objects.get(name=\"nvidia\")\n\nDeviceType(model=\"vx\", manufacturer=m).save()\nt = DeviceType.objects.get(model=\"vx\")\n\nInterfaceTemplate(name=\"eth0\", device_type=t, mgmt_only=True).save()\nInterfaceTemplate(name=\"lo\", device_type=t).save()\nfor i in range(1,3): \n\tInterfaceTemplate(name=f\"swp{i}\", device_type=t, type=\"virtual\").save()\n```\n\nNow we can put all these details together to add the two lab devices:\n\n```python\nDevice(name=\"leaf01\", device_role=role, site=site, device_type=t).save()\nleaf01 = Device.objects.get(name=\"leaf01\")\nDevice(name=\"leaf02\", device_role=role, site=site, device_type=t).save()\nleaf02 = Device.objects.get(name=\"leaf02\")\n```\n\nFinally, populate Netbox IPAM with details required to configure the two lab devices:\n\n```python\n# create mgmt vrf and prefix\nVRF(name=\"mgmt\").save()\nvrf_mgmt = VRF.objects.get(name=\"mgmt\")\nPrefix(prefix=\"192.168.200.0/24\", vrf=vrf_mgmt).save()\nprefix = Prefix.objects.get(prefix=\"192.168.200.0/24\")\n\n# get a pointer to `eth0` interface\nleaf01_eth0 = Interface.objects.get(name=\"eth0\", device=leaf01)\nleaf02_eth0 = Interface.objects.get(name=\"eth0\", device=leaf02)\n\n# create OOB IP and assign it to `eth0` interface\nIPAddress(address=\"192.168.200.2\", vrf=vrf_mgmt, assigned_object=leaf01_eth0 ).save()\nleaf01_ip = IPAddress.objects.get(address=\"192.168.200.2\")\nleaf01.primary_ip4=leaf01_ip\nleaf01.save()\n\nIPAddress(address=\"192.168.200.3\", vrf=vrf_mgmt, assigned_object=leaf02_eth0 ).save()\nleaf02_ip = IPAddress.objects.get(address=\"192.168.200.3\")\nleaf02.primary_ip4=leaf02_ip\nleaf02.save()\n\n# create `default` vrf and assign loopback ips\nVRF(name=\"default\").save()\nvrf_default = VRF.objects.get(name=\"default\")\nPrefix(prefix=\"10.0.1.0/24\", vrf=vrf_default ).save()\n\nleaf01_lo = Interface.objects.get(name=\"lo\", device=leaf01)\nleaf02_lo = Interface.objects.get(name=\"lo\", device=leaf02)\n\nIPAddress(address=\"10.0.1.11\", vrf=vrf_default, assigned_object=leaf01_lo ).save()\nIPAddress(address=\"10.0.1.12\", vrf=vrf_default, assigned_object=leaf02_lo ).save()\n```\n\nThis is all what we need to populate basic Netbox data. This can be verified using Netbox UI at [localhost:8080](http://localhost:8080).\n\n\u003e **NOTE**: We're only configuring a minimal set of details about our network and not including things like interface connections or rack layouts. Although these details are helpful, they are not relevant to this demo and can be safely skipped.\n\n## 2. Configuring AWX\n\nIn order to use Netbox as an inventory source, we need to provide a way to pass authentication details to the [nb_inventory plugin](https://docs.ansible.com/ansible/latest/collections/netbox/netbox/nb_inventory_inventory.html). To do that, add a new credential type for netbox. From [AWX dashboard](http://localhost:8081) navigate to Administration -\u003e Credential Types and add a new \"netbox\" credential type.\n```\n---\nfields:\n- type: string\n  id: netbox_api\n  label: Netbox API URL\n- type: string\n  id: netbox_token\n  label: Netbox Token\nrequired:\n- netbox_token\n- netbox_api\n```\n```\n---\nenv:\n    NETBOX_API: \"{{ netbox_api }}\n    NETBOX_TOKEN: \"{{ netbox_token }}\n\n```\n![](https://gitlab.com/nvidia-networking/systems-engineering/poc-support/netbox-awx-automation/-/raw/main/awx/credential-type.png)\n\nNow we can create a new credential object with the details of the local Netbox instance, i.e. URL `http://citc-netbox` and token `0123456789abcdef0123456789abcdef01234567`:\n\n![](https://gitlab.com/nvidia-networking/systems-engineering/poc-support/netbox-awx-automation/-/raw/main/awx/netbox-cred.png)\n\nWe also need to create a new credential to access the Cumulus Linux devices: \n\n![](https://gitlab.com/nvidia-networking/systems-engineering/poc-support/netbox-awx-automation/-/raw/main/awx/cl-creds.png)\n\n\nThe default AWX EE execution environment does not include some of the python libraries required to interact with Netbox, so we'd need to create a new one. The container image has already been pre-built, however should you decide to create a custom image, you can see how it can be done by looking at the `make ee` command. For now, you can create a new execution environment with the provided pre-built image:\n\n![](https://gitlab.com/nvidia-networking/systems-engineering/poc-support/netbox-awx-automation/-/raw/main/awx/netbox-ee.png)\n\nNow we need to tell AWX where to find our playbooks and the inventory by creating a new Project and pointing at the current git repository:\n\n![](https://gitlab.com/nvidia-networking/systems-engineering/poc-support/netbox-awx-automation/-/raw/main/awx/project.png)\n\nOnce saved, AWX will try to fetch the latest commit and should report the job status as \"Success\".\n\n![](https://gitlab.com/nvidia-networking/systems-engineering/poc-support/netbox-awx-automation/-/raw/main/awx/project-success.png)\n\nCreate a new Inventory called \"netbox\" and Navigate to the \"Sources\" tab to add this git repo as a source and tie it together with the previously created credentails:\n\n![](https://gitlab.com/nvidia-networking/systems-engineering/poc-support/netbox-awx-automation/-/raw/main/awx/inventory.png)\n\nOnce the source is created and synced, the two lab devices should appear under the \"Hosts\" tab of the inventory:\n\n![](https://gitlab.com/nvidia-networking/systems-engineering/poc-support/netbox-awx-automation/-/raw/main/awx/hosts.png)\n\n\nNow we can run our first end-to-end test by combining all of the previously configured elements in a single job template. \n\n![](https://gitlab.com/nvidia-networking/systems-engineering/poc-support/netbox-awx-automation/-/raw/main/awx/debug.png)\n\n\nThis job template will execute the \"debug\" playbook that will do the following:\n\n* Pull all information about Netbox devices (model, type, IP)\n* Pull information about all interfaces known to Netbox\n* Pull all IPAM information from Netbox\n* For each device, print all known information to `stdout`\n\nThis is how you can verify the details that have been collected by this playbook.\n\n![](https://gitlab.com/nvidia-networking/systems-engineering/poc-support/netbox-awx-automation/-/raw/main/awx/debug-success.png)\n\n\n\n## 3. Configuration modelling in Netbox\n\nIt's quite common to refer to Netbox as the \"networking source of truth\", however in reality its scope is limited to inventory and IP address management. In order to model the entire network device configuration state, we'll use a feature called [configuration context](https://netbox.readthedocs.io/en/stable/models/extras/configcontext/) that was designed to store JSON data associated with various Netbox objects. In our case, we'll use this to store a simple BGP configuration for both of our lab devices. We'll use the Netbox's [hierarchical rendering](https://netbox.readthedocs.io/en/stable/models/extras/configcontext/#hierarchical-rendering) to define common configuration for groups of network devices and use [local context](https://netbox.readthedocs.io/en/stable/models/extras/configcontext/#local-context-data) to override any device-specific settings.\n\nLet's start by defining the common data model that will be shared amongst all devices with \"leaf\" role. Connect back to the netbox shell as it was described in step #1.\n\n\n```python\nrole_context = {'bgp': {'address_family': [{'name': 'ipv4_unicast',          \n                             'redistribute': [{'type': 'connected'}]}],     \n         'asn': 65000,                                                      \n         'neighbors': [{'interface': 'swp1',                               \n                        'peergroup': 'underlay',                            \n                        'unnumbered': True},                                \n                       {'interface': 'swp2',                               \n                        'peergroup': 'underlay',                            \n                        'unnumbered': True}],                               \n         'peergroups': [{'name': 'underlay', 'remote_as': 'external'}]},    \n 'interfaces': [{'name': 'swp1'}, {'name': 'swp2'}]}\n\nrole = DeviceRole.objects.get(name=\"leaf\")\nConfigContext(name=\"leaf\", data=role_context).save()\nctx = ConfigContext.objects.get(name=\"leaf\")\nctx.roles.add(role)\n```\n\nFor each individual device, override the default BGP AS number:\n\n```python\nleaf01 = Device.objects.get(name=\"leaf01\")\nleaf02 = Device.objects.get(name=\"leaf02\")\nleaf01.local_context_data = { \"bgp\": { \"asn\": 65001 }}\nleaf02.local_context_data = { \"bgp\": { \"asn\": 65002 }}\nleaf01.save()\nleaf02.save()\n```\n\nTo check the final state that will be rendered for \"leaf01\":\n\n```python\nimport json\nprint(json.dumps(leaf01.get_config_context(), indent=2))\n```\nWhich should display:\n```python\n{\n  \"bgp\": {\n    \"asn\": 65001,\n    \"neighbors\": [\n      {\n        \"interface\": \"swp1\",\n        \"peergroup\": \"underlay\",\n        \"unnumbered\": true\n      },\n      {\n        \"interface\": \"swp2\",\n        \"peergroup\": \"underlay\",\n        \"unnumbered\": true\n      }\n    ],\n    \"peergroups\": [\n      {\n        \"name\": \"underlay\",\n        \"remote_as\": \"external\"\n      }\n    ],\n    \"address_family\": [\n      {\n        \"name\": \"ipv4_unicast\",\n        \"redistribute\": [\n          {\n            \"type\": \"connected\"\n          }\n        ]\n      }\n    ]\n  },\n  \"interfaces\": [\n    {\n      \"name\": \"swp1\"\n    },\n    {\n      \"name\": \"swp2\"\n    }\n  ]\n}\n```\n\n## 4. Using Netbox configuration context from AWX\n\nNow that we have the data models defined in AWX, we can use it to provision our lab devices. To do that, we'll create another job template in AWX and point it at the [`generate.yml`](https://gitlab.com/nvidia-networking/systems-engineering/poc-support/netbox-awx-automation/-/blob/main/ansible_collections/nvidia/cumulus/playbooks/generate.yml) playbook.\n\n\n![](https://gitlab.com/nvidia-networking/systems-engineering/poc-support/netbox-awx-automation/-/raw/main/awx/provision.png)\n\nThe new playbook will go through the following sequence of actions:\n\n1. Fetch configuration data from Netbox using the [`nb_lookup`](https://docs.ansible.com/ansible/latest/collections/netbox/netbox/nb_lookup_lookup.html) plugin.\n2. Using the collected data, generate NVUE configuration file for every device.\n3. Apply the generated configuration file.\n\nHere's an example of a jinja template that that pulls information from multiple sources and generates the final NVUE JSON configuration file.\n\n```jinja\n{% set config = dict({\"set\": dict()}) %}\n{% set loopback_ip = hostvars[inventory_hostname].netbox_ips | community.general.json_query('[?assigned_object.name==`lo`].address') | first %}\n\n{%   include './features/hostname.j2' %}\n\n{# interface config #}\n{% set _ = config['set'].update(dict({\"interface\": dict()})) %}\n{%   include './features/eth0.j2' %}\n{%   include './features/swp.j2' %}\n{%   include './features/loopback.j2' %}\n\n{# bgp config #}\n{% set _ = config['set'].update(dict({\"router\": dict()})) %}\n{%   include './features/bgp.j2' %}\n\n[\n    {{ config | to_nice_json  }}\n]\n```\n\nOnce the \"provision\" job has run succesfully, we should be able to ping between loopback interfaces of `leaf01` and `leaf02`\n\n```\ncumulus@leaf01:mgmt:~$ ping 10.0.1.12 -I 10.0.1.11 -c 2\nvrf-wrapper.sh: switching to vrf \"default\"; use '--no-vrf-switch' to disable\nPING 10.0.1.12 (10.0.1.12) from 10.0.1.11 : 56(84) bytes of data.\n64 bytes from 10.0.1.12: icmp_seq=1 ttl=64 time=0.368 ms\n64 bytes from 10.0.1.12: icmp_seq=2 ttl=64 time=0.395 ms\n\n--- 10.0.1.12 ping statistics ---\n2 packets transmitted, 2 received, 0% packet loss, time 39ms\nrtt min/avg/max/mdev = 0.368/0.381/0.395/0.023 ms\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetworkop%2Fnetbox-awx-automation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnetworkop%2Fnetbox-awx-automation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetworkop%2Fnetbox-awx-automation/lists"}