{"id":20649357,"url":"https://github.com/clay584/genie_collection","last_synced_at":"2025-04-17T01:04:22.525Z","repository":{"id":40736800,"uuid":"235932110","full_name":"clay584/genie_collection","owner":"clay584","description":"Brings Cisco Genie parse, learn, and diff capabilities to Ansible","archived":false,"fork":false,"pushed_at":"2023-02-07T23:15:13.000Z","size":615,"stargazers_count":16,"open_issues_count":7,"forks_count":8,"subscribers_count":8,"default_branch":"master","last_synced_at":"2023-08-05T17:12:06.474Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/clay584.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-01-24T03:15:24.000Z","updated_at":"2023-08-05T17:12:06.475Z","dependencies_parsed_at":"2023-01-25T04:30:09.795Z","dependency_job_id":null,"html_url":"https://github.com/clay584/genie_collection","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clay584%2Fgenie_collection","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clay584%2Fgenie_collection/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clay584%2Fgenie_collection/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clay584%2Fgenie_collection/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/clay584","download_url":"https://codeload.github.com/clay584/genie_collection/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224944513,"owners_count":17396257,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-16T17:14:01.399Z","updated_at":"2024-11-16T17:14:02.649Z","avatar_url":"https://github.com/clay584.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cisco Genie Ansible Collection (Unofficial)\n\n[![Published](https://static.production.devnetcloud.com/codeexchange/assets/images/devnet-published.svg)](https://developer.cisco.com/codeexchange/github/repo/clay584/genie_collection)\n![Ansible Version](https://img.shields.io/badge/ansible-%3E%3D2.9-blue.svg)\n![Python Version](https://img.shields.io/badge/python-%3E%3D3.4-blue.svg)\n\nThis is an Ansible collection that brings the functionality of Cisco's pyATS and Genie \nlibraries to Ansible users.\n\nFor more information on the Cisco pyATS project, see [https://developer.cisco.com/docs/pyats/\n](https://developer.cisco.com/docs/pyats/)\n\n- [Cisco Genie Ansible Collection (Unofficial)](#cisco-genie-ansible-collection--unofficial-)\n  * [What's in this Collection?](#what-s-in-this-collection-)\n    + [Learn Genie Ansible Module](#learn-genie-ansible-module)\n    + [Parse Genie Ansible Filter Plugin](#parse-genie-ansible-filter-plugin)\n  * [Prerequisites](#prerequisites)\n  * [Installation](#installation)\n- [Usage](#usage)\n  * [Learn Genie Module](#learn-genie-module)\n      - [Module Parameters](#module-parameters)\n      - [Examples](#examples)\n  * [Parse Genie Filter Plugin](#parse-genie-filter-plugin)\n    + [Ansible to Genie OS Mappings](#ansible-to-genie-os-mappings)\n      - [Usage](#usage-1)\n      - [Short Example](#short-example)\n      - [Full Example #1](#full-example--1)\n      - [Full Example #2](#full-example--2)\n    + [Generic Tabular Parsing](#generic-tabular-parsing)\n      - [How Tabular Parsing Works](#how-tabular-parsing-works)\n      - [Preparing to Use the Tabular Parser](#preparing-to-use-the-tabular-parser)\n      - [Calling the Tabular Parser in a Playbook](#calling-the-tabular-parser-in-a-playbook)\n      - [Full Example #1](#full-example--1-1)\n    + [Development](#development)\n    + [Testing](#testing)\n    + [Pushing](#pushing)\n\n## What's in this Collection?\n\n### Learn Genie Ansible Module\n\nThis is an Ansible module that allows you to `learn` a feature on a device in your Ansible playbook. \nThis is the equivalent of running `genie learn \u003cfeature\u003e` from the Genie CLI tool.\nUsing Cisco's Genie libraries, this will connect to the device, run a series of commands, and return \na data structure that conforms to an OS-agnostic data model, meaning that you could run the `learn_genie` \nmodule against an IOS, NXOS, IOS-XR, or IOS-XE device for a given feature, and the data returned will be \nin the same format. This allows for much more simple automation logic as the data structures are \nidentical regardless of device OS.\n\nThe second part of this module allows you to, again, `learn` a feature, but then also compare it \nagainst a previous run. For example, if you have a playbook, you could learn a feature, make some \ndevice configuration changes, and then the final task of the playbook, you could learn the feature again \nand compare the two using `genie diff`.\n\n### Parse Genie Ansible Filter Plugin\n\nThis is an Ansible filter plugin that can take raw CLI output and return structured data. This is the \nsame filter plugin located [here](https://galaxy.ansible.com/clay584/parse_genie). Since Ansible has \ncreated Ansible Collections, this has been added to this collection. The other project will remain unchanged for \nbackward-compatibility, but no further updates will be done to that project. All future work on `parse_genie` \nwill be in this collection.\n\n## Prerequisites\n\nThis collection will require the following on the Ansible control machine:\n\n- Python 3.4+\n- Ansible 2.9+\n- pyATS\n- Genie\n- colorama\n\n## Installation\n\nPlease follow these instructions to ensure that the filter plugin will function with your playbooks:\n\n1. Create a directory for your playbook and go into it.\n    - `mkdir my_playbook \u0026\u0026 cd my_playbook`\n2. Create a virtual environment.\n    - `python3 -m venv .venv`\n3. Activate the virtual environment.\n    - `source .venv/bin/activate`\n4. Install the required Python packages.\n    - `pip install ansible pyats genie colorama`\n5. Deactivate and reactivate the Python virtual environment.\n    - `deactivate \u0026\u0026 source .venv/bin/activate`\n6. Install Ansible Collection\n    - `ansible-galaxy collection install clay584.genie`\n\n\n# Usage\n\n## Learn Genie Module\n\nThis module is very similar to invoking the `genie learn` and `genie diff` CLI tool. In your Ansible \nplaybook, you would call the module `learn_genie` and pass in parameters as described in the following \nsection. The returned data from Genie is returned, and when using this module in your playbooks, it is \nappropriate the `register` the output from the task in your playbook. That registered output can then be \nused in subsequent tasks or plays. Also, you can use the registered output in later runs of the `learn_genie` \nmodule in order to take advantage of the `genie diff` functionality.\n\n#### A Note About Ansible Interpreter\n\nThere are some silly Ansible settings regarding localhost settings and `ansible_python_interpreter` \nand what you have to do in order to use your virtual environment on localhost when `connection: local`. \n\nYou will have to set the `ansible_python_interpreter` to the current playbook interpreter, either via \nthe ansible.cfg file or via the inventory file.\n\nHere is an example:\n\n```\nall:\n  hosts:\n    sbx-nxos-mgmt.cisco.com:\n      ansible_connection: local\n      ansible_python_interpreter: \"{{ ansible_playbook_python }}\"\n```\n\n\n[Here](https://docs.ansible.com/ansible/latest/inventory/implicit_localhost.html) is the documentation \naround this setting.\n\n#### Module Parameters\n\n![Learn Genie Parameters](https://github.com/clay584/genie_collection/raw/master/clay584/genie/learn_genie_params.png)\n\n#### Examples\n\n```\n---\n\n- hosts: all\n  gather_facts: false\n  connection: local\n  collections:\n   - clay584.genie\n  tasks:\n  - name: Learn Genie - ARP\n    learn_genie:\n      host: \"{{ ansible_host }}\"\n      port: 8181\n      protocol: ssh\n      username: admin\n      password: Admin_1234!\n      os: nxos\n      feature: arp\n    register: genie_arp1\n\n  - name: Debug Genie\n    debug:\n      msg: \"{{ genie_arp1 }}\"\n```\n\nThe above playbook would yield the following:\n\n```\n$ ansible-playbook -i inventory test.yml\n\nPLAY [all] ******************************************************************\n\nTASK [Learn Genie - ARP] ****************************************************\nok: [sbx-nxos-mgmt.cisco.com]\n\nTASK [Debug Genie] **********************************************************\nok: [sbx-nxos-mgmt.cisco.com] =\u003e \n  msg:\n    changed: false\n    failed: false\n    genie:\n      arp:\n        interfaces:\n          Ethernet1/5:\n            arp_dynamic_learning:\n              local_proxy_enable: false\n              proxy_enable: false\n          Vlan100:\n            arp_dynamic_learning:\n              local_proxy_enable: false\n              proxy_enable: false\n          Vlan101:\n            arp_dynamic_learning:\n              local_proxy_enable: false\n              proxy_enable: false\n          Vlan102:\n            arp_dynamic_learning:\n              local_proxy_enable: false\n              proxy_enable: false\n          Vlan103:\n            arp_dynamic_learning:\n              local_proxy_enable: false\n              proxy_enable: false\n          Vlan104:\n            arp_dynamic_learning:\n              local_proxy_enable: false\n              proxy_enable: false\n          Vlan105:\n            arp_dynamic_learning:\n              local_proxy_enable: false\n              proxy_enable: false\n          loopback1:\n            arp_dynamic_learning:\n              local_proxy_enable: false\n              proxy_enable: false\n          mgmt0:\n            arp_dynamic_learning:\n              local_proxy_enable: false\n              proxy_enable: false\n        statistics:\n          entries_total: 0\n          in_drops: 3387630\n          in_replies_pkts: 153\n          in_requests_pkts: 3387463\n          in_total: 0\n          incomplete_total: 0\n          out_drops: 0\n          out_gratuitous_pkts: 14\n          out_replies_pkts: 0\n          out_requests_pkts: 14\n          out_total: 28\n\nPLAY RECAP ******************************************************************\nsbx-nxos-mgmt.cisco.com    : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   \n\n```\n\nHere is an example of using the diff functionality...\n\n```\n---\n\n- hosts: all\n  gather_facts: false\n  connection: local\n  tasks:\n  - name: Learn Genie 1st Run\n    learn_genie:\n      host: \"{{ ansible_host }}\"\n      port: 8181\n      protocol: ssh\n      username: admin\n      password: Admin_1234!\n      os: nxos\n      feature: ospf\n    register: genie1\n\n  - name: Learn Genie with Diff\n    learn_genie:\n      host: \"{{ ansible_host }}\"\n      port: 8181\n      protocol: ssh\n      username: admin\n      password: Admin_1234!\n      os: nxos\n      feature: ospf\n      compare_to: \"{{ genie1 }}\"\n    diff: true\n```\n\nThe above play outputs the following:\n\n```\n$ ansible-playbook -i inventory test.yml\n\nPLAY [all] ******************************************************************\n\nTASK [Learn Genie - 1st Run] ************************************************\nok: [sbx-nxos-mgmt.cisco.com]\n\n... truncated - made some OSPF changes ...\n\nTASK [Learn Genie with Diff] ************************************************\nvrf:\n default:\n  address_family:\n   ipv4:\n    instance:\n     1:\n+      router_id: 172.16.1.0\n-      router_id: 172.16.0.1\nchanged: [sbx-nxos-mgmt.cisco.com]\n\nPLAY RECAP ******************************************************************\nsbx-nxos-mgmt.cisco.com    : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   \n\n```\n\nIf the module parameter `colors` is not set to `false` and `colorama` is installed, the diff output \nwill be colored.\n\n![Colored Diff Output](https://github.com/clay584/genie_collection/raw/master/clay584/genie/colored_diff.png)\n\n## Parse Genie Filter Plugin\n\n**ATTENTION!!! - If you run into an issue with a command failing to parse, it is possible that there is a bug in the parsing library which is maintained by Cisco. For those issues, you can open an issue [here](https://github.com/CiscoTestAutomation/genieparser/issues).**\n\nThe network genie filter takes unstructured network CLI command output from all\nCisco network operating systems, and outputs structured data. While similar to other\nnetwork CLI parsers already available (parse_cli, parse_cli_textfsm), this parser is\npowered by a very mature and robust library written by Cisco called Genie (and underlying framework pyATS).\nThis provides over 1200 parsers that transform configuration and CLI\noutput to structured data that is normalized and conforms to standard, OS-agnostic data models.\n\nThe Genie library can also serve as an engine to parse tabular and non-tabular free-form text\nusing much less code than traditional parsing requires. Therefore, it can be used to\nparse any vendor output; not just that of Cisco devices. However, that would involve writing custom parsers.\nThis release does not include the functionality to utilize custom parsers. The supported parsers are whatever\nis included in the release of Genie that the user has installed on the Ansible control machine.\n\nThe list of supported operating systems and commands, as well\nas the data's schema definitions (data models) which describe exactly what fields and\ndata types will be returned for any given command, is available from Cisco at the link below.\n\nhttps://pubhub.devnetcloud.com/media/genie-feature-browser/docs/#/parsers \n\n### Ansible to Genie OS Mappings\n\nBelow are the mappings from Ansible's `ansible_network_os` to Genie's `os`:\n\n| Ansible Network OS  | Genie OS      |\n| ------------------- | ------------- |\n| ios                 | ios, iosxe    |\n| nxos                | nxos          |\n| iosxr               | iosxr         |\n| junos               | junos         |\n\nIf you are working with IOS or IOS-XE there is ambiguity in that Ansible considers IOS and IOS-XE\nthe same and therefore the `ansible_network_os = ios`, but Genie needs to know specifically if it is\nIOS or IOS-XE in order to parse the CLI output correctly. If you pass `ansible_network_os` to this filter plugin,\nand it is equal to `ios`, parse_genie will try to parse it with Genie using `os=ios` first, and if that fails, it will\nthen try to parse it with `os=iosxe`.\n\nSo keep that in mind when creating your playbooks. It may be best to pass the real OS to the parse_genie.\nYou can do that by keeping another inventory variable or host_var to specify the Genie OS for each network device\nand using that variable as the OS for the parse_genie.\n\n#### Usage\n\nMake sure to read in the parse_genie role before you attempt to use it later in your playbook.\n\n    ...trunctated...\n    \n\t  tasks:\n\t  - name: Read in parse_genie role\n\t\tcollections:\n\t\t  - clay584.genie\n\t\t  \n    ...trunctated...\n\n**There is a stupid bug in Ansible, which the Ansible maintainers have listed as \"by design\", \nwhereby modules and such can be referenced by their short-name, but filter plugins must be reference \nby their fully qualified name. So, when using the parse_genie filter plugin, you must use it in the following way.**\n\n```\n\"{{ show_cli_output | clay584.genie.parse_genie(command='show version', os='iosxe') }}\"\n```\n\n#### Short Example\nTo convert the output of a network device CLI command, use the `parse_genie` filter as shown in this example\n(do not use abbreviated CLI commands).\n\nConverting CLI output of the `show version` command from a Cisco IOS-XE device to structured data::\n\n    {{ cli_output | clay584.genie.parse_genie(command='show version', os='iosxe') }}\n\nFor deeper abstraction, you might want to add `platform` to `parse_parse`.\n\n    {{ cli_output | clay584.genie.parse_genie(command='show environment all', os='iosxe', platform='asr1k') }}\n\nThe above example would yield the following:\n\n    {\n        \"version\": {\n            \"chassis\": \"CSR1000V\",\n            \"chassis_sn\": \"9TKUWGKX5MO\",\n            \"curr_config_register\": \"0x2102\",\n            \"disks\": {\n                \"bootflash:.\": {\n                    \"disk_size\": \"7774207\",\n                    \"type_of_disk\": \"virtual hard disk\"\n                },\n                \"webui:.\": {\n                    \"disk_size\": \"0\",\n                    \"type_of_disk\": \"WebUI ODM Files\"\n                }\n            },\n            \"hostname\": \"host-172-16-1-96\",\n            \"image_id\": \"X86_64_LINUX_IOSD-UNIVERSALK9-M\",\n            \"image_type\": \"production image\",\n            \"last_reload_reason\": \"Reload Command\",\n            \"license_level\": \"ax\",\n            \"license_type\": \"Default. No valid license found.\",\n            \"main_mem\": \"1126522\",\n            \"mem_size\": {\n                \"non-volatile configuration\": \"32768\",\n                \"physical\": \"3018840\"\n            },\n            \"next_reload_license_level\": \"ax\",\n            \"number_of_intfs\": {\n                \"Gigabit Ethernet\": \"2\"\n            },\n            \"os\": \"IOS-XE\",\n            \"platform\": \"Virtual XE\",\n            \"processor_type\": \"VXE\",\n            \"rom\": \"IOS-XE ROMMON\",\n            \"rtr_type\": \"CSR1000V\",\n            \"system_image\": \"bootflash:packages.conf\",\n            \"uptime\": \"2 minutes\",\n            \"uptime_this_cp\": \"3 minutes\",\n            \"version\": \"16.5.1b,\",\n            \"version_short\": \"16.5\"\n        }\n    }\n\n#### Full Example #1\n\nPlaybook:\n\n\t---\n\t\n\t- hosts: localhost\n\t  connection: local\n\t  collections:\n        - clay584.genie\n\t  vars:\n\t\tshow_version_output: |\n\t\t  Cisco IOS XE Software, Version 16.05.01b\n\t\t  Cisco IOS Software [Everest], Virtual XE Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 16.5.1b, RELEASE SOFTWARE (fc1)\n\t\t  Technical Support: http://www.cisco.com/techsupport\n\t\t  Copyright (c) 1986-2017 by Cisco Systems, Inc.\n\t\t  Compiled Tue 11-Apr-17 16:41 by mcpre\n\t\n\t\n\t\t  Cisco IOS-XE software, Copyright (c) 2005-2017 by cisco Systems, Inc.\n\t\t  All rights reserved.  Certain components of Cisco IOS-XE software are\n\t\t  licensed under the GNU General Public License (\"GPL\") Version 2.0.  The\n\t\t  software code licensed under GPL Version 2.0 is free software that comes\n\t\t  with ABSOLUTELY NO WARRANTY.  You can redistribute and/or modify such\n\t\t  GPL code under the terms of GPL Version 2.0.  For more details, see the\n\t\t  documentation or \"License Notice\" file accompanying the IOS-XE software,\n\t\t  or the applicable URL provided on the flyer accompanying the IOS-XE\n\t\t  software.\n\t\n\t\n\t\t  ROM: IOS-XE ROMMON\n\t\n\t\t  host-172-16-1-96 uptime is 2 minutes\n\t\t  Uptime for this control processor is 3 minutes\n\t\t  System returned to ROM by reload\n\t\t  System image file is \"bootflash:packages.conf\"\n\t\t  Last reload reason: Reload Command\n\t\n\t\n\t\n\t\t  This product contains cryptographic features and is subject to United\n\t\t  States and local country laws governing import, export, transfer and\n\t\t  use. Delivery of Cisco cryptographic products does not imply\n\t\t  third-party authority to import, export, distribute or use encryption.\n\t\t  Importers, exporters, distributors and users are responsible for\n\t\t  compliance with U.S. and local country laws. By using this product you\n\t\t  agree to comply with applicable laws and regulations. If you are unable\n\t\t  to comply with U.S. and local laws, return this product immediately.\n\t\n\t\t  A summary of U.S. laws governing Cisco cryptographic products may be found at:\n\t\t  http://www.cisco.com/wwl/export/crypto/tool/stqrg.html\n\t\n\t\t  If you require further assistance please contact us by sending email to\n\t\t  export@cisco.com.\n\t\n\t\t  License Level: ax\n\t\t  License Type: Default. No valid license found.\n\t\t  Next reload license Level: ax\n\t\n\t\t  cisco CSR1000V (VXE) processor (revision VXE) with 1126522K/3075K bytes of memory.\n\t\t  Processor board ID 9TKUWGKX5MO\n\t\t  2 Gigabit Ethernet interfaces\n\t\t  32768K bytes of non-volatile configuration memory.\n\t\t  3018840K bytes of physical memory.\n\t\t  7774207K bytes of virtual hard disk at bootflash:.\n\t\t  0K bytes of WebUI ODM Files at webui:.\n\t\n\t\t  Configuration register is 0x2102\n\t\n\t  - name: Debug Genie Filter\n\t\tdebug:\n\t\t  msg: \"{{ show_version_output | clay584.genie.parse_genie(command='show version', os='iosxe') }}\"\n\t\tdelegate_to: localhost\n\n\nOutput:\n\n\t$ ansible-playbook -i inventory debug.yml\n\t\n\tPLAY [localhost] *************************************************************************\n\t\n\tTASK [Gathering Facts] *******************************************************************\n\tok: [localhost]\n\t\n\tTASK [Debug Genie Filter] ****************************************************************\n\tok: [localhost -\u003e localhost] =\u003e {\n\t\t\"msg\": {\n\t\t\t\"version\": {\n\t\t\t\t\"chassis\": \"CSR1000V\",\n\t\t\t\t\"chassis_sn\": \"9TKUWGKX5MO\",\n\t\t\t\t\"curr_config_register\": \"0x2102\",\n\t\t\t\t\"disks\": {\n\t\t\t\t\t\"bootflash:.\": {\n\t\t\t\t\t\t\"disk_size\": \"7774207\",\n\t\t\t\t\t\t\"type_of_disk\": \"virtual hard disk\"\n\t\t\t\t\t},\n\t\t\t\t\t\"webui:.\": {\n\t\t\t\t\t\t\"disk_size\": \"0\",\n\t\t\t\t\t\t\"type_of_disk\": \"WebUI ODM Files\"\n\t\t\t\t\t}\n\t\t\t\t},\n\t\t\t\t\"hostname\": \"host-172-16-1-96\",\n\t\t\t\t\"image_id\": \"X86_64_LINUX_IOSD-UNIVERSALK9-M\",\n\t\t\t\t\"image_type\": \"production image\",\n\t\t\t\t\"last_reload_reason\": \"Reload Command\",\n\t\t\t\t\"license_level\": \"ax\",\n\t\t\t\t\"license_type\": \"Default. No valid license found.\",\n\t\t\t\t\"main_mem\": \"1126522\",\n\t\t\t\t\"mem_size\": {\n\t\t\t\t\t\"non-volatile configuration\": \"32768\",\n\t\t\t\t\t\"physical\": \"3018840\"\n\t\t\t\t},\n\t\t\t\t\"next_reload_license_level\": \"ax\",\n\t\t\t\t\"number_of_intfs\": {\n\t\t\t\t\t\"Gigabit Ethernet\": \"2\"\n\t\t\t\t},\n\t\t\t\t\"os\": \"IOS-XE\",\n\t\t\t\t\"platform\": \"Virtual XE\",\n\t\t\t\t\"processor_type\": \"VXE\",\n\t\t\t\t\"rom\": \"IOS-XE ROMMON\",\n\t\t\t\t\"rtr_type\": \"CSR1000V\",\n\t\t\t\t\"system_image\": \"bootflash:packages.conf\",\n\t\t\t\t\"uptime\": \"2 minutes\",\n\t\t\t\t\"uptime_this_cp\": \"3 minutes\",\n\t\t\t\t\"version\": \"16.5.1b,\",\n\t\t\t\t\"version_short\": \"16.5\"\n\t\t\t}\n\t\t}\n\t}\n\n\n#### Full Example #2\n\nPlaybook:\n\n\t---\n\t\n\t- hosts: csr1000v\n\t  gather_facts: False\n\t  collections:\n        - clay584.genie\n\t  tasks:\n\t  - name: Get Data From Device\n\t\tios_command:\n\t\t  commands: show arp vrf Mgmt-intf\n\t\tregister: arp_output\n\t\n\t  - name: Print Structured Data\n\t\tdebug:\n\t\t  msg: \"{{ arp_output['stdout'][0] | clay584.genie.parse_genie(command='show arp vrf Mgmt-intf', os='iosxe') }}\"\n\t\tdelegate_to: localhost\n\nOutput:\n\n\t$ ansible-playbook -i inventory playbook.yml\n\t\n\tPLAY [csr1000v] **************************************************************************\n\t\n\tTASK [Get Data From Device] **************************************************************\n\tok: [csr1000v]\n\t\n\tTASK [Print Structured Data] *************************************************************\n\tok: [csr1000v -\u003e localhost] =\u003e {\n\t\t\"msg\": {\n\t\t\t\"interfaces\": {\n\t\t\t\t\"GigabitEthernet1\": {\n\t\t\t\t\t\"ipv4\": {\n\t\t\t\t\t\t\"neighbors\": {\n\t\t\t\t\t\t\t\"172.16.1.111\": {\n\t\t\t\t\t\t\t\t\"age\": \"0\",\n\t\t\t\t\t\t\t\t\"ip\": \"172.16.1.111\",\n\t\t\t\t\t\t\t\t\"link_layer_address\": \"5e00.4004.0000\",\n\t\t\t\t\t\t\t\t\"origin\": \"dynamic\",\n\t\t\t\t\t\t\t\t\"protocol\": \"Internet\",\n\t\t\t\t\t\t\t\t\"type\": \"ARPA\"\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\"172.16.1.114\": {\n\t\t\t\t\t\t\t\t\"age\": \"-\",\n\t\t\t\t\t\t\t\t\"ip\": \"172.16.1.114\",\n\t\t\t\t\t\t\t\t\"link_layer_address\": \"5e00.4001.0000\",\n\t\t\t\t\t\t\t\t\"origin\": \"static\",\n\t\t\t\t\t\t\t\t\"protocol\": \"Internet\",\n\t\t\t\t\t\t\t\t\"type\": \"ARPA\"\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n### Generic Tabular Parsing\n\nCisco Genie has support for 1200 commands and counting, but for those show commands where there is not \na parser that has been built by Cisco, there is the generic tabular parsing functionality. For more \ninformation on the Genie tabular parsing functionality, see their [oper_fill_tabular](https://pubhub.devnetcloud.com/media/pyats-packages/docs/parsergen/tabular/tabular.html) documentation.\n\n#### How Tabular Parsing Works\n\nIn order to parse a command output when there is a parser that has been built, all that is required is the `command`, `command ouput`, and `os`. \nBut if there is not a parser built, you must specify some additional information to help the parser determine how \nto parse the command output. This additional data is two-fold:\n\n1. Headers - The column headers as shown in the command's output.\n2. Index - The key of the dictionary items that the parser will return.\n\nConsider the following example:\n\n1. Command: `show ip sla summary`\n2. Command Output:\n\n```\nIPSLAs Latest Operation Summary\nCodes: * active, ^ inactive, ~ pending\nAll Stats are in milliseconds. Stats with u are in microseconds\n\nID           Type           Destination       Stats               Return        Last\n                                                                  Code          Run\n------------------------------------------------------------------------------------------------\n*1           udp-jitter      10.0.0.2          RTT=900u           OK             20 seconds ago\n*2           icmp-echo       10.0.0.2          RTT=1              OK              3 seconds ago\n```\n3. Headers - `ID`, `Type`, `Destination`, `Stats`, `Return Code`, and `Last Run`.\n4. Index - We want to use the `ID` column as the index for this data when we get it back from the parser.\n5. Parser Output:\n\n```\n{'*1': {'Destination ': '10.0.0.2',\n        'ID ': '*1',\n        'Last Run': '20 seconds ago',\n        'Return Code': 'OK',\n        'Stats ': 'RTT=900u',\n        'Type ': 'udp-jitter'},\n '*2': {'Destination ': '10.0.0.2',\n        'ID ': '*2',\n        'Last Run': '3 seconds ago',\n        'Return Code': 'OK',\n        'Stats ': 'RTT=1',\n        'Type ': 'icmp-echo'}}\n\n```\n\n#### Preparing to Use the Tabular Parser\n\nIn order to use this tabular parser we must first construct the `headers` and `index` for a given command on \na given OS in a format that can be read into an Ansible playbook, and subsequently fed into the parse_genie filter plugin.\n\nIn order to do this, you must create a vars file in your playbook that is in the following format. It is \norganized by OS, then by command. Then under each command, the headers and index are defined. You can \ndefine as many commands as you like for each network OS as long as it is within this data structure.\n\n```\nparse_genie:\n  ios:\n    \"show ip sla summary\":\n      headers:\n        - - ID\n          - Type\n          - Destination\n          - Stats\n          - Return\n          - Last\n        - - ''\n          - ''\n          - ''\n          - ''\n          - Code\n          - Run\n      index:\n        - 0\n  iosxe:\n    \"show ip sla summary\":\n      headers:\n        - - ID\n          - Type\n          - Destination\n          - Stats\n          - Return\n          - Last\n        - - ''\n          - ''\n          - ''\n          - ''\n          - Code\n          - Run\n      index:\n        - 1\n\n```\n\nThe python equivalent of the above yaml format is:\n\n```\npython_dict = {\n  \"parse_genie\": {\n    \"ios\": {\n      \"show ip sla summary\": {\n        \"headers\": [\n          [\n            \"ID\", \n            \"Type\", \n            \"Destination\", \n            \"Stats\", \n            \"Return\", \n            \"Last\"\n          ], \n          [\n            \"\", \n            \"\", \n            \"\", \n            \"\", \n            \"Code\", \n            \"Run\"\n          ]\n        ], \n        \"index\": [\n          0\n        ]\n      }\n    }, \n    \"iosxe\": {\n      \"show ip sla summary\": {\n        \"headers\": [\n          [\n            \"ID\", \n            \"Type\", \n            \"Destination\", \n            \"Stats\", \n            \"Return\", \n            \"Last\"\n          ], \n          [\n            \"\", \n            \"\", \n            \"\", \n            \"\", \n            \"Code\", \n            \"Run\"\n          ]\n        ], \n        \"index\": [\n          1\n        ]\n      }\n    }\n  }\n}\n```\n\n#### Calling the Tabular Parser in a Playbook\n\nNow that we have defined a generic tabular command and its headers and index, we can actually call \nit from a playbook.\n\nFirst, we read in the vars file that contains the tabular command parsing metadata.\n\n```\n- name: Include vars file with generic command metadata\n  include_vars:\n    file: parse_genie_generic_commands.yml\n    name: parse_genie\n```\n\nNext, we pass the command output to `parse_genie` but with a couple of extra parameters.\n\n```\n- name: Parse generic tabular command output\n  debug:\n    msg: \"{{ command_output | parse_genie(command='show ip sla summary', os='ios', generic_tabular=True, generic_tabular_metadata=parse_genie) }}\"\n  delegate_to: localhost\n```\n\nThe resulting parsed output will show as follows:\n\n```\nok: [localhost -\u003e localhost] =\u003e {\n    \"msg\": {\n        \"*1\": {\n            \"Destination \": \"10.0.0.2\",\n            \"ID \": \"*1\",\n            \"Last Run\": \"20 seconds ago\",\n            \"Return Code\": \"OK\",\n            \"Stats \": \"RTT=900u\",\n            \"Type \": \"udp-jitter\"\n        },\n        \"*2\": {\n            \"Destination \": \"10.0.0.2\",\n            \"ID \": \"*2\",\n            \"Last Run\": \"3 seconds ago\",\n            \"Return Code\": \"OK\",\n            \"Stats \": \"RTT=1\",\n            \"Type \": \"icmp-echo\"\n        }\n    }\n}\n```\n\n#### Full Example #1\n\nPlaybook:\n\n```\n\n---\n\n- hosts: localhost\n  connection: local\n  collections:\n    - clay584.genie\n  vars:\n    out_ios_sla: |\n      IPSLAs Latest Operation Summary\n      Codes: * active, ^ inactive, ~ pending\n      All Stats are in milliseconds. Stats with u are in microseconds\n\n      ID           Type           Destination       Stats               Return        Last\n                                                                        Code          Run\n      ------------------------------------------------------------------------------------------------\n      *1           udp-jitter      10.0.0.2          RTT=900u           OK             20 seconds ago\n      *2           icmp-echo       10.0.0.2          RTT=1              OK              3 seconds ago\n\n  tasks:\n    - name: Include vars file that has generic tabular command metadata\n      include_vars:\n        file: parse_genie_generic_commands.yml\n        name: parse_genie\n\n    - name: Test Genie Filter for generic tabular data\n      debug:\n        msg: \"{{ out_ios_sla | clay584.genie.parse_genie(command='test show ip sla summary', os='ios', generic_tabular=True, generic_tabular_metadata=parse_genie) }}\"\n      delegate_to: localhost\n\n```\n\n`parse_genie_generic_commands.yml` contents:\n\n```\n\n---\n\nparse_genie:\n  ios:\n    \"test show ip sla summary\":\n      headers:\n        - - ID\n          - Type\n          - Destination\n          - Stats\n          - Return\n          - Last\n        - - ''\n          - ''\n          - ''\n          - ''\n          - Code\n          - Run\n      index:\n        - 0\n  iosxe:\n    \"test show ip sla summary\":\n      headers:\n        - - ID\n          - Type\n          - Destination\n          - Stats\n          - Return\n          - Last\n        - - ''\n          - ''\n          - ''\n          - ''\n          - Code\n          - Run\n      index:\n        - 1\n\n```\n\nPlaybook Output:\n\n```\n\nPLAY [localhost] ******************************************************************************************************************************************************************************************************************************************************************\n\nTASK [Gathering Facts] ************************************************************************************************************************************************************************************************************************************************************\nok: [localhost]\n\nTASK [Include Parse Genie Role] ***************************************************************************************************************************************************************************************************************************************************\n\nTASK [Include vars] ***************************************************************************************************************************************************************************************************************************************************************\nok: [localhost]\n\nTASK [Test Genie Filter for generic tabular data] *********************************************************************************************************************************************************************************************************************************\nok: [localhost -\u003e localhost] =\u003e {\n    \"msg\": {\n        \"*1\": {\n            \"Destination \": \"10.0.0.2\",\n            \"ID \": \"*1\",\n            \"Last Run\": \"20 seconds ago\",\n            \"Return Code\": \"OK\",\n            \"Stats \": \"RTT=900u\",\n            \"Type \": \"udp-jitter\"\n        },\n        \"*2\": {\n            \"Destination \": \"10.0.0.2\",\n            \"ID \": \"*2\",\n            \"Last Run\": \"3 seconds ago\",\n            \"Return Code\": \"OK\",\n            \"Stats \": \"RTT=1\",\n            \"Type \": \"icmp-echo\"\n        }\n    }\n}\n\nPLAY RECAP ************************************************************************************************************************************************************************************************************************************************************************\nlocalhost                  : ok=3    changed=0    unreachable=0    failed=0   \n\n```\n\n\n\n### Development\n\nSet up your development environment:\n\n1. Clone the repo and go into it. `git clone https://github.com/clay584/parse_genie.git \u0026\u0026 cd parse_genie`\n2. Create a virtual environment. `python3 -m venv .venv`\n3. Activate the virtual environment. `source .venv/bin/activate`\n4. Install Ansible. `pip install ansible`\n5. Install Genie and pyATS. `pip install genie`\n6. Install yamllint. `pip install yamllint`\n\n### Testing\n\nRun these commands to test locally:\n\n1. Lint all of the YAML files. `yamllint -c yamllint_config.yml *`\n2. Run the test playbook. `ansible-playbook tests/test.yml --connection=local -i tests/inventory`\n\n### Pushing\n\nAnsible Galaxy works on tags.\n\n1. `git commit -m\"whatever'`\n2. `git tag -a X.X.X` - where X.X.X is a symantec versioning number.\n3. `git push origin master`\n4. `git push X.X.X`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclay584%2Fgenie_collection","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fclay584%2Fgenie_collection","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclay584%2Fgenie_collection/lists"}