{"id":30277319,"url":"https://github.com/allenrobel/n9kv-kvm","last_synced_at":"2025-08-16T11:55:59.784Z","repository":{"id":305610531,"uuid":"1023349348","full_name":"allenrobel/n9kv-kvm","owner":"allenrobel","description":"Bringup a small VXLAN lab with Cisco Nexus Dashboard and Cisco Nexus9000v (aka n9kv) using Ubuntu 24.04.2 LTS virtualization stack.","archived":false,"fork":false,"pushed_at":"2025-08-14T20:40:12.000Z","size":21288,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-14T21:23:10.128Z","etag":null,"topics":["chrony","cisco","dnsmasq","libvirt","n9kv","nd","nexus","nexus9000v","qemu","ubuntu"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/allenrobel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-21T03:16:58.000Z","updated_at":"2025-08-14T20:40:16.000Z","dependencies_parsed_at":"2025-08-14T21:07:48.609Z","dependency_job_id":"912acf6f-8c08-4702-85fa-88ef303823c7","html_url":"https://github.com/allenrobel/n9kv-kvm","commit_stats":null,"previous_names":["allenrobel/n9kv-kvm"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/allenrobel/n9kv-kvm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allenrobel%2Fn9kv-kvm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allenrobel%2Fn9kv-kvm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allenrobel%2Fn9kv-kvm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allenrobel%2Fn9kv-kvm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/allenrobel","download_url":"https://codeload.github.com/allenrobel/n9kv-kvm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allenrobel%2Fn9kv-kvm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270708137,"owners_count":24631979,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-16T02:00:11.002Z","response_time":91,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chrony","cisco","dnsmasq","libvirt","n9kv","nd","nexus","nexus9000v","qemu","ubuntu"],"created_at":"2025-08-16T11:55:54.457Z","updated_at":"2025-08-16T11:55:59.758Z","avatar_url":"https://github.com/allenrobel.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Summary\n\nBringup a small multi-site VXLAN lab with Cisco Nexus Dashboard and Cisco Nexus9000v\n(aka n9kv) using Ubuntu 24.04.2 LTS virtualization stack.\n\n[Topology](#topology-built-by-this-repository)\n\nNOTE: You'll need a Cisco account to download Nexus Dashboard and Nexus9000v images.\n\n## Hardware Requirements\n\n- At least 500GB disk (preferrably 1TB)\n- At least 256GB RAM (preferrably 512GB)\n\n## Software Environment\n\nThis repository has been tested with the software versions listed below.\n\nA note about the `Installation` links below.  Components should be installed\nin the order they appear in this document since some components depend on\npreviously-installed components.  The intent of the `Installation` links\nis twofold:\n\n- Provide an overview of what lies ahead\n- Provide easy reference access to specific sections after the project is up and running\n\nIt's assumed Ubuntu 24.04.2 LTS is already installed on hardware that\nmeets the [Hardware Requirements](#hardware-requirements) and on which\n[KVM is supported](#kvm-support).\n\n- [Cisco Nexus Dashboard](https://www.cisco.com/c/en/us/support/data-center-analytics/nexus-dashboard/series.html)\n  - nd-dk9.3.2.1e.qcow2\n  - [Installation](./docs/nd_installation.md)\n- [Cisco Nexus9000v](https://www.cisco.com/c/en/us/td/docs/dcn/nx-os/nexus9000/103x/n9000v-n9300v-9500v/cisco-nexus-9000v-9300v-9500v-guide-release-103x.html)\n  - nexus9300v64.10.3.8.M.qcow2\n  - nexus9500v64.10.5.3.F.qcow2\n  - [Installation](./docs/n9kv_bringup.md)\n- Ubuntu\n  - [24.04.2 LTS](https://ubuntu.com/desktop)\n- Python\n  - [3.13.5](https://www.python.org/downloads/release/python-3135/)\n  - The stock Python 3.12 on Ubuntu 24.04.2 LTS should also work\n  - [Installation](#install-python-313)\n- [Ansible](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html#installing-and-upgrading-ansible-with-pip)\n  - 2.18.7\n  - [Installation](./docs/clone_prepare_repo.md)\n- NDFC Ansible Collection\n  - [Installation](./docs/install_ansible_collection.md)\n- [QEMU](https://www.qemu.org)\n  - qemu-system-x86_64 version 8.2.2\n  - [Installation](#install-qemu-and-libvirt-virtualization-stack)\n- [OVMF](https://wiki.ubuntu.com/UEFI/OVMF) (used for nk9v BIOS)\n  - [Installation](#install-ovmf)\n- [Cockpit](https://cockpit-project.org)\n  - Optional (for monitoring n9kv VMs)\n  - Version 343\n  - [Installation (work in progress)](https://github.com/allenrobel/n9kv-kvm/tree/main/cockpit)\n- [dnsmasq](https://wiki.debian.org/dnsmasq)\n  - DNS server (for ND)\n  - 2.90-2ubuntu0.1\n  - [Installation](#dnsmasq-installation-and-configuration)\n- [chrony](https://chrony-project.org)\n  - NTP server (for ND)\n  - chrony/noble-updates,now 4.5-1ubuntu4.2\n  - [Installation](#chrony-installation-and-configuration)\n- [debootstrap](https://launchpad.net/ubuntu/noble/amd64/debootstrap)\n  - Create LXC host containers for end-to-end network testing hosts (H1 and H2 in the topology shown further below)\n  - 1.0.134ubuntu1\n- `libvirt-daemon-driver-lxc`\n  - LXC support for libvirt for H1 and H2 network testing endpoint hosts\n  - 10.0.0-2ubuntu8.8\n\n## Install and Setup\n\nWe've arranged the steps below so that dependencies for later steps are met by earlier steps.\nFor best results, follow the steps below in order.\n\n## KVM Support\n\nCheck if KVM is supported. If this returns error(s) things are not going to work for you.\n\n```bash\nsudo apt install cpu-checker\nkvm-ok\n```\n\n## Install Python 3.13\n\nI use Python 3.13, but the stock Python 3.12 on Ubuntu 24.04.2 LTS should be fine.\n\nTo install Python 3.13, do the following.  Add the deadsnakes PPA.\nThis PPA contains more recent Python versions packaged for Ubuntu.\n\n```bash\nsudo add-apt-repository ppa:deadsnakes/ppa\nsudo apt update\n\n# Install\nsudo apt install python3.13\n\n# Install additional packages, especially python3.13-venv which we use further below\nsudo apt install python3.13-venv python3.13-dev\n```\n\n## Install QEMU and libvirt Virtualization Stack\n\nYou'll need the virtualization stack consisting of qemu and libvirt.\nInstall them as follows.\n\n```bash\nsudo apt update\nsudo apt install qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils virt-manager\n```\n\nIf you don't want to run virsh and other virtualization commands as root, and you want\nto run Virtual Machine Manager as a normal user (not root) add yourself to the libvirt\ngroup.\n\n```bash\n# Run the following as a non-root user with sudo access.\nsudo usermod -aG libvirt $USER\nsudo usermod -aG kvm $USER\nnewgrp libvirt\nsudo systemctl enable --now libvirtd\n\n# Check libvirt status\nsudo systemctl status libvirtd\n\n# Assuming you've executed the above, you can run the virt-manager GUI (Virtual Machine Manager) as a normal user.\nvirt-manager\n```\n\n## Install OVMF\n\nYou'll need OVMF for the nexus9000v BIOS\n\n```bash\nsudo apt install ovmf\n```\n\n## Clone and Prepare the `n9kv-kvm` Repository\n\nFollow the link below to clone this project's repository.\n\n[Clone and Prepare Repository](./docs/clone_prepare_repo.md)\n\n## Install the NDFC Ansible Collection\n\nWe'll need this collection to run playbooks later that fix\nnexus9000v inter-switch mac address issues.\n\nFollow the steps outlined in\n[Install NDFC Ansible Collection](./docs/install_ansible_collection.md)\n\n## Setup Bridges\n\nFollow the link below to configure the bridges used for this project.\n\n[Setup Bridges](./docs/bridges.md)\n\n## dnsmasq Installation and Configuration\n\nFollow the link below to install and configure `dnsmasq` for this project.\n\n[Install, Configure, and Manage dnsmasq](./docs/dnsmasq.md)\n\n## chrony Installation and Configuration\n\nFollow the link below to install and configure `chrony` for this project.\n\n[Install, Configure, and Manage chrony](./docs/chrony.md)\n\n## Nexus Dashboard Installation\n\nFollow the link below to install Nexus Dashboard.\n\n[Install Nexus Dashboard](./docs/nd_installation.md)\n\n## Create Nexus Dashboard Fabrics\n\nFollow the link below to create the Nexus Dashboard fabrics used in this project.\n\n[Create ISN, SITE1, and SITE2 Fabrics](./docs/nd_create_fabrics.md)\n\n## nexus9000v Initial Configuration and Bringup\n\nFollow the link below to configure and bringup the nexus9000v VMs for this project.\n\n[nexus9000v Configuration and Startup](./docs/n9kv_bringup.md)\n\n## Add nexus9000v Switches to Nexus Dashboard Fabrics\n\n[Add nexus9000v Switches](./docs/nd_add_switches.md)\n\n## Fix duplicate mac addresses on interswitch-links\n\nYou'll notice that the nexus9000v switches are complaining about bridge\ndisputes, etc, on their Eth1/1-2 interaces, and they are unable to peer.\n\nFollow this link to fix this.\n\n[nexus9000v Fix Interface Mac Addresses](./docs/n9kv_fix_interface_mac_addresses.md)\n\n## Install libvert LXC Support\n\nWe are using LXC-style containers, running under libvirt, for network testing endpoint hosts.\n\nBefore running their creation scripts, we need to install LXC support for libvirt.\n\nFollow this link to complete this step.\n\n[libvirt LXC Driver Installation](./docs/install_libvirt_lxc_driver.md)\n\n## Topology built by this repository\n\n- Three fabrics\n  - ISN (inter-site network)\n    - 1x Edge Router (ER)\n  - SITE1 (VxLAN)\n    - 1x Border Spines (S1)\n    - 1x Leaf (L1)\n    - 1x Host (H1)\n  - SITE2 (VxLAN)\n    - 1x Border Spines (S2)\n    - 1x Leaf (L2)\n    - 1x Host (H2)\n\n```mermaid\ngraph TB\n    subgraph ISN[\"ISN Fabric (Inter-Site Network)\"]\n        ER[Edge Router - ER]\n    end\n\n    subgraph SITE1[\"SITE1\"]\n        S1[Border Spine - S1]\n        L1[Leaf - L1]\n        H1[Host - H1]\n        \n        %% SITE1 fabric connections\n        S1 --- L1\n        L1 --- H1\n    end\n\n    subgraph SITE2[\"SITE2\"]\n        S2[Border Spine - S2]\n        L2[Leaf - L2]\n        H2[Host - H2]\n        \n        %% SITE2 fabric connections\n        S2 --- L2\n        L2 --- H2\n    end\n\n    %% Inter-fabric connection\n    ER --- S1\n    ER --- S2\n\n    %% Styling\n    classDef fabricBox fill:#e1f5fe,stroke:#01579b,stroke-width:2px\n    classDef edgeRouter fill:#fff3e0,stroke:#e65100,stroke-width:1px,color:#000000\n    classDef borderSpine fill:#f3e5f5,stroke:#4a148c,stroke-width:1px,color:#000000\n    classDef leaf fill:#a8f3c8,stroke:#2e7d32,stroke-width:1px,color:#000000\n    classDef host fill:#e8f2a0,stroke:#2e7d32,stroke-width:1px,color:#000000\n\n    class ER edgeRouter\n    class S1,S2 borderSpine\n    class L1,L2 leaf\n    class H1,H2 host\n        \n```\n\n## Project Structure\n\n```bash\narobel@Allen-M4 n9kv-kvm % tree\n.\n├── cockpit\n│   ├── cockpit.png\n│   ├── README.md\n│   └── usr\n│       ├── local\n│       │   └── bin\n│       │       ├── nexus9000v_monitor.py\n│       │       ├── nexus9000v-monitor.service\n│       │       ├── nexus9000v-monitor.timer\n│       │       └── README.md\n│       └── share\n│           └── cockpit\n│               └── nexus9000v\n│                   ├── index.html\n│                   ├── manifest.json\n│                   ├── nexus-monitor-dark-theme.css\n│                   ├── nexus-monitor-light-theme.css\n│                   ├── nexus-monitor.css\n│                   ├── nexus-monitor.js\n│                   └── README.md\n├── config\n│   ├── ansible\n│   │   ├── dynamic_inventory.py\n│   │   ├── interface_mac_addresses_S1.yaml\n│   │   ├── interface_mac_addresses_S2.yaml\n│   │   ├── networks_site1.yaml\n│   │   ├── nxos_startup_config.j2\n│   │   ├── overlay_site1.yaml\n│   │   ├── overlay_site2.yaml\n│   │   └── startup_config_iso.yaml\n│   ├── bridges\n│   │   ├── 99-bridges.yaml\n│   │   ├── bridge.conf\n│   │   ├── bridges_config.sh\n│   │   ├── bridges_down.sh\n│   │   └── bridges_monitor.sh\n│   ├── H1\n│   │   ├── create_h1.sh\n│   │   ├── H1.xml\n│   │   └── manage_h1.sh\n│   ├── H2\n│   │   ├── create_h2.sh\n│   │   ├── H2.xml\n│   │   └── manage_h2.sh\n│   └── qemu\n│       ├── n9kv_qemu_ER_cdrom.sh\n│       ├── n9kv_qemu_ER.sh\n│       ├── n9kv_qemu_L1.sh\n│       ├── n9kv_qemu_L2.sh\n│       ├── n9kv_qemu_S1.sh\n│       ├── n9kv_qemu_S2.sh\n│       ├── nd_qemu_321e.sh\n│       └── nd_qemu_EFT.sh\n├── docs\n│   ├── bridges.md\n│   ├── chrony.md\n│   ├── clone_prepare_repo.md\n│   ├── dnsmasq.md\n│   ├── images\n│   │   ├── nd3\n│   │   │   ├── 01_cluster_bringup.png\n│   │   │   ├── 02_node_details.png\n│   │   │   ├── 03_edit_node.png\n│   │   │   ├── 04_cluster_bringup.png\n│   │   │   ├── 05_deployment_mode.png\n│   │   │   ├── 06_external_service_ips.png\n│   │   │   ├── 07_cluster_bringup.png\n│   │   │   ├── 08_summary.png\n│   │   │   ├── 09_warning.png\n│   │   │   ├── 10_progress.png\n│   │   │   ├── 11_login.png\n│   │   │   ├── 12_progress.png\n│   │   │   ├── 13_meet_nexus_dashboard.png\n│   │   │   ├── 14_getting_started_map.png\n│   │   │   └── add_switches\n│   │   │       ├── 00_manage_inventory.png\n│   │   │       ├── 01_inventory.png\n│   │   │       ├── 02_pick_a_fabric.png\n│   │   │       ├── 03_select_fabric.png\n│   │   │       ├── 04_seed_switch_details.png\n│   │   │       ├── 05_discovery_results.png\n│   │   │       ├── 06_wait.png\n│   │   │       ├── 07_wait.png\n│   │   │       ├── 08_inventory.png\n│   │   │       ├── 09_pick_a_fabric.png\n│   │   │       ├── 10_select_fabric.png\n│   │   │       ├── 11_seed_switch_details.png\n│   │   │       ├── 12_warning_dialog.png\n│   │   │       ├── 13_discovery_results.png\n│   │   │       ├── 14_switches_reboot.png\n│   │   │       ├── 15_wait_for_switch_added.png\n│   │   │       ├── 16_add_switches.png\n│   │   │       ├── 17_pick_a_fabric.png\n│   │   │       ├── 18_select_fabric.png\n│   │   │       ├── 19_seed_switch_details.png\n│   │   │       ├── 20_warning.png\n│   │   │       ├── 21_discovery_results.png\n│   │   │       ├── 22_switches_reboot.png\n│   │   │       ├── 23_wait_for_switch_added.png\n│   │   │       ├── 24_wait_for_discovery_ok.png\n│   │   │       ├── 25_set_role.png\n│   │   │       ├── 26_select_role.png\n│   │   │       ├── 27_warning.png\n│   │   │       ├── 28_border_spine_switch_role.png\n│   │   │       ├── 29_select_role.png\n│   │   │       ├── 30_warning.png\n│   │   │       ├── 31_set_role.png\n│   │   │       ├── 32_select_role.png\n│   │   │       ├── 33_warning.png\n│   │   │       ├── 34_wait.png\n│   │   │       ├── 35_all_switches_ready.png\n│   │   │       ├── 36_manage_fabrics.png\n│   │   │       ├── 37_site1_fabric.png\n│   │   │       ├── 38_recalculate_and_deploy.png\n│   │   │       ├── 39_wait.png\n│   │   │       ├── 40_deploy.png\n│   │   │       ├── 41_wait.png\n│   │   │       ├── 42_close.png\n│   │   │       └── 43_close_window.png\n│   │   ├── nd4\n│   │   │   ├── 01_journey.png\n│   │   │   ├── 02_basic_information.png\n│   │   │   ├── 04_node_details_cluster_connectivity.png\n│   │   │   ├── 05_persistent_ips.png\n│   │   │   ├── 06_persistent_ips_added.png\n│   │   │   ├── 07_summary.png\n│   │   │   ├── 08_summary_error.png\n│   │   │   ├── 09_login.png\n│   │   │   ├── 10_cluster_install.png\n│   │   │   ├── 11_system_software.png\n│   │   │   ├── 12_release_details.png\n│   │   │   └── 13_whats_new.png\n│   │   └── ndfc\n│   │       ├── 00_first_access.png\n│   │       ├── 01_introduction.png\n│   │       ├── 02_journey.png\n│   │       ├── 03_operational_modes.png\n│   │       ├── 04_feature_selection.png\n│   │       ├── 05_summary.png\n│   │       ├── 06_controller_service_setup.png\n│   │       ├── 07_journey.png\n│   │       ├── 08_features_updated.png\n│   │       ├── 09_intro_set_credentials_behind.png\n│   │       ├── 10_set_credentials_intro_behind.png\n│   │       ├── 11_lan_credentials_management.png\n│   │       ├── 12_set_credentials.png\n│   │       └── 13_success.png\n│   ├── install_ansible_collection.md\n│   ├── install_libvirt_lxc_driver.md\n│   ├── n9kv_bringup.md\n│   ├── n9kv_fix_interface_mac_addresses.md\n│   ├── nd_add_switches.md\n│   ├── nd_bringup_cli.md\n│   ├── nd_create_fabrics.md\n│   ├── nd_installation.md\n│   ├── nd3_add_switches.md\n│   ├── nd3_bringup_web.md\n│   ├── nd3_fabrics_bringup.md\n│   ├── nd4_add_switches.md\n│   ├── nd4_bringup_web.md\n│   ├── nd4_fabrics_bringup.md\n│   ├── ndfc_bringup_web.md\n│   └── topology.mmd\n├── env\n│   ├── env_ansible.sh\n│   ├── env_libvirt.sh\n│   └── env_python.sh\n├── monitor\n│   └── show_nd_interfaces\n├── pyproject.toml\n├── README.md\n├── tmp_remove\n│   └── detour.md\n└── uv.lock\n\n23 directories, 152 files\narobel@Allen-M4 n9kv-kvm %\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallenrobel%2Fn9kv-kvm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fallenrobel%2Fn9kv-kvm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallenrobel%2Fn9kv-kvm/lists"}