https://github.com/niloysh/testbed-automator
Scripts for automating deployment of 5G testbed
https://github.com/niloysh/testbed-automator
k8s multus ovs-cni testbed
Last synced: about 2 months ago
JSON representation
Scripts for automating deployment of 5G testbed
- Host: GitHub
- URL: https://github.com/niloysh/testbed-automator
- Owner: niloysh
- License: mit
- Created: 2023-10-25T04:22:55.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2025-10-22T05:07:39.000Z (8 months ago)
- Last Synced: 2025-10-22T07:10:49.528Z (8 months ago)
- Topics: k8s, multus, ovs-cni, testbed
- Language: Shell
- Homepage:
- Size: 9.54 MB
- Stars: 15
- Watchers: 1
- Forks: 11
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-telco - testbed-automator - 12]` - Scripts for automating deployment of 5G testbeds with Open5GS, free5GC, and UERANSIM. (Lab & Testbeds / Ready-to-Use Environments)
README
# testbed-automator
This project provides scripts to automate the setup and teardown of a Kubernetes cluster for deploying a 5G core network.
This repo is designed to be used with [free5gc-k8s](https://github.com/niloysh/free5gc-k8s) and [open5gs-k8s](https://github.com/niloysh/open5gs-k8s).
# Requirements
- Supported OS: **Ubuntu 22.04 LTS** (recommended); Ubuntu 20.04 is compatible but unmaintained.
- Minimum hardware: **4 cores, 4GB RAM**, and internet access.
# Quick Start
```bash
# Clone the repository
git clone https://github.com/niloysh/testbed-automator.git
cd testbed-automator
# Run the installation script
./install.sh
```
**Note**: Run scripts as a regular user (not as root).
The `install.sh` script should take approximately 5 mintues to run to completion. On successful deployment, you should see all pods in **Running** state as shown below.
# Scripts Overview
Descriptions of the scripts in this repo are given below.
- `install.sh` : Automates the deployment of a single-node k8s cluster, configures cluster, installs various CNIs, configures ovs bridges and sets everything up for deployment of 5G core.
- `uninstall.sh`: Reverts changes made by install.sh, removing the Kubernetes cluster and configurations.
- `cleanup.sh`: Removes remants (old configuration files, CNIs) etc from previous installations of Kubernetes. Use this after `uninstall.sh` before running `install.sh` again.
- `worker-join-token.sh`: Run this on the **master** node to print out the command for adding worker nodes to the cluster.
**Note**: Run scripts as regular user with sudo permissions if needed. Do not use **root**.
# Multi-node deployment
To deploy a multi-node Kubernetes cluster, start by setting up the master node using install.sh, and then add worker nodes.
1. **Set up the Master Node**:
Run install.sh on the intended master node as shown in the [Quick Start](#quick-start) section.
2. **Set up each Worker Node**: On each worker node, run the install.sh script with the `--worker` flag as follows.
```bash
# Clone the repository
git clone https://github.com/niloysh/testbed-automator.git
cd testbed-automator
# Run the installation script for the worker node
sudo ./install.sh --worker
```
3. **Generate Join Token**:
On the master node, execute:
```bash
sudo ./worker-join-token.sh
```
This will output a command that includes the token and IP address for the worker node to join the cluster.
4. **Join the Worker Nodes**:
On each worker node, run (with sudo) the join command generated by worker-join-token.sh on the master node. The command will look something like this:
```bash
sudo kubeadm join : --token --discovery-token-ca-cert-hash
```
5. **Verify Node Status**:
Back on the master node, check the status of all nodes:
```bash
kubectl get nodes
```
Each worker node should appear with the status **Ready**.
6. **Connect every worker node to the master node using VXLAN tunnels**:
```bash
sudo python3 setup-tunnels.py
```
**Note**: The `setup-tunnels.py` script is a work in progress and may not work as expected, as it makes several assumptions such as passwordless SSH being enabled. The idea is to create VXLAN tunnels between worker nodes and the master node to enable communication between them for the secondary Multus interface. If the script does not work, you can manually create VXLAN tunnels between the nodes.
# Useful Resources and Commands
This project relies on several networking and Kubernetes tools that are essential for 5G deployments. Below are resources to help understand these tools better, along with some common commands.
## Key Technologies
- [Kubernetes](https://kubernetes.io/): The orchestrator for containerized applications.
- [Multus CNI](https://github.com/k8snetworkplumbingwg/multus-cni): Enables multiple network interfaces in Kubernetes.
- [OpenVSwitch (OVS)](https://www.openvswitch.org/): A virtual switch for network virtualization.
- [OVS-CNI](https://github.com/k8snetworkplumbingwg/ovs-cni): A plugin that integrates OpenVSwitch with Kubernetes.
## Common Kubernetes Commands
```bash
# View all nodes in the cluster
kubectl get nodes
# View all pods in the default namespace
kubectl get pods
# View detailed info about a specific pod
kubectl describe pod
# List services in the namespace
kubectl get svc
# Delete a pod
kubectl delete pod
```
## Useful OVS Commands
```bash
# Show OVS bridges
ovs-vsctl show
# Add a new bridge
sudo ovs-vsctl add-br
# Delete an OVS bridge
sudo ovs-vsctl del-br
```
# Troubleshooting
## Common issues
- **Check Logs**: Each function logs output to help trace errors. Look for “RED” messages indicating critical failures.
- **Network Requirements**: Some components require network access (e.g., Helm, Kubernetes packages). Ensure the server can access the internet during the installation process.
- **Resource Limits**: Verify system resources meet minimum requirements. Kubernetes installations may fail with less than the recommended CPU and RAM.
For additional help, see [troubleshooting.md](troubleshooting.md)
# Contributing
Contributions are welcome! Please open an issue or submit a pull request. For larger changes, open an issue first to discuss the proposed updates.
## License
This repository is licensed under the [MIT License](LICENSE).