https://github.com/opennetworkinglab/int-host-reporter
User space agent and eBPF programs that provide INT support to end hosts
https://github.com/opennetworkinglab/int-host-reporter
Last synced: 7 months ago
JSON representation
User space agent and eBPF programs that provide INT support to end hosts
- Host: GitHub
- URL: https://github.com/opennetworkinglab/int-host-reporter
- Owner: opennetworkinglab
- Created: 2021-05-27T14:41:22.000Z (about 5 years ago)
- Default Branch: main
- Last Pushed: 2022-01-20T19:16:02.000Z (over 4 years ago)
- Last Synced: 2024-06-19T16:48:35.455Z (almost 2 years ago)
- Language: Go
- Homepage:
- Size: 811 KB
- Stars: 14
- Watchers: 6
- Forks: 1
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSES/Apache-2.0.txt
Awesome Lists containing this project
README
# INT Host Reporter
The In-Band Network Telemetry (INT) standard is a network telemetry solution providing an in-depth, per-packet network visibility.
So far, the INT standard has been mainly implemented by the network switches (this implementation is somewhere referred as "switch-INT").
The INT Host Reporter is the implementation of the "host-INT" approach - the concept of extending the In-Band Network Telemetry (INT)
support to the end hosts.
The INT Host Reporter is tightly integrated with Kubernetes and generates an INT report (flow or drop report) for each packet being sent between (virtual) network interfaces managed by the Kubernetes Container Network Interface (CNI).
The INT reports are furhter sent to the INT collector that gathers INT reports from different network devices (e.g. switches, hosts) and shows the end-to-end network statistics.
This enables observing E2E network flows traversing the Kubernetes cluster (e.g. Pod-to-Pod communication or packets to/from Kubernetes Services).
## Overview
`INT Host Reporter` implements the "host-INT" approach by using a combination of the eBPF code running in the Linux kernel
and a Go application running as an userspace agent. The diagram below shows the high-level design of the host-INT solution, which is independent of the Kubernetes CNI being used.
As depicted in the diagram, the system consists of two pieces:
- the **INT-aware eBPF programs** are attached to both TC Ingress and TC Egress hooks. The ingress eBPF program
does a basic packet pre-processing and collects ingress metadata that is further stored in a shared BPF map.
The egress eBPF program reads the per-packet metadata and generates a data plane report by pushing an event to the
`BPF_PERF_EVENT_ARRAY` map .
- the **INT Host Reporter** application listens to the events from the `BPF_PERF_EVENT_ARRAY` map, converts
the data plane reports into INT reports and sends the INT reports to the INT collector.
As mentioned before, the INT Host Reporter works in the CNI-independent fashion, so the target goal is to enable using INT Host Reporter with _any_ Kubernetes CNI.
So far, we have tested the INT Host Reporter with [Calico](https://docs.projectcalico.org/getting-started/kubernetes/) and [Cilium](https://cilium.io/) and DeepInsight from Intel as an INT collector.
## Building INT Host Reporter
To build the INT Host Reporter image from scratch run the below command from the main directory:
```bash
$ docker build -t : .
```
## Downloading INT Host Reporter image
Alternatively, the Docker image of INT Host Reporter can be downloaded from the ONF registry.
```bash
$ docker pull opennetworking/int-host-reporter:latest
```
## Deployment guide
### Requirements
- Kernel version v5.11 or higher.
- `CAP_SYS_ADMIN` privileges with access to the host network (`hostNetwork: true`).
- Access to the eBPF filesystem (`/sys/fs/bpf`).
- INT Host Reporter exposes REST API on `tcp/4048` port. Make sure this port is accessible.
### Install the K8s cluster
The installation of a Kubernetes cluster is basically out of scope of this document.
You should follow the instructions to deploy the Kubernetes cluster using the installer of your choice ([see Kubernetes documentation](https://kubernetes.io/docs/setup/)).
### Deploy INT Host Reporter
#### Using Kubernetes deployment files
Edit `deployment/kubernetes/inthostreporter.yaml` and configure the following variables:
- `CNI` should define the Kubernetes CNI being used for the cluster. We currently support the following values: `cilium`, `calico-ebpf`, `calico-iptables`.
- `COLLECTOR` should point to the address of the INT collector (e.g. 192.168.50.99:32766).
- `DATA_INTERFACE` should be set to the name of physical interface that has the `status.hostIP` assigned.
For instance, if Kubernetes API server is advertised on `192.168.99.20`, the data interface is the interface that has this IP address assigned.
Next, prepare the INT watchlist file (modify `configs/watchlist.yaml` if needed) and deploy it as ConfigMap.
`$ kubectl create -n kube-system configmap watchlist-conf --from-file=./configs/watchlist.yaml`
Then, run the below command to deploy the INT Host Reporter.
`$ kubectl apply -f deployment/kubernetes/inthostreporter.yaml`
Verify that the `int-host-reporter` is in the Running state on each node:
```bash
$ kubectl get pods --all-namespaces -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
kube-system int-host-reporter-jpdl2 1/1 Running 0 9m11s 10.79.233.238 worker2
kube-system int-host-reporter-ljwvs 1/1 Running 0 9m11s 10.68.235.172 worker1
kube-system int-host-reporter-x48ps 1/1 Running 0 9m11s 10.67.219.106 kubemaster
```
For the CNIs that we have tested, there are no CNI-specific configuration steps required for the INT Host Reporter to work properly.
Once the INT Host Reporter is successfully deployed, it should start sending INT reports to the collector.
You can uninstall `int-host-reporter` using:
`$ kubectl delete -f deployment/kubernetes/inthostreporter.yaml`
#### Using Helm
We also provide Helm charts to deploy INT Host Reporter.
You can customize your deployment by changing configuration in `deployments/helm/int-host-reporter/Values.yaml`.
To deploy `int-host-reporter` run the below command:
`$ helm install --namespace ./deployments/helm/int-host-reporter [-f .yaml]`
To uninstall the Helm deployment run:
`$ helm uninstall --namespace `
## Using INT Host Reporter with DeepInsight
If the INT Host Reporter has been successfully deployed, it will start generating and sending INT reports to the INT collector.
So far, the Host INT Reporter has been tested with DeepInsight (DI) - the INT collector provided by Intel.
DeepInsight requires the additional configuration step to start visualizing network statistics.
The additional configuration step is to upload the DI topology file - the JSON file that describes
the topology of a network. In particular, the DI topology should have the following sections defined:
- `switches` - the list of switches in the network. In the case of host-INT, the virtual switch (CNI datapath) is also defined as the switch.
Thus, this section should contain each network switch plus all the hosts, where the Kubernetes is running on.
- `hosts` - the list of hosts in the network. In the case of host-INT, the `hosts` section should contain all servers attached to the network,
except those being used as K8s nodes.
- `subnets` - the list of subnets in the network. In the case of host-INT, it should contain the subnet used to interconnect Kubernetes nodes, as well as
the all the Pod subnets (the range of IP addresses, from which the Pod IPs are assigned from). Typically, a CNI will use a single IP subnet per Kubernetes worker,
so there should be at least as many subnets defined as the number of Kubernetes workers.
- `links` - the list of links in the network. It represents links between 2 switches, between a switch and a host, or between a switch and a subnet -
the last way of representing a link is an abstraction introduced by DeepInsight and means that we can access a subnet via a given interface of a switch.
In the case of host-INT, Pods are not defined as `hosts`, but we use `subnet` to represent the group of Pods attached to the network. Therefore,
the `links` section should contain the links between a virtual switch and a Pod's subnet for each virtual interface configured on the Kubernetes node.
As a reference, we provide a sample DI topology file in in the `examples/deepinsight/` directory. The sample DI topology file is visualized below. The dashed lines
represents the abstraction of connections between switch and subnet.
However, building the DI topology file manually is time-consuming and error-prone. Therefore, we have created the `./di gen-topology` script
to automate this process. You can find the guide how to use this script in [the sdfabric-utils repository](https://github.com/opennetworkinglab/sdfabric-utils) (member-only, please reach out to your ONF representative to get access).
The `./di gen-topology` scipt leverages INT Host Reporter's `GET /api/v1/topology` API exposed on each
K8s nodes to retrieve information about local links.
## Current limitations
- Only IPv4 endpoints are supported.
- INT Host Reporter only supports UDP/TCP packets; ICMP packets are not reported.
- System flows (e.g. traffic between Kubernetes agents) are not reported.
- INT Host Reporeter doesn't work with CNIs making use of XDP hook yet.
If you want to use INT Host Reporter with Calico-eBPF or Cilium please make sure that XDP acceleration is disabled.
## License
The user space components of INT Host Reporter are licensed under the [Apache License, Version 2.0](LICENSES/Apache-2.0.txt). The BPF code
is licensed under the [General Public License, Version 2.0](LICENSES/GPL-2.0-only.txt).