{"id":25495949,"url":"https://github.com/opennetworkinglab/int-host-reporter","last_synced_at":"2025-11-09T21:30:21.602Z","repository":{"id":45597608,"uuid":"371406496","full_name":"opennetworkinglab/int-host-reporter","owner":"opennetworkinglab","description":"User space agent and eBPF programs that provide INT support to end hosts","archived":false,"fork":false,"pushed_at":"2022-01-20T19:16:02.000Z","size":830,"stargazers_count":14,"open_issues_count":3,"forks_count":1,"subscribers_count":6,"default_branch":"main","last_synced_at":"2024-06-19T16:48:35.455Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/opennetworkinglab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSES/Apache-2.0.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-05-27T14:41:22.000Z","updated_at":"2023-04-16T13:04:05.000Z","dependencies_parsed_at":"2022-09-11T16:22:31.451Z","dependency_job_id":null,"html_url":"https://github.com/opennetworkinglab/int-host-reporter","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opennetworkinglab%2Fint-host-reporter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opennetworkinglab%2Fint-host-reporter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opennetworkinglab%2Fint-host-reporter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opennetworkinglab%2Fint-host-reporter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/opennetworkinglab","download_url":"https://codeload.github.com/opennetworkinglab/int-host-reporter/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239576797,"owners_count":19662113,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-19T00:52:46.944Z","updated_at":"2025-11-09T21:30:21.549Z","avatar_url":"https://github.com/opennetworkinglab.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!--\nCopyright 2021-present Open Networking Foundation\nSPDX-License-Identifier: Apache-2.0\n--\u003e\n\n# INT Host Reporter\n\nThe In-Band Network Telemetry (INT) standard is a network telemetry solution providing an in-depth, per-packet network visibility. \nSo far, the INT standard has been mainly implemented by the network switches (this implementation is somewhere referred as \"switch-INT\"). \n\nThe INT Host Reporter is the implementation of the \"host-INT\" approach - the concept of extending the In-Band Network Telemetry (INT)\nsupport to the end hosts. \nThe INT Host Reporter is tightly integrated with Kubernetes and generates an INT report (flow or drop report) for each packet being sent between (virtual) network interfaces managed by the Kubernetes Container Network Interface (CNI). \nThe INT reports are furhter sent to the INT collector that gathers INT reports from different network devices (e.g. switches, hosts) and shows the end-to-end network statistics.\nThis enables observing E2E network flows traversing the Kubernetes cluster (e.g. Pod-to-Pod communication or packets to/from Kubernetes Services).\n\n## Overview\n\n`INT Host Reporter` implements the \"host-INT\" approach by using a combination of the eBPF code running in the Linux kernel \nand a Go application running as an userspace agent. The diagram below shows the high-level design of the host-INT solution, which is independent of the Kubernetes CNI being used.\n\n![Design](docs/static/images/design.png?raw=true \"High-level design of CNI-independent host-INT\")\n\nAs depicted in the diagram, the system consists of two pieces:\n\n- the **INT-aware eBPF programs** are attached to both TC Ingress and TC Egress hooks. The ingress eBPF program\ndoes a basic packet pre-processing and collects ingress metadata that is further stored in a shared BPF map.\n  The egress eBPF program reads the per-packet metadata and generates a data plane report by pushing an event to the \n  `BPF_PERF_EVENT_ARRAY` map .\n- the **INT Host Reporter** application listens to the events from the `BPF_PERF_EVENT_ARRAY` map, converts\nthe data plane reports into INT reports and sends the INT reports to the INT collector. \n  \nAs mentioned before, the INT Host Reporter works in the CNI-independent fashion, so the target goal is to enable using INT Host Reporter with _any_ Kubernetes CNI. \nSo far, we have tested the INT Host Reporter with [Calico](https://docs.projectcalico.org/getting-started/kubernetes/) and [Cilium](https://cilium.io/) and DeepInsight from Intel as an INT collector. \n\n## Building INT Host Reporter\n\nTo build the INT Host Reporter image from scratch run the below command from the main directory:\n\n```bash\n$ docker build -t \u003cIMAGE-NAME\u003e:\u003cTAG\u003e .\n```\n\n## Downloading INT Host Reporter image\n\nAlternatively, the Docker image of INT Host Reporter can be downloaded from the ONF registry.\n\n```bash\n$ docker pull opennetworking/int-host-reporter:latest\n```\n\n## Deployment guide\n\n### Requirements\n\n- Kernel version v5.11 or higher. \n- `CAP_SYS_ADMIN` privileges with access to the host network (`hostNetwork: true`).\n- Access to the eBPF filesystem (`/sys/fs/bpf`).\n- INT Host Reporter exposes REST API on `tcp/4048` port. Make sure this port is accessible.\n\n### Install the K8s cluster\n\nThe installation of a Kubernetes cluster is basically out of scope of this document. \nYou should follow the instructions to deploy the Kubernetes cluster using the installer of your choice ([see Kubernetes documentation](https://kubernetes.io/docs/setup/)).\n\n### Deploy INT Host Reporter\n\n#### Using Kubernetes deployment files\n\nEdit `deployment/kubernetes/inthostreporter.yaml` and configure the following variables:\n\n- `CNI` should define the Kubernetes CNI being used for the cluster. We currently support the following values: `cilium`, `calico-ebpf`, `calico-iptables`.\n- `COLLECTOR` should point to the address of the INT collector (e.g. 192.168.50.99:32766).\n- `DATA_INTERFACE` should be set to the name of physical interface that has the `status.hostIP` assigned. \n  For instance, if Kubernetes API server is advertised on `192.168.99.20`, the data interface is the interface that has this IP address assigned.\n\nNext, prepare the INT watchlist file (modify `configs/watchlist.yaml` if needed) and deploy it as ConfigMap. \n\n`$ kubectl create -n kube-system configmap watchlist-conf --from-file=./configs/watchlist.yaml`\n\nThen, run the below command to deploy the INT Host Reporter.\n\n`$ kubectl apply -f deployment/kubernetes/inthostreporter.yaml`\n\nVerify that the `int-host-reporter` is in the Running state on each node:\n\n```bash\n$ kubectl get pods --all-namespaces -o wide\nNAMESPACE     NAME                                       READY   STATUS    RESTARTS   AGE     IP              NODE         NOMINATED NODE   READINESS GATES\nkube-system   int-host-reporter-jpdl2                    1/1     Running   0          9m11s   10.79.233.238   worker2      \u003cnone\u003e           \u003cnone\u003e\nkube-system   int-host-reporter-ljwvs                    1/1     Running   0          9m11s   10.68.235.172   worker1      \u003cnone\u003e           \u003cnone\u003e\nkube-system   int-host-reporter-x48ps                    1/1     Running   0          9m11s   10.67.219.106   kubemaster   \u003cnone\u003e           \u003cnone\u003e\n```\n\nFor the CNIs that we have tested, there are no CNI-specific configuration steps required for the INT Host Reporter to work properly.\nOnce the INT Host Reporter is successfully deployed, it should start sending INT reports to the collector.\n\nYou can uninstall `int-host-reporter` using:\n\n`$ kubectl delete -f deployment/kubernetes/inthostreporter.yaml`\n\n#### Using Helm\n\nWe also provide Helm charts to deploy INT Host Reporter. \nYou can customize your deployment by changing configuration in `deployments/helm/int-host-reporter/Values.yaml`.\n\nTo deploy `int-host-reporter` run the below command:\n\n`$ helm install --namespace \u003cNAMESPACE\u003e \u003cNAME\u003e ./deployments/helm/int-host-reporter [-f \u003cCUSTOM-VALUES\u003e.yaml]`\n\nTo uninstall the Helm deployment run:\n\n`$ helm uninstall --namespace \u003cNAMESPACE\u003e \u003cNAME\u003e`\n\n## Using INT Host Reporter with DeepInsight\n\nIf the INT Host Reporter has been successfully deployed, it will start generating and sending INT reports to the INT collector.\nSo far, the Host INT Reporter has been tested with DeepInsight (DI) - the INT collector provided by Intel. \nDeepInsight requires the additional configuration step to start visualizing network statistics.  \n\nThe additional configuration step is to upload the DI topology file - the JSON file that describes \nthe topology of a network. In particular, the DI topology should have the following sections defined:\n\n- `switches` - the list of switches in the network. In the case of host-INT, the virtual switch (CNI datapath) is also defined as the switch.\nThus, this section should contain each network switch plus all the hosts, where the Kubernetes is running on.\n- `hosts` - the list of hosts in the network. In the case of host-INT, the `hosts` section should contain all servers attached to the network,\nexcept those being used as K8s nodes.\n- `subnets` - the list of subnets in the network. In the case of host-INT, it should contain the subnet used to interconnect Kubernetes nodes, as well as\nthe all the Pod subnets (the range of IP addresses, from which the Pod IPs are assigned from). Typically, a CNI will use a single IP subnet per Kubernetes worker,\n  so there should be at least as many subnets defined as the number of Kubernetes workers.\n- `links` - the list of links in the network. It represents links between 2 switches, between a switch and a host, or between a switch and a subnet -\nthe last way of representing a link is an abstraction introduced by DeepInsight and means that we can access a subnet via a given interface of a switch.\nIn the case of host-INT, Pods are not defined as `hosts`, but we use `subnet` to represent the group of Pods attached to the network. Therefore,\nthe `links` section should contain the links between a virtual switch and a Pod's subnet for each virtual interface configured on the Kubernetes node.\n\nAs a reference, we provide a sample DI topology file in in the `examples/deepinsight/` directory. The sample DI topology file is visualized below. The dashed lines\nrepresents the abstraction of connections between switch and subnet. \n\n![The visualization of a sample DI topology](docs/static/images/di-topo.png?raw=true \"The visualization of a sample DI topology\")\n\nHowever, building the DI topology file manually is time-consuming and error-prone. Therefore, we have created the `./di gen-topology` script\nto automate this process. You can find the guide how to use this script in [the sdfabric-utils repository](https://github.com/opennetworkinglab/sdfabric-utils) (member-only, please reach out to your ONF representative to get access).\n\nThe `./di gen-topology` scipt leverages INT Host Reporter's `GET /api/v1/topology` API exposed on each\nK8s nodes to retrieve information about local links. \n\n## Current limitations \n\n- Only IPv4 endpoints are supported.\n- INT Host Reporter only supports UDP/TCP packets; ICMP packets are not reported.\n- System flows (e.g. traffic between Kubernetes agents) are not reported.\n- INT Host Reporeter doesn't work with CNIs making use of XDP hook yet. \n  If you want to use INT Host Reporter with Calico-eBPF or Cilium please make sure that XDP acceleration is disabled.\n\n## License\n\nThe user space components of INT Host Reporter are licensed under the [Apache License, Version 2.0](LICENSES/Apache-2.0.txt). The BPF code\nis licensed under the [General Public License, Version 2.0](LICENSES/GPL-2.0-only.txt).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopennetworkinglab%2Fint-host-reporter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopennetworkinglab%2Fint-host-reporter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopennetworkinglab%2Fint-host-reporter/lists"}