{"id":13839330,"url":"https://github.com/openshift/sriov-network-operator","last_synced_at":"2025-05-16T04:04:36.984Z","repository":{"id":37237911,"uuid":"187282661","full_name":"openshift/sriov-network-operator","owner":"openshift","description":"SR-IOV Network Operator","archived":false,"fork":false,"pushed_at":"2025-05-12T07:29:10.000Z","size":103263,"stargazers_count":132,"open_issues_count":3,"forks_count":111,"subscribers_count":19,"default_branch":"main","last_synced_at":"2025-05-12T07:38:29.655Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openshift.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-05-17T21:18:21.000Z","updated_at":"2025-05-12T06:34:27.000Z","dependencies_parsed_at":"2023-12-24T21:33:33.881Z","dependency_job_id":"7640c503-7866-4ace-80c0-12cb3ee311c9","html_url":"https://github.com/openshift/sriov-network-operator","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openshift%2Fsriov-network-operator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openshift%2Fsriov-network-operator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openshift%2Fsriov-network-operator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openshift%2Fsriov-network-operator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openshift","download_url":"https://codeload.github.com/openshift/sriov-network-operator/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254464893,"owners_count":22075570,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-04T17:00:19.265Z","updated_at":"2025-05-16T04:04:31.976Z","avatar_url":"https://github.com/openshift.png","language":"Go","readme":"# sriov-network-operator\n\nThe Sriov Network Operator is designed to help the user to provision and configure SR-IOV CNI plugin and Device plugin in the Openshift cluster.\n\n## Motivation\n\nSR-IOV network is an optional feature of an Openshift cluster. To make it work, it requires different components to be provisioned and configured accordingly. It makes sense to have one operator to coordinate those relevant components in one place, instead of having them managed by different operators. And also, to hide the complexity, we should provide an elegant user interface to simplify the process of enabling SR-IOV.\n\n## Features\n\n- Initialize the supported SR-IOV NIC types on selected nodes.\n- Provision/upgrade SR-IOV device plugin executable on selected node.  \n- Provision/upgrade SR-IOV CNI plugin executable on selected nodes.\n- Manage configuration of SR-IOV device plugin on host.\n- Generate net-att-def CRs for SR-IOV CNI plugin\n- Supports operation in a virtualized Kubernetes deployment\n  - Discovers VFs attached to the Virtual Machine (VM)\n  - Does not require attached of associated PFs\n  - VFs can be associated to SriovNetworks by selecting the appropriate PciAddress as the RootDevice in the SriovNetworkNodePolicy\n\n## Quick Start\n\nFor more detail on installing this operator, refer to the [quick-start](doc/quickstart.md) guide.\n\n## API\n\nThe SR-IOV network operator introduces following new CRDs:\n\n- SriovNetwork\n\n- OVSNetwork\n\n- SriovNetworkNodeState\n\n- SriovNetworkNodePolicy\n\n### SriovNetwork\n\nA custom resource of SriovNetwork could represent the a layer-2 broadcast domain where some SR-IOV devices are attach to. It is primarily used to generate a NetworkAttachmentDefinition CR with an SR-IOV CNI plugin configuration. \n\nThis SriovNetwork CR also contains the ‘resourceName’ which is aligned with the ‘resourceName’ of SR-IOV device plugin. One SriovNetwork obj maps to one ‘resoureName’, but one ‘resourceName’ can be shared by different SriovNetwork CRs.\n\nThis CR should be managed by cluster admin. Here is an example:\n\n```yaml\napiVersion: sriovnetwork.openshift.io/v1\nkind: SriovNetwork\nmetadata:\n  name: example-network\n  namespace: example-namespace\nspec:\n  ipam: |\n    {\n      \"type\": \"host-local\",\n      \"subnet\": \"10.56.217.0/24\",\n      \"rangeStart\": \"10.56.217.171\",\n      \"rangeEnd\": \"10.56.217.181\",\n      \"routes\": [{\n        \"dst\": \"0.0.0.0/0\"\n      }],\n      \"gateway\": \"10.56.217.1\"\n    }\n  vlan: 0\n  resourceName: intelnics\n```\n\n#### Chaining CNI metaplugins\n\nIt is possible to add additional capabilities to the device configured via the SR-IOV configuring optional metaplugins.\n\nIn order to do this, the `metaPlugins` field must contain the array of one or more additional configurations used to build a [network configuration list](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration-lists), as per the following example:\n\n```yaml\napiVersion: sriovnetwork.openshift.io/v1\nkind: SriovNetwork\nmetadata:\n  name: example-network\n  namespace: example-namespace\nspec:\n  ipam: |\n    {\n      \"type\": \"host-local\",\n      \"subnet\": \"10.56.217.0/24\",\n      \"rangeStart\": \"10.56.217.171\",\n      \"rangeEnd\": \"10.56.217.181\",\n      \"routes\": [{\n        \"dst\": \"0.0.0.0/0\"\n      }],\n      \"gateway\": \"10.56.217.1\"\n    }\n  vlan: 0\n  resourceName: intelnics\n  metaPlugins : |\n    {\n      \"type\": \"tuning\",\n      \"sysctl\": {\n        \"net.core.somaxconn\": \"500\"\n      }\n    },\n    {\n      \"type\": \"vrf\",\n      \"vrfname\": \"red\"\n    }\n```\n\n### OVSNetwork\n\nA custom resource of OVSNetwork could represent the a layer-2 broadcast domain attached to Open vSwitch that works in HW-offloading mode. \nIt is primarily used to generate a NetworkAttachmentDefinition CR with an OVS CNI plugin configuration. \n\nThe OVSNetwork CR also contains the `resourceName` which is aligned with the `resourceName` of SR-IOV device plugin. One OVSNetwork obj maps to one `resourceName`, but one `resourceName` can be shared by different OVSNetwork CRs.\n\nIt is expected that `resourceName` contains name of the resource pool which holds Virtual Functions of a NIC in the switchdev mode. \nA Physical function of the NIC should be attached to an OVS bridge before any workload which uses OVSNetwork starts.\n\nExample:\n\n```yaml\napiVersion: sriovnetwork.openshift.io/v1\nkind: OVSNetwork\nmetadata:\n  name: example-network\n  namespace: example-namespace\nspec:\n  ipam: |\n    {\n      \"type\": \"host-local\",\n      \"subnet\": \"10.56.217.0/24\",\n      \"rangeStart\": \"10.56.217.171\",\n      \"rangeEnd\": \"10.56.217.181\",\n      \"routes\": [{\n        \"dst\": \"0.0.0.0/0\"\n      }],\n      \"gateway\": \"10.56.217.1\"\n    }\n  vlan: 100\n  bridge: my-bridge\n  mtu: 2500\n  resourceName: switchdevnics\n```\n\n### SriovNetworkNodeState\n\nThe custom resource to represent the SR-IOV interface states of each host, which should only be managed by the operator itself.\n\n- The ‘spec’ of this CR represents the desired configuration which should be applied to the interfaces and SR-IOV device plugin.\n- The ‘status’ contains current states of those PFs (baremetal only), and the states of the VFs. It helps user to discover SR-IOV network hardware on node, or attached VFs in the case of a virtual deployment.\n\nThe spec is rendered by sriov-policy-controller, and consumed by sriov-config-daemon. Sriov-config-daemon is responsible for updating the ‘status’ field to reflect the latest status, this information can be used as input to create SriovNetworkNodePolicy CR.\n\nAn example of SriovNetworkNodeState CR:\n\n```yaml\napiVersion: sriovnetwork.openshift.io/v1\nkind: SriovNetworkNodeState\nmetadata:\n  name: worker-node-1\n  namespace: sriov-network-operator\nspec:\n  interfaces:\n  - deviceType: vfio-pci\n    mtu: 1500\n    numVfs: 4\n    pciAddress: 0000:86:00.0\nstatus:\n  interfaces:\n  - deviceID: \"1583\"\n    driver: i40e\n    mtu: 1500\n    numVfs: 4\n    pciAddress: 0000:86:00.0\n    maxVfs: 64\n    vendor: \"8086\"\n    Vfs:\n      - deviceID: 154c\n      driver: vfio-pci\n      pciAddress: 0000:86:02.0\n      vendor: \"8086\"\n      - deviceID: 154c\n      driver: vfio-pci\n      pciAddress: 0000:86:02.1\n      vendor: \"8086\"\n      - deviceID: 154c\n      driver: vfio-pci\n      pciAddress: 0000:86:02.2\n      vendor: \"8086\"\n      - deviceID: 154c\n      driver: vfio-pci\n      pciAddress: 0000:86:02.3\n      vendor: \"8086\"\n  - deviceID: \"1583\"\n    driver: i40e\n    mtu: 1500\n    pciAddress: 0000:86:00.1\n    maxVfs: 64\n    vendor: \"8086\"\n```\n\nFrom this example, in status field, the user can find out there are 2 SRIOV capable NICs on node 'work-node-1'; in spec field, user can learn what the expected configure is generated from the combination of SriovNetworkNodePolicy CRs.  In the virtual deployment case, a single VF will be associated with each device.\n\n### SriovNetworkNodePolicy\n\nThis CRD is the key of SR-IOV network operator. This custom resource should be managed by cluster admin, to instruct the operator to:\n\n1. Render the spec of SriovNetworkNodeState CR for selected node, to configure the SR-IOV interfaces.  In virtual deployment, the VF interface is read-only.\n2. Deploy SR-IOV CNI plugin and device plugin on selected node.\n3. Generate the configuration of SR-IOV device plugin.\n\nAn example of SriovNetworkNodePolicy CR:\n\n```yaml\napiVersion: sriovnetwork.openshift.io/v1\nkind: SriovNetworkNodePolicy\nmetadata:\n  name: policy-1\n  namespace: sriov-network-operator\nspec:\n  deviceType: vfio-pci\n  mtu: 1500\n  nicSelector:\n    deviceID: \"1583\"\n    rootDevices:\n    - 0000:86:00.0\n    vendor: \"8086\"\n  nodeSelector:\n    feature.node.kubernetes.io/network-sriov.capable: \"true\"\n  numVfs: 4\n  priority: 90\n  resourceName: intelnics\n```\n\nIn this example, user selected the nic from vendor '8086' which is intel, device module is '1583' which is XL710 for 40GbE, on nodes labeled with 'network-sriov.capable' equals 'true'. Then for those PFs, create 4 VFs each, set mtu to 1500 and the load the vfio-pci driver to those virtual functions.  \n\nIn a virtual deployment: \n- The mtu of the PF is set by the underlying virtualization platform and cannot be changed by the sriov-network-operator.\n- The numVfs parameter has no effect as there is always 1 VF\n- The deviceType field depends upon whether the underlying device/driver is [native-bifurcating or non-bifurcating](https://doc.dpdk.org/guides/howto/flow_bifurcation.html) For example, the supported Mellanox devices support native-bifurcating drivers and therefore deviceType should be netdevice (default).  The support Intel devices are non-bifurcating and should be set to vfio-pci.\n\n#### Multiple policies\n\nWhen multiple SriovNetworkNodeConfigPolicy CRs are present, the `priority` field\n(0 is the highest priority) is used to resolve any conflicts. Conflicts occur\nonly when same PF is referenced by multiple policies. The final desired\nconfiguration is saved in `SriovNetworkNodeState.spec.interfaces`.\n\nPolicies processing order is based on priority (lowest first), followed by `name`\nfield (starting from `a`). Policies with same **priority** or **non-overlapping\nVF groups** (when #-notation is used in pfName field) are merged, otherwise only\nthe highest priority policy is applied. In case of same-priority policies and\noverlapping VF groups, only the last processed policy is applied.\n\nWhen using #-notation to define VF group, no actions are taken on virtual functions that\nare not mentioned in any policy (e.g. if a policy defines a `vfio-pci` device group for a device, when \nit is deleted the VF are not reset to the default driver).\n\n#### Externally Manage virtual functions\n\nWhen `ExternallyManage` is request on a policy the operator will only skip the virtual function creation.\nThe operator will only bind the virtual functions to the requested driver and expose them via the device plugin.\nAnother difference when this field is requested in the policy is that when this policy is removed the operator\nwill not remove the virtual functions from the policy.\n\n*Note:* This means the user must create the virtual functions before they apply the policy or the webhook will reject\nthe policy creation.\n\nIt's possible to use something like nmstate kubernetes-nmstate or just a simple systemd file to create\nthe virtual functions on boot.\n\nThis feature was created to support deployments where the user want to use some of the virtual funtions for the host\ncommunication like storage network or out of band managment and the virtual functions must exist on boot and not only\nafter the operator and config-daemon are running.\n\n#### Disabling SR-IOV Config Daemon plugins\n\nIt is possible to disable SR-IOV network operator config daemon plugins in case their operation\nis not needed or un-desirable.\n\nAs an example, some plugins perform vendor specific firmware configuration\nto enable SR-IOV (e.g `mellanox` plugin). certain deployment environments may prefer to perform such configuration\nonce during node provisioning, while ensuring the configuration will be compatible with any sriov network node policy\ndefined for the particular environment. This will reduce or completely eliminate the need for reboot of nodes during SR-IOV\nconfigurations by the operator.\n\nThis can be done by setting SriovOperatorConfig `default` CR `spec.disablePlugins` with the list of desired plugins\nto disable.\n\n**Example**:\n\n```yaml\napiVersion: sriovnetwork.openshift.io/v1\nkind: SriovOperatorConfig\nmetadata:\n  name: default\n  namespace: sriov-network-operator\nspec:\n  ...\n  disablePlugins:\n    - mellanox\n  ...\n```\n\n\u003e **NOTE**: Currently only `mellanox` plugin can be disabled.\n\n### Parallel draining\n\nIt is possible to drain more than one node at a time using this operator.\n\nThe configuration is done via the SriovNetworkNodePool, selecting a number of nodes using the node selector and how many\nnodes in parallel from the pool the operator can drain in parallel. maxUnavailable can be a number or a percentage.\n\n\u003e **NOTE**: every node can only be part of one pool, if a node is selected by more than one pool, then it will not be drained\n\n\u003e **NOTE**: If a node is not part of any pool it will have a default configuration of maxUnavailable 1\n\n**Example**:\n\n```yaml\napiVersion: sriovnetwork.openshift.io/v1\nkind: SriovNetworkPoolConfig\nmetadata:\n  name: worker\n  namespace: sriov-network-operator\nspec:\n  maxUnavailable: 2\n  nodeSelector:\n    matchLabels:\n      node-role.kubernetes.io/worker: \"\"\n```\n\n## Feature Gates\n\nFeature gates are used to enable or disable specific features in the operator.\n\n\u003e **NOTE**: As features mature and graduate to stable status, default settings may change, and feature gates might be removed in future releases. Keep this in mind when configuring feature gates and ensure your environment is compatible with any updates.\n\n### Available Feature Gates\n\n1. **Parallel NIC Configuration** (`parallelNicConfig`)\n  - **Description:** Allows the configuration of NICs in parallel, which can potentially reduce the time required for network setup.\n  - **Default:** Disabled\n\n2. **Resource Injector Match Condition** (`resourceInjectorMatchCondition`)\n  - **Description:** Switches the resource injector's webhook failure policy from \"Ignore\" to \"Fail\" by utilizing the `MatchConditions` feature introduced in Kubernetes 1.28. This ensures the webhook only targets pods with the `k8s.v1.cni.cncf.io/networks` annotation, improving reliability without affecting other pods.\n  - **Default:** Disabled\n\n3. **Metrics Exporter** (`metricsExporter`)\n  - **Description:** Enables the metrics exporter on the same node where the config-daemon is running. This helps in collecting and exporting metrics related to SR-IOV network devices.\n  - **Default:** Disabled\n\n4. **Manage Software Bridges** (`manageSoftwareBridges`)\n  - **Description:** Allows the operator to manage software bridges. This feature gate is useful for environments where bridge management is required.\n  - **Default:** Disabled\n\n5. **Mellanox Firmware Reset** (`mellanoxFirmwareReset`)\n  - **Description:** Enables the firmware reset via `mstfwreset` before a system reboot. This feature is specific to Mellanox network devices and is used to ensure that the firmware is properly reset during system maintenance.\n  - **Default:** Disabled\n\n### Enabling Feature Gates\n\nTo enable a feature gate, add it to your configuration file or command line with the desired state. For example, to enable the `resourceInjectorMatchCondition` feature gate, you would specify:\n\n```yaml\napiVersion: sriovnetwork.openshift.io/v1\nkind: SriovOperatorConfig\nmetadata:\n  name: default\n  namespace: sriov-network-operator\nspec:\n  featureGates:\n    resourceInjectorMatchCondition: true\n  ...\n```\n\n## Components and design\n\nThis operator is split into 2 components:\n\n- controller\n- sriov-config-daemon\n\nThe controller is responsible for:\n\n1. Read the SriovNetworkNodePolicy CRs and SriovNetwork CRs as input.\n2. Render the manifests for SR-IOV CNI plugin and device plugin daemons.\n3. Render the spec of SriovNetworkNodeState CR for each node.\n\nThe sriov-config-daemon is responsible for:\n\n1. Discover the SRIOV NICs on each node, then sync the status of SriovNetworkNodeState CR.\n2. Take the spec of SriovNetworkNodeState CR as input to configure those NICs.\n\n## Workflow\n\n![SRIOV Network Operator work flow](doc/images/workflow.png)\n","funding_links":[],"categories":["NetWork"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenshift%2Fsriov-network-operator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenshift%2Fsriov-network-operator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenshift%2Fsriov-network-operator/lists"}