{"id":22499735,"url":"https://github.com/opiproject/opi-intel-bridge","last_synced_at":"2025-07-04T08:04:48.840Z","repository":{"id":79220506,"uuid":"605564083","full_name":"opiproject/opi-intel-bridge","owner":"opiproject","description":"OPI gRPC to Intel bridge third party repo","archived":false,"fork":false,"pushed_at":"2025-03-17T08:12:34.000Z","size":697,"stargazers_count":7,"open_issues_count":11,"forks_count":8,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-09T05:41:43.547Z","etag":null,"topics":["bridge","dpu","golang","intel","inventory","ipsec","opi","spdk","storage"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/opiproject.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-23T12:30:23.000Z","updated_at":"2025-03-17T08:12:39.000Z","dependencies_parsed_at":null,"dependency_job_id":"e73e3b20-46b0-457a-8532-99a3cac1eaee","html_url":"https://github.com/opiproject/opi-intel-bridge","commit_stats":{"total_commits":572,"total_committers":8,"mean_commits":71.5,"dds":"0.36888111888111885","last_synced_commit":"6380e13ff74239f5a68eab3f481dec591b33280d"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/opiproject/opi-intel-bridge","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opiproject%2Fopi-intel-bridge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opiproject%2Fopi-intel-bridge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opiproject%2Fopi-intel-bridge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opiproject%2Fopi-intel-bridge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/opiproject","download_url":"https://codeload.github.com/opiproject/opi-intel-bridge/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opiproject%2Fopi-intel-bridge/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263472279,"owners_count":23471812,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bridge","dpu","golang","intel","inventory","ipsec","opi","spdk","storage"],"created_at":"2024-12-06T22:15:36.296Z","updated_at":"2025-07-04T08:04:48.812Z","avatar_url":"https://github.com/opiproject.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# OPI gRPC to Intel SDK bridge\n\n[![Linters](https://github.com/opiproject/opi-intel-bridge/actions/workflows/linters.yml/badge.svg)](https://github.com/opiproject/opi-intel-bridge/actions/workflows/linters.yml)\n[![CodeQL](https://github.com/opiproject/opi-intel-bridge/actions/workflows/codeql.yml/badge.svg)](https://github.com/opiproject/opi-intel-bridge/actions/workflows/codeql.yml)\n[![tests](https://github.com/opiproject/opi-intel-bridge/actions/workflows/go.yml/badge.svg)](https://github.com/opiproject/opi-intel-bridge/actions/workflows/go.yml)\n[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/7999/badge)](https://www.bestpractices.dev/projects/7999)\\\n[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/opiproject/opi-intel-bridge/badge)](https://securityscorecards.dev/viewer/?platform=github.com\u0026org=opiproject\u0026repo=opi-intel-bridge)\n[![codecov](https://codecov.io/gh/opiproject/opi-intel-bridge/branch/main/graph/badge.svg)](https://codecov.io/gh/opiproject/opi-intel-bridge)\n[![Go Report Card](https://goreportcard.com/badge/github.com/opiproject/opi-intel-bridge)](https://goreportcard.com/report/github.com/opiproject/opi-intel-bridge)\n\n[Download 🚀](https://github.com/orgs/opiproject/packages?repo_name=opi-intel-bridge) ·\n[Report issue 🐞](https://github.com/opiproject/opi-intel-bridge/issues/new/choose) ·\n[Contribute 👋](#i-want-to-contribute)\n\n----\n\nThis is an Intel bridge to OPI APIs. Currently it supports storage APIs and is subject to extension for other domains including inventory, ipsec and networking.\nThe intel-opi-bridge (further bridge) acts as a gRPC server for xPU management and configuration. See [Intel xPU (IPU)](https://www.intel.com/content/www/us/en/products/details/network-io/ipu.html) for more details about IPU.\n\nThe diagram below illustrates main system components of an exemplary xPU NVMe-oF initiator deployment. The bridge (in blue) runs on an xPU and translates OPI API commands to appropriate sequences of Intel SDK instructions. Ultimately, two emulated NVMe/PCIe storage devices are exposed to the host. These devices are backed by an \"over Fabrics\"-connection to some remote storage backends while the host has an illusion of accessing locally attached storage and can run standard and unmodified apps/drivers to access it. Storage-related services and computations are offloaded from the host and accelerated on the xPU itself.\n\n![opi-intel-bridge system overview initiator](doc/images/opi-intel-bridge_system-overview-initiator.png \"opi-intel-bridge system overview initiator\")\\\n*Fig. 1 - System components in NVMe-oF initiator scenario.*\n\nNext to the initiator mode, the xPU can also be run in a target mode and serve as an acceleration and virtualization layer for locally attached SSD drives (JBOF usage model). Note that in this case xPU is not required to be plugged into a physical server and runs headless as a low-power, low-cost storage appliance. Customized storage services can be run on the xPU as well to realize arbitrary storage operations.\nIn the diagram below a standard NVMe/TCP driver is used on a host to access physical storage which is exposed by the xPU over network via two NVMe namespaces.\n\n![opi-intel-bridge system overview target](doc/images/opi-intel-bridge_system-overview-target.png \"opi-intel-bridge system overview target\")\\\n*Fig. 2 - System components in NVMe-oF target scenario.*\n\nCombining both above usage scenarios results in xPU acting as both initiator and target to accelerate NVMe-oF communication on both ends of a network connection - see the diagram below. Hyperconverged use cases are possible if two xPUs are plugged into the same physical server\n\n![opi-intel-bridge system overview initiator and target 1](doc/images/opi-intel-bridge_system-overview-initiator-and-target.png \"opi-intel-bridge system overview initiator and target 1\")\\\n*Fig. 3 - System components in a combined NVMe-oF initiator and target scenario with two independent xPUs.*\n\nor if an xPU is run in initiator and target mode simultaneously.\n\n![opi-intel-bridge system overview initiator and target 2](doc/images/opi-intel-bridge_system-overview-initiator-and-target-simultaneously.png \"opi-intel-bridge system overview initiator and target 2\")\\\n*Fig. 4 - System components in a combined NVMe-oF initiator and target scenario with one xPU running in both modes simultaneously.*\n\n## Quickstart\n\nThis section outlines the basic steps to get you up-and-running with the bridge and shows some examples of its usage to expose storage devices to the host, set bandwidth/rate limiters on them or enable data-at-rest crypto. The steps are mainly executed as gRPC commands to the bridge but may also involve some host-side interactions or initial xPU-side setup.\n\n\u003e **Note** \\\nIt is assumed that Intel IPU is already properly set up to be used with Intel OPI bridge.\n\nThe following variables are used throughout this document:\n\n| Variable         | Description                                                                                                                                        |\n| ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |\n| BRIDGE_IP        | opi-intel-bridge gRPC/HTTP listening IP address e.g. 10.10.10.10 or localhost                                                                      |\n| BRIDGE_PORT      | opi-intel-bridge gRPC listening port e.g. 50051                                                                                                    |\n| BRIDGE_ADDR      | BRIDGE_IP:BRIDGE_PORT                                                                                                                              |\n| BRIDGE_HTTP_PORT | opi-intel-bridge http gateway port e.g. 8082                                                                                                       |\n| NVME_PF_BDF      | physical function PCI address e.g. 0000:3b:00.1 for Nvme                                                                                           |\n| NVME_VF_BDF      | virtual function PCI address e.g. 0000:40:00.0 can be found in pf's virtfn\\\u003cX\\\u003e where X equals to virtual_function in CreateNvmeController minus 1 |\n| BLK_PF_BDF       | physical function PCI address e.g. 0000:af:01.0 for virtio-blk                                                                                     |\n| TARGET_IP        | storage target ip address                                                                                                                          |\n| TARGET_PORT      | storage target port                                                                                                                                |\n| JAEGER_PORT      | Jaeger port                                                                                                                                        |\n| REDIS_PORT       | Redis port                                                                                                                                         |\n\n### Build and import\n\nTo build the solution execute\n\n```bash\ngo build -v -o /opi-intel-bridge-storage ./cmd/storage\n```\n\nTo import the bridge within another go package or module use\n\n```go\nimport \"github.com/opiproject/opi-intel-bridge-storage/pkg/frontend\"\n```\n\n### Usage\n\nBefore initiating the bridge, the [Redis](https://redis.io/) and [Jaeger](https://www.jaegertracing.io/) services must be operational. To specify non-standard ports for these services, use the `--help` command with the binary to find out which parameters needs to be passed.\n\nAccording to this guideline, the Docker should be installed and appropriately configured in compliance with your organization's policies, such as the utilization of a Proxy, to ensure its proper functioning.\n\nBring up SSA app according to the steps described in IPU SDK documentation.\n\nMake sure `/var/tmp/spdk.sock` is created.\n\nOn xPU run\n\n```bash\n$ docker run --rm -it -v /var/tmp/:/var/tmp/ -p $BRIDGE_PORT:$BRIDGE_PORT -p $BRIDGE_HTTP_PORT:$BRIDGE_HTTP_PORT -p $JAEGER_PORT:$JAEGER_PORT -p $REDIS_PORT:$REDIS_PORT ghcr.io/opiproject/opi-intel-bridge:main /opi-intel-bridge-storage -grpc_port=$BRIDGE_PORT -http_port=$BRIDGE_HTTP_PORT\n\n2023/09/12 20:29:05 TLS files are not specified. Use insecure connection.\n2023/09/12 20:29:05 Connection to SPDK will be via: unix detected from /var/tmp/spdk.sock\n2023/09/12 20:29:05 gRPC server listening at [::]:50051\n2023/09/12 20:29:05 HTTP Server listening at 8082\n```\n\nTo send commands to the bridge, grpc_cli tool is used. It can be used as a containerized or a native version. If containerized version is preferable, then an alias can be defined as follows\n\n```bash\nalias grpc_cli=\"docker run --network=host --rm -it namely/grpc-cli\"\n```\n\nOn management machine run below command to check bridge availability and reflection capabilities\n\n```bash\n$ grpc_cli ls --json_input --json_output $BRIDGE_ADDR\n\ngrpc.reflection.v1alpha.ServerReflection\nopi_api.inventory.v1.InventorySvc\nopi_api.security.v1.IPsec\nopi_api.storage.v1.AioControllerService\nopi_api.storage.v1.FrontendNvmeService\nopi_api.storage.v1.FrontendVirtioBlkService\nopi_api.storage.v1.FrontendVirtioScsiService\nopi_api.storage.v1.MiddleendEncryptionService\nopi_api.storage.v1.MiddleendQosVolumeService\nopi_api.storage.v1.NvmeRemoteControllerService\nopi_api.storage.v1.NullDebugService\n```\n\nrun either gRPC or HTTP requests\n\n```bash\n# gRPC requests\n# Nvme PF creation\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR CreateNvmeSubsystem \"{nvme_subsystem : {spec : {nqn: 'nqn.2022-09.io.spdk:opitest2', serial_number: 'myserial2', model_number: 'mymodel2', max_namespaces: 11} }, nvme_subsystem_id : 'subsystem2' }\"\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR ListNvmeSubsystems \"{}\"\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR GetNvmeSubsystem \"{name : 'nvmeSubsystems/subsystem2'}\"\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR CreateNvmeController \"{parent: 'nvmeSubsystems/subsystem2', nvme_controller : {spec : {nvme_controller_id: 2, pcie_id : {physical_function : 0, virtual_function : 0, port_id: 0}, max_nsq:5, max_ncq:5, 'trtype': 'NVME_TRANSPORT_TYPE_PCIE' } }, nvme_controller_id : 'controller1'}\"\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR ListNvmeControllers \"{parent : 'nvmeSubsystems/subsystem2'}\"\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR GetNvmeController \"{name : 'nvmeSubsystems/subsystem2/nvmeControllers/controller1'}\"\n\n# Nvme VF creation on PF0\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR CreateNvmeSubsystem \"{nvme_subsystem : {spec : {nqn: 'nqn.2022-09.io.spdk:opitest3', serial_number: 'mev-opi-serial', model_number: 'mev-opi-model', max_namespaces: 11} }, nvme_subsystem_id : 'subsystem03' }\"\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR CreateNvmeController \"{parent: 'nvmeSubsystems/subsystem03', nvme_controller : {spec : {nvme_controller_id: 2, pcie_id : {physical_function : 0, virtual_function : 3, port_id: 0 }, max_nsq:5, max_ncq:5, 'trtype': 'NVME_TRANSPORT_TYPE_PCIE' } }, nvme_controller_id : 'controller3'}\"\n\n# Create Nvme/TCP controller\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR CreateNvmeSubsystem \"{nvme_subsystem : {spec : {nqn: 'nqn.2022-09.io.spdk:opitest4', serial_number: 'myserial2', model_number: 'mymodel2', max_namespaces: 11} }, nvme_subsystem_id : 'subsystem4' }\"\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR CreateNvmeController \"{'parent':'nvmeSubsystems/subsystem4','nvme_controller':{'spec':{'nvme_controller_id':2,'fabrics_id':{'traddr': '127.0.0.1', trsvcid: '4421', adrfam: 'NVME_ADDRESS_FAMILY_IPV4'}, 'max_nsq':5,'max_ncq':5, 'trtype': 'NVME_TRANSPORT_TYPE_TCP'}},'nvme_controller_id':'controller4'}\"\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR GetNvmeController \"{name : 'nvmeSubsystems/subsystem4/nvmeControllers/controller4'}\"\n\n# Connect to remote storage-target\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR CreateNvmeRemoteController \"{nvme_remote_controller : {multipath: 'NVME_MULTIPATH_MULTIPATH'}, nvme_remote_controller_id: 'nvmetcp12'}\"\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR ListNvmeRemoteControllers \"{}\"\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR GetNvmeRemoteController \"{name: 'nvmeRemoteControllers/nvmetcp12'}\"\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR CreateNvmePath \"{parent: 'nvmeRemoteControllers/nvmetcp12', nvme_path : {traddr:'11.11.11.2', trtype:'NVME_TRANSPORT_TYPE_TCP', fabrics:{subnqn:'nqn.2016-06.com.opi.spdk:target0', trsvcid:'4444', adrfam:'NVME_ADDRESS_FAMILY_IPV4', hostnqn:'nqn.2014-08.org.nvmexpress:uuid:feb98abe-d51f-40c8-b348-2753f3571d3c'}}, nvme_path_id: 'nvmetcp12path0'}\"\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR ListNvmePaths \"{parent : 'nvmeRemoteControllers/nvmetcp12'}\"\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR GetNvmePath \"{name: 'nvmeRemoteControllers/nvmetcp12/nvmePaths/nvmetcp12path0'}\"\n\n# Connect to local PCIe storage-target\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR CreateNvmeRemoteController \"{nvme_remote_controller : {multipath: 'NVME_MULTIPATH_DISABLE'}, nvme_remote_controller_id: 'nvmepcie13'}\"\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR CreateNvmePath \"{parent: 'nvmeRemoteControllers/nvmepcie13', nvme_path : {traddr:'0000:01:00.0', trtype:'NVME_TRANSPORT_TYPE_PCIE'}, nvme_path_id: 'nvmepcie13path0'}\"\n\n# Virtio-blk PF creation (virtio-blk requires a volume, that's why it is created after connection to storage-target)\ngrpc_cli --json_input --json_output call $BRIDGE_ADDR CreateVirtioBlk \"{virtio_blk_id: 'virtioblk0', virtio_blk : { volume_name_ref: 'nvmetcp12n0', pcie_id: { physical_function: '0', virtual_function: '0', port_id: '0'}}}\"\n\n# Create QoS volume\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR CreateQosVolume \"{'qos_volume' : {'volume_name_ref' :'nvmetcp12n1', 'limits' : { 'max': {'rw_iops_kiops': 3} } }, 'qos_volume_id' : 'qosnvmetcp12n1' }\"\n\n# Create encrypted volume\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR CreateEncryptedVolume \"{'encrypted_volume': { 'cipher': 'ENCRYPTION_TYPE_AES_XTS_128', 'volume_name_ref': 'nvmetcp12n1', 'key': 'MDAwMTAyMDMwNDA1MDYwNzA4MDkwYTBiMGMwZDBlMGY='}, 'encrypted_volume_id': 'encnvmetcp12n1' }\"\n\n# Create namespace\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR CreateNvmeNamespace \"{parent: 'nvmeSubsystems/subsystem2', nvme_namespace : {spec : {volume_name_ref : 'nvmetcp12n1', 'host_nsid' : '10', uuid:'1b4e28ba-2fa1-11d2-883f-b9a761bde3fb', nguid: '1b4e28ba-2fa1-11d2-883f-b9a761bde3fb', eui64: 1967554867335598546 } }, nvme_namespace_id: 'namespace1'}\"\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR ListNvmeNamespaces \"{parent : 'nvmeSubsystems/subsystem2'}\"\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR GetNvmeNamespace \"{name : 'nvmeSubsystems/subsystem2/nvmeNamespaces/namespace1'}\"\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR StatsNvmeNamespace \"{name : 'nvmeSubsystems/subsystem2/nvmeNamespaces/namespace1'}\"\n\n# Delete namespace\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR DeleteNvmeNamespace \"{name : 'nvmeSubsystems/subsystem2/nvmeNamespaces/namespace1'}\"\n\n# Delete encrypted volume\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR DeleteEncryptedVolume \"{'name': 'volumes/encnvmetcp12n1'}\"\n\n# Delete QoS volume\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR DeleteQosVolume \"{name : 'volumes/qosnvmetcp12n1'}\"\n\n# Delete virtio-blk PF\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR DeleteVirtioBlk \"{name: 'volumes/virtioblk0'}\"\n\n# Disconnect from local PCIe storage-target\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR DeleteNvmePath \"{name: 'nvmeRemoteControllers/nvmepcie13/nvmePaths/nvmepcie13path0'}\"\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR DeleteNvmeRemoteController \"{name: 'nvmeRemoteControllers/nvmepcie13'}\"\n\n# Disconnect from remote storage-target\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR DeleteNvmePath \"{name: 'nvmeRemoteControllers/nvmetcp12/nvmePaths/nvmetcp12path0'}\"\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR DeleteNvmeRemoteController \"{name: 'nvmeRemoteControllers/nvmetcp12'}\"\n\n# Delete Nvme VF\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR DeleteNvmeController \"{name : 'nvmeSubsystems/subsystem03/nvmeControllers/controller3'}\"\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR DeleteNvmeSubsystem \"{name : 'nvmeSubsystems/subsystem03'}\"\n\n# Delete Nvme PF\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR DeleteNvmeController \"{name : 'nvmeSubsystems/subsystem2/nvmeControllers/controller1'}\"\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR DeleteNvmeSubsystem \"{name : 'nvmeSubsystems/subsystem2'}\"\n\n# Delete Nvme/TCP\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR DeleteNvmeController \"{name : 'nvmeSubsystems/subsystem4/nvmeControllers/controller4'}\"\ngrpc_cli call --json_input --json_output $BRIDGE_ADDR DeleteNvmeSubsystem \"{name : 'nvmeSubsystems/subsystem4'}\"\n```\n\n```bash\n# HTTP requests\n# inventory\ncurl -kL http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/inventory/1/inventory/2\n\n# Nvme\n# create\ncurl -X POST -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeRemoteControllers?nvme_remote_controller_id=nvmetcp12 -d '{\"multipath\": \"NVME_MULTIPATH_MULTIPATH\"}'\ncurl -X POST -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeRemoteControllers/nvmetcp12/nvmePaths?nvme_path_id=nvmetcp12path0 -d \"{\\\"traddr\\\":\\\"$TARGET_IP\\\", \\\"trtype\\\":\\\"NVME_TRANSPORT_TYPE_TCP\\\", \\\"fabrics\\\":{\\\"subnqn\\\":\\\"nqn.2016-06.com.opi.spdk:target0\\\", \\\"trsvcid\\\":\\\"$TARGET_PORT\\\", \\\"adrfam\\\":\\\"NVME_ADDRESS_FAMILY_IPV4\\\", \\\"hostnqn\\\":\\\"nqn.2014-08.org.nvmexpress:uuid:feb98abe-d51f-40c8-b348-2753f3571d3c\\\"}}\"\ncurl -X POST -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeSubsystems?nvme_subsystem_id=subsys0 -d '{\"spec\": {\"nqn\": \"nqn.2022-09.io.spdk:opitest1\"}}'\ncurl -X POST -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeSubsystems/subsys0/nvmeNamespaces?nvme_namespace_id=namespace0 -d '{\"spec\": {\"volume_name_ref\": \"Malloc0\", \"host_nsid\": 10}}'\ncurl -X POST -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeSubsystems/subsys0/nvmeControllers?nvme_controller_id=ctrl0 -d '{\"spec\": {\"trtype\": \"NVME_TRANSPORT_TYPE_TCP\", \"fabrics_id\":{\"traddr\": \"127.0.0.1\", \"trsvcid\": \"4421\", \"adrfam\": \"NVME_ADDRESS_FAMILY_IPV4\"}}}'\n# get\ncurl -X GET -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeRemoteControllers/nvmetcp12\ncurl -X GET -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeRemoteControllers/nvmetcp12/nvmePaths/nvmetcp12path0\ncurl -X GET -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeSubsystems/subsys0\ncurl -X GET -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeSubsystems/subsys0/nvmeNamespaces/namespace0\ncurl -X GET -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeSubsystems/subsys0/nvmeControllers/ctrl0\n# list\ncurl -X GET -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeRemoteControllers\ncurl -X GET -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeRemoteControllers/nvmetcp12/nvmePaths\ncurl -X GET -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeSubsystems\ncurl -X GET -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeSubsystems/subsys0/nvmeNamespaces\ncurl -X GET -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeSubsystems/subsys0/nvmeControllers\n# stats\ncurl -X GET -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeRemoteControllers/nvmetcp12:stats\ncurl -X GET -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeRemoteControllers/nvmetcp12/nvmePaths/nvmetcp12path0:stats\ncurl -X GET -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeSubsystems/subsys0:stats\ncurl -X GET -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeSubsystems/subsys0/nvmeNamespaces/namespace0:stats\ncurl -X GET -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeSubsystems/subsys0/nvmeControllers/ctrl0:stats\n# update\ncurl -X PATCH -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeRemoteControllers/nvmetcp12 -d '{\"multipath\": \"NVME_MULTIPATH_MULTIPATH\"}'\ncurl -X PATCH -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeRemoteControllers/nvmetcp12/nvmePaths/nvmetcp12path0 -d \"{\\\"traddr\\\":\\\"$TARGET_ADDR\\\", \\\"trtype\\\":\\\"NVME_TRANSPORT_TYPE_TCP\\\", \\\"fabrics\\\":{\\\"subnqn\\\":\\\"nqn.2016-06.com.opi.spdk:target0\\\", \\\"trsvcid\\\":\\\"$TARGET_PORT\\\", \\\"adrfam\\\":\\\"NVME_ADDRESS_FAMILY_IPV4\\\", \\\"hostnqn\\\":\\\"nqn.2014-08.org.nvmexpress:uuid:feb98abe-d51f-40c8-b348-2753f3571d3c\\\"}}\"\ncurl -X PATCH -k http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeSubsystems/subsys0/nvmeNamespaces/namespace0 -d '{\"spec\": {\"volume_name_ref\": \"Malloc1\", \"host_nsid\": 10}}'\ncurl -X PATCH -k http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeSubsystems/subsys0/nvmeControllers/ctrl0 -d '{\"spec\": {\"trtype\": \"NVME_TRANSPORT_TYPE_TCP\", \"fabrics_id\":{\"traddr\": \"127.0.0.1\", \"trsvcid\": \"4421\", \"adrfam\": \"NVME_ADDRESS_FAMILY_IPV4\"}}}'\n# delete\ncurl -X DELETE -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeSubsystems/subsys0/nvmeControllers/ctrl0\ncurl -X DELETE -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeSubsystems/subsys0/nvmeNamespaces/namespace0\ncurl -X DELETE -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeSubsystems/subsys0\ncurl -X DELETE -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeRemoteControllers/nvmetcp12/nvmePaths/nvmetcp12path0\ncurl -X DELETE -f http://$BRIDGE_IP:$BRIDGE_HTTP_PORT/v1/nvmeRemoteControllers/nvmetcp12\n```\n\nTo observe Nvme devices on host:\n\nAfter PF is created\n\n```bash\n# Bind driver to PF\nmodprobe nvme\ncd /sys/bus/pci/devices/$NVME_PF_BDF\necho 'nvme' \u003e ./driver_override\necho $NVME_PF_BDF \u003e /sys/bus/pci/drivers/nvme/bind\n\n# Allocate resources and prepare for VF creation\necho 0 \u003e ./sriov_drivers_autoprobe\necho 4 \u003e ./sriov_numvfs\n```\n\nAfter VF is created\n\n```bash\ncd /sys/bus/pci/devices/$NVME_PF_BDF\necho 'nvme' \u003e ./virtfn0/driver_override\necho $NVME_VF_BDF \u003e /sys/bus/pci/drivers/nvme/bind\n```\n\nBefore VF is deleted\n\n```bash\ncd /sys/bus/pci/devices/$NVME_PF_BDF\necho $NVME_VF_BDF \u003e /sys/bus/pci/drivers/nvme/unbind\necho '(null)' \u003e ./virtfn2/driver_override\n```\n\nBefore PF is deleted\n\n```bash\ncd /sys/bus/pci/devices/$NVME_PF_BDF\necho $NVME_PF_BDF \u003e /sys/bus/pci/drivers/nvme/unbind\necho '(null)' \u003e ./driver_override\n```\n\nTo observe Virtio-blk devices on host:\n\nAfter PF is created\n\n```bash\necho 1 \u003e /sys/bus/pci/devices/$BLK_PF_BDF/remove\necho 1 \u003e /sys/bus/pci/rescan\n```\n\n### Mutual TLS setup\n\nIn order to pass configuration data to the xPU securely and only by authenticated/allowed clients it is recommended to secure the gRPC port with mutual TLS.\n\n\u003e **Warning** \\\nThe steps outlined below use self-signed certificates and serve only demonstration purposes. It is up to the integrator of the solution to securely provision the keys and certificates to the server and follow up-to-date crypto recommendations, e.g. [NIST TLS Guidelines](https://csrc.nist.gov/publications/detail/sp/800-52/rev-2/final) and [NIST Cryptographic Standards and Guidelines](https://csrc.nist.gov/Projects/Cryptographic-Standards-and-Guidelines).\n\nThe following variables are used in the instructions below:\n\n| Variable    | Description                                                                         |\n| ----------- | ----------------------------------------------------------------------------------- |\n| SAN_IP      | subject alternative name in form of ip address e.g. 10.10.10.10 for TLS certificate |\n| SERVER_CERT | server certificate file path e.g. /etc/opi/server-cert.pem                          |\n| SERVER_KEY  | server key file path e.g. /etc/opi/server-key.pem                                   |\n| CA_CERT     | server CA certificate file path e.g. /etc/opi/ca-cert.pem                           |\n| CLIENT_CERT | client certificate file path e.g. /etc/opi/client-cert.pem                          |\n| CLIENT_KEY  | client key file path e.g. /etc/opi/client-key.pem                                   |\n\n#### Generate certificates/keys\n\nThis section describes how to generate TLS self-signed certificates. The root level Certificate Authority (CA) is used to generate server-side key and cert files, and client-side key and cert files. This results in a 1-depth level certificate chain, which will suffice for verification and validation purposes but may not provide sufficient security for production systems. It is highly recommended to use well-known CAs, and generate certificates at multiple depth levels in order to conform to higher security standards.\n\n```bash\n# create config files\necho \"subjectAltName=IP:$SAN_IP\" \u003e server-ext.cnf\necho \"subjectAltName=IP:$SAN_IP\" \u003e client-ext.cnf\n# generate CA certificate\nopenssl req -x509 -newkey rsa:4096 -days 365 -nodes -keyout ca-key.pem -out ca-cert.pem -sha384\n# generate server private key and signing request\nopenssl req -newkey rsa:4096 -nodes -keyout server-key.pem -out server-req.pem -sha384\n# use CA's private key to get signed server certificate\nopenssl x509 -req -in server-req.pem -days 365 -CA ca-cert.pem -CAkey ca-key.pem -CAcreateserial -out server-cert.pem -extfile server-ext.cnf\n# generate client private key and signing request\nopenssl req -newkey rsa:4096 -nodes -keyout client-key.pem -out client-req.pem -sha384\n# use CA's private key to get signed client certificate\nopenssl x509 -req -in client-req.pem -days 365 -CA ca-cert.pem -CAkey ca-key.pem -CAcreateserial -out client-cert.pem -extfile client-ext.cnf\n```\n\ncopy server related certificates/keys to a dedicated secure location on xPU\n\n```bash\nmkdir /etc/opi\ncp server-cert.pem /etc/opi\ncp server-key.pem /etc/opi\ncp ca-cert.pem /etc/opi\n```\n\nmake sure to follow the principle of least privilege for access permissions and change ownership to a dedicated user.\n\n##### Run server\n\nRun bridge binary specifying TLS-related server key/certificate and CA cert\n\n```bash\n./opi-intel-bridge-storage -grpc_port=$BRIDGE_PORT -http_port=$BRIDGE_HTTP_PORT -tls $SERVER_CERT:$SERVER_KEY:$CA_CERT\n```\n\nfor container\n\n```bash\ndocker run --network=host -v \"/var/tmp:/var/tmp\" -v \"/etc/opi:/etc/opi\" ghcr.io/opiproject/opi-intel-bridge:main /opi-intel-bridge -grpc_port=$BRIDGE_PORT -http_port=$BRIDGE_HTTP_PORT -tls $SERVER_CERT:$SERVER_KEY:$CA_CERT\n```\n\n##### Send client commands\n\nTo send by means of not-containerized grpc_cli:\n\n```bash\nGRPC_DEFAULT_SSL_ROOTS_FILE_PATH=\"$CA_CERT\" grpc_cli --json_input --json_output --channel_creds_type=ssl --ssl_client_cert=\"$CLIENT_CERT\" --ssl_client_key=\"$CLIENT_KEY\" call $BRIDGE_ADDR ListAioControllers \"{}\"\n```\n\n## I Want To Contribute\n\nThis project welcomes contributions and suggestions.  We are happy to have the Community involved via submission of **Issues and Pull Requests** (with substantive content or even just fixes). We are hoping for the documents, test framework, etc. to become a community process with active engagement.  PRs can be reviewed by any number of people, and a maintainer may accept.\n\nSee [CONTRIBUTING](https://github.com/opiproject/opi/blob/main/CONTRIBUTING.md) and [GitHub Basic Process](https://github.com/opiproject/opi/blob/main/Policies/doc-github-rules.md) for more details.\n\nFor reporting security vulnerabilities see [SECURITY.md](https://github.com/opiproject/opi-intel-bridge/blob/main/SECURITY.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopiproject%2Fopi-intel-bridge","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopiproject%2Fopi-intel-bridge","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopiproject%2Fopi-intel-bridge/lists"}