{"id":18802307,"url":"https://github.com/oracle-quickstart/oci-beegfs-beeond-rdma","last_synced_at":"2025-09-02T11:31:36.504Z","repository":{"id":106379903,"uuid":"263390016","full_name":"oracle-quickstart/oci-beegfs-beeond-rdma","owner":"oracle-quickstart","description":"Deploy BeeGFS BeeOND (BeeGFS ON Demand) on Oracle Cloud Infrastructure HPC 100Gbps RDMA Clustered Network","archived":false,"fork":false,"pushed_at":"2020-10-18T10:10:57.000Z","size":1033,"stargazers_count":5,"open_issues_count":0,"forks_count":4,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-11-07T22:38:41.677Z","etag":null,"topics":["oracle-led"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oracle-quickstart.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-05-12T16:22:58.000Z","updated_at":"2024-04-09T06:50:37.000Z","dependencies_parsed_at":null,"dependency_job_id":"e5c81110-6558-4390-9e18-cd70b34b2856","html_url":"https://github.com/oracle-quickstart/oci-beegfs-beeond-rdma","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oracle-quickstart%2Foci-beegfs-beeond-rdma","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oracle-quickstart%2Foci-beegfs-beeond-rdma/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oracle-quickstart%2Foci-beegfs-beeond-rdma/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oracle-quickstart%2Foci-beegfs-beeond-rdma/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oracle-quickstart","download_url":"https://codeload.github.com/oracle-quickstart/oci-beegfs-beeond-rdma/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":231780352,"owners_count":18425544,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["oracle-led"],"created_at":"2024-11-07T22:27:24.569Z","updated_at":"2025-09-02T11:31:36.442Z","avatar_url":"https://github.com/oracle-quickstart.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# oci-beegfs-beeond-rdma\nDeploy BeeGFS BeeOND (BeeGFS ON Demand) on Oracle Cloud Infrastructure HPC 100Gbps RDMA Clustered Network. The template will do the following: \n\n- Provision a private clustered network using HPC baremeta nodes and 100 Gbps RDMA.  Use this template to deploy BeeGFS BeeOND on HPC nodes with clustered networking.   \n- Create a BeeGFS BeeOND parallel filesystem using all the nodes in the cluster by leveraging the local NVMe SSD on each HPC node to create a single filesystem namespace.\n- By default, all nodes will have a BeeGFS BeeOND Storage service \u0026 BeeOND client service running.  The first node, will also run Management and Metadata service, in addition to storage and client service.   You can override the default behavior using Terraform variables, see below [Customize-the-template](https://github.com/oracle-quickstart/oci-beegfs-beeond-rdma#customize-the-template) section. \n- By default, the filesystem is designed to use 100Gbps RDMA for filesystem traffic along with your HPC application traffic.  If we want to use 100Gbps RDMA only for your compute application traffic, then you can configure BeeOND to use 25Gbps network on the HPC nodes.      \n\n## BeeGFS BeeOND (ON-Demand) License\nBeeGFS/BeeOND requires license, reach out to [thinkparQ](https://thinkparq.com/) for license details. This github code is open source and its users responsibility to get the required license.\n\n## What is BeeGFS BeeOND (ON-Demand)\n![](./images/What_is_BeeOND.png)\n*Quoted from https://www.beegfs.io/wiki/BeeOND#whatis\n\n\n## High Level Architecture\n###  Using RDMA network: \nThe filesystem is designed to use 100Gbps RDMA for filesystem traffic along with your HPC application traffic.\n![](./images/BeeOND_RDMA_OCI_High_Level_Arch.png)\n\n####  RDMA network and mount point:\n![](./images/Output-01.png)\n![](./images/Output-02.png)\n\n\n\n###  Using TCP network: \nThe filesystem is designed to use 25Gbps TCP network for filesystem traffic, while the 100Gbps RDMA is used for HPC application traffic.    Set **use_beegfs_over_rdma to false** in terraform.tfvars or variables.tf file.   \n![](./images/BeeOND_TCP_OCI_High_Level_Arch.png)\n\n####  TCP (25Gbps) network and mount point:\n![](./images/Output-04.png)\n![](./images/Output-03.png)\n\n\n## Prerequisites\nFirst off you'll need to do some pre deploy setup.  That's all detailed [here](https://github.com/oracle/oci-quickstart-prerequisites).\n\n## Clone the Terraform template\nNow, you'll want a local copy of this repo.  You can make that with the commands:\n\n    git clone https://github.com/oracle-quickstart/oci-beegfs-beeond-rdma.git\n    cd oci-beegfs-beeond-rdma/\n    ls\n\n## Customize the template \nCreate a terraform.tfvars file and set values as per your needs.  We recommend to use terraform.tfvars to override values in variables.tf file.   Update values based on your AD,  # of nodes in a cluster, etc.    \n\n- By default,  **use_beegfs_over_rdma is set to true** in variables.tf, so RDMA network will be used for BeeOND filesystem traffic.  \n- Set **use_marketplace_image=true** in your terraform.tfvars file as shown below, To use OCI  HPC Cluster Network functionality, you need to use a custom marketplace image created by OCI HPC team.  The custom marketplace image is built using Oracle Linux UEK 7.6 image with OFED4.6.x drivers for 100Gbps RDMA cluster network and OCI-HPC utils.  We recommend using OS image using Oracle Linux UEK,  however if you need CentOS+OCI  HPC Cluster Network functionality, then reach out to OCI HPC team.  \n- The variable,  **use_standard_image** only applies to what image to use for bastion node and doesn't apply for hpc node image.  If you plan to use a custom image for bastion, then set **use_standard_image** to false and set the **custom_bastion_image** variable to an Image OCID value. (ocid1.image.oc1.xxxxxxxxxx) \n\n```\n           cat terraform.tfvars\n           ad=\"kWVD:UK-LONDON-1-AD-1\"\n           bastion_ad=\"kWVD:UK-LONDON-1-AD-1\"\n           node_count=3\n           boot_volume_size=\"100\"\n           use_existing_vcn=false\n           use_custom_name=false\n           use_marketplace_image=true\n           metadata_node_count=2\n           storage_node_count=2\n           io500=false\n           use_standard_image=true\n\n```\n\nIn above example,  on a 3 node (**node_count=3**) BeeOND cluster, I am asking for 2 Metadata servers (**metadata_node_count=2**) instead of default of 1.  Also I am asking for only 2 Storage servers (**storage_node_count=2**) instead of the default of creating Storage servers on all nodes of the BeeOND cluster (In this example: 3)\n\n\n\n## Deployment and Post Deployment\nDeploy using standard Terraform commands\n\n        terraform init\n        terraform plan\n        terraform apply \n\n## Contributing\n\nThis project welcomes contributions from the community. Before submitting a pull request, please [review our contribution guide](./CONTRIBUTING.md)\n\n## Security\n\nPlease consult the [security guide](./SECURITY.md) for our responsible security vulnerability disclosure process\n\n## License\n\nCopyright (c) 2023 Oracle and/or its affiliates.\n\nReleased under the Apache License Version 2.0, as shown at\n\u003chttp://www.apache.org/licenses/\u003e.\n\n![](./images/TF-apply.PNG)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foracle-quickstart%2Foci-beegfs-beeond-rdma","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foracle-quickstart%2Foci-beegfs-beeond-rdma","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foracle-quickstart%2Foci-beegfs-beeond-rdma/lists"}