{"id":18802190,"url":"https://github.com/oracle-quickstart/oci-hpc-runbook-starccm","last_synced_at":"2025-10-12T01:25:09.119Z","repository":{"id":42370834,"uuid":"368722533","full_name":"oracle-quickstart/oci-hpc-runbook-starccm","owner":"oracle-quickstart","description":null,"archived":false,"fork":false,"pushed_at":"2022-04-12T14:19:59.000Z","size":2159,"stargazers_count":3,"open_issues_count":1,"forks_count":3,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-07-26T23:44:54.901Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"upl-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oracle-quickstart.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-05-19T02:31:43.000Z","updated_at":"2024-01-26T07:00:13.000Z","dependencies_parsed_at":"2022-09-11T12:31:29.082Z","dependency_job_id":null,"html_url":"https://github.com/oracle-quickstart/oci-hpc-runbook-starccm","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/oracle-quickstart/oci-hpc-runbook-starccm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oracle-quickstart%2Foci-hpc-runbook-starccm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oracle-quickstart%2Foci-hpc-runbook-starccm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oracle-quickstart%2Foci-hpc-runbook-starccm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oracle-quickstart%2Foci-hpc-runbook-starccm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oracle-quickstart","download_url":"https://codeload.github.com/oracle-quickstart/oci-hpc-runbook-starccm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oracle-quickstart%2Foci-hpc-runbook-starccm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279009770,"owners_count":26084645,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-11T02:00:06.511Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T22:26:56.348Z","updated_at":"2025-10-12T01:25:09.072Z","avatar_url":"https://github.com/oracle-quickstart.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# \u003cimg src=\"https://github.com/oracle-quickstart/oci-hpc-runbook-StarCCM/raw/main/Images/STARCCM_logo.png\" height=\"60\"\u003e Siemens Simcenter STAR-CCM+ Runbook\n\n# Introduction\nSimcenter STAR-CCM+ is a complete multiphysics solution for the simulation of products and designs. This Runbook will take you through the process of deploying a Simcenter STAR-CCM+ cluster on Oracle Cloud with low latency networking between the compute nodes. Running Simcenter STAR-CCM+ on Oracle Cloud is quite straightforward, follow along this guide for all the tips and tricks.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://github.com/oracle-quickstart/oci-hpc-runbook-StarCCM/raw/main/Images/Screenshot 2019-07-09 at 14.50.19.png\" height=\"300\" \u003e\n \u003c/p\u003e\n \n## Prerequisites\n\n- Permission to `manage` the following types of resources in your Oracle Cloud Infrastructure tenancy: `vcns`, `internet-gateways`, `route-tables`, `network-security-groups`, `subnets`, and `instances`.\n\n- Quota to create the following resources: 1 VCN, 2 subnets, 1 Internet Gateway, 1 NAT Gateway, 1 Service Gateway, 3 route rules, and minimum 2 compute instances in instance pool or cluster network (plus bastion host).\n\nIf you don't have the required permissions and quota, contact your tenancy administrator. See [Policy Reference](https://docs.cloud.oracle.com/en-us/iaas/Content/Identity/Reference/policyreference.htm), [Service Limits](https://docs.cloud.oracle.com/en-us/iaas/Content/General/Concepts/servicelimits.htm), [Compartment Quotas](https://docs.cloud.oracle.com/iaas/Content/General/Concepts/resourcequotas.htm).\n\n## Deploy Using Oracle Resource Manager\n\n1. Click [![Deploy to Oracle Cloud](https://oci-resourcemanager-plugin.plugins.oci.oraclecloud.com/latest/deploy-to-oracle-cloud.svg)](https://cloud.oracle.com/resourcemanager/stacks/create?region=home\u0026zipUrl=https://github.com/oracle-quickstart/oci-hpc-runbook-starccm/releases/latest/download/oci-hpc-runbook-starccm-stack-latest.zip)\n\n    If you aren't already signed in, when prompted, enter the tenancy and user credentials.\n\n2. Review and accept the terms and conditions.\n\n3. Select the region where you want to deploy the stack.\n\n4. Follow the on-screen prompts and instructions to create the stack.\n\n5. After creating the stack, click **Terraform Actions**, and select **Plan**.\n\n6. Wait for the job to be completed, and review the plan.\n\n    To make any changes, return to the Stack Details page, click **Edit Stack**, and make the required changes. Then, run the **Plan** action again.\n\n7. If no further changes are necessary, return to the Stack Details page, click **Terraform Actions**, and select **Apply**. \n\n## Deploy Using the Terraform CLI\n\n### Clone the Module\nNow, you'll want a local copy of this repo. You can make that with the commands:\n\n    git clone https://github.com/oracle-quickstart/oci-hpc-runbook-starccm.git\n    cd oci-hpc-runbook-starccm\n    ls\n\n### Set Up and Configure Terraform\n\n1. Complete the prerequisites described [here](https://github.com/cloud-partners/oci-prerequisites).\n\n2. Create a `terraform.tfvars` file, and specify the following variables:\n\n```\n# Authentication\ntenancy_ocid         = \"\u003ctenancy_ocid\u003e\"\nuser_ocid            = \"\u003cuser_ocid\u003e\"\nfingerprint          = \"\u003cfinger_print\u003e\"\nprivate_key_path     = \"\u003cpem_private_key_path\u003e\"\n\n# Region\nregion = \"\u003coci_region\u003e\"\n\n# Availablity Domain \nad = \"\u003cavailablity doman\u003e\" # for example \"GrCH:US-ASHBURN-AD-1\"\n\n# Bastion \nbastion_ad               = \"\u003cavailablity doman\u003e\" # for example \"GrCH:US-ASHBURN-AD-1\"\nbastion_boot_volume_size = \"\u003cbastion_boot_volume_size\u003e\" # for example 50\nbastion_shape            = \"\u003cbastion_shape\u003e\" # for example \"VM.Standard.E3.Flex\"\nboot_volume_size         = \"\u003cboot_volume_size\u003e\" # for example 100\nnode_count               = \"\u003cnode_count\u003e\" # for example 2\nssh_key                  = \"\u003cssh_key\u003e\"\ntargetCompartment        = \"\u003ctargetCompartment\u003e\" \nuse_custom_name          = false\nuse_existing_vcn         = false\nuse_marketplace_image    = true\nuse_standard_image       = true\ncluster_network          = false\ninstance_pool_shape      = \"\u003cinstance_pool_shape\u003e\" # for example VM.Standard.E3.Flex\nstarccm_binaries         = \"\u003cstarccm_binaries\u003e\" # for example https://objectstorage.us-phoenix-1.oraclecloud.com/p/CTYj(...)F7V/n/hpc/b/HPC_APPS/o/STAR-CCM+15.02.009_01_linux-x86_64-2.12_gnu7.1\"\nstarccm_version          = \"\u003cstarccm_version\u003e\" # for example \"STAR-CCM+15.02.009_01_linux-x86_64-2.12_gnu7.1\"\n\n\n````\n\n### Create the Resources\nRun the following commands:\n\n    terraform init\n    terraform plan\n    terraform apply\n\n### Destroy the Deployment\nWhen you no longer need the deployment, you can run this command to destroy the resources:\n\n    terraform destroy\n    \n# Architecture\n![](https://github.com/oracle-quickstart/oci-hpc-runbook-fluent/blob/main/images/architecture-hpc.png \"Architecture for Running StarCCM+ in OCI\")\nThe architecture for this runbook is as follow, we have one small machine (bastion) that you will connect into. The compute nodes will be on a separate private network linked with RDMA RoCE v2 networking. The bastion will be accesible through SSH from anyone with the key (or VNC if you decide to enable it). Compute nodes will only be accessible through the bastion inside the network. This is made possible with 1 Virtual Cloud Network with 2 subnets, one public and one private.\n\nThe above baseline infrastructure provides the following specifications:\n-\tNetworking\n    -\t1 x 100 Gbps RDMA over converged ethernet (ROCE) v2\n    -\tLatency as low as 1.5 µs\n-\tHPC Compute Nodes (BM.HPC2.36)\n    -\t6.4 TB Local NVME SSD storage per node\n    -\t36 cores per node\n    -\t384 GB memory per node\n    \n# Upload StarCCM+ binaries to Object Storage\n1. Log In\n\nYou can start by logging in the Oracle Cloud console. If this is the first time, instructions to do so are available [here](https://docs.cloud.oracle.com/iaas/Content/GSG/Tasks/signingin.htm).\nSelect the region in which you wish to create your Object Storage Bucket. Click on the current region in the top right dropdown list to select another one. \n\n2. Go to Buckets by clicking on  \u003cimg src=\"https://github.com/oracle-quickstart/oci-hpc-runbook-lsdyna/blob/main/images/menu.png?raw=true\" height=\"30\"\u003e  and selecting **Storage**  \u003cimg src=\"https://github.com/oracle-quickstart/oci-hpc-runbook-lsdyna/blob/main/images/Storage%20option.png?raw=true\" height=\"130\"\u003e  \u003e **Buckets**  \u003cimg src=\"https://github.com/oracle-quickstart/oci-hpc-runbook-lsdyna/blob/main/images/Buckets.png?raw=true\" height=\"70\"\u003e\n\n3. Create a bucket by clicking  \u003cimg src=\"https://github.com/oracle-quickstart/oci-hpc-runbook-lsdyna/blob/main/images/Create%20bucket.png?raw=true\" height=\"30\"\u003e. Give your bucket a name and select the storage tier and encryption.\n\n4. Once the bucket has been created, upload an object (binary) to the bucket by clicking **Upload**  \u003cimg src=\"https://github.com/oracle-quickstart/oci-hpc-runbook-lsdyna/blob/main/images/Upload%20Object.png?raw=true\" height=\"90\"\u003e  under **Objects**.\n\n5. Create a Pre-Authenitcated Request (PAR) using the following steps:\n\n\t- Click on  \u003cimg src=\"https://github.com/oracle-quickstart/oci-hpc-runbook-lsdyna/blob/main/images/par%20menu.png?raw=true\" height=\"40\"\u003e  for the object, then select  \u003cimg src=\"https://github.com/oracle-quickstart/oci-hpc-runbook-lsdyna/blob/main/images/Create%20PAR%20button%20from%20menu.png?raw=true\" height=\"30\"\u003e \n\n\t- Select  \u003cimg src=\"https://github.com/oracle-quickstart/oci-hpc-runbook-lsdyna/blob/main/images/Object%20option%20PAR%20menu.png?raw=true\" height=\"100\"\u003e  for the **Pre-Authenticated Request Target** and then select an access type.\n\n\t- Click  \u003cimg src=\"https://github.com/oracle-quickstart/oci-hpc-runbook-lsdyna/blob/main/images/Create%20PAR.png?raw=true\" height=\"30\"\u003e\n\n\t- Be sure to copy the PAR URL by clicking \u003cimg src=\"https://github.com/oracle-quickstart/oci-hpc-runbook-lsdyna/blob/main/images/Copy.png?raw=true\" height=\"30\"\u003e before closing because you will **NOT** have access to the URL again.  \n\n6. Add this PAR to the starccm_binaries variable.\n\n# Running STAR-CCM+\n# Install Intel MPI 2018 librairies\n\nRun those commands on every node. \n```\ncd /nfs/cluster\nwget https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB\nsudo rpm --import GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB\nsudo yum-config-manager --add-repo=https://yum.repos.intel.com/mpi\nsudo yum install -y intel-mpi-2018.4-057 intel-mpi-samples-2018.4-274\n```\n\t\n# Running StarCCM+\n\n1. Navigate to Bastion - Find the public IP address of your remote host after the deployment job has finished:\n\u003cdetails\u003e\n\t\u003csummary\u003eResource Manager\u003c/summary\u003e\n\t\u003cp\u003e\u003c/p\u003e\n\tIf you deployed your stack via Resource Manager, find the public IP address of the compute node at the bottom of the CLI console logs.\n\t\u003cp\u003e\u003c/p\u003e\n\u003c/details\u003e\n\u003cdetails\u003e\n\t\u003csummary\u003eCommand Line\u003c/summary\u003e\n\t\u003cp\u003e\u003c/p\u003e\n\tIf you deployed your stack via Command Line, find the public IP address of the compute node at the bottom of the console logs on the \u003cb\u003eLogs\u003c/b\u003e page, or on the \u003cb\u003eOutputs\u003c/b\u003e page.\n\t\u003cp\u003e\u003c/p\u003e\n\u003c/details\u003e\n\n2. SSH into your bastion host \n```\nssh -i PRIVATE KEY PATH opc@IP_ADDRESS\n```\n\n3. SSH into cluster\n```\nssh hpc-node-1\n```\n\n3. Go to your model\n```\ncd /nfs/scratch/starccm/work\n```\n\n7. To run, navigate to and run this [script](https://github.com/oracle-quickstart/oci-hpc-runbook-lsdyna/blob/main/Resources/LSDYNA_3car.sh)\n```\n/nfs/cluster/lsdyna/work\n```\nBe sure to set execution permission before running the [script](https://github.com/oracle-quickstart/oci-hpc-runbook-starccm/blob/main/Resources/StarCCM_lemans_poly_17m.sh).\n\nExample:\n```\nchmod +x script.sh\n```\n\nPlease make sure to update the script with your specific variables `line 17` - `line 28`\n\n# Benchmark Example\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://github.com/oracle-quickstart/oci-hpc-runbook-StarCCM/blob/main/Images/lemans.png\" height=\"300\"\u003e\n\u003c/p\u003e\nPerformances of STAR-CCM+ are often measured using the LeMans benchmark with 17 and 105 Millions cells. The next graphs are showing how using more nodes impact the runtime, with a scaling really close to 100%. RDMA network, which has not been discussed in this runbook, only start to differentiate versus regular TCP runs if the Cells / Core ratio starts to go down.  \n\n## 17 Millions Cells\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://github.com/oracle-quickstart/oci-hpc-runbook-StarCCM/blob/main/Images/RunTime_17M.png\" height=\"350\"\u003e\n\u003cimg src=\"https://github.com/oracle-quickstart/oci-hpc-runbook-StarCCM/blob/main/Images/scaling_17M.png\" height=\"350\"\u003e\n\u003c/p\u003e\n\n## 105 Millions Cells\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://github.com/oracle-quickstart/oci-hpc-runbook-StarCCM/blob/main/Images/RunTime_105M.png\" height=\"350\"\u003e\n\u003cimg src=\"https://github.com/oracle-quickstart/oci-hpc-runbook-StarCCM/blob/main/Images/Scaling_105M.png\" height=\"350\"\u003e\n\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foracle-quickstart%2Foci-hpc-runbook-starccm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foracle-quickstart%2Foci-hpc-runbook-starccm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foracle-quickstart%2Foci-hpc-runbook-starccm/lists"}