{"id":26129500,"url":"https://github.com/mathworks-ref-arch/matlab-parallel-server-with-aws-batch","last_synced_at":"2025-10-16T00:23:24.976Z","repository":{"id":100806921,"uuid":"219993489","full_name":"mathworks-ref-arch/matlab-parallel-server-with-aws-batch","owner":"mathworks-ref-arch","description":"Stand up a MATLAB Parallel Server AWS Batch configuration using CloudFormation","archived":false,"fork":false,"pushed_at":"2024-05-02T15:17:12.000Z","size":613,"stargazers_count":7,"open_issues_count":1,"forks_count":0,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-05-03T02:07:25.027Z","etag":null,"topics":["matlab-aws"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mathworks-ref-arch.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-11-06T12:38:28.000Z","updated_at":"2024-05-02T15:17:05.000Z","dependencies_parsed_at":"2024-04-23T10:59:31.454Z","dependency_job_id":null,"html_url":"https://github.com/mathworks-ref-arch/matlab-parallel-server-with-aws-batch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathworks-ref-arch%2Fmatlab-parallel-server-with-aws-batch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathworks-ref-arch%2Fmatlab-parallel-server-with-aws-batch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathworks-ref-arch%2Fmatlab-parallel-server-with-aws-batch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathworks-ref-arch%2Fmatlab-parallel-server-with-aws-batch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mathworks-ref-arch","download_url":"https://codeload.github.com/mathworks-ref-arch/matlab-parallel-server-with-aws-batch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242917951,"owners_count":20206516,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["matlab-aws"],"created_at":"2025-03-10T19:49:12.378Z","updated_at":"2025-10-16T00:23:24.970Z","avatar_url":"https://github.com/mathworks-ref-arch.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# MATLAB Parallel Server with AWS Batch\n\nMATLAB\u0026reg; Parallel Server\u0026trade; with AWS\u0026reg; Batch is a solution for [batch processing MATLAB jobs](https://www.mathworks.com/help/parallel-computing/batch-processing.html) using Amazon\u0026reg; Web Services (AWS). This repository helps you automate the process of launching an AWS Batch cluster in your AWS account. \n\n[AWS Batch](https://aws.amazon.com/batch/) is a service for running batch computing jobs on AWS. AWS Batch dynamically provisions, manages, monitors, and terminates [Amazon EC2\u0026reg;](https://aws.amazon.com/ec2/) instances based on the volume and resource requirements of the submitted jobs.  Both [On-Demand](https://aws.amazon.com/ec2/pricing/on-demand/) and [Spot](https://aws.amazon.com/ec2/spot/) EC2 instances are supported. AWS Batch jobs run as containerized applications on Elastic Container Service (ECS) container instances. For more information about ECS containers, see [What is Amazon Elastic Container Service?](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html).\n\nFor information about the architecture of this solution, see [Learn About Cluster Architecture](#learn-about-cluster-architecture).\n\nMATLAB Parallel Server with AWS Batch does not support [communicating jobs](https://www.mathworks.com/help/parallel-computing/introduction.html). To run communicating jobs, you can run MATLAB Parallel Server in AWS using the built-in MATLAB Job Scheduler instead of AWS Batch using this GitHub\u0026reg; repository: [MATLAB Parallel Server on AWS](https://github.com/mathworks-ref-arch/matlab-parallel-server-on-aws). For a simpler but less customizable method of launching a MATLAB Parallel Server cluster in AWS, try [MathWorks Cloud Center](https://www.mathworks.com/help/cloudcenter/mathworks-cloud-center.html) instead.\n\n# Requirements\n\nBefore starting, you must have the following:\n\n* MATLAB Parallel Server license. For more information on how to configure your license for cloud use, see [MATLAB Parallel Server on the Cloud](https://www.mathworks.com/help/install/license/licensing-for-mathworks-products-running-on-the-cloud.html).\n\n* MATLAB and Parallel Computing Toolbox\u0026trade; on your desktop.\n\n* An AWS account with required permissions. To see what is required, look at the [example policy](matlab-parallel-server-with-aws-batch-admin-iam-policy.json). For more information about the services used, see [Learn About Cluster Architecture](#learn-about-cluster-architecture).\n\n# Costs\nYou are responsible for the cost of the AWS services used when you create cloud resources using this repository. Resource settings, such as instance type, affect the cost of deployment. For cost estimates, see the pricing pages for each AWS service you use. Prices are subject to change.\n\n# Deployment Steps\n\nTo view instructions for deploying the MATLAB Parallel Server with AWS Batch reference architecture, select a MATLAB release:\n\n| Release |\n| ------- |\n| [R2025b](releases/R2025b/README.md) |\n| [R2025a](releases/R2025a/README.md) |\n| [R2024b](releases/R2024b/README.md) |\n| [R2024a](releases/R2024a/README.md) |\n| [R2023b](releases/R2023b/README.md) |\n| [R2023a](releases/R2023a/README.md) |\n| [R2022b](releases/R2022b/README.md) |\n| [R2022a](releases/R2022a/README.md) |\n| [R2021b](releases/R2021b/README.md) |\n| [R2021a](releases/R2021a/README.md) |\n| [R2020b](releases/R2020b/README.md) |\n| [R2020a](releases/R2020a/README.md) |\n| [R2019b](releases/R2019b/README.md) |\n\n\n\n# Learn About Cluster Architecture\n\nThis diagram illustrates the cluster architecture and operation of MATLAB Parallel Server with AWS Batch. When you use the [AWS CloudFormation templates](https://aws.amazon.com/cloudformation/) in this repository, it creates the AWS Batch Cluster and the following resources. For more information about each resource, see the [AWS CloudFormation template reference](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-reference.html).\n\n![Cluster Architecture](img/MATLAB-Parallel-Server-with-AWS-Batch-architecture.png?raw=true)\n\n*Figure 1: Cluster Architecture*\n\n### AWS Batch Resources\n* Job Definition (AWS::Batch::JobDefinition)\nThe job definition specifies how a job is run. It defines the following:\n    * The Docker container image to use.\n    * The number of virtual CPUS (vCPUS), GPUs, and the amount of memory a job requires.\n    * Where to mount the data volume containing the MATLAB installation.\n* Compute environment (AWS::Batch::ComputeEnvironment)\nThe compute environment used to run jobs. It defines the following:\n  * The EC2 pricing model to use (On-Demand or Spot).\n  * The launch template to use for EC2 instances.\n  * The EC2 instance types to use.\n  * The maximum number of vCPUs that can run simultaneously.\n  * That the cluster scales down to zero vCPUs when no jobs are in the queue.\n* Job Queue (AWS::Batch::JobQueue)\nThe job queue to which jobs are submitted. Jobs submitted to the queue run in the defined compute environment.\n\n### Storage Resources\n* Amazon S3\u0026trade; Bucket for data transfer (AWS::S3::Bucket)\nThe S3 bucket used for data transfer between the MATLAB client and the MATLAB workers running in AWS Batch.\n\n### Compute Resources\n* Launch template (AWS::EC2::LaunchTemplate)\nThe launch template for EC2 instances launched by AWS Batch. Used to mount a MATLAB installation to the EC2 instance.\n* Security Group (AWS::EC2::SecurityGroup)\nThe security group for EC2 instances launched by AWS Batch.\n\n### Identity and Access Management (IAM) Resources\n* IAM Role for container instances (AWS::IAM::Role)\nRole that allows containers launched by AWS Batch to access Amazon S3 and Amazon Elastic Container Service.\n\n* IAM Instance Profile for container instances (AWS::IAM::InstanceProfile)\nProfile that associates containers launched by AWS Batch with the IAM Role for container instances.\n\n* IAM Role for AWS Batch (AWS::IAM::Role)\nRole that provides AWS Batch with the necessary permissions to use other AWS services to run jobs.\n\n* IAM Role for Spot (AWS::IAM::Role)\nRole that provides AWS Batch with the necessary permissions to run jobs using EC2 Spot.\n\n* IAM Role for Lambda function for multiplication (AWS::IAM::Role)\nRole that provides permissions to run the Lambda function for multiplication.\n\n* IAM role for deleting S3 bucket contents (AWS::IAM::Role)\nRole that provides permissions to run the Lambda function to empty S3 bucket.\n\n### Lambda resources\n* Lambda function for multiplication (AWS::Lambda::Function)\nLambda function that multiplies two numbers together. Used to calculate the compute environment's maximum number of vCPUs.\n\n* Custom resource for multiplication (Custom::MultiplyFunction)\nCustom resource that uses the Lambda function for multiplication to calculate the compute environment's maximum number of vCPUs. Multiplies the input parameters \"MaxWorkers\" and \"NumVCPUsPerWorker\" together at stack creation.\n\n* Lambda function to empty S3 bucket (AWS::Lambda::Function)\nLambda function that empties the S3 bucket at stack deletion. When empty, the S3 bucket can be automatically deleted by Cloud Formation.\n\n* Custom resource to empty S3 bucket (Custom::EmptyBuckets)\nCustom resource that uses the Lambda function to empty S3 bucket at stack deletion.\n\n## FAQ\n\n### What can I do with MATLAB Parallel Server?\n\nParallel Computing Toolbox and MATLAB Parallel Server software let you solve computationally and data-intensive programs using MATLAB and Simulink on computer clusters, clouds, and grids. Parallel processing constructs such as parallel-for loops and code blocks, distributed arrays, parallel numerical algorithms, and message-passing functions let you implement task-parallel and data-parallel algorithms at a high level in MATLAB. To learn more, see the documentation: [Parallel Computing Toolbox](https://www.mathworks.com/help/parallel-computing) and [MATLAB Parallel Server](https://www.mathworks.com/help/matlab-parallel-server/). \n\n### What is Amazon Web Services (AWS)?\n\nAWS is a set of cloud services which allow you to build, deploy, and manage applications hosted in Amazon’s global network of data centers. Find out more about the range of [cloud-based products offered by AWS](https://aws.amazon.com/products/). Services deployed in AWS can be created, managed, and deleted using the AWS Management Console. For more information about the AWS Management Console, see [AWS Management Console](https://docs.aws.amazon.com/awsconsolehelpdocs/). \n\n# Enhancement Request\nProvide suggestions for additional features or capabilities using the following link: [MATLAB and Simulink in the Cloud](https://www.mathworks.com/solutions/cloud.html).\n\n# Technical Support\nIf you require assistance or have a request for additional features or capabilities, contact [MathWorks Technical Support](https://www.mathworks.com/support/contact_us.html).\n\n----\n\nCopyright 2020-2025 The MathWorks, Inc.\n\n----","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmathworks-ref-arch%2Fmatlab-parallel-server-with-aws-batch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmathworks-ref-arch%2Fmatlab-parallel-server-with-aws-batch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmathworks-ref-arch%2Fmatlab-parallel-server-with-aws-batch/lists"}