Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.
Awesome Lists | Featured Topics | Projects
https://github.com/comet-ml/terraform-aws-comet

Last synced: 7 days ago
JSON representation
Host: GitHub
URL: https://github.com/comet-ml/terraform-aws-comet
Owner: comet-ml
Created: 2023-06-07T02:42:49.000Z (over 1 year ago)
Default Branch: main
Last Pushed: 2024-04-08T15:23:24.000Z (7 months ago)
Last Synced: 2024-04-08T18:52:41.924Z (7 months ago)
Language: HCL
Size: 74.4 MB
Stars: 1
Watchers: 4
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md
Awesome Lists containing this project

README

        # Comet Infrastructure Deployment: Terraform on AWS

Terraform module for deploying infrastructure components to run CometML.

### Deployment

**Prerequisites:**

- [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) installed

- [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) installed

  - Access to an AWS account and credentials that allow you to create resources

  - Ensure `AWS_ACCESS_KEY_ID` and `AWS_SECRET_KEY_ID` environment variables are set in your session

- [Terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli) installed

**Infrastructure Deployment:**

- Follow the steps below to deploy directly from the GitHub repository.

  - Clone the repository to your local machine: `git clone https://github.com/comet-ml/terraform_aws_comet.git`

  - Move into the deployment directory: `cd terraform-aws-comet`

  - Initialize the directory: `terraform init`

  - Within terraform.tfvars, set your module toggles to enable the desired infrastructure components and set any related inputs

  - Provision the resources: `terraform apply`

**A note on state management:**

- This configuration stores the Terraform state locally by default. To store the state file remotely in S3, a `backend` block can be nested within the `terraform` block inside versions.tf if applying directly from this configuration, or within your `terraform` block if calling the module. Below is an example of such a configuration:

```

terraform {

  backend "s3" {

    bucket = "mybucket"

    key    = "path/to/my/key"

    region = "us-east-1"

  }

}

```

- More on state management in S3 can be found [here](https://developer.hashicorp.com/terraform/language/settings/backends/s3)

## Requirements

| Name | Version |

|------|---------|

|  [terraform](#requirement\_terraform) | >= 1.0 |

|  [aws](#requirement\_aws) | ~> 5.1 |

|  [helm](#requirement\_helm) | ~> 2.10 |

|  [kubernetes](#requirement\_kubernetes) | ~> 2.21 |

## Providers

| Name | Version |

|------|---------|

|  [aws](#provider\_aws) | 5.8.0 |

## Modules

| Name | Source | Version |

|------|--------|---------|

|  [comet\_ec2](#module\_comet\_ec2) | ./modules/comet_ec2 | n/a |

|  [comet\_ec2\_alb](#module\_comet\_ec2\_alb) | ./modules/comet_ec2_alb | n/a |

|  [comet\_eks](#module\_comet\_eks) | ./modules/comet_eks | n/a |

|  [comet\_elasticache](#module\_comet\_elasticache) | ./modules/comet_elasticache | n/a |

|  [comet\_rds](#module\_comet\_rds) | ./modules/comet_rds | n/a |

|  [comet\_s3](#module\_comet\_s3) | ./modules/comet_s3 | n/a |

|  [comet\_vpc](#module\_comet\_vpc) | ./modules/comet_vpc | n/a |

## Resources

| Name | Type |

|------|------|

| [aws_eks_cluster_auth.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) | data source |

## Inputs

| Name | Description | Type | Default | Required |

|------|-------------|------|---------|:--------:|

|  [availability\_zones](#input\_availability\_zones) | List of availability zones from region | `list(string)` | `null` | no |

|  [comet\_ec2\_ami\_type](#input\_comet\_ec2\_ami\_type) | Operating system type for the EC2 instance AMI | `string` | `"ubuntu22"` | no |

|  [comet\_ec2\_instance\_count](#input\_comet\_ec2\_instance\_count) | Number of EC2 instances to provision | `number` | `1` | no |

|  [comet\_ec2\_instance\_type](#input\_comet\_ec2\_instance\_type) | Instance type for the EC2 instance | `string` | `"m5.4xlarge"` | no |

|  [comet\_ec2\_key](#input\_comet\_ec2\_key) | Name of the SSH key to configure on the EC2 instance | `string` | `null` | no |

|  [comet\_ec2\_volume\_size](#input\_comet\_ec2\_volume\_size) | Size, in gibibytes (GiB), for the EC2 instance root volume | `number` | `1024` | no |

|  [comet\_ec2\_volume\_type](#input\_comet\_ec2\_volume\_type) | EBS volume type for the EC2 instance root volume | `string` | `"gp2"` | no |

|  [comet\_private\_subnets](#input\_comet\_private\_subnets) | List of private subnets IDs from existing VPC to provision resources in | `list(string)` | `null` | no |

|  [comet\_public\_subnets](#input\_comet\_public\_subnets) | List of public subnets IDs from existing VPC to provision resources in | `list(string)` | `null` | no |

|  [comet\_vpc\_id](#input\_comet\_vpc\_id) | ID of an existing VPC to provision resources in | `string` | `null` | no |

|  [eks\_aws\_cloudwatch\_metrics](#input\_eks\_aws\_cloudwatch\_metrics) | Enables AWS Cloudwatch Metrics in the EKS cluster | `bool` | `true` | no |

|  [eks\_aws\_load\_balancer\_controller](#input\_eks\_aws\_load\_balancer\_controller) | Enables the AWS Load Balancer Controller in the EKS cluster | `bool` | `true` | no |

|  [eks\_cert\_manager](#input\_eks\_cert\_manager) | Enables cert-manager in the EKS cluster | `bool` | `false` | no |

|  [eks\_cluster\_name](#input\_eks\_cluster\_name) | Name for EKS cluster | `string` | `"comet-eks"` | no |

|  [eks\_cluster\_version](#input\_eks\_cluster\_version) | Kubernetes version of the EKS cluster | `string` | `"1.27"` | no |

|  [eks\_external\_dns](#input\_eks\_external\_dns) | Enables ExternalDNS in the EKS cluster | `bool` | `false` | no |

|  [eks\_external\_dns\_r53\_zones](#input\_eks\_external\_dns\_r53\_zones) | Route 53 zones for external-dns to have access to | `list(string)` | 
[
  "arn:aws:route53:::hostedzone/XYZ"
] | no |

|  [eks\_mng\_ami\_type](#input\_eks\_mng\_ami\_type) | AMI family to use for the EKS nodes | `string` | `"AL2_x86_64"` | no |

|  [eks\_mng\_desired\_size](#input\_eks\_mng\_desired\_size) | Desired number of nodes in EKS cluster | `number` | `3` | no |

|  [eks\_mng\_disk\_size](#input\_eks\_mng\_disk\_size) | Size of the storage disks for nodes in EKS cluster | `number` | `500` | no |

|  [eks\_mng\_max\_size](#input\_eks\_mng\_max\_size) | Maximum number of nodes in EKS cluster | `number` | `6` | no |

|  [eks\_mng\_name](#input\_eks\_mng\_name) | Name for the EKS managed nodegroup | `string` | `"mng"` | no |

|  [eks\_node\_types](#input\_eks\_node\_types) | Node instance types for EKS managed node group | `list(string)` | [
  "m5.4xlarge"
] | no |

|  [elasticache\_allow\_from\_sg](#input\_elasticache\_allow\_from\_sg) | Security group from which to allow connections to ElastiCache, to use when provisioning with existing compute | `string` | `null` | no |

|  [elasticache\_auth\_token](#input\_elasticache\_auth\_token) | Auth token for ElastiCache | `string` | `null` | no |

|  [elasticache\_engine](#input\_elasticache\_engine) | Engine type for ElastiCache cluster | `string` | `"redis"` | no |

|  [elasticache\_engine\_version](#input\_elasticache\_engine\_version) | Version number for ElastiCache engine | `string` | `"7.1.0"` | no |

|  [elasticache\_instance\_type](#input\_elasticache\_instance\_type) | ElastiCache instance type | `string` | `"cache.r4.xlarge"` | no |

|  [elasticache\_num\_cache\_nodes](#input\_elasticache\_num\_cache\_nodes) | Number of nodes in the ElastiCache cluster | `number` | `1` | no |

|  [elasticache\_param\_group\_name](#input\_elasticache\_param\_group\_name) | Name for the ElastiCache cluster parameter group | `string` | `"default.redis5.0"` | no |

|  [elasticache\_transit\_encryption](#input\_elasticache\_transit\_encryption) | Enable transit encryption for ElastiCache | `bool` | `false` | no |

|  [enable\_ec2](#input\_enable\_ec2) | Toggles the comet\_ec2 module, to provision EC2 resources for running Comet | `bool` | n/a | yes |

|  [enable\_ec2\_alb](#input\_enable\_ec2\_alb) | Toggles the comet\_ec2\_alb module, to provision an ALB in front of the EC2 instance | `bool` | n/a | yes |

|  [enable\_eks](#input\_enable\_eks) | Toggles the comet\_eks module, to provision EKS resources for running Comet | `bool` | n/a | yes |

|  [enable\_elasticache](#input\_enable\_elasticache) | Toggles the comet\_elasticache module for provisioning Comet Redis on elasticache | `bool` | n/a | yes |

|  [enable\_rds](#input\_enable\_rds) | Toggles the comet\_rds module for provisioning Comet RDS database | `bool` | n/a | yes |

|  [enable\_s3](#input\_enable\_s3) | Toggles the comet\_s3 module for provisioning Comet S3 bucket | `bool` | n/a | yes |

|  [enable\_vpc](#input\_enable\_vpc) | Toggles the comet\_vpc module, to provision a new VPC for hosting the Comet resources | `bool` | n/a | yes |

|  [environment](#input\_environment) | Deployment environment, i.e. dev/stage/prod, etc | `string` | `"dev"` | no |

|  [rds\_allow\_from\_sg](#input\_rds\_allow\_from\_sg) | Security group from which to allow connections to RDS, to use when provisioning with existing compute | `string` | `null` | no |

|  [rds\_backup\_retention\_period](#input\_rds\_backup\_retention\_period) | Days specified for RDS snapshotretention period | `number` | `7` | no |

|  [rds\_database\_name](#input\_rds\_database\_name) | Name for the application database in RDS | `string` | `"logger"` | no |

|  [rds\_engine](#input\_rds\_engine) | Engine type for RDS database | `string` | `"aurora-mysql"` | no |

|  [rds\_engine\_version](#input\_rds\_engine\_version) | Engine version number for RDS database | `string` | `"5.7"` | no |

|  [rds\_iam\_db\_auth](#input\_rds\_iam\_db\_auth) | Enables IAM auth for the database in RDS | `bool` | `true` | no |

|  [rds\_instance\_count](#input\_rds\_instance\_count) | Number of RDS instances in the database cluster | `number` | `2` | no |

|  [rds\_instance\_type](#input\_rds\_instance\_type) | Instance type for RDS database | `string` | `"db.r5.xlarge"` | no |

|  [rds\_preferred\_backup\_window](#input\_rds\_preferred\_backup\_window) | Backup window for RDS | `string` | `"07:00-09:00"` | no |

|  [rds\_root\_password](#input\_rds\_root\_password) | Root password for RDS database | `string` | n/a | yes |

|  [rds\_storage\_encrypted](#input\_rds\_storage\_encrypted) | Enables encryption for RDS storage | `bool` | `true` | no |

|  [region](#input\_region) | AWS region to provision resources in | `string` | n/a | yes |

|  [s3\_bucket\_name](#input\_s3\_bucket\_name) | Name for S3 bucket | `string` | n/a | yes |

|  [single\_nat\_gateway](#input\_single\_nat\_gateway) | Controls whether single NAT gateway used for all public subnets | `bool` | `true` | no |

|  [ssl\_certificate\_arn](#input\_ssl\_certificate\_arn) | ARN of the ACM certificate to use for the ALB | `string` | `null` | no |

## Outputs

| Name | Description |

|------|-------------|

|  [comet\_alb\_dns\_name](#output\_comet\_alb\_dns\_name) | DNS name of the ALB fronting the Comet EC2 instance |

|  [comet\_ec2\_instance](#output\_comet\_ec2\_instance) | ID of the Comet EC2 instance |

|  [comet\_ec2\_public\_ip](#output\_comet\_ec2\_public\_ip) | EIP associated with the Comet EC2 instance |

|  [configure\_kubectl](#output\_configure\_kubectl) | Configure kubectl: run the following command to update your kubeconfig with the newly provisioned cluster. |

|  [mysql\_host](#output\_mysql\_host) | Endpoint for the RDS instance |

|  [region](#output\_region) | Region resources are provisioned in |