{"id":18830656,"url":"https://github.com/materializeinc/terraform-aws-rds-privatelink","last_synced_at":"2025-08-30T17:35:49.584Z","repository":{"id":171031793,"uuid":"639931183","full_name":"MaterializeInc/terraform-aws-rds-privatelink","owner":"MaterializeInc","description":"A Terraform module for configuring RDS with AWS PrivateLink.","archived":false,"fork":false,"pushed_at":"2024-10-16T18:20:33.000Z","size":30,"stargazers_count":2,"open_issues_count":2,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-11-08T01:49:45.792Z","etag":null,"topics":["privatelink","rds","terraform","terraform-module"],"latest_commit_sha":null,"homepage":"https://registry.terraform.io/modules/MaterializeInc/rds-privatelink","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MaterializeInc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-12T14:53:42.000Z","updated_at":"2024-10-22T22:17:28.000Z","dependencies_parsed_at":"2024-10-18T08:46:27.846Z","dependency_job_id":"056a69e4-540a-4de1-9224-ff82803688cf","html_url":"https://github.com/MaterializeInc/terraform-aws-rds-privatelink","commit_stats":null,"previous_names":["materializeinc/terraform-aws-rds-privatelink"],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaterializeInc%2Fterraform-aws-rds-privatelink","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaterializeInc%2Fterraform-aws-rds-privatelink/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaterializeInc%2Fterraform-aws-rds-privatelink/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaterializeInc%2Fterraform-aws-rds-privatelink/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MaterializeInc","download_url":"https://codeload.github.com/MaterializeInc/terraform-aws-rds-privatelink/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":231822692,"owners_count":18431803,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["privatelink","rds","terraform","terraform-module"],"created_at":"2024-11-08T01:49:51.157Z","updated_at":"2025-08-30T17:35:49.565Z","avatar_url":"https://github.com/MaterializeInc.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Materialize + PrivateLink + RDS\n\n\u003e [!WARNING]\n\u003e This is provided on a best-effort basis and Materialize cannot offer support for this module\n\nThis repository contains a Terraform module that configures a PrivateLink endpoint for existing Amazon RDS PostgreSQL or MySQL databases to connect to Materialize.\n\nThe module creates the following resources:\n- Target group for each RDS instance\n- Network Load Balancer for the RDS instances\n- TCP listener for the NLB to forward traffic to the target groups\n- A VPC endpoint service for your RDS instances\n- Lambda Function to check and update the IP addresses of the RDS instances in the NLB target groups\n- IAM Role and Policy to give the Lambda function necessary permissions\n- Event Source Mapping, Event Rule, and Target: Triggers the Lambda function every _n_ minutes\n- Lambda Permission: Allows the event to invoke the Lambda function\n\n## Important Remarks\n\n\u003e [!NOTE]\n\u003e The RDS instances need to be private. If your RDS instances are public, there is no need to use PrivateLink.\n\n\u003e [!NOTE]\n\u003e When using Aurora, the RDS instance needs to be a **writer** instance as the reader instances will not work.\n\n- The RDS instances must be in the same VPC as the PrivateLink endpoint.\n- Review this module with your Cloud Security team to ensure that it meets your security requirements.\n- Finally, after the Terraform module has been applied, you will need to make sure that the Target Groups health checks are passing. As the NLB does not have security groups, you will need to make sure that the NLB is able to reach the RDS instances by allowing the subnet CIDR blocks in the security groups of the RDS instances.\n\n- Cross-region connections:\n    To connect to an AWS PrivateLink endpoint service in a different region to the one where your Materialize environment is deployed, you need to set the `mz_supported_regions` variable to include the region where the Materialize instance is deployed.\n    For same-region connections, you can leave the `mz_supported_regions` variable empty.\n\nTo override the default AWS provider variables, you can export the following environment variables:\n\n```bash\nexport AWS_PROFILE=\u003cyour_aws_profile\u003e # eg. default\nexport AWS_CONFIG_FILE=\u003cyour_aws_config_file\u003e # eg. [\"~/.aws/config\"]\nexport AWS_REGION=\u003cyour_aws_region\u003e # eg. us-east-1\n```\n\n## Usage\n\n### Variables\n\nStart by copying the `terraform.tfvars.example` file to `terraform.tfvars` and filling in the variables:\n\n```\ncp terraform.tfvars.example terraform.tfvars\n```\n\n| Name                        | Description | Type | Example | Required |\n|-----------------------------|-------------|:----:|:-----:|:-----:|\n| `mz_rds_instance_names`     | The name and listener port of the RDS instances | list | `{ name = \"instance1\", listener_port = 5001 }` | yes |\n| `mz_rds_vpc_id`             | The VPC ID of the RDS instance | string | `'vpc-1234567890abcdef0'` | yes |\n| `mz_acceptance_required`    | Whether or not to require manual acceptance of new connections | bool | `true` | no |\n| `schedule_expression`       | [The scheduling expression](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_rule#schedule_expression). For example, `cron(0 20 * * ? *)` | string | `'rate(5 minutes)'` | no |\n| `cross_zone_load_balancing` | Enables cross zone load balancing for the NLB | bool | `false` | no |\n| `mz_supported_regions`      | Only required for cross-region connections. The regions where the Materialize instance is deployed | list | `[\"us-east-1\"]` | no |\n\n### Apply the Terraform Module\n\n```\nterraform apply\n```\n\n### Output\n\nAfter the Terraform module has been applied, you will see the following output.\n\nYou can follow the instructions in the output to configure the PrivateLink endpoint and the database connections in Materialize.\n\nFirst, you will need to create the PrivateLink endpoint in Materialize:\n\n```sql\nmz_rds_private_link_endpoint_sql = \u003c\u003cEOT\n    -- Create the private link endpoint in Materialize\n    CREATE CONNECTION privatelink_svc TO AWS PRIVATELINK (\n        SERVICE NAME 'com.amazonaws.vpce.us-east-1.vpce-svc-1234567890abcdef0',\n        AVAILABILITY ZONES (\"use1-az1\", \"use1-az2\")\n    );\n\n    -- Get the allowed principals for the VPC endpoint service\n    SELECT principal\n        FROM mz_aws_privatelink_connections plc\n        JOIN mz_connections c ON plc.id = c.id\n        WHERE c.name = 'privatelink_svc';\n\nEOT\n```\n\nAfter that, you will need to create the database connections in Materialize. If you have multiple RDS instances, you will see multiple SQL statements:\n\n```sql\nmz_rds_database_connection_sql   = {\n    rds-instance-name = \u003c\u003c-EOT\n          -- Create a secret for the password for rds-instance-name\n              CREATE SECRET rds-instance-name_dbpass AS 'YOUR_DB_PASSWORD_FOR_rds-instance-name';\n              -- Create the connection to the RDS instance using the listener port\n              CREATE CONNECTION rds-instance-name_db_conn TO POSTGRES (\n                  HOST 'rds-instance-name.ctthmav6dsti.us-east-1.rds.amazonaws.com',\n                  PORT 5001,\n                  DATABASE postgres,\n                  USER postgres,\n                  PASSWORD SECRET rds-instance-name_dbpass,\n                  AWS PRIVATELINK privatelink_svc\n              );\n      EOT\n}\n```\n\nNote: For MySQL instances, replace `POSTGRES` with `MYSQL` in the connection creation SQL, and adjust the `DATABASE` and `USER` fields accordingly.\n\n### Output details: Configure Materialize\n\nOnce the Terraform module has been applied, you can configure Materialize to connect to the RDS instances using the PrivateLink endpoint:\n\n- Connect to the Materialize instance using `psql`\n- Run the SQL statement from the output of the `terraform apply` command to configure the PrivateLink connection, example:\n\n```sql\nCREATE CONNECTION privatelink_svc TO AWS PRIVATELINK (\n        SERVICE NAME 'com.amazonaws.vpce.us-east-1.vpce-svc-1234567890abcdef0',\n        AVAILABILITY ZONES (\"use1-az1\", \"use1-az2\")\n    );\n```\n\n\u003e Change the `privatelink_svc` to the name of the connection you want to use.\n\n- Get the allowed principals for the VPC endpoint service\n\n```sql\nSELECT principal\n    FROM mz_aws_privatelink_connections plc\n    JOIN mz_connections c ON plc.id = c.id\n    WHERE c.name = 'privatelink_svc';\n```\n\n- Add the allowed principals to the Endpoint Service configuration in the AWS console\n\n- Finally, run the last SQL statement from the output of the `terraform apply` command to create the database connection which will use the PrivateLink endpoint. If you have multiple RDS instances, you will see multiple SQL statements:\n\nFor PostgreSQL instances:\n\n```sql\n-- Create the connection to the PostgreSQL RDS instance\nCREATE CONNECTION pg_connection TO POSTGRES (\n    HOST 'instance.foo000.us-west-1.rds.amazonaws.com',\n    PORT 5432,\n    DATABASE postgres,\n    USER postgres,\n    PASSWORD SECRET pgpass,\n    AWS PRIVATELINK privatelink_svc\n);\n```\n\nFor MySQL instances:\n\n```sql\n-- Create the connection to the MySQL RDS instance\nCREATE CONNECTION mysql_connection TO MYSQL (\n    HOST 'mysql-instance.foo000.us-west-1.rds.amazonaws.com',\n    PORT 3306,\n    USER your_mysql_user,\n    PASSWORD SECRET mysql_dbpass,\n    AWS PRIVATELINK privatelink_svc\n);\n```\n\nAfter that go to your AWS console and check that the VPC endpoint service has a pending connection request from the Materialize instance which you can approve.\n\nAfter the connection request has been approved, you can create a database source in Materialize using the `pg_connection` or `mysql_connection` connection.\n\n## Materialize Documentation\n\nYou can also follow the [Materialize documentation](https://materialize.com/docs/ops/network-security/privatelink/) for more information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaterializeinc%2Fterraform-aws-rds-privatelink","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmaterializeinc%2Fterraform-aws-rds-privatelink","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaterializeinc%2Fterraform-aws-rds-privatelink/lists"}