{"id":25115766,"url":"https://github.com/devops-rob/aws-lambda-in-vpc","last_synced_at":"2025-04-02T11:41:31.840Z","repository":{"id":110462510,"uuid":"180244861","full_name":"devops-rob/AWS-lambda-in-vpc","owner":"devops-rob","description":"Tutorial - running Lambda functions inside a VPC with internet access","archived":false,"fork":false,"pushed_at":"2019-04-12T08:57:11.000Z","size":8583,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-08T02:35:31.286Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/devops-rob.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-04-08T22:52:33.000Z","updated_at":"2019-09-19T13:26:31.000Z","dependencies_parsed_at":"2023-04-30T19:01:28.888Z","dependency_job_id":null,"html_url":"https://github.com/devops-rob/AWS-lambda-in-vpc","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devops-rob%2FAWS-lambda-in-vpc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devops-rob%2FAWS-lambda-in-vpc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devops-rob%2FAWS-lambda-in-vpc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devops-rob%2FAWS-lambda-in-vpc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/devops-rob","download_url":"https://codeload.github.com/devops-rob/AWS-lambda-in-vpc/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246811194,"owners_count":20837745,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-08T02:34:06.914Z","updated_at":"2025-04-02T11:41:31.389Z","avatar_url":"https://github.com/devops-rob.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Lambda in VPC\n\n## What are lambdas\n\nAWS Lambda enables you to execute your code without having to run a server.  To put in simply, it's a serverless technology.  This doesn't mean there aren't any servers involved in the setup.  Instead, AWS manage the server workloads so you don't have to.\n\nSome of the main benefits of using lambdas are:\n\n1. Less support overheads - as the server workloads are not your responsibility, you don't have to worry about patching them or the general health of the servers (high memory or high CPU utilization or storage usage).  You can just trust that your lambdas will always have a stable server to run on.\n2. Cost of running vs Always on VMs - With EC2 instances running a web application, they are generally always on so as to make the web application available at all times.  EC2 instances charge for every second that the VM is running, meaning your potentially could have quite a large bill come the end of the month for your server work loads.  You will be charged regardless of whether anyone is using your application or not.  There are also other costs to consider such as storage costs and static IP addresses.  Lambdas only charge you for calls to the lambda which run for more than 3 seconds.  This means that if your application is made up of small short running functions, lambdas could significantly reduce your operational costs.\n\n## What are VPCs\n\nA VPC (Virtual private cloud), is a virtual private network in AWS.  It works in the same way as any other network, allowing you to subnet, create VPNs, configure routing via Route Tables and manage security of the network using ACLs (Access Control List) and Security Groups.  You can also implement NAT (Network Address Translation) for your private subnets\n\n## Use cases for using Lambdas inside of VPCs / Lambda default behaviour\n\nBy default, lambdas have internet access allowing them to communicate with any public resources.  This works well for many use cases but there are times when you will need your lambda to access resources inside your VPC.  An example of this is would be a requirement to access a VPN tunnel inside a VPC.  \n\nYou would need to add the VPC configuration to the lambda in order for this to work. By adding the VPC configuration to the lambda, it will loose its internet connectvity that lambdas usually have by default\n\n## Demo\n\nIn this demo, we will look at how to configure a lambda to work within a VPC, as well as grant it internet access to allow it to access resources outside of the VPC.\n\n## Architecture overview and explanantion\n\nTo implement this demo, we will require a:\n\n  1. VPC\n  2. Public subnet and a private subnet inside the VPC\n  3. NAT gateway\n  4. Internet gateway\n  5. Elastic IP\n  6. Route table for each of the subnets\n  7. Security group x 2\n  8. Lambda function (a simple helloworld sample function generated from SAM template will suffice)\n\nThe VPC will consists of 2 subnets, one public and one private with an internet gateway associated with the VPC.  \n\nThe public subnet will have a NAT gateway associated with it and the route table for this subnet will have a default route pointing at the internet gateway.\n\nThe private subnet's route table will have a default route pointing to the NAT gateway that is associated with the public subnet.\n\nFor simplicity, we will use the following subnetting CIDR blocks for the different networks:\n\n* **VPC** - 10.150.0.0/16\n* **Public subnet** - 10.150.1.0/24\n* **Private subnet** - 10.150.2.0/24\n\nAgain, for simplicity sake, we will create two security groups which allows all traffic via all ports\n\nThe diagram below is a visual representation of the architecture\n\n![Architecture Diagram](images/network_architecture.png \"Architecture Diagram\")\n\n## Method Statement\n\nTo follow along this tutorial, you will need a few pre-requisites:\n\n  1. AWS CLI\n  2. AWS SAM\n  3. dotnetcore 2.1\n  4. jq\n  \nWe will be using AWS CloudFormation code that I have written to deploy the network infrastructure required for this demo.  This will require you to have AWS configure setup with your access key and secret key.  If you do not have this set up, log in to your AWS console and generate a key pair for your user in the IAM section.\n\nThe following instrutions assume you are in the src folder.\n\n### Step 1\n\nThe first thing we need to do is create an s3 bucket for our cloudformation scripts.  Run the following line of code subsituting the bucket name and region for one of your choosing:\n\n```bash\naws s3api create-bucket \\\n--bucket \u003cinsert unique bucket name here\u003e --region \u003cinsert region here\u003e \\\n--create-bucket-configuration LocationConstraint=\u003cinsert region here\u003e \\\n--acl private\n```\n\nNow that the s3 bucket has been created, we need to apply a policy to the bucket to control access to it.  We'll do this using a policy document which is a json file.  You will need to edit the json file with your S3 bucket name or create your own policy document for your needs.  Once edited, run the following command\n\n```bash\naws s3api put-bucket-policy \\\n--bucket \u003cinsert unique bucket name here\u003e \\\n--policy file://iam_policy.json\n```\n\n### Step 2\n\nWe now need to package our code and push it to the s3 bucket we have just created.  To do this we will run the following line of code:\n\n```bash\naws cloudformation package --template-file infrastructure.yml \\\n--s3-bucket \u003cinsert unique bucket name here\u003e \\\n--output-template-file output.yml \u003e packaged-template.yml\n```\n\nThis step is essentially building our code into an artifact that is then pushed to our artifact repository, which in this case is the S3 bucket we have just created.  You will see two new files created:\n\n1. output.yml\n2. packaged-template.yml\n\n### Step 3\n\nThe next step is to deploy the core infrastructure that will be required to demonstrate lambdas working in a VPC.  We will deploy this using the output.yaml file generated from the previous step.  Run the following command changing the path to your output.yml\n\n```bash\naws cloudformation deploy \\\n--template-file /path/to/output.yml \\\n--stack-name DevOpsRob-infrastructure\n```\n\nThis step has deployed the key infrastructure components needed for the purposes of this demo.  There are some outputs from this cloudformation template that we will need for the lambda to work inside of the VPC we have just created which are:\n\n  1. PrivateSubnetId\n  2. SecurityGroupAId\n  3. SecurityGroupBId\n\n  You will also need to make a note of the s3 bucket name created in this code.  You can obtain that from the outputs tab in the cloudformation section of the AWS console.  The output you are looking for is S3BucketId.  \n  **Please note that the bucket name is different to the one you created in step 1.**\n\n### Step 4\n\nNow that we have our core infrastructure deployed, we can now start working with the lambdas.  We will generate a helloworld lambda in C# using the SAM cli tool by running the following command:\n\n```bash\nsam init --runtime dotnet\n```\n\nThis will create a project structure in a folder called sam-app.  This is a simple Helloworld application that also gives us the IP address that the lambda is using by connecting to an external service outside of the VPC.\n\n### Step 5\n\nWe now need to add the VPC configuration to the SAM template that has been created in the previous command.  In the template.yaml file, there is one resouce called HelloWorldFunction.  This is the lambda function that we will add the VPC configuration to.  We will also add some permissions to allow the lambda function to create Network interfaces to enable connectivity.  Add the following block of code to the function underneath the Events block of code but in line with it.  Yaml is space and tab sensitive so it's important it is in line with the Events block of code.\n\n```yml\nVpcConfig:\n    SubnetIds:\n    - !ImportValue PrivateSubnetId\n    SecurityGroupIds:\n    - !ImportValue SecurityGroupAId\n    - !ImportValue SecurityGroupBId\nPolicies:\n    - Statement:\n        Action:\n        - ec2:CreateNetworkInterface\n        - ec2:DeleteNetworkInterface\n        - ec2:DescribeInstances\n        - ec2:AttachNetworkInterface\n        - ec2:DescribeNetworkInterfaces\n        - autoscaling:CompleteLifecycleAction\n        Effect: \"Allow\"\n        Resource: \"*\"\n```\n\nDon't forget to save this file.\n\n### Step 6\n\nWe need to build our function code to prepare it for packaging.  Inside the sam-app folder, there is a shell script called build.sh.  We firstly need to move into the sam-app directory and ensure we have the permissions to run the build script by running the following command:\n\n```bash\ncd sam-app \u0026\u0026 chmod 755 build.sh\n```\n\nThen run the script by entering the following on the commandline:\n\n```bash\n./build.sh\n```\n\nWhen this has finished building you will see a similar output on the commandline to the following picture:\n\n![Build Output](images/build-output.png \"Build Output\")\n\n### Step 7\n\nThe next step is to package our newly edited SAM template in a similar way to what we did in step 2.  Run the following command\n\n```bash\nsam package --output-template-file packaged.yaml \\\n--s3-bucket \u003cinsert s3 bucket name here\u003e\n```\n\nThe s3 bucket specifed in the above command is the bucket name you noted down in step3.\n\n### Step 8\n\nThe final step is to deploy our lambda by running the following command:\n\n```bash\naws cloudformation deploy \\\n--template-file /path/to/packaged.yaml \\\n--stack-name helloworld \\\n--capabilities CAPABILITY_IAM\n```\n\n## Testing\n\nWith our lambda function now sucessfully deployed, the time has come to test it to make sure it is part of the VPC but can still access resources outside of the VPC.  \n\nWe will start by examining the c# code for the helloworld program that was generated to get an understanding of what it is doing.  Lets look in the program.cs file that is located in src \u003e example-sam-app \u003e src.  \n\nIn this file, we have a class called Function which does two things:\n\n1. Makes a call to \u003chttp://checkip.amazonaws.com\u003e to get the IP address of the function.  The endpoint being called is an external service.\n2. Prints \"hello world\" along with the IP address obtained from the above method.\n\nWe now know that the lambda will make a call to an endpoint which is not inside the VPC to obtain the IP address.  We also know that the Elastic IP address that was provisioned in the CloudFormation code is associated with the NAT gateway which is inside the VPC.\n\nTo test the lambda, run the following block of code:\n\n```bash\ngwid=$(aws apigateway get-rest-apis | jq -r '.items[] | .id')\nregion=\"eu-west-1\"\n\nendpoint=\"https://${gwid}.execute-api.${region}.amazonaws.com/Prod/hello\"\n\ncurl $endpoint\n```\n\nor alternatively you can run this script in the prepared bash script with the following commands:\n\n```bash\ncd src/tests/ \u0026\u0026 chmod 755 lambda-test.sh \u0026\u0026 ./lambda-test.sh\n```\n\n**It's worth pointing out that there is a cold start situation whereby a lambda call that hasn't been made in the last 45 minutes or so can take around 30 seconds to complete though this is beyond the scope of this tutorial.**\n\nRunning this code should return a json response like below:\n\n```json\n{\"message\":\"hello world\",\"location\":\"ELASTIC_IP_ADDRESS\"}\n```\n\nCheck the IP address in this reponse matches the Elastic IP provisoned in the CloudFormation code.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevops-rob%2Faws-lambda-in-vpc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdevops-rob%2Faws-lambda-in-vpc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevops-rob%2Faws-lambda-in-vpc/lists"}