{"id":19205354,"url":"https://github.com/evgeny-nik/project_aws_lambda_app","last_synced_at":"2026-05-07T14:33:52.937Z","repository":{"id":247957126,"uuid":"827231786","full_name":"Evgeny-Nik/project_aws_lambda_app","owner":"Evgeny-Nik","description":"a project to learn the usage of AWS Lambda and AWS API Gateway ","archived":false,"fork":false,"pushed_at":"2024-07-14T09:01:44.000Z","size":89,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-04T14:48:41.886Z","etag":null,"topics":["aws-api-gateway","aws-lambda","aws-s3-bucket","docker","flask-application","git-hook","github-actions","python3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Evgeny-Nik.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-11T08:46:42.000Z","updated_at":"2024-07-14T09:01:47.000Z","dependencies_parsed_at":"2024-11-09T13:12:39.556Z","dependency_job_id":"7deedd24-12a7-4a37-8073-7d5657e70b82","html_url":"https://github.com/Evgeny-Nik/project_aws_lambda_app","commit_stats":null,"previous_names":["evgeny-nik/project_aws_lambda_app"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Evgeny-Nik%2Fproject_aws_lambda_app","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Evgeny-Nik%2Fproject_aws_lambda_app/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Evgeny-Nik%2Fproject_aws_lambda_app/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Evgeny-Nik%2Fproject_aws_lambda_app/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Evgeny-Nik","download_url":"https://codeload.github.com/Evgeny-Nik/project_aws_lambda_app/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240277266,"owners_count":19775859,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws-api-gateway","aws-lambda","aws-s3-bucket","docker","flask-application","git-hook","github-actions","python3"],"created_at":"2024-11-09T13:12:27.692Z","updated_at":"2026-05-07T14:33:47.912Z","avatar_url":"https://github.com/Evgeny-Nik.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Flask App Interaction with AWS Lambda using AWS API Gateway REST API\n\n## Project Overview\n\nThis project demonstrates a CI/CD (Continuous Integration/Continuous Delivery) pipeline using GitHub Actions, Docker, and the integration of a Flask App with AWS Lambda and AWS API Gateway.\n\n## Application\n\nThis project uses a Flask web application that communicates with AWS Lambda functions via HTTP requests to a REST API on AWS API Gateway, using Gunicorn as a WSGI server.\n\n## Workflow Overview\n\nThere are two workflows:\n1. To handle the CI/CD cycle of the web_app.\n2. To handle the CI/CD cycle of the Lambda functions.\n\n### 1. CI/CD for Flask App\n\nThis workflow builds and deploys a Flask application that communicates with AWS Lambda functions. It includes:\n\n- **Build and Tag Docker Image**: Builds a Docker image and tags it with the current version and latest.\n- **Login to Docker Hub**: Logs into Docker Hub using provided credentials.\n- **Push Docker Images**: Pushes the tagged Docker images to Docker Hub.\n\n#### Trigger\n\nThis workflow is triggered on a push event to the `master` branch in the following paths:\n```yaml\non:\n  push:\n    branches:\n      - 'master'\n    paths:\n      - 'web_app/**'\n      - '.github/workflows/web_app_on_push_workflow.yaml'\n      - '!**/README.md'\n```\nIt is also integrated with a git post-commit hook to automatically bump the version of the app on each commit.\n\n### 2. CI/CD for Lambda Functions\n\nThis workflow zips and uploads Lambda functions to S3 and updates them. It includes:\n\n- **List and Filter Changed Python Files**: Identifies and lists changed Python files.\n- **Run Tests on Changed Files**: Runs tests on the identified changed files.\n- **Zip and Upload Lambda Function**: Zips the changed files, uploads them to S3, and updates the corresponding Lambda functions.\n\n#### Trigger\n\nThis workflow is triggered on a push event to the `master` branch in the following paths:\n```yaml\non:\n  push:\n    branches:\n      - 'master'\n    paths:\n      - '**/lambda_function.py'\n      - '.github/workflows/lambdas_on_push_workflow.yaml'\n      - '!**/README.md'\n```\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003ch2\u003eReproducing the Project\u003c/h2\u003e\u003c/summary\u003e\n\n## Lambda Functions:\n\n### Overview\n\n- **lambda-Backup**\n  - `lambda_function.py`: Runs on an EventBridge cron job to back up the S3 bucket containing the Lambda functions daily, weekly, and monthly.\n\n| Required Environment Variables | Description                         |\n|--------------------------------|-------------------------------------|\n| `SOURCE_BUCKET`                | Name of Lambda functions' S3 Bucket |\n| `BACKUP_BUCKET`                | Name of backup S3 Bucket            |\n\n| Required Trigger | Type    |\n|------------------|---------|\n| `EventBridge`    | Daily   |\n| `EventBridge`    | Weekly  |\n| `EventBridge`    | Monthly |\n\n- **csv_to_excel**\n  - `lambda_function.py`: Converts CSV files to Excel format.\n\n- **lambda_discord_msg**\n  - `lambda_function.py`: Sends messages to a Discord channel via a webhook URL.\n\n| Required Environment Variables | Description         |\n|--------------------------------|---------------------|\n| `DISCORD_URL`                  | Discord Webhook URL |\n\n- **gitlab_create_user**\n  - `lambda_function.py`: Uses a Google Sheet to obtain user credentials, creates a new user, group, and project in GitLab, and adds the user to the new group as a reporter, using the GitLab API.\n\n| Required Environment Variables | Description                                      |\n|--------------------------------|--------------------------------------------------|\n| `GITLAB_TOKEN`                 | GitLab PAT to manage users, groups, and projects |\n| `GITLAB_INSTANCE_ID`           | Your AWS instance ID, used to retrieve public IP |\n| `GITLAB_GROUP_NAME`            | Generic String                                   |\n| `GITLAB_GROUP_DESC`            | Generic String                                   |\n\n| Required Google Sheets fields |\n|-------------------------------|\n| `email`                       |\n| `password`                    |\n| `username`                    |\n| `name`                        |\n\n- **gitlab_new_project**\n  - `lambda_function.py`: Creates a Python script from a template, uploads it to S3, and provides the user with the download link. The script then creates a new directory with a blank file of the user's choosing, a new project in GitLab using the GitLab API, and pushes the file to the newly created project.\n\n| Required Environment Variables | Description                                       |\n|--------------------------------|---------------------------------------------------|\n| `BUCKET_NAME`                  | AWS S3 bucket that stores executable script       |\n| `REGION`                       | AWS region where your S3 bucket is located        |\n| `GITLAB_TOKEN`                 | GitLab PAT to create a project                    |\n| `GITLAB_INSTANCE_ID`           | Your AWS instance ID, used to retrieve public IP  |\n| `GITLAB_USER`                  | Username that will own the project                |\n\n- **wikipedia_func**\n  - `lambda_function.py`: Fetches the selected subject's top Wikipedia pages and stores the information in an S3 file for history.\n\n| Required Environment Variables | Description                                   |\n|--------------------------------|-----------------------------------------------|\n| `BUCKET_NAME`                  | AWS S3 bucket that stores wiki search history |\n| `FILE_KEY`                     | Key to history file                           |\n\n### Setting Up AWS Lambda Function\n\n1. **Create a Lambda Function**:\n   - Navigate to Lambda in the AWS Management Console.\n   - Click \"Create Function\" and choose \"Author from scratch\".\n   - Enter the function name and select the default runtime (e.g., Python 3.12).\n2. **Set Environment Variables**:\n   - In the Configuration tab, select \"Environment variables\".\n   - Add the required key-value pairs.\n3. **Set Permissions**:\n   - In the Configuration tab, select \"Permissions\".\n   - Ensure the Lambda function has the necessary IAM role with permissions to execute and log to CloudWatch. Remember to grant least privileged permissions.\n4. **Set Timeout**:\n   - In the Configuration tab, select \"General configuration\".\n   - Set the timeout so the Lambda function has enough time to execute.\n\n### Setting Up AWS API Gateway\n\n1. **Create an API**:\n   - Navigate to API Gateway in the AWS Management Console.\n   - Click \"Create API\" and select \"REST API\".\n2. **Create a Resource**:\n   - Define the paths for your resources.\n3. **Create a Method**:\n   - Choose an HTTP method (e.g., GET, POST) and set it up with the appropriate integration request/response configurations.\n4. **Integrate with Lambda**:\n   - In the Integration section, select \"Lambda Function\" and choose your Lambda function.\n5. **Set up Binary Types**:\n   - Configure the API Settings \u003e Binary media types as needed.\n6. **Deploy API**:\n   - Deploy the API to the stage of your choosing and get the URLs you need for the `.env` file.\n\n### Flask Application\n\n**web_app**\n  - `app.py`: A simple Flask web application demonstrating AWS Lambda integration using AWS API Gateway REST API.\n\n## Web_App Dockerfile\n\n- **Base Image**: Uses `python:alpine3.19` for a lightweight Python environment.\n- **Work Directory**: Sets the working directory to `/home/lambda_app_user`.\n- **Install Dependencies**: Copies `requirements.txt` and installs Python dependencies without cache.\n- **Copy Source Code**: Copies the source code to the working directory.\n- **Run Application**: Uses Gunicorn to run the Flask application with 4 workers, binding to port 8000.\n\n## Setup\n\n1. **Clone the Repository**:\n    ```bash\n    git clone https://github.com/Evgeny-Nik/project_aws_lambda_app.git\n    cd project_aws_lambda_app\n    ```\n2. **Setup the .env file**:\n    ```bash\n    touch web_app/.env\n    ```\n\n#### Environment Variables Example (.env-example)\n\nThe `.env` file in your `web_app` directory should have the following example values:\n\n```\nLAMBDA_CSV=\"\u003cAWS_API_Gateway_Link_goes_here\u003e/csv\"\nLAMBDA_DISCORD=\"\u003cAWS_API_Gateway_Link_goes_here\u003e/discord\"\nLAMBDA_GITLAB_USER=\"\u003cAWS_API_Gateway_Link_goes_here\u003e/gitlab_create_user\"\nLAMBDA_GITLAB_PROJECT=\"\u003cAWS_API_Gateway_Link_goes_here\u003e/gitlab_create_project\"\nLAMBDA_WIKI=\"\u003cAWS_API_Gateway_Link_goes_here\u003e/wiki\"\n```\n\n3. **Trigger the workflow to build the app**:\n   - Push changes to the `master` branch.\n   - See triggers [here](#trigger).\n\n4. **Deploy the app to the environment of your choosing**:\n   - To run the web app locally:\n     ```bash\n     docker run ${DOCKERHUB_USERNAME}/github_app:latest\n     ```\n\n\u003c/details\u003e\n\n### GitHub Actions Plugins Used\n\n- **actions/checkout@v3**: Checks out the repository to the runner.\n- **docker/login-action@v3**: Logs into Docker Hub.\n- **docker/build-push-action@v3**: Builds and pushes Docker images.\n- **aws-actions/configure-aws-credentials@v2**: Configures AWS credentials.\n- **actions/upload-artifact@v2**: Uploads artifacts to be shared between jobs.\n- **actions/download-artifact@v2**: Downloads artifacts generated by other jobs.\n\n## To-Do List\n\n- [ ] Create manifest files.\n- [ ] Set up deployment stage to deploy to Kubernetes (Helm/ArgoCD or both).\n- [ ] Integrate HTTPS support for the web app via Let's Encrypt.\n\n## Links\n- [csv_to_excel README](csv_to_excel/README.md)\n- [gitlab_user_create README](gitlab_user_create/README.md)\n- [gitlab_new_project](gitlab_new_project/README.md)\n- [lambda_discord_msg README](lambda_discord_msg/README.md)\n- [lambda-Backup](lambda-Backup/README.md)\n- [wikipedia_func](wikipedia_func/README.md)\n- [web_app README](web_app/README.md)\n- [DockerHub Project Registry](https://hub.docker.com/repository/docker/evgenyniko/lambda_app)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fevgeny-nik%2Fproject_aws_lambda_app","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fevgeny-nik%2Fproject_aws_lambda_app","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fevgeny-nik%2Fproject_aws_lambda_app/lists"}