{"id":13821363,"url":"https://github.com/jmfuchs/aws-guardduty-processing","last_synced_at":"2025-05-16T12:33:15.355Z","repository":{"id":217089151,"uuid":"135103969","full_name":"jmfuchs/aws-guardduty-processing","owner":"jmfuchs","description":"A serverless application for processing AWS GuardDuty findings.","archived":false,"fork":false,"pushed_at":"2018-09-07T15:53:47.000Z","size":237,"stargazers_count":11,"open_issues_count":0,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-11-19T21:36:03.112Z","etag":null,"topics":["aws","elasticsearch","guardduty","lambda","logging","serverless-framework"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jmfuchs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-05-28T03:13:01.000Z","updated_at":"2021-03-15T00:32:26.000Z","dependencies_parsed_at":"2024-01-15T16:27:52.197Z","dependency_job_id":"cbac966b-1d41-4e1a-865d-6af7113c3709","html_url":"https://github.com/jmfuchs/aws-guardduty-processing","commit_stats":null,"previous_names":["jmfuchs/aws-guardduty-processing"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmfuchs%2Faws-guardduty-processing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmfuchs%2Faws-guardduty-processing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmfuchs%2Faws-guardduty-processing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmfuchs%2Faws-guardduty-processing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jmfuchs","download_url":"https://codeload.github.com/jmfuchs/aws-guardduty-processing/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254530644,"owners_count":22086651,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","elasticsearch","guardduty","lambda","logging","serverless-framework"],"created_at":"2024-08-04T08:01:20.756Z","updated_at":"2025-05-16T12:33:12.241Z","avatar_url":"https://github.com/jmfuchs.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Processing Guardduty Findings with Lambda\n\nThis serverless application creates the necessary resources and integrations for processing [Amazon GuardDuty](https://aws.amazon.com/guardduty/) findings. The below architecture showcases how logs are post-processed, and pushed to Elasticsearch.\n\n## Architecture\n\n![Log-Architecture](images/aws-gd-processing-arch.png)\n\n## Prerequisites\n\nBelow are the necessary prerequisites:\n\n*\t[AWS Account](https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/)\n*\t[Serverless Framework](https://serverless.com/)\n*\tAn [Amazon Elasticsearch](https://aws.amazon.com/elasticsearch-service/) cluster.  This application builds on [aws-cloudtrail-processing](https://github.com/jmfuchs/aws-cloudtrail-processing) in an effort to consolidate more AWS native log sources but can also be run independently.\n\n### Cloud9 Environment\n\nIf you have trouble installing any of the prerequisites or dependencies, you can spin up an [AWS Cloud9](https://aws.amazon.com/cloud9/) environment, which is a cloud-based IDE that comes prepackaged with a number of essential packages.  After which you can run the following command to install the Serverless Framework.\n\n```\nnpm install -g serverless\n```\n\n## Set Variables\n\nClone the repo and open up **environment/dev.yml** and add in the appropiate variables. \n\n\u003e  You can add new files for different environments.\n\n## Enrich Logs\n\nSince the Lambda Function parses through the GuardDuty finding, you can enrich the finding by adding additional information to complement data points and add context.  This can either be static data hardcoded in the Lambda function or it can be dynamically pulled from a DynamoDB table or 3rd party threat intelligence feed. \n\nOpen **guardduty.py**, find the section of code below, and add additional metadata as necessary:\n\n```\n############# Add additional metadata to event #############\n            \n# Example: Add AWS Account type\ni[\"accountType\"] = \"Production\" \n\n############################################################\n``` \n\n\n## Install Dependencies\n\nEnsure you are in the aws-ct-processing directory and run the following to install the dependencies:\n\n```\npip install -r requirements.txt -t ./\n```\n\n## Deploy\n\nTo deploy the serverless application, run the following command:\n\n```\nsls deploy -s dev -r us-west-2\n```\n\n\u003e If you've created different environment files that reference other aws profiles or you want to deploy to different regions, you can replace *dev* and *us-west-2* as necessary.\n\n## Access\n\nAfter to the application has been successfully deployed you can view the logs in Kibana by doing the following:\n\n* Go to the [AWS Elasticsearch](https://us-west-2.console.aws.amazon.com/es/home?region=us-west-2) console.\n* Click on the Domain that starts with **sls-aws-ct-processing**.\n* Click the link next to **Kibana**.\n\nOnce you are in Kibana:\n\n* Click **Management** in the left Navigation.\n* Click **Index Patterns**\n* For Step 1 (Define index pattern) type **logs-\\***.\n* For Step 2 (Configure Settings) select **@timestamp**.\n* Click **Create Index Pattern**.\n* On the left navigation, click **Discover** to view your events.\n\n## Cleanup\n\nBelow are the steps to cleanup this application:\n\n* \tRun the below command to delete the serverless application\n\n\t```\n\tsls remove  -s dev -r us-west-2\n\t```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjmfuchs%2Faws-guardduty-processing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjmfuchs%2Faws-guardduty-processing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjmfuchs%2Faws-guardduty-processing/lists"}