{"id":15208877,"url":"https://github.com/kevinndungu-source/amazon_emr_serverless_demonstration","last_synced_at":"2026-01-19T22:31:38.488Z","repository":{"id":229475846,"uuid":"770395603","full_name":"kevinndungu-source/Amazon_EMR_Serverless_Demonstration","owner":"kevinndungu-source","description":"Explore the capabilities of Amazon EMR Serverless by processing semi-structured review data with Apache Spark, showcasing efficient big data analysis without managing clusters.","archived":false,"fork":false,"pushed_at":"2024-06-19T16:17:47.000Z","size":569,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-13T08:16:47.069Z","etag":null,"topics":["apache-spark","bigdatacloud","bigdatainfrastructure","dataprocessing","emrserverless","python","sql-query"],"latest_commit_sha":null,"homepage":"https://drive.google.com/file/d/16BrMDEi1JACxEJZfeDUAg90ulZR1V2Ol/view?usp=drive_link","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kevinndungu-source.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-11T13:34:24.000Z","updated_at":"2024-06-19T16:17:49.000Z","dependencies_parsed_at":"2024-06-20T04:21:49.596Z","dependency_job_id":"27692000-adcf-4462-bb0b-17a088d0ccc1","html_url":"https://github.com/kevinndungu-source/Amazon_EMR_Serverless_Demonstration","commit_stats":null,"previous_names":["kevinndungu-source/emr_serverless_demonstration_resources","kevinndungu-source/amazon_emr_serverless_demonstration_resources","kevinndungu-source/amazon_emr_serverless_demonstration"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kevinndungu-source%2FAmazon_EMR_Serverless_Demonstration","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kevinndungu-source%2FAmazon_EMR_Serverless_Demonstration/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kevinndungu-source%2FAmazon_EMR_Serverless_Demonstration/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kevinndungu-source%2FAmazon_EMR_Serverless_Demonstration/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kevinndungu-source","download_url":"https://codeload.github.com/kevinndungu-source/Amazon_EMR_Serverless_Demonstration/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247585789,"owners_count":20962400,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-spark","bigdatacloud","bigdatainfrastructure","dataprocessing","emrserverless","python","sql-query"],"created_at":"2024-09-28T07:02:55.105Z","updated_at":"2026-01-19T22:31:38.442Z","avatar_url":"https://github.com/kevinndungu-source.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Amazon Elastic Map Reduce (EMR) Serverless Demonstration\n\n---\n\n\u003cimg width=\"1180\" alt=\"Amazon-EMR-Serverless\" src=\"https://github.com/kevinndungu-source/EMR_Serverless_Demonstration_Resources/assets/114335263/0b67b0b1-eacc-4101-ba35-90e31b7d8fb9\"\u003e\n\n---\n\nThis project showcases the utilization of Amazon EMR Serverless for running a sample Spark job to process semi-structured review data. The goal is to demonstrate the capabilities of Amazon EMR Serverless in efficiently processing and analyzing big data workloads.\nOverview\nAmazon EMR (Elastic MapReduce) Serverless is a serverless big data processing service that enables you to run Apache Spark applications without managing clusters. In this demonstration, we leverage EMR Serverless to process semi-structured review data stored in JSON format and derive insights from the analysis.\n\n---\n\n## Project Structure\n**1.\tScripts:**\n - **reviews.py**: Python script for processing the review data.\n - **script_arguments**: Additional script arguments used during the EMR Serverless application setup.\n\n**2.\tSample Dataset:**\n - **dataset_en_dev.json**: Semi-structured review data in JSON format.\n\n---\n\n## How to Use\n**1.\tSetup Amazon EMR Serverless:**\n -\tConfigure an S3 bucket to store output files and logs.\n -\tCreate an IAM role with appropriate permissions for EMR Serverless.\n\n**2.\tRun Spark Job:**\n -\tExecute the sample Spark job using Amazon EMR Serverless.\n -\tProvide necessary script arguments during application setup.\n\n**3.\tAnalyze Data with Amazon Athena:**\n -\tLink Amazon Athena to the output folder in the S3 bucket containing processed Parquet data.\n -\tRun SQL queries in Amazon Athena to analyze the processed data and derive insights.\n\n---\n\n## Additional Resources\n1. For detailed documentation and insights, refer to this project's documentation document [link](https://drive.google.com/file/d/16BrMDEi1JACxEJZfeDUAg90ulZR1V2Ol/view?usp=drive_link).\n2.\tTo replicate the project or explore the code, refer to this GitHub repository code section.\n\n---\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkevinndungu-source%2Famazon_emr_serverless_demonstration","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkevinndungu-source%2Famazon_emr_serverless_demonstration","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkevinndungu-source%2Famazon_emr_serverless_demonstration/lists"}