{"id":14065633,"url":"https://github.com/sahays/serverless-analytics","last_synced_at":"2025-07-29T21:32:35.492Z","repository":{"id":123688687,"uuid":"158275443","full_name":"sahays/serverless-analytics","owner":"sahays","description":"AWS Serverless Analytics using Amazon S3, Athena, Glue, and QuickSight","archived":false,"fork":false,"pushed_at":"2020-02-06T03:10:45.000Z","size":16880,"stargazers_count":5,"open_issues_count":0,"forks_count":5,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-08-14T07:09:20.325Z","etag":null,"topics":["athena","aws-cli","data-catalog","dataset","glue","quicksight","transform-data","visualization"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sahays.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2018-11-19T18:58:53.000Z","updated_at":"2023-12-13T00:54:06.000Z","dependencies_parsed_at":null,"dependency_job_id":"5cb49eaf-c9c2-4725-ab9a-70663672a703","html_url":"https://github.com/sahays/serverless-analytics","commit_stats":{"total_commits":30,"total_committers":1,"mean_commits":30.0,"dds":0.0,"last_synced_commit":"794d6e496a76d5ea8107ca60d94d41b7becc536d"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sahays%2Fserverless-analytics","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sahays%2Fserverless-analytics/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sahays%2Fserverless-analytics/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sahays%2Fserverless-analytics/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sahays","download_url":"https://codeload.github.com/sahays/serverless-analytics/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228052501,"owners_count":17862102,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["athena","aws-cli","data-catalog","dataset","glue","quicksight","transform-data","visualization"],"created_at":"2024-08-13T07:04:36.337Z","updated_at":"2024-12-04T05:30:34.878Z","avatar_url":"https://github.com/sahays.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Overview\nCome and learn how to leverage Amazon S3, Amazon Glue, Amazon Athena and Amazon QuickSight to upload a dataset to your central data lake, automate the creation of a data catalog, transform data to a compressed columnar format that allows to speed up and reduce the cost of your processes, query the data using standard SQL and create and share rich web-based visualizations. Do this all without having to manage clusters or having to spin up a single instance.\n\nhttps://reinvent.awsevents.com/learn/builders-sessions/\n\nBuilders Sessions are 60-minute small groups sessions with up to five customers and one AWS expert, who is there to help, answer questions, and provide guidance. It’s just you, your laptop, and the AWS expert.\n\nEach builders session begins with a short explanation or demonstration of what you are going to build. There will not be any formal presentation. Once the demonstration is complete, you will use your laptop to experiment and build with the AWS expert.\n\n# Architecture\n![Architecture](screenshots/arch.png)\n\n# Walkthrough\n## [Step 1: Upload a dataset to Amazon S3](step-one.md)\n## [Step 2: Automate the creation of a Data Catalog](step-two.md)\n## [Step 3: Transform data into a compressed columnar format](step-three.md)\n## [Step 4: Query the data using standard SQL](step-four.md)\n## [Step 5: Create and Share rich web-based visualizations](step-five.md)\n\n\n# Appendix:\nInstalling the AWS CLI\nhttps://docs.aws.amazon.com/cli/latest/userguide/installing.html\n\nI prefer to use Homebrew for a Mac\nhttps://github.com/aws/aws-cli/issues/727\n\nConfiguring the AWS CLI\nhttps://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html\n\nCopy objects between S3 buckets\nhttps://aws.amazon.com/premiumsupport/knowledge-center/move-objects-s3-bucket/\n\nAWS Glue Built-in classifiers\nhttps://docs.aws.amazon.com/glue/latest/dg/add-classifier.html#classifier-built-in\n\nParquet data format\nhttps://parquet.apache.org/documentation/latest/\n\nSetting up QuickSight\nhttps://docs.aws.amazon.com/quicksight/latest/user/signing-in.html\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsahays%2Fserverless-analytics","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsahays%2Fserverless-analytics","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsahays%2Fserverless-analytics/lists"}