https://github.com/chrisammon3000/serverless-streaming-reddit-pipeline
Serverless big data streaming application using a fan-out architecture with Lambda and SQS to fetch data from Reddit forums, stream to Kinesis Firehose and store in an S3 data lake.
https://github.com/chrisammon3000/serverless-streaming-reddit-pipeline
athena aws aws-cloudformation aws-lambda kinesis-firehose reddit-api serverless sqs
Last synced: 7 months ago
JSON representation
Serverless big data streaming application using a fan-out architecture with Lambda and SQS to fetch data from Reddit forums, stream to Kinesis Firehose and store in an S3 data lake.
- Host: GitHub
- URL: https://github.com/chrisammon3000/serverless-streaming-reddit-pipeline
- Owner: chrisammon3000
- Created: 2020-10-14T16:21:05.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2023-07-11T23:17:43.000Z (about 2 years ago)
- Last Synced: 2025-01-22T16:14:36.160Z (9 months ago)
- Topics: athena, aws, aws-cloudformation, aws-lambda, kinesis-firehose, reddit-api, serverless, sqs
- Language: Python
- Homepage:
- Size: 994 KB
- Stars: 3
- Watchers: 1
- Forks: 0
- Open Issues: 0