{"id":24750634,"url":"https://github.com/fabioba/sparkify-data-pipeline","last_synced_at":"2026-04-15T08:38:39.895Z","repository":{"id":63583248,"uuid":"567329070","full_name":"fabioba/sparkify-data-pipeline","owner":"fabioba","description":"This project is the final project of the the Data Pipeline - Udacity module.","archived":false,"fork":false,"pushed_at":"2022-11-22T10:18:50.000Z","size":246,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-23T03:41:29.893Z","etag":null,"topics":["airflow","aws","docker-compose"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fabioba.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-11-17T15:01:37.000Z","updated_at":"2022-11-22T21:24:33.000Z","dependencies_parsed_at":"2023-01-23T09:01:07.217Z","dependency_job_id":null,"html_url":"https://github.com/fabioba/sparkify-data-pipeline","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/fabioba/sparkify-data-pipeline","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabioba%2Fsparkify-data-pipeline","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabioba%2Fsparkify-data-pipeline/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabioba%2Fsparkify-data-pipeline/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabioba%2Fsparkify-data-pipeline/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fabioba","download_url":"https://codeload.github.com/fabioba/sparkify-data-pipeline/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabioba%2Fsparkify-data-pipeline/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31833836,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-15T07:17:56.427Z","status":"ssl_error","status_checked_at":"2026-04-15T07:17:30.007Z","response_time":63,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["airflow","aws","docker-compose"],"created_at":"2025-01-28T09:08:32.126Z","updated_at":"2026-04-15T08:38:39.851Z","avatar_url":"https://github.com/fabioba.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SPARKIFY-DATA-PIPELINE\n\n## Business Context\nA music streaming company, `Sparkify`, has decided that it is time to introduce more automation and monitoring to their data warehouse ETL pipelines and come to the conclusion that the best tool to achieve this is Apache Airflow.\n\n## Prerequisites\n* Connect Airflow and AWS\n* Connect Airflow to the AWS Redshift Cluster\n\n## Source Data\nThe source data resides in S3 and needs to be processed in Sparkify's data warehouse in Amazon Redshift. The source datasets consist of JSON logs that tell about user activity in the application and JSON metadata about the songs the users listen to.\n* Log data: s3://udacity-dend/log_data\n* Song data: s3://udacity-dend/song_data\n\n\n## Workflow\n![image](docs/images/dag.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffabioba%2Fsparkify-data-pipeline","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffabioba%2Fsparkify-data-pipeline","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffabioba%2Fsparkify-data-pipeline/lists"}