{"id":19164068,"url":"https://github.com/pyladiesams/etl-intermediate-dec2020","last_synced_at":"2026-04-20T04:04:13.132Z","repository":{"id":55608261,"uuid":"320086524","full_name":"pyladiesams/ETL-intermediate-dec2020","owner":"pyladiesams","description":"Talks co-organized with PyAmsterdam about ETL/ELT in action in the cloud","archived":false,"fork":false,"pushed_at":"2020-12-30T09:53:19.000Z","size":543,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-01-03T22:14:08.825Z","etag":null,"topics":["aws","aws-glue","docker","etl","gcp"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pyladiesams.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-12-09T21:41:46.000Z","updated_at":"2020-12-30T09:53:22.000Z","dependencies_parsed_at":"2022-08-15T04:20:19.240Z","dependency_job_id":null,"html_url":"https://github.com/pyladiesams/ETL-intermediate-dec2020","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":"pyladiesams/workshoptopic-mmmYYYY","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyladiesams%2FETL-intermediate-dec2020","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyladiesams%2FETL-intermediate-dec2020/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyladiesams%2FETL-intermediate-dec2020/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyladiesams%2FETL-intermediate-dec2020/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pyladiesams","download_url":"https://codeload.github.com/pyladiesams/ETL-intermediate-dec2020/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240245885,"owners_count":19771029,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","aws-glue","docker","etl","gcp"],"created_at":"2024-11-09T09:17:48.256Z","updated_at":"2026-04-20T04:04:13.090Z","avatar_url":"https://github.com/pyladiesams.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# [ONLINE] aggregate(PyLadies, PyAmsterdam, ETL).byebye(2020)\n### Level: Intermediate - Advanced\n\n## Meetup description\nThis meetup is co-organized with [PyAmsterdam](https://py.amsterdam/), and it's all about ETL/ELT in action in the cloud.\n\n### Talk 1 - Data Processing with Python: a Containerized, Scheduled, and Monitored Pipeline on the Google Cloud Platform\n\nDuring the talk, you will learn how to containerize a Python pipeline for data processing and execute it as a scheduled job that sends email messages for critical errors. The presentation can be find [here](https://github.com/pyladiesams/ETL-intermediate-dec2020/blob/master/presentations/pyladies-amsterdam-etl-slides.pdf). The repo with code [here](https://github.com/nancyirisarri/pyladies-amsterdam-etl).\n\n### Talk 2 - AWS Glue: the good, the bad, the ugly\n\nAWS Glue offers Serverless ETL pipeline and workflows, while it can be rather simple to start using it via console it quickly become non trivial for deployment via CI/CD pipeline. Management of multiple environments, working with the pipeline state and keeping your code DRY is a challenge. During the talk I will walk you through guts of AWS Glue and show my approach to more streamlined deployment. The presentation can be find [here](http://slides.com/oglopzelgo/aws-glue-pyamsterdam). The repo with code [here](https://github.com/1oglop1/aws-glue-monorepo-style).\n\n## Video record\n[Video record](https://bit.ly/3rza70O)\n\n## Credits\nThis event was set up by @pyladiesams, @PyAmsterdam, @nancyirisarri and @1oglop1\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpyladiesams%2Fetl-intermediate-dec2020","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpyladiesams%2Fetl-intermediate-dec2020","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpyladiesams%2Fetl-intermediate-dec2020/lists"}