https://github.com/pocket/data-flows

Pocket data flows orchestrated using Prefect
https://github.com/pocket/data-flows

cdktf prefect

Last synced: about 1 year ago
JSON representation

Pocket data flows orchestrated using Prefect

• Host: GitHub
• URL: https://github.com/pocket/data-flows
• Owner: Pocket
• License: apache-2.0
• Created: 2021-11-24T17:23:37.000Z (over 4 years ago)
• Default Branch: main-v2
• Last Pushed: 2024-05-28T19:14:42.000Z (about 2 years ago)
• Last Synced: 2024-05-29T07:16:04.151Z (about 2 years ago)
• Topics: cdktf, prefect
• Language: Python
• Homepage:
• Size: 3.49 MB
• Stars: 37
• Watchers: 16
• Forks: 12
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE
  • Code of conduct: CODE_OF_CONDUCT.md
  • Codeowners: .github/CODEOWNERS

Awesome Lists containing this project

README

          
# data-flows

Data flows orchestrated using [Prefect](https://prefect.io) Version 2.

This project is comprised of the the following subprojects:

- [.aws](.aws)

    This contains the Terraform CDK code for the AWS resources needed for this project.  It also contains a custom docker image for the Prefect v2 agent.

- [.circleci](.circleci)

    This contains CircleCI configuration for managing the CICD process for this project.

- [common-utils](common-utils)

    This contains common Python utilities to use for Prefect v2 flow development and deployment.

- [data-products](data-products)

    This contains the Python code for Prefect v2 flows owned by the Data Products Team.

This is built as a **monorepo**, which means new subprojects for Python code can be added as needed including for teams that want to build their own flows on Prefect v2.