Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/anna-geller/prefect-deployment-patterns
Code examples showing flow deployment to various types of infrastructure
https://github.com/anna-geller/prefect-deployment-patterns
automation aws data data-engineering data-engineering-infrastructure data-engineering-pipeline data-engineering-team data-products data-science dataflow dataflow-ops orchestration pipeline prefect python serverless serverless-framework
Last synced: 16 days ago
JSON representation
Code examples showing flow deployment to various types of infrastructure
- Host: GitHub
- URL: https://github.com/anna-geller/prefect-deployment-patterns
- Owner: anna-geller
- License: apache-2.0
- Created: 2022-07-24T11:28:23.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-01-11T16:58:30.000Z (almost 2 years ago)
- Last Synced: 2023-04-27T19:33:17.465Z (over 1 year ago)
- Topics: automation, aws, data, data-engineering, data-engineering-infrastructure, data-engineering-pipeline, data-engineering-team, data-products, data-science, dataflow, dataflow-ops, orchestration, pipeline, prefect, python, serverless, serverless-framework
- Language: Python
- Homepage:
- Size: 249 KB
- Stars: 64
- Watchers: 3
- Forks: 8
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# DataflowOps Best Practices & Examples
Code examples showing:
- how to use Prefect
- flow deployments to various types of infrastructure
## Blog posts about using and deploying Prefect
- [Declarative Dataflow Deployments with Prefect Make CI/CD a Breeze](https://medium.com/the-prefect-blog/declarative-dataflow-deployments-with-prefect-make-ci-cd-a-breeze-fe77bdbb58d4)
- [Prefect 2.3.0 adds support for flows defined in Docker Images and GitHub Repositories](https://medium.com/the-prefect-blog/prefect-2-3-0-adds-support-for-flows-defined-in-docker-images-and-github-repositories-79a8797a7371)
- [Scheduled vs. Event-driven Data Pipelines — Orchestrate Anything with Prefect](https://medium.com/the-prefect-blog/scheduled-vs-event-driven-data-pipelines-orchestrate-anything-with-prefect-b915e6adc3ba)
- [Event-driven Data Pipelines with AWS Lambda, Prefect and GitHub Actions](https://medium.com/the-prefect-blog/event-driven-data-pipelines-with-aws-lambda-prefect-and-github-actions-b3d9f84b1309)
- [Serverless Real-Time Data Pipelines on AWS with Prefect, ECS and GitHub Actions](https://medium.com/the-prefect-blog/serverless-real-time-data-pipelines-on-aws-with-prefect-ecs-and-github-actions-1737c80da3f5)
- [Deploy Prefect Pipelines with Python | by Jeff Hale](https://medium.com/the-prefect-blog/deploy-prefect-pipelines-with-python-perfect-68c944a3a89f)
- [Orchestrate Your Data Science Project with Prefect 2.0 | by Khuyen Tran](https://medium.com/the-prefect-blog/orchestrate-your-data-science-project-with-prefect-2-0-4118418fd7ce)
- [Sending Slack Notifications in Python with Prefect | by Khuyen Tran](https://medium.com/the-prefect-blog/sending-slack-notifications-in-python-with-prefect-840a895f81c)
- [Orchestrating Airbyte with Prefect 2.0 | by Nate Nowack](https://medium.com/the-prefect-blog/orchestrating-airbyte-with-prefect-2-0-35501997a974)
- [Migrating Databricks Tasks from Prefect 1 to Prefect 2 | by Andrew Huang](https://medium.com/the-prefect-blog/migrating-databricks-tasks-from-prefect-1-to-prefect-2-6f1187145d18)
## Repository templates with CI/CD workflows
[](https://github.com/anna-geller/dataflow-ops)
[](https://github.com/anna-geller/dataflow-ops-aws-eks)
[](https://github.com/anna-geller/prefect-aws-lambda)
[](https://github.com/anna-geller/prefect-streaming)
[](https://github.com/anna-geller/prefect-docker-deployment)
[](https://github.com/rpeden/prefect-docker-compose)
[](https://github.com/khuyentran1401/prefect-mlops-recipes)
[](https://github.com/khuyentran1401/prefect-course)
## Useful Discourse topics
- [Prefect Deployments FAQ](https://discourse.prefect.io/t/prefect-deployments-faq/1467)
- [Prefect Logging FAQ](https://discourse.prefect.io/t/prefect-logging-faq/1476)
- [Prefect Agents and Work-Queues FAQ](https://discourse.prefect.io/t/agents-and-work-queues-faq/1468/5)
## Questions?
Reach out via [Discourse](https://discourse.prefect.io/) or [Slack](https://prefect.io/slack)
