https://github.com/outerbounds/metaflow-pyspark

EXPERIMENTAL PySpark decorator for Metaflow
https://github.com/outerbounds/metaflow-pyspark

Last synced: about 1 year ago
JSON representation

EXPERIMENTAL PySpark decorator for Metaflow

• Host: GitHub
• URL: https://github.com/outerbounds/metaflow-pyspark
• Owner: outerbounds
• License: apache-2.0
• Created: 2022-07-29T16:56:17.000Z (almost 4 years ago)
• Default Branch: main
• Last Pushed: 2025-02-11T16:57:21.000Z (over 1 year ago)
• Last Synced: 2025-04-02T04:23:31.429Z (over 1 year ago)
• Language: Python
• Size: 22.5 KB
• Stars: 7
• Watchers: 4
• Forks: 1
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-metaflow - metaflow-pyspark - PySpark decorator for Metaflow steps. Experimental, low adoption. (Distributed Compute & Training)

README

          

# EXPERIMENTAL `@pyspark` decorator for Metaflow

see `example/sparkflow.py` for an example