https://github.com/spotify/pythonflow

:snake: Dataflow programming for python.
https://github.com/spotify/pythonflow

dataflow machine-learning python

Last synced: 10 months ago
JSON representation

:snake: Dataflow programming for python.

• Host: GitHub
• URL: https://github.com/spotify/pythonflow
• Owner: spotify
• License: apache-2.0
• Created: 2017-09-05T17:52:33.000Z (over 8 years ago)
• Default Branch: master
• Last Pushed: 2023-05-23T17:41:54.000Z (over 2 years ago)
• Last Synced: 2025-03-28T23:04:51.207Z (10 months ago)
• Topics: dataflow, machine-learning, python
• Language: Python
• Homepage: http://pythonflow.readthedocs.io/en/latest/
• Size: 222 KB
• Stars: 289
• Watchers: 73
• Forks: 48
• Open Issues: 4
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# Pythonflow: Dataflow programming for python. [![Build Status](https://travis-ci.org/spotify/pythonflow.svg)](https://travis-ci.org/spotify/pythonflow) [![Documentation Status](https://readthedocs.org/projects/pythonflow/badge/?version=latest)](http://pythonflow.readthedocs.io/en/latest/?badge=latest)

Pythonflow is a simple implementation of [dataflow programming](https://en.wikipedia.org/wiki/Dataflow_programming) for python. Users of [Tensorflow](https://www.tensorflow.org/) will immediately be familiar with the syntax.

At Spotify, we use Pythonflow in data preprocessing pipelines for machine learning models because

* it automatically caches computationally expensive operations,

* any part of the computational graph can be easily evaluated for debugging purposes,

* it allows us to distribute data preprocessing across multiple machines.

See the [documentation](http://pythonflow.readthedocs.io/en/latest/) for details.