https://github.com/excamera/pipelines
https://github.com/excamera/pipelines
Last synced: 7 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/excamera/pipelines
- Owner: excamera
- Created: 2016-05-07T19:29:27.000Z (almost 10 years ago)
- Default Branch: master
- Last Pushed: 2016-05-07T20:04:27.000Z (almost 10 years ago)
- Last Synced: 2025-01-08T12:13:17.371Z (about 1 year ago)
- Language: Python
- Size: 4.88 KB
- Stars: 1
- Watchers: 4
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Pipelines
The masterLambda is a general purpose lambda, which takes a video, chunks it, calls the WORKER LAMBDA on each chunk, gathers the result of all the WORKER LAMBDAS and calls the reducerLambda to join them. It then notifies completion on SNS.
The reducerLambda, takes a list of S3 keys, downloads the keys, and joins them -if they are txt files then simply appends them to a long file. If they are video chunks it uses ffmpeg concat to rejoin them.
The taggerLambda, imageclassifyLambda and the grayscaleLambdas are WORKER LAMBDAS, which take in an S3 key as input, operate on it, produce an output chunk or text, write it back to S3 and give its key as output.