Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/superhighfives/lysterfield-lake-pipeline
Video generator for the Lysterfield Lake project.
https://github.com/superhighfives/lysterfield-lake-pipeline
ai art creative-coding generative music
Last synced: about 2 months ago
JSON representation
Video generator for the Lysterfield Lake project.
- Host: GitHub
- URL: https://github.com/superhighfives/lysterfield-lake-pipeline
- Owner: superhighfives
- Created: 2023-11-29T21:12:28.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2023-12-11T12:24:04.000Z (about 1 year ago)
- Last Synced: 2024-04-13T23:42:34.008Z (9 months ago)
- Topics: ai, art, creative-coding, generative, music
- Language: Python
- Homepage: https://lysterfieldlake.com/
- Size: 61.8 MB
- Stars: 3
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Lysterfield Lake (Pipeline)
Video generator for the Lysterfield Lake project.
🚨 **Please note:** 🚨 This repository is largely spaghetti code, intended purely as a resource for anyone looking to dive into aspects of how [Lysterfield Lake](https://lysterfieldlake.com/) was created. It represents the tasks completed on a Mac, while the AI heavy lifting was completed using [Cog](https://github.com/replicate/cog) and a PC with a RTX 3060 GPU. Files specific to that process are in `/pc-settings/` That said, it should paint some of the picture (see what I did there?) of how the project works.
> The client for the app is available at [superhighfives/lysterfield-lake](https://github.com/superhighfives/lysterfield-lake)
✋ You can [learn more about how the project works here](https://medium.com/@superhighfives/lysterfield-lake-71345aa8c016).
## Overview
The pipeline for Lysterfield Lake is made up of a collection of bash scripts. They run a mixture of python scripts, and unix applications (like ffmpeg) to output videos and images.
On the python side, data is partially passed to and from the AI models using [Cog](https://github.com/replicate/cog). It runs three models, which it expects at the following locations on your network:
| Model | Replicate | GitHub | Location |
| ------------- | ------------------------------------------------------------------------ | --------------------------------------------------------------------- | -------------------------------- |
| DiffusionCLIP | [gwang-kim/diffusionclip](https://replicate.com/gwang-kim/diffusionclip) | [gwang-kim/DiffusionCLIP](https://github.com/gwang-kim/DiffusionCLIP) | Running on http://localhost:5000 |
| ZoeDepth | [cjwbw/zoedepth](https://replicate.com/cjwbw/zoedepth) | [chenxwh/ZoeDepth](https://github.com/chenxwh/ZoeDepth) | Running on http://localhost:5005 |
| Real-ESRGAN | [cjwbw/real-esrgan](https://replicate.com/cjwbw/real-esrgan) | [xinntao/Real-ESRGAN](https://github.com/xinntao/Real-ESRGAN) | Running on http://localhost:5010 |
The other models should be added to the root folder, in the paths referenced in the source files.
## Toolchain
- Cog ([Github](https://github.com/replicate/cog))
- ZoeDepth (Depthifier): [Replicate](https://replicate.com/cjwbw/zoedepth), [GitHub](https://github.com/chenxwh/ZoeDepth)
- Real-ESRGAN (Resizer): [Replicate](https://replicate.com/cjwbw/real-esrgan), [GitHub](https://github.com/xinntao/Real-ESRGAN)
- Robust Video Matting (Alphafier): [Replicate](https://replicate.com/arielreplicate/robust_video_matting), [GitHub](https://github.com/PeterL1n/RobustVideoMatting)
- Deforum Stable Diffusion (Dreamer): [Replicate](https://replicate.com/deforum/deforum_stable_diffusion), [GitHub](https://github.com/deforum/stable-diffusion)
- Diffusion CLIP (Stylizer): [Replicate](https://replicate.com/gwang-kim/diffusionclip), [GitHub](https://github.com/gwang-kim/DiffusionCLIP)
- Inpaint Anything (Inpainter): [GitHub](https://github.com/geekyutao/Inpaint-Anything/)