https://github.com/etched-ai/open-oasis
Inference script for Oasis 500M
https://github.com/etched-ai/open-oasis
Last synced: 8 months ago
JSON representation
Inference script for Oasis 500M
- Host: GitHub
- URL: https://github.com/etched-ai/open-oasis
- Owner: etched-ai
- License: mit
- Created: 2024-10-31T09:10:45.000Z (about 1 year ago)
- Default Branch: master
- Last Pushed: 2024-11-08T21:49:23.000Z (about 1 year ago)
- Last Synced: 2024-11-08T22:38:18.752Z (about 1 year ago)
- Language: Python
- Size: 28.2 MB
- Stars: 1,199
- Watchers: 21
- Forks: 88
- Open Issues: 18
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- ai-game-devtools - Oasis
README
# Oasis 500M
Oasis is an interactive world model developed by [Decart](https://www.decart.ai/) and [Etched](https://www.etched.com/). Based on diffusion transformers, Oasis takes in user keyboard input and generates gameplay in an autoregressive manner. We release the weights for Oasis 500M, a downscaled version of the model, along with inference code for action-conditional frame generation.
For more details, see our [joint blog post](https://oasis-model.github.io/) to learn more.
And to use the most powerful version of the model, be sure to check out the [live demo](https://oasis.us.decart.ai/) as well!
## Setup
```
git clone https://github.com/etched-ai/open-oasis.git
cd open-oasis
# Install pytorch
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121
# Install other dependencies
pip install einops diffusers timm av
```
## Download the model weights
Inside the `open-oasis/` directory, run:
```
huggingface-cli login
huggingface-cli download Etched/oasis-500m oasis500m.safetensors # DiT checkpoint
huggingface-cli download Etched/oasis-500m vit-l-20.safetensors # ViT VAE checkpoint
```
## Basic Usage
We include a basic inference script that loads a prompt frame from a video and generates additional frames conditioned on actions.
```
python generate.py
# Or specify path to checkpoints:
python generate.py --oasis-ckpt --vae-ckpt
```
Use a custom image prompt:
```
python generate.py --prompt-path
```
The resulting video will be saved to `video.mp4`. Here's are some examples of a generation from this 500M model!
