{"id":50852041,"url":"https://github.com/albertoelopez/ai2videogen-","last_synced_at":"2026-06-14T14:05:26.463Z","repository":{"id":351028065,"uuid":"1209251328","full_name":"albertoelopez/ai2videogen-","owner":"albertoelopez","description":"Local AI video generation — Wan 2.2 GGUF + LTX-Video via ComfyUI API. RTX 4090 optimized. No cloud.","archived":false,"fork":false,"pushed_at":"2026-04-13T08:47:43.000Z","size":20,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-14T14:04:06.292Z","etag":null,"topics":["ai","comfyui","gguf","image-to-video","open-source","rtx-4090","video-generation","wan2"],"latest_commit_sha":null,"homepage":"https://github.com/albertoelopez/ai2videogen-","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/albertoelopez.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-13T08:36:50.000Z","updated_at":"2026-04-13T08:47:48.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/albertoelopez/ai2videogen-","commit_stats":null,"previous_names":["albertoelopez/ai2videogen-"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/albertoelopez/ai2videogen-","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/albertoelopez%2Fai2videogen-","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/albertoelopez%2Fai2videogen-/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/albertoelopez%2Fai2videogen-/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/albertoelopez%2Fai2videogen-/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/albertoelopez","download_url":"https://codeload.github.com/albertoelopez/ai2videogen-/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/albertoelopez%2Fai2videogen-/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34324004,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-14T02:00:07.365Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","comfyui","gguf","image-to-video","open-source","rtx-4090","video-generation","wan2"],"created_at":"2026-06-14T14:05:25.736Z","updated_at":"2026-06-14T14:05:26.449Z","avatar_url":"https://github.com/albertoelopez.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"hero.svg\" alt=\"ai2videogen\" width=\"100%\"/\u003e\n\u003c/p\u003e\n\n# ai2videogen\n\nLocal AI video generation pipeline — animate images with **Wan 2.2 GGUF**, generate with **LTX-Video**, orchestrated via **ComfyUI API**. Optimized for RTX 4090 + 24 GB VRAM. No cloud. No API keys.\n\n---\n\n## Quick start\n\n```bash\n# Check ComfyUI is running\npython cli.py status\n\n# Start ComfyUI if it isn't\npython cli.py start\n\n# Animate an image\npython cli.py animate photo.png --prompt \"head turns slowly, soft light\"\n\n# Portrait, fast mode\npython cli.py animate photo.png --prompt \"...\" --portrait --fast\n\n# Web UI\npython app.py        # opens at http://localhost:7860\n```\n\n---\n\n## CLI (`cli.py`)\n\n```bash\npython cli.py animate IMAGE --prompt \"...\" [options]\n```\n\n| Flag | Default | Notes |\n|------|---------|-------|\n| `--model` | `wan-gguf` | `wan-gguf` · `ltx` · `wan-fp8` |\n| `--portrait` | off | Sets 480×832 |\n| `--fast` | off | 41 frames, 15 steps (~4 min) |\n| `--frames` | `81` | Must be `4n+1` |\n| `--steps` | `20` | |\n| `--seed` | random | |\n| `--output` | `./outputs` | |\n\n```bash\npython cli.py status   # check if ComfyUI is running\npython cli.py start    # start ComfyUI on port 8189\n```\n\n---\n\n## Web UI (`app.py`)\n\n```bash\npip install gradio\npython app.py\n```\n\nOpens at **http://localhost:7860** — upload an image, describe the motion, hit Generate.\n\n---\n\n## Scripts\n\n### `wan_gguf_generate.py` — Wan 2.2 GGUF Q4_K_M (recommended)\n\nTwo-pass HIGH + LOW noise image-to-video using 4-bit quantized 14B models. Best quality on a single 4090.\n\n```bash\npython wan_gguf_generate.py image.png --prompt \"describe the motion\" --output ./outputs\n```\n\n```bash\n# Portrait\npython wan_gguf_generate.py photo.jpg --prompt \"...\" --width 480 --height 832\n\n# Whole folder\npython wan_gguf_generate.py ./images/ --prompt \"...\" --output ./clips\n\n# Shorter clip, faster run (~4 min)\npython wan_gguf_generate.py image.png --prompt \"...\" --frames 41\n```\n\n| Flag | Default | Notes |\n|------|---------|-------|\n| `--frames` | `81` | Must be `4n+1`. 81 ≈ 3.4 s at 24 fps |\n| `--steps` | `20` | More steps = better quality, slower |\n| `--width` / `--height` | `832` / `480` | Swap for portrait |\n| `--cfg` | `6.0` | Lower = more motion |\n| `--seed` | random | Fix for reproducible results |\n| `--blocks-to-swap` | `30` | CPU offload blocks (14B has 40 total) |\n\n---\n\n### `ltx_generate.py` — LTX-Video 2 (fast)\n\nLighter alternative — fits entirely in VRAM without block swapping. Faster but lower fidelity than Wan 2.2.\n\n```bash\npython ltx_generate.py image.png --prompt \"woman walks through neon city\" --output ./clips\n```\n\n---\n\n### `generate.py` — Wan 2.2 fp8 + VBVR workflow\n\nFull-precision fp8 pipeline with VBVR LoRA. Requires fp8 safetensors models (larger download, higher VRAM).\n\n```bash\npython generate.py image.png --prompt \"...\" --pipeline 3\n```\n\nPipelines: `1` = Standard HIGH + VBVR LoRA · `2` = SNR Hybrid · `3` = VBVR SNR Hybrid (best)\n\n---\n\n## Setup\n\n### 1. ComfyUI\n\n```bash\ncd ~/ComfyUI\npython main.py --port 8189 --cuda-malloc --use-pytorch-cross-attention\n```\n\n\u003e Do **not** use `--highvram` with two-pass GGUF workflows — it keeps both models in VRAM simultaneously and causes OOM.\n\n### 2. Required models\n\n**Wan 2.2 GGUF** → `ComfyUI/models/unet/`\n```bash\nhf download bullerwins/Wan2.2-I2V-A14B-GGUF \\\n  wan2.2_i2v_high_noise_14B_Q4_K_M.gguf \\\n  wan2.2_i2v_low_noise_14B_Q4_K_M.gguf \\\n  --local-dir ComfyUI/models/unet/\n```\n\n**VAE** → `ComfyUI/models/vae/wan_2.1_vae.safetensors`\n\n**T5 text encoder** → `ComfyUI/models/text_encoders/umt5_xxl_fp16.safetensors`\n\n### 3. ComfyUI extensions\n\n```bash\ncd ComfyUI/custom_nodes\ngit clone https://github.com/kijai/ComfyUI-WanVideoWrapper\ngit clone https://github.com/city96/ComfyUI-GGUF\ngit clone https://github.com/kijai/ComfyUI-KJNodes\ngit clone https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite\n```\n\n---\n\n## VRAM budget (RTX 4090, Q4_K_M)\n\n| Component | VRAM | Notes |\n|-----------|------|-------|\n| Active GGUF model (half blocks on GPU) | ~5 GB | 30/40 blocks swapped to RAM |\n| T5 encoder | offloaded | CPU after text encoding |\n| VAE | ~0.1 GB | |\n| Peak during sampling | **~6 GB** | |\n\nPeak RAM: ~15 GB (T5 + model offload)\n\n---\n\n## Coming in Phase 2\n\n- Full pipeline: prompt → scenes → images (FLUX.1) → video clips → audio (MusicGen) → final MP4\n- Bring your own images and audio\n- Multi-scene orchestration\n\n---\n\n## Requirements\n\n- Python 3.11+\n- PyTorch 2.10+ with CUDA 12.8\n- ComfyUI with WanVideoWrapper, GGUF, KJNodes, VideoHelperSuite extensions\n- RTX 4090 or equivalent (24 GB VRAM recommended)\n\n```bash\npip install sageattention  # recommended — reduces attention VRAM\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falbertoelopez%2Fai2videogen-","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falbertoelopez%2Fai2videogen-","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falbertoelopez%2Fai2videogen-/lists"}