{"id":27602803,"url":"https://github.com/lllyasviel/FramePack","last_synced_at":"2025-04-22T19:02:12.597Z","repository":{"id":288391738,"uuid":"965159463","full_name":"lllyasviel/FramePack","owner":"lllyasviel","description":"Lets make video diffusion practical!","archived":false,"fork":false,"pushed_at":"2025-04-17T18:20:38.000Z","size":42,"stargazers_count":1651,"open_issues_count":28,"forks_count":54,"subscribers_count":22,"default_branch":"main","last_synced_at":"2025-04-17T22:22:46.514Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lllyasviel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-12T14:45:22.000Z","updated_at":"2025-04-17T22:22:21.000Z","dependencies_parsed_at":"2025-04-20T07:31:03.498Z","dependency_job_id":null,"html_url":"https://github.com/lllyasviel/FramePack","commit_stats":null,"previous_names":["lllyasviel/framepack"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lllyasviel%2FFramePack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lllyasviel%2FFramePack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lllyasviel%2FFramePack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lllyasviel%2FFramePack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lllyasviel","download_url":"https://codeload.github.com/lllyasviel/FramePack/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250306607,"owners_count":21408925,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-22T19:01:06.880Z","updated_at":"2025-04-22T19:02:12.572Z","avatar_url":"https://github.com/lllyasviel.png","language":"Python","funding_links":[],"categories":["Python","视频 Video","视频生成_补帧_摘要","Poster","App"],"sub_categories":["资源传输下载"],"readme":"\u003cp align=\"center\"\u003e\n    \u003cimg src=\"https://github.com/user-attachments/assets/2cc030b4-87e1-40a0-b5bf-1b7d6b62820b\" width=\"300\"\u003e\n\u003c/p\u003e\n\n# FramePack\n\nOfficial implementation and desktop software for [\"Packing Input Frame Context in Next-Frame Prediction Models for Video Generation\"](https://lllyasviel.github.io/frame_pack_gitpage/).\n\nLinks: [**Paper**](https://arxiv.org/abs/2504.12626), [**Project Page**](https://lllyasviel.github.io/frame_pack_gitpage/)\n\nFramePack is a next-frame (next-frame-section) prediction neural network structure that generates videos progressively. \n\nFramePack compresses input contexts to a constant length so that the generation workload is invariant to video length.\n\nFramePack can process a very large number of frames with 13B models even on laptop GPUs.\n\nFramePack can be trained with a much larger batch size, similar to the batch size for image diffusion training.\n\n**Video diffusion, but feels like image diffusion.**\n\n# Notes\n\nNote that this GitHub repository is the only official FramePack website. We do not have any web services. All other websites are spam and fake, including but not limited to `framepack.co`, `frame_pack.co`, `framepack.net`, `frame_pack.net`, `framepack.ai`, `frame_pack.ai`, `framepack.pro`, `frame_pack.pro`, `framepack.cc`, `frame_pack.cc`,`framepackai.co`, `frame_pack_ai.co`, `framepackai.net`, `frame_pack_ai.net`, `framepackai.pro`, `frame_pack_ai.pro`, `framepackai.cc`, `frame_pack_ai.cc`, and so on. Again, they are all spam and fake. **Do not pay money or download files from any of those websites.**\n\n# Requirements\n\nNote that this repo is a functional desktop software with minimal standalone high-quality sampling system and memory management.\n\n**Start with this repo before you try anything else!**\n\nRequirements:\n\n* Nvidia GPU in RTX 30XX, 40XX, 50XX series that supports fp16 and bf16. The GTX 10XX/20XX are not tested.\n* Linux or Windows operating system.\n* At least 6GB GPU memory.\n\nTo generate 1-minute video (60 seconds) at 30fps (1800 frames) using 13B model, the minimal required GPU memory is 6GB. (Yes 6 GB, not a typo. Laptop GPUs are okay.)\n\nAbout speed, on my RTX 4090 desktop it generates at a speed of 2.5 seconds/frame (unoptimized) or 1.5 seconds/frame (teacache). On my laptops like 3070ti laptop or 3060 laptop, it is about 4x to 8x slower.\n\nIn any case, you will directly see the generated frames since it is next-frame(-section) prediction. So you will get lots of visual feedback before the entire video is generated.\n\n# Installation\n\n**Windows**:\n\n[\u003e\u003e\u003e Click Here to Download One-Click Package (CUDA 12.6 + Pytorch 2.6) \u003c\u003c\u003c](https://github.com/lllyasviel/FramePack/releases/download/windows/framepack_cu126_torch26.7z)\n\nAfter you download, you uncompress, use `update.bat` to update, and use `run.bat` to run.\n\nNote that running `update.bat` is important, otherwise you may be using a previous version with potential bugs unfixed.\n\n![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/c49bd60d-82bd-4086-9859-88d472582b94)\n\nNote that the models will be downloaded automatically. You will download more than 30GB from HuggingFace.\n\n**Linux**:\n\nWe recommend having an independent Python 3.10.\n\n    pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126\n    pip install -r requirements.txt\n\nTo start the GUI, run:\n\n    python demo_gradio.py\n\nNote that it supports `--share`, `--port`, `--server`, and so on.\n\nThe software supports PyTorch attention, xformers, flash-attn, sage-attention. By default, it will just use PyTorch attention. You can install those attention kernels if you know how. \n\nFor example, to install sage-attention (linux):\n\n    pip install sageattention==1.0.6\n\nHowever, you are highly recommended to first try without sage-attention since it will influence results, though the influence is minimal.\n\n# GUI\n\n![ui](https://github.com/user-attachments/assets/8c5cdbb1-b80c-4b7e-ac27-83834ac24cc4)\n\nOn the left you upload an image and write a prompt.\n\nOn the right are the generated videos and latent previews.\n\nBecause this is a next-frame-section prediction model, videos will be generated longer and longer.\n\nYou will see the progress bar for each section and the latent preview for the next section.\n\nNote that the initial progress may be slower than later diffusion as the device may need some warmup.\n\n# Sanity Check\n\nBefore trying your own inputs, we highly recommend going through the sanity check to find out if any hardware or software went wrong. \n\nNext-frame-section prediction models are very sensitive to subtle differences in noise and hardware. Usually, people will get slightly different results on different devices, but the results should look overall similar. In some cases, if possible, you'll get exactly the same results.\n\n## Image-to-5-seconds\n\nDownload this image:\n\n\u003cimg src=\"https://github.com/user-attachments/assets/f3bc35cf-656a-4c9c-a83a-bbab24858b09\" width=\"150\"\u003e\n\nCopy this prompt:\n\n`The man dances energetically, leaping mid-air with fluid arm swings and quick footwork.`\n\nSet like this:\n\n(all default parameters, with teacache turned off)\n![image](https://github.com/user-attachments/assets/0071fbb6-600c-4e0f-adc9-31980d540e9d)\n\nThe result will be:\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\" width=\"300\"\u003e\n      \u003cvideo \n        src=\"https://github.com/user-attachments/assets/bc74f039-2b14-4260-a30b-ceacf611a185\" \n        controls \n        style=\"max-width:100%;\"\u003e\n      \u003c/video\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e\n      \u003cem\u003eVideo may be compressed by GitHub\u003c/em\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n**Important Note:**\n\nAgain, this is a next-frame-section prediction model. This means you will generate videos frame-by-frame or section-by-section.\n\n**If you get a much shorter video in the UI, like a video with only 1 second, then it is totally expected.** You just need to wait. More sections will be generated to complete the video.\n\n## Know the influence of TeaCache and Quantization\n\nDownload this image:\n\n\u003cimg src=\"https://github.com/user-attachments/assets/42293e30-bdd4-456d-895c-8fedff71be04\" width=\"150\"\u003e\n\nCopy this prompt:\n\n`The girl dances gracefully, with clear movements, full of charm.`\n\nSet like this:\n\n![image](https://github.com/user-attachments/assets/4274207d-5180-4824-a552-d0d801933435)\n\nTurn off teacache:\n\n![image](https://github.com/user-attachments/assets/53b309fb-667b-4aa8-96a1-f129c7a09ca6)\n\nYou will get this:\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\" width=\"300\"\u003e\n      \u003cvideo \n        src=\"https://github.com/user-attachments/assets/04ab527b-6da1-4726-9210-a8853dda5577\" \n        controls \n        style=\"max-width:100%;\"\u003e\n      \u003c/video\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e\n      \u003cem\u003eVideo may be compressed by GitHub\u003c/em\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\nNow turn on teacache:\n\n![image](https://github.com/user-attachments/assets/16ad047b-fbcc-4091-83dc-d46bea40708c)\n\nAbout 30% users will get this (the other 70% will get other random results depending on their hardware):\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\" width=\"300\"\u003e\n      \u003cvideo \n        src=\"https://github.com/user-attachments/assets/149fb486-9ccc-4a48-b1f0-326253051e9b\" \n        controls \n        style=\"max-width:100%;\"\u003e\n      \u003c/video\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e\n      \u003cem\u003eA typical worse result.\u003c/em\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\nSo you can see that teacache is not really lossless and sometimes can influence the result a lot.\n\nWe recommend using teacache to try ideas and then using the full diffusion process to get high-quality results.\n\nThis recommendation also applies to sage-attention, bnb quant, gguf, etc., etc.\n\n## Image-to-1-minute\n\n\u003cimg src=\"https://github.com/user-attachments/assets/820af6ca-3c2e-4bbc-afe8-9a9be1994ff5\" width=\"150\"\u003e\n\n`The girl dances gracefully, with clear movements, full of charm.`\n\n![image](https://github.com/user-attachments/assets/8c34fcb2-288a-44b3-a33d-9d2324e30cbd)\n\nSet video length to 60 seconds:\n\n![image](https://github.com/user-attachments/assets/5595a7ea-f74e-445e-ad5f-3fb5b4b21bee)\n\nIf everything is in order you will get some result like this eventually.\n\n60s version:\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\" width=\"300\"\u003e\n      \u003cvideo \n        src=\"https://github.com/user-attachments/assets/c3be4bde-2e33-4fd4-b76d-289a036d3a47\" \n        controls \n        style=\"max-width:100%;\"\u003e\n      \u003c/video\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e\n      \u003cem\u003eVideo may be compressed by GitHub\u003c/em\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n6s version:\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\" width=\"300\"\u003e\n      \u003cvideo \n        src=\"https://github.com/user-attachments/assets/37fe2c33-cb03-41e8-acca-920ab3e34861\" \n        controls \n        style=\"max-width:100%;\"\u003e\n      \u003c/video\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e\n      \u003cem\u003eVideo may be compressed by GitHub\u003c/em\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n# More Examples\n\nMany more examples are in [**Project Page**](https://lllyasviel.github.io/frame_pack_gitpage/).\n\nBelow are some more examples that you may be interested in reproducing.\n\n---\n\n\u003cimg src=\"https://github.com/user-attachments/assets/99f4d281-28ad-44f5-8700-aa7a4e5638fa\" width=\"150\"\u003e\n\n`The girl dances gracefully, with clear movements, full of charm.`\n\n![image](https://github.com/user-attachments/assets/0e98bfca-1d91-4b1d-b30f-4236b517c35e)\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\" width=\"300\"\u003e\n      \u003cvideo \n        src=\"https://github.com/user-attachments/assets/cebe178a-09ce-4b7a-8f3c-060332f4dab1\" \n        controls \n        style=\"max-width:100%;\"\u003e\n      \u003c/video\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e\n      \u003cem\u003eVideo may be compressed by GitHub\u003c/em\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n---\n\n\u003cimg src=\"https://github.com/user-attachments/assets/853f4f40-2956-472f-aa7a-fa50da03ed92\" width=\"150\"\u003e\n\n`The girl suddenly took out a sign that said “cute” using right hand`\n\n![image](https://github.com/user-attachments/assets/d51180e4-5537-4e25-a6c6-faecae28648a)\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\" width=\"300\"\u003e\n      \u003cvideo \n        src=\"https://github.com/user-attachments/assets/116069d2-7499-4f38-ada7-8f85517d1fbb\" \n        controls \n        style=\"max-width:100%;\"\u003e\n      \u003c/video\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e\n      \u003cem\u003eVideo may be compressed by GitHub\u003c/em\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n---\n\n\u003cimg src=\"https://github.com/user-attachments/assets/6d87c53f-81b2-4108-a704-697164ae2e81\" width=\"150\"\u003e\n\n`The girl skateboarding, repeating the endless spinning and dancing and jumping on a skateboard, with clear movements, full of charm.`\n\n![image](https://github.com/user-attachments/assets/c2cfa835-b8e6-4c28-97f8-88f42da1ffdf)\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\" width=\"300\"\u003e\n      \u003cvideo \n        src=\"https://github.com/user-attachments/assets/d9e3534a-eb17-4af2-a8ed-8e692e9993d2\" \n        controls \n        style=\"max-width:100%;\"\u003e\n      \u003c/video\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e\n      \u003cem\u003eVideo may be compressed by GitHub\u003c/em\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n---\n\n\u003cimg src=\"https://github.com/user-attachments/assets/6e95d1a5-9674-4c9a-97a9-ddf704159b79\" width=\"150\"\u003e\n\n`The girl dances gracefully, with clear movements, full of charm.`\n\n![image](https://github.com/user-attachments/assets/7412802a-ce44-4188-b1a4-cfe19f9c9118)\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\" width=\"300\"\u003e\n      \u003cvideo \n        src=\"https://github.com/user-attachments/assets/e1b3279e-e30d-4d32-b55f-2fb1d37c81d2\" \n        controls \n        style=\"max-width:100%;\"\u003e\n      \u003c/video\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e\n      \u003cem\u003eVideo may be compressed by GitHub\u003c/em\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n---\n\n\u003cimg src=\"https://github.com/user-attachments/assets/90fc6d7e-8f6b-4f8c-a5df-ee5b1c8b63c9\" width=\"150\"\u003e\n\n`The man dances flamboyantly, swinging his hips and striking bold poses with dramatic flair.`\n\n![image](https://github.com/user-attachments/assets/1dcf10a3-9747-4e77-a269-03a9379dd9af)\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\" width=\"300\"\u003e\n      \u003cvideo \n        src=\"https://github.com/user-attachments/assets/aaa4481b-7bf8-4c64-bc32-909659767115\" \n        controls \n        style=\"max-width:100%;\"\u003e\n      \u003c/video\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e\n      \u003cem\u003eVideo may be compressed by GitHub\u003c/em\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n---\n\n\u003cimg src=\"https://github.com/user-attachments/assets/62ecf987-ec0c-401d-b3c9-be9ffe84ee5b\" width=\"150\"\u003e\n\n`The woman dances elegantly among the blossoms, spinning slowly with flowing sleeves and graceful hand movements.`\n\n![image](https://github.com/user-attachments/assets/396f06bc-e399-4ac3-9766-8a42d4f8d383)\n\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\" width=\"300\"\u003e\n      \u003cvideo \n        src=\"https://github.com/user-attachments/assets/f23f2f37-c9b8-45d5-a1be-7c87bd4b41cf\" \n        controls \n        style=\"max-width:100%;\"\u003e\n      \u003c/video\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e\n      \u003cem\u003eVideo may be compressed by GitHub\u003c/em\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n---\n\n\u003cimg src=\"https://github.com/user-attachments/assets/4f740c1a-2d2f-40a6-9613-d6fe64c428aa\" width=\"150\"\u003e\n\n`The young man writes intensely, flipping papers and adjusting his glasses with swift, focused movements.`\n\n![image](https://github.com/user-attachments/assets/c4513c4b-997a-429b-b092-bb275a37b719)\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\" width=\"300\"\u003e\n      \u003cvideo \n        src=\"https://github.com/user-attachments/assets/62e9910e-aea6-4b2b-9333-2e727bccfc64\" \n        controls \n        style=\"max-width:100%;\"\u003e\n      \u003c/video\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e\n      \u003cem\u003eVideo may be compressed by GitHub\u003c/em\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n---\n\n# Prompting Guideline\n\nMany people would ask how to write better prompts. \n\nBelow is a ChatGPT template that I personally often use to get prompts:\n\n    You are an assistant that writes short, motion-focused prompts for animating images.\n\n    When the user sends an image, respond with a single, concise prompt describing visual motion (such as human activity, moving objects, or camera movements). Focus only on how the scene could come alive and become dynamic using brief phrases.\n\n    Larger and more dynamic motions (like dancing, jumping, running, etc.) are preferred over smaller or more subtle ones (like standing still, sitting, etc.).\n\n    Describe subject, then motion, then other things. For example: \"The girl dances gracefully, with clear movements, full of charm.\"\n\n    If there is something that can dance (like a man, girl, robot, etc.), then prefer to describe it as dancing.\n\n    Stay in a loop: one image in, one motion prompt out. Do not explain, ask questions, or generate multiple options.\n\nYou paste the instruct to ChatGPT and then feed it an image to get prompt like this:\n\n![image](https://github.com/user-attachments/assets/586c53b9-0b8c-4c94-b1d3-d7e7c1a705c3)\n\n*The man dances powerfully, striking sharp poses and gliding smoothly across the reflective floor.*\n\nUsually this will give you a prompt that works well. \n\nYou can also write prompts yourself. Concise prompts are usually preferred, for example:\n\n*The girl dances gracefully, with clear movements, full of charm.*\n\n*The man dances powerfully, with clear movements, full of energy.*\n\nand so on.\n\n# Cite\n\n    @article{zhang2025framepack,\n        title={Packing Input Frame Contexts in Next-Frame Prediction Models for Video Generation},\n        author={Lvmin Zhang and Maneesh Agrawala},\n        journal={Arxiv},\n        year={2025}\n    }\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flllyasviel%2FFramePack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flllyasviel%2FFramePack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flllyasviel%2FFramePack/lists"}