{"id":23233750,"url":"https://github.com/frankchieng/ComfyUI_Aniportrait","last_synced_at":"2025-08-19T18:35:19.120Z","repository":{"id":230862552,"uuid":"780294568","full_name":"frankchieng/ComfyUI_Aniportrait","owner":"frankchieng","description":"Unofficial implementation of AniPortrait custom node in ComfyUI","archived":false,"fork":false,"pushed_at":"2024-09-13T10:41:16.000Z","size":33738,"stargazers_count":51,"open_issues_count":5,"forks_count":9,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-09-14T00:26:40.302Z","etag":null,"topics":["comfyui","comfyui-nodes","comfyui-workflow"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/frankchieng.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-01T06:52:26.000Z","updated_at":"2024-09-13T10:41:20.000Z","dependencies_parsed_at":"2024-06-26T04:24:58.684Z","dependency_job_id":null,"html_url":"https://github.com/frankchieng/ComfyUI_Aniportrait","commit_stats":null,"previous_names":["frankchieng/comfyui_aniportrait"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frankchieng%2FComfyUI_Aniportrait","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frankchieng%2FComfyUI_Aniportrait/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frankchieng%2FComfyUI_Aniportrait/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frankchieng%2FComfyUI_Aniportrait/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/frankchieng","download_url":"https://codeload.github.com/frankchieng/ComfyUI_Aniportrait/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230367780,"owners_count":18215325,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["comfyui","comfyui-nodes","comfyui-workflow"],"created_at":"2024-12-19T03:02:02.511Z","updated_at":"2025-08-19T18:35:19.100Z","avatar_url":"https://github.com/frankchieng.png","language":"Python","funding_links":[],"categories":["Workflows (3395) sorted by GitHub Stars","All Workflows Sorted by GitHub Stars"],"sub_categories":[],"readme":"#### Updates:\n① Implement the frame_interpolation to speed up generation\n\n② Modify the current code and support chain with the [VHS nodes](https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite), i just found that comfyUI IMAGE type requires the torch float32 datatype, and AniPortrait heavily used numpy of image unit8 datatype,so i just changed my mind from my own image/video upload and generation nodes to the prevelance SOTA VHS image/video upload and video combined nodes,it WYSIWYG and inteactive well and instantly render the result\n- ✅ [2024/04/09] raw video to pose video with reference image(aka self-driven)\n- ✅ [2024/04/09] audio driven\n- ✅ [2024/04/09] face reenacment\n- ✅ [2024/04/22] implement audio2pose model and [pre-trained weight](https://huggingface.co/ZJYang/AniPortrait/tree/main) for audio2video,the face reenacment and audio2video workflow has been modified, currently  inference up to a maximum length of 10 seconds has been supported,you can experiment with the length hyperparameter.\n\nU can contact me thr ![twitter_1](https://github.com/frankchieng/ComfyUI_Aniportrait/assets/130369523/27b4fcae-e50c-477d-86f4-dacf7fd052f4)[twitter](https://twitter.com/kurtqian) ![wechat_1](https://github.com/frankchieng/ComfyUI_Aniportrait/assets/130369523/b95cd0a2-4188-4eb3-b1de-5f6eeab71045) Weixin：GalaticKing\n\n\n### audio driven combined with reference image and reference video\n![截图 2024-08-30 12-04-53](https://github.com/user-attachments/assets/10b73c50-a046-41d5-abd1-5ea40a23ad3a)\n[audio2video workflow](https://github.com/frankchieng/ComfyUI_Aniportrait/blob/main/assets/audio2video_workflow.json)\n\u003ctable class=\"center\"\u003e\n\u003ctr\u003e\n    \u003ctd width=100% style=\"border: none\"\u003e\n        \u003cvideo controls autoplay loop src=\"https://github.com/frankchieng/ComfyUI_Aniportrait/assets/130369523/1ab454ef-13ca-4262-9206-383470a8408d\" muted=\"false\"\u003e\u003c/video\u003e\n    \u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\n### raw video to pose video with reference image\n![pose2video](https://github.com/frankchieng/ComfyUI_Aniportrait/assets/130369523/882e3685-ee13-4798-9f90-d195d6595a97)\n\u003ctable class=\"center\"\u003e\n\u003ctr\u003e\n    \u003ctd width=100% style=\"border: none\"\u003e\n        \u003cvideo controls autoplay loop src=\"https://github.com/frankchieng/ComfyUI_Aniportrait/assets/130369523/edaa9907-8720-4435-8529-405c96a2e66d\" muted=\"false\"\u003e\u003c/video\u003e\n    \u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\n### face reenacment\n![face_reenacment](https://github.com/frankchieng/ComfyUI_Aniportrait/assets/130369523/82f2ae7c-b7c2-49a7-8f13-4456ebff55e6)\n[video2video workflow](https://github.com/frankchieng/ComfyUI_Aniportrait/blob/main/assets/face_reenacment_workflow.json)\n\u003ctable class=\"center\"\u003e\n\u003ctr\u003e\n    \u003ctd width=100% style=\"border: none\"\u003e\n        \u003cvideo controls autoplay loop src=\"https://github.com/frankchieng/ComfyUI_Aniportrait/assets/130369523/0a9e46fb-33b8-4787-b8ca-b70301240752\" muted=\"false\"\u003e\u003c/video\u003e\n    \u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\nThis is unofficial implementation of AniPortrait in ComfyUI custom_node,cuz i have routine jobs,so i will update this project when i have time\n\u003e [Aniportrait_pose2video.json](https://github.com/frankchieng/ComfyUI_Aniportrait/blob/main/assets/pose2video_workflow.json)\n\n\u003e [Audio driven](https://github.com/frankchieng/ComfyUI_Aniportrait/blob/main/assets/audio2video_workflow.json)\n\n\u003e [face reenacment](https://github.com/frankchieng/ComfyUI_Aniportrait/blob/main/assets/face_reenacment_workflow.json)\n\nyou should run\n```shell\ngit clone https://github.com/frankchieng/ComfyUI_Aniportrait.git\n```\nthen run \n```shell\npip install -r requirements.txt\n```\ndownload the pretrained models\n\u003e [StableDiffusion V1.5](https://huggingface.co/runwayml/stable-diffusion-v1-5)\n\n\u003e [sd-vae-ft-mse](https://huggingface.co/stabilityai/sd-vae-ft-mse)\n\n\u003e [image_encoder](https://huggingface.co/lambdalabs/sd-image-variations-diffusers/tree/main/image_encoder)\n\n\u003e [wav2vec2-base-960h](https://huggingface.co/facebook/wav2vec2-base-960h) \n\ndownload the weights:\n\u003e [denoising_unet.pth](https://huggingface.co/ZJYang/AniPortrait/tree/main)\n\u003e [reference_unet.pth](https://huggingface.co/ZJYang/AniPortrait/tree/main)\n\u003e [pose_guider.pth](https://huggingface.co/ZJYang/AniPortrait/tree/main)\n\u003e [motion_module.pth](https://huggingface.co/ZJYang/AniPortrait/tree/main)\n\u003e [audio2mesh.pt](https://huggingface.co/ZJYang/AniPortrait/tree/main)\n\u003e [audio2pose.pt](https://huggingface.co/ZJYang/AniPortrait/tree/main)\n\u003e [film_net_fp16.pt](https://huggingface.co/ZJYang/AniPortrait/tree/main)\n```text\n./pretrained_model/\n|-- image_encoder\n|   |-- config.json\n|   `-- pytorch_model.bin\n|-- sd-vae-ft-mse\n|   |-- config.json\n|   |-- diffusion_pytorch_model.bin\n|   `-- diffusion_pytorch_model.safetensors\n|-- stable-diffusion-v1-5\n|   |-- feature_extractor\n|   |   `-- preprocessor_config.json\n|   |-- model_index.json\n|   |-- unet\n|   |   |-- config.json\n|   |   `-- diffusion_pytorch_model.bin\n|   `-- v1-inference.yaml\n|-- wav2vec2-base-960h\n|   |-- config.json\n|   |-- feature_extractor_config.json\n|   |-- preprocessor_config.json\n|   |-- pytorch_model.bin\n|   |-- README.md\n|   |-- special_tokens_map.json\n|   |-- tokenizer_config.json\n|   `-- vocab.json\n|-- audio2mesh.pt\n|-- audio2pose.pt\n|-- denoising_unet.pth\n|-- motion_module.pth\n|-- pose_guider.pth\n|-- reference_unet.pth\n|-- film_net_fp16.pt\n```\n\nTips :\nThe intermediate audio file will be generated and deleted,the raw video to pose video with audio and pose2video mp4 file will be located in the output directory of ComfyUI\nthe original uploaded mp4 video requires square size like 512x512, otherwise the result will be weird \n#### I've updated diffusers from 0.24.x to 0.26.2,so the diffusers/models/embeddings.py classname of PositionNet changed to GLIGENTextBoundingboxProjection and CaptionProjection changed to PixArtAlphaTextProjection,you should pay attention to it and modify the corresponding python files like src/models/transformer_2d.py if you installed the lower version of diffusers \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrankchieng%2FComfyUI_Aniportrait","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffrankchieng%2FComfyUI_Aniportrait","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrankchieng%2FComfyUI_Aniportrait/lists"}