{"id":18806959,"url":"https://github.com/rudy2steiner/photomaker","last_synced_at":"2025-04-13T19:13:10.520Z","repository":{"id":217531406,"uuid":"744141014","full_name":"rudy2steiner/PhotoMaker","owner":"rudy2steiner","description":"PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding","archived":false,"fork":false,"pushed_at":"2024-11-07T17:22:26.000Z","size":17945,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-11-07T18:28:48.185Z","etag":null,"topics":["maker","photo"],"latest_commit_sha":null,"homepage":"https://www.photomaker.co","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rudy2steiner.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":null,"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"lfx_crowdfunding":null,"custom":["https://www.buymeacoffee.com/lvwzhen"]}},"created_at":"2024-01-16T17:46:39.000Z","updated_at":"2024-11-07T17:22:31.000Z","dependencies_parsed_at":"2024-11-07T18:35:02.986Z","dependency_job_id":null,"html_url":"https://github.com/rudy2steiner/PhotoMaker","commit_stats":null,"previous_names":["rudy2steiner/photomaker"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rudy2steiner%2FPhotoMaker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rudy2steiner%2FPhotoMaker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rudy2steiner%2FPhotoMaker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rudy2steiner%2FPhotoMaker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rudy2steiner","download_url":"https://codeload.github.com/rudy2steiner/PhotoMaker/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223602742,"owners_count":17171952,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["maker","photo"],"created_at":"2024-11-07T22:50:08.323Z","updated_at":"2024-11-07T22:50:08.962Z","avatar_url":"https://github.com/rudy2steiner.png","language":"TypeScript","funding_links":["https://www.buymeacoffee.com/lvwzhen"],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://photo-maker.github.io/assets/logo.png\" height=100\u003e\n\n\u003c/p\u003e\n\n\u003c!-- ## \u003cdiv align=\"center\"\u003e\u003cb\u003ePhotoMaker\u003c/b\u003e\u003c/div\u003e --\u003e\n\u003cdiv align=\"center\"\u003e\n\n## PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding\n[[Paper](https://huggingface.co/papers/2312.04461)] \u0026emsp; [[Project Page](https://photo-maker.github.io)] \u0026emsp; [[Model Card](https://huggingface.co/TencentARC/PhotoMaker)] \u003cbr\u003e\n\n[[🤗 Demo (Realistic)](https://huggingface.co/spaces/TencentARC/PhotoMaker)] \u0026emsp; [[🤗 Demo (Stylization)](https://huggingface.co/spaces/TencentARC/PhotoMaker-Style)] \u003cbe\u003e\n\nIf the ID fidelity is not enough for you, please try our [stylization application](https://huggingface.co/spaces/TencentARC/PhotoMaker-Style), you may be pleasantly surprised.\n\u003c/div\u003e\n\n\n---\n\nOfficial implementation of **[PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding](https://huggingface.co/papers/2312.04461)**.\n\n### 🌠  **Key Features:**\n\n1. Rapid customization **within seconds**, with no additional LoRA training.\n2. Ensures impressive ID fidelity, offering diversity, promising text controllability, and high-quality generation.\n3. Can serve as an **Adapter** to collaborate with other Base Models alongside LoRA modules in community.\n\n\n---\n\n\u003cdiv align=\"center\"\u003e\n\n![photomaker_demo_fast](https://github.com/TencentARC/PhotoMaker/assets/21050959/e72cbf4d-938f-417d-b308-55e76a4bc5c8)\n\u003c/div\u003e\n\n\n## 🚩 **New Features/Updates**\n- ✅ Jan. 15, 2024. We release PhotoMaker.\n\n---\n\n## 🔥 **Examples**\n\n\n### Realistic generation\n\n- [![Huggingface PhotoMaker](https://img.shields.io/static/v1?label=Demo\u0026message=Huggingface%20Gradio\u0026color=orange)](https://huggingface.co/spaces/TencentARC/PhotoMaker)\n- [**PhotoMaker notebook demo**](photomaker_demo.ipynb)\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://cdn-uploads.huggingface.co/production/uploads/6285a9133ab6642179158944/BYBZNyfmN4jBKBxxt4uxz.jpeg\" height=450\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://cdn-uploads.huggingface.co/production/uploads/6285a9133ab6642179158944/9KYqoDxfbNVLzVKZzSzwo.jpeg\" height=450\u003e\n\u003c/p\u003e\n\n### Stylization generation\n\nNote: only change the base model and add the LoRA modules for better stylization\n\n- [![Huggingface PhotoMaker-Style](https://img.shields.io/static/v1?label=Demo\u0026message=Huggingface%20Gradio\u0026color=orange)](https://huggingface.co/spaces/TencentARC/PhotoMaker-Style)\n- [**PhotoMaker-Style notebook demo**](photomaker_style_demo.ipynb)\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://cdn-uploads.huggingface.co/production/uploads/6285a9133ab6642179158944/du884lcjpqqjnJIxpATM2.jpeg\" height=450\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://cdn-uploads.huggingface.co/production/uploads/6285a9133ab6642179158944/-AC7Hr5YL4yW1zXGe_Izl.jpeg\" height=450\u003e\n\u003c/p\u003e\n\n# 🔧 Dependencies and Installation\n\n- Python \u003e= 3.8 (Recommend to use [Anaconda](https://www.anaconda.com/download/#linux) or [Miniconda](https://docs.conda.io/en/latest/miniconda.html))\n- [PyTorch \u003e= 2.0.0](https://pytorch.org/)\n```bash\npip install -r requirements.txt\n```\n\n# ⏬ Download Models\nThe model will be automatically downloaded through following two lines:\n\n```python\nfrom huggingface_hub import hf_hub_download\nphotomaker_path = hf_hub_download(repo_id=\"TencentARC/PhotoMaker\", filename=\"photomaker-v1.bin\", repo_type=\"model\")\n```\n\nYou can also choose to download manually from this [url](https://huggingface.co/TencentARC/PhotoMaker).\n\n# 💻 How to Test\n\n## Use like [diffusers](https://github.com/huggingface/diffusers)\n\n- Dependency\n```py\nimport torch\nimport os\nfrom diffusers.utils import load_image\nfrom diffusers import EulerDiscreteScheduler\nfrom photomaker.pipeline import PhotoMakerStableDiffusionXLPipeline\n\n### Load base model\npipe = PhotoMakerStableDiffusionXLPipeline.from_pretrained(\n    base_model_path,  # can change to any base model based on SDXL\n    torch_dtype=torch.bfloat16, \n    use_safetensors=True, \n    variant=\"fp16\"\n).to(device)\n\n### Load PhotoMaker checkpoint\npipe.load_photomaker_adapter(\n    os.path.dirname(photomaker_path),\n    subfolder=\"\",\n    weight_name=os.path.basename(photomaker_path),\n    trigger_word=\"img\"  # define the trigger word\n)     \n\npipe.scheduler = EulerDiscreteScheduler.from_config(pipe.scheduler.config)\n\n### Also can cooperate with other LoRA modules\n# pipe.load_lora_weights(os.path.dirname(lora_path), weight_name=lora_model_name, adapter_name=\"xl_more_art-full\")\n# pipe.set_adapters([\"photomaker\", \"xl_more_art-full\"], adapter_weights=[1.0, 0.5])\n\npipe.fuse_lora()\n```\n\n- Input ID Images\n```py\n### define the input ID images\ninput_folder_name = './examples/newton_man'\nimage_basename_list = os.listdir(input_folder_name)\nimage_path_list = sorted([os.path.join(input_folder_name, basename) for basename in image_basename_list])\n\ninput_id_images = []\nfor image_path in image_path_list:\n    input_id_images.append(load_image(image_path))\n```\n\n\u003cdiv align=\"center\"\u003e\n\n\u003ca href=\"https://github.com/TencentARC/PhotoMaker/assets/21050959/01d53dfa-7528-4f09-a1a5-96b349ae7800\" align=\"center\"\u003e\u003cimg style=\"margin:0;padding:0;\" src=\"https://github.com/TencentARC/PhotoMaker/assets/21050959/01d53dfa-7528-4f09-a1a5-96b349ae7800\"/\u003e\u003c/a\u003e\n\u003c/div\u003e\n\n- Generation\n```py\n# Note that the trigger word `img` must follow the class word for personalization\nprompt = \"a half-body portrait of a man img wearing the sunglasses in Iron man suit, best quality\"\nnegative_prompt = \"(asymmetry, worst quality, low quality, illustration, 3d, 2d, painting, cartoons, sketch), open mouth, grayscale\"\ngenerator = torch.Generator(device=device).manual_seed(42)\nimages = pipe(\n    prompt=prompt,\n    input_id_images=input_id_images,\n    negative_prompt=negative_prompt,\n    num_images_per_prompt=1,\n    num_inference_steps=num_steps,\n    start_merge_step=10,\n    generator=generator,\n).images[0]\ngen_images.save('out_photomaker.png')\n```\n\n\u003cdiv align=\"center\"\u003e\n\n\u003ca href=\"https://github.com/TencentARC/PhotoMaker/assets/21050959/703c00e1-5e50-4c19-899e-25ee682d2c06\" align=\"center\"\u003e\u003cimg width=400 style=\"margin:0;padding:0;\" src=\"https://github.com/TencentARC/PhotoMaker/assets/21050959/703c00e1-5e50-4c19-899e-25ee682d2c06\"/\u003e\u003c/a\u003e\n\n\u003c/div\u003e\n\n## Start a local gradio demo\nRun the following command:\n\n```python\npython gradio_demo/app.py\n```\n\nYou could customize this script in [this file](gradio_demo/app.py).\n\n## Usage Tips:\n- Upload more photos of the person to be customized to improve ID fidelty. If the input is Asian face(s), maybe consider adding 'asian' before the class word, e.g., `asian woman img`\n- When stylizing, does the generated face look too realistic? Adjust the Style strength to 30-50, the larger the number, the less ID fidelty, but the stylization ability will be better. You could also try out other base models or LoRAs with good stylization effects.\n- For faster speed, reduce the number of generated images and sampling steps. However, please note that reducing the sampling steps may compromise the ID fidelity.\n\n# 🤗 Acknowledgements\n- T2I-Adapter is co-hosted by Tencent ARC Lab and Nankai University [MCG-NKU](https://mmcheng.net/cmm/).\n- Inspired from many excellent demos and repos, including [IP-Adapter](https://github.com/tencent-ailab/IP-Adapter), [multimodalart/Ip-Adapter-FaceID](https://huggingface.co/spaces/multimodalart/Ip-Adapter-FaceID), [FastComposer](https://github.com/mit-han-lab/fastcomposer), and [T2I-Adapter](https://github.com/TencentARC/T2I-Adapter). Thanks for their great works!\n- Thanks for Venus team in Tencent PCG for their feedback and suggestions.\n\n# Disclaimer\nThis project strives to positively impact the domain of AI-driven image generation. Users are granted the freedom to create images using this tool, but they are expected to comply with local laws and utilize it in a responsible manner. The developers do not assume any responsibility for potential misuse by users.\n\n# BibTeX\nIf you find PhotoMaker useful for your research and applications, please cite using this BibTeX:\n\n```bibtex\n@article{li2023photomaker,\n  title={PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding},\n  author={Li, Zhen and Cao, Mingdeng and Wang, Xintao and Qi, Zhongang and Cheng, Ming-Ming and Shan, Ying},\n  booktitle={arXiv preprint arxiv:2312.04461},\n  year={2023}\n}\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frudy2steiner%2Fphotomaker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frudy2steiner%2Fphotomaker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frudy2steiner%2Fphotomaker/lists"}