{"id":28059996,"url":"https://github.com/wisconsinaivision/yochameleon","last_synced_at":"2026-01-27T17:38:45.624Z","repository":{"id":290829061,"uuid":"829089675","full_name":"WisconsinAIVision/YoChameleon","owner":"WisconsinAIVision","description":"🦎 Yo'Chameleon: Your Personalized Chameleon (CVPR 2025)","archived":false,"fork":false,"pushed_at":"2025-05-05T17:48:33.000Z","size":11147,"stargazers_count":128,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-12T08:36:26.920Z","etag":null,"topics":["chameleon","cvpr","cvpr2025","large-language-models","large-multimodal-models","llms","lmms","personalization","personalized","personalized-generation"],"latest_commit_sha":null,"homepage":"https://thaoshibe.github.io/YoChameleon/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/WisconsinAIVision.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-07-15T18:26:30.000Z","updated_at":"2025-05-07T07:21:05.000Z","dependencies_parsed_at":"2025-04-30T19:46:32.539Z","dependency_job_id":"44160165-86c0-4e53-9209-6bf4b7ac7452","html_url":"https://github.com/WisconsinAIVision/YoChameleon","commit_stats":null,"previous_names":["thaoshibe/yochameleon"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/WisconsinAIVision/YoChameleon","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WisconsinAIVision%2FYoChameleon","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WisconsinAIVision%2FYoChameleon/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WisconsinAIVision%2FYoChameleon/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WisconsinAIVision%2FYoChameleon/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/WisconsinAIVision","download_url":"https://codeload.github.com/WisconsinAIVision/YoChameleon/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WisconsinAIVision%2FYoChameleon/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28817128,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-27T12:25:15.069Z","status":"ssl_error","status_checked_at":"2026-01-27T12:25:05.297Z","response_time":168,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chameleon","cvpr","cvpr2025","large-language-models","large-multimodal-models","llms","lmms","personalization","personalized","personalized-generation"],"created_at":"2025-05-12T08:36:23.846Z","updated_at":"2026-01-27T17:38:45.618Z","avatar_url":"https://github.com/WisconsinAIVision.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Yo'Chameleon (Unofficial)\n\n###  [Project Page](https://thaoshibe.github.io/YoChameleon) | [arXiv](https://arxiv.org/abs/2504.20998) | [HF Datasets](https://huggingface.co/datasets/thaoshibe/Mini-YoChameleon-Data)\n\n\n\u003cimg src=\"./images/yochameleon-bo.png\" alt=\"YoChameleon\" width=\"1000\"\u003e\n\n---\n**🦎 Yo’Chameleon: Personalized Vision and Language Generation**\u003cbr\u003e\nThao Nguyen\u003csup\u003e1, 2\u003c/sup\u003e, Krishna Kumar Singh\u003csup\u003e2\u003c/sup\u003e, Jing Shi\u003csup\u003e2\u003c/sup\u003e, Trung Bui\u003csup\u003e2\u003c/sup\u003e, Yong Jae Lee\u003csup\u003e1, ¶\u003c/sup\u003e, Yuheng Li\u003csup\u003e2, ¶\u003c/sup\u003e\u003cbr\u003e\n*\u003csup\u003e1\u003c/sup\u003eUniversity of Wisconsin-Madison, \u003csup\u003e2\u003c/sup\u003eAdobe Research*\u003cbr\u003e\n\n(⭑.ᐟ *This is [Yo'LLaVA](https://thaoshibe.github.io/YoLLaVA/) meets [Chameleon](https://arxiv.org/abs/2405.09818)!* ⭑.ᐟ)\n\n\u003cimg src=\"./images/yochameleon.png\" alt=\"YoChameleon\" width=\"200\"\u003e\n\n\u003e Large Multimodal Models (e.g., GPT-4, Gemini, Chameleon) have evolved into powerful tools with millions of users. However, they remain generic models and lack personalized knowledge of specific user concepts.\nPrevious work has explored personalization for text generation, yet it remains unclear how these methods can be adapted to new modalities, such as image generation. In this paper, we introduce Yo'Chameleon, the first attempt to study personalization for large multimodal models.\nGiven 3-5 images of a particular concept, Yo'Chameleon leverages soft-prompt tuning to embed subject-specific information to (i) answer questions about the subject and (ii) recreate pixel-level details to produce images of the subject in new contexts. Yo'Chameleon is trained with (i) a self-prompting optimization mechanism to balance performance across multiple modalities, and (ii) a ``soft-positive\" image generation approach to enhanc6e image quality in a few-shot setting.\nOur qualitative and quantitative analyses reveal that Yo'Chameleon can learn concepts more efficiently using fewer tokens and effectively encode visual attributes, outperforming prompting baselines.\n\n*(¶: equal advising)*\n\n---\n\n##### Table of Contents\n\n1. [**Getting Started**](#-getting-started)\n1. [**Creating Dataset**](#-creating-dataset)\n1. [**Train**](#-train)\n1. [**Test**](#-test)\n1. [**Evaluation**](#-evaluation): [Detailed Caption](#detailed-caption), [Facial Similarity Scores](#facial-similarity-scores), [CLIP Image-to-Image Similarity](#clip-image-to-image-similarity), [Recognition Accuracy](#recognition-accuracy)\n1. [**Acknowledgements**](#-acknowledgements)\n\n### 🚀 Getting Started\n\n```\n# Clone the repository\ngit clone https://github.com/thaoshibe/YoChameleon.git\ncd YoChameleon\n\n# Install via pip\nconda create -n yochameleon\nconda acitvate yochameleon\nconda install pytorch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 -c pytorch\n\npip install -r requirements.txt\n\n# Or run the bash script\nbash install.sh\n```\n\n### Quick Start\n\nThis provide a quick start to train the model with the provided dataset.\n\n***1. Download Mini-YoChameleon Data***\n\nHere we provide a quick start with a mini-version of the YoChameleon training dataset. Please download it via HuggingFace Datasets.\n\n```\ngit lfs install\ngit clone git@hf.com:datasets/thaoshibe/Mini-YoChameleon-Data\n\n# optional, if you also want to see Yo'LLaVA data\ngit clone git@hf.com:datasets/thaoshibe/Mini-YoChameleon-Data\n\n```\n\n***2. Train***\n\nRun this script to quick train -- You should be able to monitor the training with visualization via WanDB.\n\n```\npython train.py --config config/basic.yaml\n```\n\n### 🛠️ Creating dataset\n\n```\n#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@\n#\n#   bash scripts provided in `scripts/create_data` folder\n#   \n#   scripts/create_data\n#   ├── retrieve.sh               # retrieve negative examples\n#   ├── recognition.sh            # recognition data (100 hard neg, 100 easy neg, \u0026 positive)\n#   ├── create_soft_positive.sh   # image generation data\n#   └── text_only_data.sh         # call GPT-4o for text-only response data\n#\n#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@\n\n#\n# Remember to check and fill the relative path in the script before running\n#\n\nbash scripts/create_soft_positive.sh\n\n```\n\n\u003cdetails\u003e\n\u003csummary\u003e Retrieve negative examples \u003c/summary\u003e\n\n```\ncd create_training_data/conversation_data\nNAMES=(\"bo\" \"duck-banana\" \"marie-cat\" \"pusheen-cup\" \"thuytien\"\n       \"brown-duck\" \"dug\" \"mydieu\" \"shiba-black\" \"tokyo-keyboard\"\n       \"butin\" \"elephant\" \"neurips-cup\" \"shiba-gray\" \"toodles-galore\"\n       \"cat-cup\" \"fire\" \"nha-tho-hanoi\" \"shiba-sleep\" \"viruss\"\n       \"chua-thien-mu\" \"henry\" \"nha-tho-hcm\" \"shiba-yellow\" \"water\"\n       \"ciin\" \"khanhvy\" \"oong\" \"thao\" \"willinvietnam\"\n       \"denisdang\" \"lamb\" \"phuc-map\" \"thap-but\" \"yellow-duck\"\n       \"dragon\" \"mam\" \"pig-cup\" \"thap-cham\" \"yuheng\")\n\nfor NAME in \"${NAMES[@]}\"; do\n  INPUT_FOLDER=\"/mnt/localssd/code/data/yochameleon-data/train/${NAME}\"\n  SAVE_FOLDER=\"${INPUT_FOLDER}/negative_example\"\n  LIMIT=5000 # Number of negative examples to retrieve\n  echo \"Processing folder: ${NAME}\"\n  \n  python create_training_data/retrieve_negative/load_similar_example.py \\\n    --input_folder $INPUT_FOLDER \\\n    --save_folder $SAVE_FOLDER \\\n    --limit $LIMIT \\\n    --origin \"l2\"\ndone\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e Create recognition data \u003c/summary\u003e\n\n```\ncd create_training_data/conversation_data\n# List of names or folders to process\nNAMES=(\"bo\" \"duck-banana\" \"marie-cat\" \"pusheen-cup\" \"thuytien\"\n       \"brown-duck\" \"dug\" \"mydieu\" \"shiba-black\" \"tokyo-keyboard\"\n       \"butin\" \"elephant\" \"neurips-cup\" \"shiba-gray\" \"toodles-galore\"\n       \"cat-cup\" \"fire\" \"nha-tho-hanoi\" \"shiba-sleep\" \"viruss\"\n       \"chua-thien-mu\" \"henry\" \"nha-tho-hcm\" \"shiba-yellow\" \"water\"\n       \"ciin\" \"khanhvy\" \"oong\" \"thao\" \"willinvietnam\"\n       \"denisdang\" \"lamb\" \"phuc-map\" \"thap-but\" \"yellow-duck\"\n       \"dragon\" \"mam\" \"pig-cup\" \"thap-cham\" \"yuheng\")\n\n# Loop through each folder\nfor NAME in \"${NAMES[@]}\"; do\n  # Define the positive image folder based on the name\n  POSITIVE_IMAGE_FOLDER=\"/mnt/localssd/code/data/yochameleon-data/train/${NAME}\"\n  \n  # Define the negative image folder (assuming it's fixed or can vary similarly)\n  NEGATIVE_IMAGE_FOLDER=\"/mnt/localssd/code/data/yochameleon-data/train/${NAME}/negative_example\"\n  \n  # Define the output file path for the JSON result\n  OUTPUT_FILE=\"/mnt/localssd/code/data/yochameleon-data/train/${NAME}/json\"\n  \n  # Log which folder is being processed\n  echo \"Processing folder: ${NAME}\"\n  \n  # Execute the Python script with the required arguments\n  python create_conversation.py \\\n    --positive_image_folder \"$POSITIVE_IMAGE_FOLDER\" \\\n    --negative_image_folder \"$NEGATIVE_IMAGE_FOLDER\" \\\n    --output_file \"$OUTPUT_FILE\" \\\n    --limit_positive 5 \\\n    --limit_negative 100\ndone\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e Simple conversation data \u003c/summary\u003e \n\n```\ncd create_training_data/dense_caption\n\n# List of names or folders to process -- For human\nNAMES=(\"thuytien\" \"viruss\" \"ciin\" \"khanhvy\" \"oong\" \"thao\" \"willinvietnam\" \"denisdang\" \"phuc-map\" \"yuheng\")\n\n# Loop through each folder\nfor NAME in \"${NAMES[@]}\"; do\n  # Define the positive image folder based on the name\n  POSITIVE_IMAGE_FOLDER=\"/mnt/localssd/code/data/yochameleon-data/train/${NAME}\"\n  \n  # Define the negative image folder (assuming it's fixed or can vary similarly)\n  NEGATIVE_IMAGE_FOLDER=\"/mnt/localssd/code/data/yochameleon-data/train/${NAME}/negative_example\"\n  \n  # Define the output file path for the JSON result\n  OUTPUT_FILE=\"/mnt/localssd/code/data/yochameleon-data/train/${NAME}/json\"\n  \n  # Log which folder is being processed\n  echo \"Processing folder: ${NAME}\"\n  \n  # Execute the Python script with the required arguments\n  python gpt4o-api.py \\\n    --input_image_folder \"$POSITIVE_IMAGE_FOLDER\" \\\n    --prompt_file_path ./system-prompts/text-conversation.txt \\\n    --output_file \"$OUTPUT_FILE\" \\\n    --text_conversation \\\n    --human \\\n    --limit 5\ndone\n\n# List of names or folders to process -- For object\nNAMES=(\"bo\" \"duck-banana\" \"marie-cat\" \"pusheen-cup\"\n       \"brown-duck\" \"dug\" \"mydieu\" \"shiba-black\" \"tokyo-keyboard\"\n       \"butin\" \"elephant\" \"neurips-cup\" \"shiba-gray\" \"toodles-galore\"\n       \"cat-cup\" \"fire\" \"nha-tho-hanoi\" \"shiba-sleep\"\n       \"chua-thien-mu\" \"henry\" \"nha-tho-hcm\" \"shiba-yellow\" \"water\"\n       \"lamb\" \"thap-but\" \"yellow-duck\"\n       \"dragon\" \"mam\" \"pig-cup\" \"thap-cham\")\n\n# Loop through each folder\nfor NAME in \"${NAMES[@]}\"; do\n  # Define the positive image folder based on the name\n  POSITIVE_IMAGE_FOLDER=\"/mnt/localssd/code/data/yochameleon-data/train/${NAME}\"\n  \n  # Define the negative image folder (assuming it's fixed or can vary similarly)\n  NEGATIVE_IMAGE_FOLDER=\"/mnt/localssd/code/data/yochameleon-data/train/${NAME}/negative_example\"\n  \n  # Define the output file path for the JSON result\n  OUTPUT_FILE=\"/mnt/localssd/code/data/yochameleon-data/train/${NAME}/json\"\n  \n  # Log which folder is being processed\n  echo \"Processing folder: ${NAME}\"\n  \n  # Execute the Python script with the required arguments\n  python gpt4o-api.py \\\n    --input_image_folder \"$POSITIVE_IMAGE_FOLDER\" \\\n    --prompt_file_path ./system-prompts/text-conversation.txt \\\n    --output_file \"$OUTPUT_FILE\" \\\n    --text_conversation \\\n    --limit 5\ndone\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e Image generation data \u003c/summary\u003e\n\n```\ncd create_training_data/retrieve_negative\n# List of names or folders to process\nNAMES=(\"bo\" \"duck-banana\" \"marie-cat\" \"pusheen-cup\" \"thuytien\"\n       \"brown-duck\" \"dug\" \"mydieu\" \"shiba-black\" \"tokyo-keyboard\"\n       \"butin\" \"elephant\" \"neurips-cup\" \"shiba-gray\" \"toodles-galore\"\n       \"cat-cup\" \"fire\" \"nha-tho-hanoi\" \"shiba-sleep\" \"viruss\"\n       \"chua-thien-mu\" \"henry\" \"nha-tho-hcm\" \"shiba-yellow\" \"water\"\n       \"ciin\" \"khanhvy\" \"oong\" \"thao\" \"willinvietnam\"\n       \"denisdang\" \"lamb\" \"phuc-map\" \"thap-but\" \"yellow-duck\"\n       \"dragon\" \"mam\" \"pig-cup\" \"thap-cham\" \"yuheng\")\n\n# Loop through each folder\nfor NAME in \"${NAMES[@]}\"; do\n  # Define the positive image folder based on the name\n  POSITIVE_IMAGE_FOLDER=\"/mnt/localssd/code/data/yochameleon-data/train/${NAME}\"\n  \n  # Define the output file path for the JSON result\n  OUTPUT_FILE=\"/mnt/localssd/code/data/yochameleon-data/train/${NAME}/json\"\n  \n  # Log which folder is being processed\n  echo \"Processing folder: ${NAME}\"\n  \n  # Execute the Python script with the required arguments\n  python create_conversation_by_ranking.py \\\n    --input_folder \"$POSITIVE_IMAGE_FOLDER\" \\\n    --save_folder \"$OUTPUT_FILE\" \\\n    --version image_gen_positive_only \\\n    --num_of_real_images 100 \\ch\n    --token_length 16 \\\n    --spacing 16\ndone\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e Soft positive data \u003c/summary\u003e\n\n```\ncd create_training_data/retrieve_negative\n# List of names or folders to process\n# NAMES=(\"bo\" \"duck-banana\" \"marie-cat\" \"pusheen-cup\" \"thuytien\"\n#        \"brown-duck\" \"dug\" \"mydieu\" \"shiba-black\" \"tokyo-keyboard\"\n#        \"butin\" \"elephant\" \"neurips-cup\" \"shiba-gray\" \"toodles-galore\"\n#        \"cat-cup\" \"fire\" \"nha-tho-hanoi\" \"shiba-sleep\" \"viruss\"\n#        \"chua-thien-mu\" \"henry\" \"nha-tho-hcm\" \"shiba-yellow\" \"water\"\n#        \"ciin\" \"khanhvy\" \"oong\" \"thao\" \"willinvietnam\"\n#        \"denisdang\" \"lamb\" \"phuc-map\" \"thap-but\" \"yellow-duck\"\n#        \"dragon\" \"mam\" \"pig-cup\" \"thap-cham\" \"yuheng\")\n\nNAMES=(\"bo\" \"mam\" \"thuytien\" \"viruss\" \"ciin\" \"khanhvy\" \"oong\" \"thao\" \"willinvietnam\" \"denisdang\" \"phuc-map\" \"yuheng\")\n# NAMES=(\"bo\")\n# Loop through each folder\nfor NAME in \"${NAMES[@]}\"; do\n  # Define the positive image folder based on the name\n  POSITIVE_IMAGE_FOLDER=\"/mnt/localssd/code/data/yochameleon-data/train/${NAME}\"\n  NEGATIVE_IMAGE_FOLDER=\"/mnt/localssd/code/data/yochameleon-data/train/${NAME}/negative_example\"\n  # Define the output file path for the JSON result\n  OUTPUT_FILE=\"/mnt/localssd/code/data/yochameleon-data/train/${NAME}/json\"\n  \n  # Log which folder is being processed\n  echo \"Processing folder: ${NAME}\"\n  \n  # Execute the Python script with the required arguments\n  python create_conversation_by_ranking.py \\\n    --input_folder \"$POSITIVE_IMAGE_FOLDER\" \\\n    --save_folder \"$OUTPUT_FILE\" \\\n    --version '2000' \\\n    --num_of_real_images -100 \\\n    --token_length 16 \\\n    --spacing 1 \\\n    --negative_image True \\\n    --limit_negative 2000\ndone\n```\n\u003c/details\u003e\n\u003cimg src=\"./images/soft-positive.png\" alt=\"YoChameleon\" width=\"500\"\u003e\n\n---\n\n### 🧑‍🏫 Train\n\n```\n#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@\n#\n#   ATTENTION: PLEASE CHECK/EDIT THE CONFIG FILE BEFORE RUNNING (IF NEEDED)\n#\n#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@\n\npython train.py --config config/basic.yaml\n```\n\nIf do you NOT want to use the `wandb` for logging (e.g., for debugging), you can turn off by\n\n```\npython train.py --config config/basic.yaml --no_wandb\n```\n\nOr multiple concept training bash script are given in `scripts` folder\n\n```\nbash train.sh\n```\n\n### 🧪 Test\n\n```\n# This test will generated \"A photo of \u003csks\u003e\" and saved to `./generated_images`\n\npython test.py --config config/basic.yaml \n# (Or optionally, you can also provide override arguments: e.g., python test.py --config config/basic.yaml --exp_name 1000 --savedir ../ckpt --sks_name \"mam\")\n\n# Bash script for test\n\nbash scripts/test.sh\n```\n\n\n### 📊 Evaluation\n\n#### Detailed Captions\n\nDetailed captions for each subject in [Yo'LLaVA datasets](https://github.com/WisconsinAIVision/YoLLaVA) are given in [baselines/subject-detailed-captions.json](./baselines/subject-detailed-captions.json).\n\nFor example, the detailed caption for `bo` is given as follows:\n\n```\n\"bo\": \"\u003csks\u003e is a charming cinnamon-colored Shiba Inu with cream accents and a cheerful personality, appears in various indoor and outdoor settings—posing on rugs, floors, and sidewalks. Often seen with a playful expression or tongue out, this Shiba enjoys relaxing, smiling for the camera, and is sometimes accompanied by a plush toy or sitting attentively in anticipation of a walk.\"\n```\n\u003cimg src=\"https://thaoshibe.github.io/visii/images/1_0.png\" alt=\"Bo\" width=\"300\"\u003e\n\n#### Facial similarity scores\n\n```\npython insightface_verify.py --real_folder /path/to/real/folder --fake_folder /path/to/fake/folder\n```\n\n\nOr edit the file `bash scripts/eval/eval_facial_sim.sh`.\n\n\u003cdetails\u003e\n\u003csummary\u003e facial similarity compute between fake/ real folders\u003c/summary\u003e\n\n```\n#!/bin/bash\ncd ../evaluation/\n\nEXP_FOLDER=\"64-5000\"\nFAKE_FOLDER_BASE=\"/sensei-fs/users/thaon/code/generated_images\"\n\n# Define the real folder\nREAL_FOLDER=\"/mnt/localssd/code/data/yollava-data/train/thao\"\n\n# Define an array of fake folders\nFAKE_FOLDERS=(\n    # Local and aligned folders\n    # \"/mnt/localssd/code/data/dathao_algined\"\n    # \"/mnt/localssd/code/data/yollava-data/train/khanhvy\"\n    # \"/mnt/localssd/code/data/yollava-data/train/thao/negative_example\"\n    # Generated image sets using FAKE_FOLDER_BASE\n    \"${FAKE_FOLDER_BASE}/${EXP_FOLDER}/1000\"\n    \"${FAKE_FOLDER_BASE}/${EXP_FOLDER}/2000\"\n    \"${FAKE_FOLDER_BASE}/${EXP_FOLDER}/3000\"\n    \"${FAKE_FOLDER_BASE}/${EXP_FOLDER}/4000\"\n    \"${FAKE_FOLDER_BASE}/${EXP_FOLDER}/4050\"\n    \"${FAKE_FOLDER_BASE}/${EXP_FOLDER}/4100\"\n    \"${FAKE_FOLDER_BASE}/${EXP_FOLDER}/4150\"\n    \"${FAKE_FOLDER_BASE}/${EXP_FOLDER}/4200\"\n)\n\n# Loop through each fake folder and run the Python evaluation script\nfor FAKE_FOLDER in \"${FAKE_FOLDERS[@]}\"\ndo\n    echo \"Running evaluation with fake folder: $FAKE_FOLDER\"\n    python insightface_verify.py --real_folder \"$REAL_FOLDER\" --fake_folder \"$FAKE_FOLDER\"\ndone\n\necho \"All evaluations completed!\"\n\n```\n\u003c/details\u003e\n\n#### CLIP Image-to-Image Similarity\n\n```\npython clip_image_similarity.py --real_folder /path/to/real/folder --fake_folder /path/to/fake/folder\n```\n\n\u003cdetails\u003e\n\u003csummary\u003e clip similarity score between fake/ real folders\u003c/summary\u003e\n\n```\n#!/bin/bash\ncd ../evaluation/\n\nEXP_FOLDER=\"64-5000\"\nFAKE_FOLDER_BASE=\"/sensei-fs/users/thaon/code/generated_images\"\n\n# Define the real folder\nREAL_FOLDER=\"/mnt/localssd/code/data/yollava-data/train/thao\"\n\n# Define an array of fake folders\nFAKE_FOLDERS=(\n    # Local and aligned folders\n    # \"/mnt/localssd/code/data/dathao_algined\"\n    # \"/mnt/localssd/code/data/yollava-data/train/khanhvy\"\n    # \"/mnt/localssd/code/data/yollava-data/train/thao/negative_example\"\n    # Generated image sets using FAKE_FOLDER_BASE\n    \"${FAKE_FOLDER_BASE}/${EXP_FOLDER}/1000\"\n    \"${FAKE_FOLDER_BASE}/${EXP_FOLDER}/2000\"\n    \"${FAKE_FOLDER_BASE}/${EXP_FOLDER}/3000\"\n    \"${FAKE_FOLDER_BASE}/${EXP_FOLDER}/4000\"\n    \"${FAKE_FOLDER_BASE}/${EXP_FOLDER}/4050\"\n    \"${FAKE_FOLDER_BASE}/${EXP_FOLDER}/4100\"\n    \"${FAKE_FOLDER_BASE}/${EXP_FOLDER}/4150\"\n    \"${FAKE_FOLDER_BASE}/${EXP_FOLDER}/4200\"\n)\n\n# Loop through each fake folder and run the Python evaluation script\nfor FAKE_FOLDER in \"${FAKE_FOLDERS[@]}\"\ndo\n    echo \"Running evaluation with fake folder: $FAKE_FOLDER\"\n    python clip_image_similarity.py --real_folder \"$REAL_FOLDER\" --fake_folder \"$FAKE_FOLDER\"\ndone\n\necho \"All evaluations completed!\"\n\n```\n\u003c/details\u003e\n\n#### Recognition Accuracy\n\n```\npython evaluation/recognition.py --config ./config/recog.yaml --sks_name \"thao\" --iteration 15\n```\n---\n\n\u003c!-- ##### TODO\n\n- [ ] Emu3-Gen related\n  - [ ] Now only support train for image generation (dataloader support image only, not self-prompting) --\u003e\n\n\n### 🖊️ Citation\n\n\n```\n@inproceedings{yochameleon,\n  author = {Thao Nguyen and Krishna Kumar Singh and Jing Shi and Trung Bui and Yong Jae Lee and Yuheng Li},\n  title = {Yo\\textquotesingle Chameleon: Personalized Vision and Language Generation},\n  year = {2025},\n  booktitle = {Proceedings of the {IEEE} Conference on Computer Vision and Pattern Recognition (CVPR)}\n}\n\n@inproceedings{yollava,\n author = {Nguyen, Thao and Liu, Haotian and Li, Yuheng and Cai, Mu and Ojha, Utkarsh and Lee, Yong Jae},\n booktitle = {Advances in Neural Information Processing Systems},\n title = {Yo\\textquotesingle LLaVA: Your Personalized Language and Vision Assistant},\n year = {2024}\n}\n```\n\n\n### 🤗 Acknowledgements\n\nThis project will not be possible without the following open-source projects:\n- [Chameleon: Mixed-Modal Early-Fusion Foundation Models](https://github.com/facebookresearch/chameleon)\n- [Anole: An Open, Autoregressive and Native Multimodal Models for Interleaved Image-Text Generation](https://gair-nlp.github.io/anole/)\n- [Emu3: Next-Token Prediction is All You Need](https://github.com/baaivision/Emu3/tree/main)\n- and amazing HuggingFace's community: [Chamleon on HuggingFace](https://huggingface.co/docs/transformers/en/model_doc/chameleon), [Anole on HuggingFace](https://github.com/huggingface/transformers/pull/32013), [Emu3 on HuggingFace](https://github.com/huggingface/transformers/pull/33770)\n\n\u003c!-- \nI would like to express my gratitude to my Adobe Research's mentors: [Dr. Krishna](https://krsingh.cs.ucdavis.edu/), [Dr. Jing Shi](https://jshi31.github.io/jingshi/), and [Dr. Trung Bui](https://sites.google.com/site/trungbuistanford/) for their discussions. Special thanks to my advisor, [Prof. Yong Jae Lee](https://pages.cs.wisc.edu/~yongjaelee/), who provided endless insights and guidance for this project (as always).\n\nA big shout-out to my primary fellow mentee [Sicheng Mo](https://sichengmo.github.io/)—he taught me so much about coding. Without him, I’d still be using TensorBoard instead of WanDB! (Also, he has wonderful taste in food and restaurants.) \u003cbr\u003e\nAdditionally, thanks to (technically-not) mentor [Fangzhou Mu](https://fmu2.github.io/) for hosting many Friday dinners and board game nights during the summer 🥓🍣🍱 (though, he’s not a fan of Thai foods —meh~).  \n\nAnd finally, saving the best for last: I couldn’t have completed this project without the unwavering support (and pushes) of my main Adobe `juan` mentor, [Dr. Yuheng Li](https://yuheng-li.github.io/) :xixi:. Thank you so much! --\u003e\n\n\n\n  ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣀⣀⣀⣀⠀⠀⠀⢀⡤⠤⠤⣄⠀⣀⣀⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀\n  ⠀⠀⠀⠀⠀⠀⠀⠀⣀⣀⠤⢴⣴⠒⠉⠹⣴⣏⠀⠀⠀⡀⠈⢇⠀⠀⣼⠀⠀⠀⠘⣶⠇⠀⢨⢃⡾⠓⠲⢤⠀⠀⠀⠀⠀⠀\n  ⠀⠀⠀⣀⠤⠔⠒⠙⣯⣇⠀⠈⣿⣇⠀⠀⣿⣿⣿⠀⠀⣷⠀⠘⡄⠀⣿⠀⠀⠀⠀⢹⠀⠀⢸⡏⠇⠀⢀⠇⣀⠤⠒⠒⠤⣄\n  ⢰⡖⠉⠀⠀⠀⠀⣀⣸⣿⠀⠀⠉⠉⠀⠀⢸⠁⣿⠀⠈⠉⠁⠀⢱⠀⣿⠀⠀⣦⠀⠀⠀⠀⣿⡸⠀⠀⠘⠉⠀⠀⣀⣤⣴⠟\n  ⢼⢣⣀⣴⡀⠀⠘⡿⠏⠗⡆⠀⠠⣶⡆⠀⠸⡄⡏⠀⠀⣶⣷⠀⠀⢧⣿⠀⠀⣿⡆⠀⠀⢸⣿⠃⠀⢰⡄⠀⠐⡿⠛⠋⠀⠀\n  ⠘⢿⡿⢿⣧⠀⠀⢳⠀⢸⠸⠀⠀⢹⣧⢀⣀⣷⣧⣤⣤⠛⣏⣦⣤⣾⣿⢦⣤⣿⢸⣄⣀⣼⡏⠀⢠⡟⡇⠀⠀⡇⠀⠀⠀⠀\n  ⠀⠀⠀⠀⢏⢇⠀⠀⣣⠀⣆⣷⣶⣿⣿⡿⠿⠿⢷⡿⠟⣠⠟⠋⠛⢿⡛⠛⠿⡼⠿⠿⢿⣿⣿⣶⠞⡅⢸⠀⠀⢸⠀⠀⠀⠀\n  ⠀⠀⠀⠀⠘⣾⣿⣿⠇⢠⣟⠉⠙⠷⡿⠀⠀⠀⢸⢀⡼⠁⠀⣀⠀⠀⠹⡄⡼⡇⠀⠀⡜⣸⡏⠙⠢⣧⣾⣦⣀⢸⠀⠀⠀⠀\n  ⠀⠀⠀⠀⠀⠈⠀⠀⠀⢿⣿⣷⣦⡀⠀⠀⠀⠀⣇⡾⠀⠀⣼⣿⢷⠀⠀⢻⢱⠀⠀⢀⣿⡿⠀⠀⢠⠋⢻⡿⠿⣏⠀⠀⠀⠀\n  ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠳⣿⣿⠆⠀⠀⢸⡏⡇⠀⠀⡏⡟⡟⠀⠀⢸⡸⠀⠀⢸⣿⠃⠀⠀⡜⡰⢩⠃⠀⠈⣱⠀⠀⠀\n  ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⢹⠀⠀⠀⢸⠀⡇⠀⠀⠙⠋⠀⠀⢀⡏⡇⠀⠀⠘⠋⠀⠀⣰⣱⢣⠇⠀⠀⣰⠃⠀⠀⠀\n  ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⡘⡎⠀⠀⠀⡏⣿⣧⡀⠀⠀⠀⠀⢀⣾⣷⡇⠀⠀⠀⠀⠀⢠⣯⣧⣾⣦⣄⣰⠃⠀⠀⠀⠀\n  ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣧⣧⣤⣶⣶⠃⠘⢿⣿⣷⣶⣶⣾⠟⠉⣿⣿⣦⣄⣀⣠⣴⢏⣽⠋⠉⠙⢿⠁⠀⠀⠀⠀⠀\n  ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠘⠛⠛⠛⠋⠁⠀⠀⠀⠉⠉⠉⠉⠀⠀⠀⠈⠛⠻⠿⠟⠋⠁⣿⣿⣦⣀⣀⡼⠀⠀⠀⠀⠀⠀\n  ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠙⠛⠛⠛⠁⠀⠀⠀⠀⠀⠀⠀\n\n\n\u003c!-- *Finally, I've wrapped up this project -- I'll go home and hug my pets now ☕,  hehe (.❛ ᴗ ❛.)* --\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwisconsinaivision%2Fyochameleon","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwisconsinaivision%2Fyochameleon","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwisconsinaivision%2Fyochameleon/lists"}