{"id":27693209,"url":"https://github.com/saarinii/poseperfect","last_synced_at":"2025-10-12T22:13:12.156Z","repository":{"id":284371445,"uuid":"867663792","full_name":"saarinii/PosePerfect","owner":"saarinii","description":" PosePerfect is a Python project that uses generative AI for object pose editing. It detects and segments objects using YOLOv5 and SAM, then modifies their pose with Stable Diffusion inpainting. The project outputs segmented or inpainted images and includes simple setup and usage scripts.","archived":false,"fork":false,"pushed_at":"2025-03-25T14:25:15.000Z","size":6638,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-25T15:34:46.354Z","etag":null,"topics":["ai-based-pose-modification","image-manipulation","inpainting","object-segmentation","output","pose-editing","sam-for-segmentation","segmentation","stable-diffusion-inpainting","yolov5-for-object-detection"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/saarinii.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-04T13:40:19.000Z","updated_at":"2025-03-25T14:25:18.000Z","dependencies_parsed_at":"2025-03-25T15:45:05.812Z","dependency_job_id":null,"html_url":"https://github.com/saarinii/PosePerfect","commit_stats":null,"previous_names":["saarinii/poseperfect"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/saarinii%2FPosePerfect","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/saarinii%2FPosePerfect/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/saarinii%2FPosePerfect/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/saarinii%2FPosePerfect/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/saarinii","download_url":"https://codeload.github.com/saarinii/PosePerfect/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250822427,"owners_count":21492902,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-based-pose-modification","image-manipulation","inpainting","object-segmentation","output","pose-editing","sam-for-segmentation","segmentation","stable-diffusion-inpainting","yolov5-for-object-detection"],"created_at":"2025-04-25T12:56:21.856Z","updated_at":"2025-10-12T22:13:12.072Z","avatar_url":"https://github.com/saarinii.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PosePerfect\nPosePerfect is a Python project that allows you to edit the pose of an object within a scene using cutting-edge generative AI techniques. This project focuses on two key tasks:\n\n## Object Segmentation: \n### PosePerfectT1.ipynb (.ipynb for quicker view of the code and the output), PosePerfectTask1/run.py (to run the code)\n\nWhat it does: Isolates the target object from the background image.\n\nHow it works: Takes an input image and a text prompt specifying the object class (e.g., chair, table).\n\nOutput: A segmented image with a red mask highlighting the object's boundaries.\n\nThis code utilizes YOLOv5 for object detection and the Segment Anything Model (SAM) for segmentation to detect, segment, and visually highlight specified objects in images.\n\n### Key Components\n#### Library Imports:\n\nOpenCV (cv2): For image processing (reading, writing).\n\nNumPy (np): For numerical operations on image data.\n\nTorch: To load and run the YOLOv5 and SAM models.\n\nMatplotlib (plt): For displaying images.\n\n#### Configuration:\n\nThe model path (CHECKPOINT_PATH) and type (MODEL_TYPE) are set, and the device is configured to use a GPU if available.\n\n#### Models:\n\nSAM is loaded using its checkpoint and set to the specified device.\nAn instance of SamAutomaticMaskGenerator is created for segmentation.\nThe YOLOv5 model is loaded from Ultralytics’ repository as a pre-trained model.\n\n#### Image Display Function:\n\nThe display_image function reads an image, converts it from BGR to RGB, and displays it using Matplotlib.\n\n#### Detection and Segmentation Function:\n\nThe function detects objects in the image and checks for the target object.\nIf detected, it extracts the region of interest (ROI), segments it using SAM, and applies a red highlight before saving and displaying the final image.\n\n![chair](https://github.com/user-attachments/assets/ec8ac2c6-1ce0-4379-8c3d-1a7f555ea1be)\n![highlighted_image1](https://github.com/user-attachments/assets/95d69d57-763c-46e0-a61e-2f6257182526)\n![highlighted_chair_red](https://github.com/user-attachments/assets/5d895fc3-542d-4361-be6f-b4c483629832)\n\n\n## SetUp\n1. Open VSCode \n2. Copy the given command\n```shell\ngit clone https://github.com/saarinii/PosePerfect.git\n```\n3. Navigate to the project directory:\n```shell\ncd PosePerfectTask1\n```\n4. Install the required packages:\n```shell\npip install -r requirements.txt\n```\n5. You can now run the script as follows::\n```shell\npython run.py --image ./example.jpg --class \"chair\" --output ./generated.png\n```\nReplace ./example.jpg with the path to your input image, \"chair\" with the object class you want to segment, and ./generated.png with the desired output path.\n\n## Pose Editing: \n### PosePerfectT2.ipynb (.ipynb for quicker view of the code and the output), PosePerfectTask2/run.py (to run the code)\nWhat it does:\nDetects a target object in an image, segments it, and then uses inpainting to modify the image based on the mask of the detected object.\n\nHow it works:\nThe script accepts an input image and a text prompt specifying the object class (e.g., chair, table). It uses YOLOv5 to detect the specified object, SAM to generate a mask around the object, and Stable Diffusion inpainting to modify the image based on the mask.\n\nOutput:\nAn inpainted image where the specified object is detected, segmented, and modified (e.g., filled in or removed) using Stable Diffusion. The output is saved as a new image file.\n\n### Key Components\n\n#### Library Imports:\n\nOpenCV (cv2): For image processing (reading, writing, and displaying).\nNumPy (np): For numerical operations on image data.\nTorch: For loading and running the YOLOv5, SAM, and Stable Diffusion models.\nPIL (Pillow): For handling images in the inpainting process.\nDiffusers: To handle the Stable Diffusion inpainting pipeline.\n\n#### Configuration:\n\nThe script checks if CUDA is available to run models on a GPU.\nIt specifies a pre-trained YOLOv5 model from Ultralytics for object detection.\nThe Segment Anything Model (SAM) is loaded using a checkpoint, with its type (e.g., vit_h) specified.\n\n#### Models:\n\nYOLOv5: Loaded from Ultralytics, this pre-trained model detects objects in the input image and extracts the region of interest (ROI) for further segmentation.\nSAM (Segment Anything Model): SAM is used to generate a mask around the detected object, allowing for precise segmentation.\nStable Diffusion Inpainting: Uses the diffusers library to apply inpainting on the segmented area, modifying the image based on the generated mask.\nDetection and Segmentation Function:\n\nDetecting Objects:\nThe script reads an image and detects objects using YOLOv5, identifying the specified object class by its label (e.g., chair). The detected object's coordinates are used to extract the ROI.\n\nSegmentation:\nSAM is employed to generate a mask for the object within the ROI, and the best mask (highest confidence) is selected for further processing.\n\nInpainting:\nThe mask is applied to the image, and Stable Diffusion is used to perform inpainting on the segmented region. This modifies the image based on the prompt and mask (e.g., removing or replacing the object).\n\n#### Image Saving and Output:\nThe script saves the segmented mask and the final inpainted image as separate output files, ensuring you get both the segmented and modified versions of the input image.\n\n\n![chair](https://github.com/user-attachments/assets/ee374a3d-8f7c-430e-a3a2-94301abb5a97)\n![isolated_chair](https://github.com/user-attachments/assets/a2310af0-6922-494f-8164-12f29c160730)\n![isolated_chair (1)](https://github.com/user-attachments/assets/2ebdfd22-f601-4fbd-b600-c83dd7eba415)\n\n![rotated_chair (3)](https://github.com/user-attachments/assets/1f350b35-5f22-42e7-a663-05728f09177d)\n\n![highlighted_chair_white](https://github.com/user-attachments/assets/8c7b1ddf-d1a8-401a-bf62-f8f9697fd091)\n\n\n## SetUp\n1. Open VSCode \n2. Copy the given command\n```shell\ngit clone https://github.com/saarinii/PosePerfect.git\n```\n3. Navigate to the project directory:\n```shell\ncd PosePerfectTask2\n```\n4. Install the required packages:\n```shell\npip install -r requirements.txt\n```\n5. You can now run the script as follows::\n```shell\npython run.py --image ./example.jpg --class \"chair\" --azimuth +72 --polar +0 --output ./generated.png\n```\nReplace ./example.jpg with the path to your input image, \"chair\" with the object class you want to segment, ./generated.png with the desired output path, and '--azimuth +72 --polar +0' with the angle you want to rotate the object to. \n## My trials\nTo look more into my learning journey and what all the experimenting and trial and errors I had to do to make this project and a lot of my mistakes head to \n### AvatarAssignmentAllMyTrys.ipynb \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsaarinii%2Fposeperfect","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsaarinii%2Fposeperfect","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsaarinii%2Fposeperfect/lists"}