{"id":16443189,"url":"https://github.com/andreihar/segpalette","last_synced_at":"2026-04-04T00:03:41.389Z","repository":{"id":233410102,"uuid":"786632350","full_name":"andreihar/segpalette","owner":"andreihar","description":"Instance Recolouring by Palette Generation after Instance Segmentation","archived":false,"fork":false,"pushed_at":"2024-08-27T23:56:29.000Z","size":6165,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-12T18:44:30.318Z","etag":null,"topics":["color-palette","computer-vision","image-processing","recolor","segmentation"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/andreihar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-15T02:10:27.000Z","updated_at":"2024-08-27T23:59:36.000Z","dependencies_parsed_at":"2024-08-28T00:55:59.215Z","dependency_job_id":"dfb2ad02-991d-4eb2-a999-099e369fea02","html_url":"https://github.com/andreihar/segpalette","commit_stats":null,"previous_names":["andreihar/cmpt461","andreihar/segpalette"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/andreihar/segpalette","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andreihar%2Fsegpalette","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andreihar%2Fsegpalette/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andreihar%2Fsegpalette/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andreihar%2Fsegpalette/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/andreihar","download_url":"https://codeload.github.com/andreihar/segpalette/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andreihar%2Fsegpalette/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":275208474,"owners_count":25424032,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-15T02:00:09.272Z","response_time":75,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["color-palette","computer-vision","image-processing","recolor","segmentation"],"created_at":"2024-10-11T09:19:40.769Z","updated_at":"2025-12-29T00:22:37.906Z","avatar_url":"https://github.com/andreihar.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!-- PROJECT LOGO --\u003e\n\u003cdiv align=\"center\"\u003e\n  \u003ca href=\"https://github.com/andreihar/segpalette\"\u003e\n    \u003cimg src=\"readme/logo.svg\" alt=\"Logo\" width=\"80\" height=\"80\"\u003e\n  \u003c/a\u003e\n  \n# SegPalette\n\n\n\n\u003c!-- PROJECT SHIELDS --\u003e\n[![Contributors][contributors-badge]][contributors]\n[![Licence][licence-badge]][licence]\n\n**Instance Recolouring by Palette Generation after Instance Segmentation**\n\nA tool for recolouring images by generating segmented instances and applying customised colour palettes to enhance visual appeal.\n\n\u003cdiv style=\"position: relative; max-width: 600px;\"\u003e\n  \u003ca href=\"https://youtu.be/7FBLZdGhSWw\"\u003e\n    \u003cimg src=\"https://img.youtube.com/vi/7FBLZdGhSWw/maxresdefault.jpg\" style=\"width: 100%;\"\u003e\n    \u003cdiv style=\"position: absolute; top: 0; left: 0; width: 100%; height: 100%; background: #FFFFFF99;\"\u003e\u003c/div\u003e\n    \u003cdiv style=\"position: absolute; top: 50%; left: 50%; transform: translate(-50%, -50%);\"\u003e\n      \u003cimg src=\"https://upload.wikimedia.org/wikipedia/commons/4/49/View-playback_Gion_simple.svg\" style=\"width: 75px\"\u003e\n    \u003c/div\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\n\u003c/div\u003e\n\n\n\n---\n\n\n\n\u003c!-- TABLE OF CONTENTS --\u003e\n\u003cdetails open\u003e\n  \u003csummary\u003eTable of Contents\u003c/summary\u003e\n  \u003col\u003e\n    \u003cli\u003e\n      \u003ca href=\"#about-the-project\"\u003eAbout The Project\u003c/a\u003e\n      \u003cul\u003e\n        \u003cli\u003e\u003ca href=\"#built-with\"\u003eBuilt With\u003c/a\u003e\u003c/li\u003e\n      \u003c/ul\u003e\n    \u003c/li\u003e\n    \u003cli\u003e\n      \u003ca href=\"#install\"\u003eInstall\u003c/a\u003e\n      \u003cul\u003e\n        \u003cli\u003e\u003ca href=\"#prerequisites\"\u003ePrerequisites\u003c/a\u003e\u003c/li\u003e\n        \u003cli\u003e\u003ca href=\"#frontend-setup\"\u003eFrontend Setup\u003c/a\u003e\u003c/li\u003e\n        \u003cli\u003e\u003ca href=\"#backend-setup\"\u003eBackend Setup\u003c/a\u003e\u003c/li\u003e\n      \u003c/ul\u003e\n    \u003c/li\u003e\n    \u003cli\u003e\n      \u003ca href=\"#functional-areas\"\u003eFunctional Areas\u003c/a\u003e\n      \u003cul\u003e\n        \u003cli\u003e\u003ca href=\"#instance-segmentation\"\u003eInstance Segmentation\u003c/a\u003e\u003c/li\u003e\n        \u003cli\u003e\u003ca href=\"#colour-palette-generation\"\u003eColour Palette Generation\u003c/a\u003e\u003c/li\u003e\n        \u003cli\u003e\u003ca href=\"#recolouring\"\u003eRecolouring\u003c/a\u003e\u003c/li\u003e\n        \u003cli\u003e\u003ca href=\"#web-application\"\u003eWeb Application\u003c/a\u003e\u003c/li\u003e\n        \u003cli\u003e\n          \u003ca href=\"#matlab-recolouring\"\u003eMATLAB Recolouring\u003c/a\u003e\n          \u003cul\u003e\n            \u003cli\u003e\u003ca href=\"#run\"\u003eRun\u003c/a\u003e\u003c/li\u003e\n          \u003c/ul\u003e\n        \u003c/li\u003e\n      \u003c/ul\u003e\n    \u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#contributors\"\u003eContributors\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#licence\"\u003eLicence\u003c/a\u003e\u003c/li\u003e\n  \u003c/ol\u003e\n\u003c/details\u003e\n\n\n\n\u003c!-- ABOUT THE PROJECT --\u003e\n## About The Project\n\nSegPalette is the final project created by a group of students for the course in Computational Photography and Image Manipulation during the Spring semester of 2024. The project was designed with the goal of exploring advanced image processing techniques to create an intuitive tool for selective image recolouring, aimed at both casual users and professionals in fields like fashion design, home decor, and digital art.\n\nThe primary motivation behind SegPalette was to address the limitations of existing recolouring tools, which often lack the precision required for segment-specific colour adjustments. By using cutting-edge technologies like the Segment Anything Model (SAM) from Meta AI and integrating them into a user-friendly web application, SegPalette allows users to easily recolour individual objects within an image without altering the rest of the scene. This level of control opens up new possibilities for creative expression, enabling users to experiment with colours in a way that is both accessible and visually accurate.\n\nDespite challenges, including the incomplete implementation of the recolouring feature within the web app, the team successfully demonstrated the feasibility of their approach through MATLAB simulations. SegPalette stands as a testament to the team's innovative spirit and technical skills, offering a promising foundation for future development in selective image recolouring.\n\n### Built With\n\n* [![ReactJS][react-badge]][react]\n* [![Flask][flask-badge]][flask]\n* [![PyTorch][pytorch-badge]][pytorch]\n\n\n\n\u003c!-- INSTALL --\u003e\n## Install\n\n### Prerequisites\n- Node.js (v14.x or later)\n- Python (v3.7 or later)\n\n### Frontend Setup\n```bash\ncd client\nnpm install\nnpm run dev\n```\n\n### Backend Setup\n\nDownload pre-trained SAM model sam_vit_h_4b8939.pth from [here](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth) and place it in the `server` directory.\n\nInstall dependencies and run the server\n\n```bash\ncd server\npip install -r requirements.txt\npython app.py\n```\n\nThe website can be accessed through the URL `http://localhost:5173/`.\n\n\n\n\u003c!-- FUNCTIONAL AREAS --\u003e\n## Functional Areas\n\n### Instance Segmentation\n\nInstance segmentation within SegPalette is crucial for precise recolouring, enabling colour changes to be applied exclusively to selected objects without altering the entire image. This process primarily utilises the Segment Anything Model (SAM) developed by Meta AI, a robust vision transformer model known for generating highly accurate segmentation masks across diverse and complex image types. SAM's ability to delineate object boundaries with precision ensures that even in cluttered or intricate scenes, individual segments can be effectively isolated for targeted recolouring.\n\nTo enhance user flexibility, SegPalette also supports the use of pre-existing segmentation data through COCO segmentation files, allowing for custom segmentations tailored to specific objects or scenarios. After obtaining the segmentation masks—whether generated by SAM or provided via COCO files—these are decoded using Run-Length Encoding (RLE) to produce binary masks, which precisely define the boundaries of each object for the subsequent recolouring process.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"readme/segmentation.jpg\" alt=\"Segmentation of Interior\" height=\"300\"\u003e\n\u003c/p\u003e\n\n### Colour Palette Generation\n\nThe colour palette for each segmented instance is generated using a K-Means clustering algorithm. K-Means is a widely used method for partitioning data into distinct clusters, which in this context refers to grouping similar colours found within a segment. In SegPalette, we set the number of clusters (k) to 6, which strikes a balance between capturing the variety of colours within a segment and keeping the palette manageable for users to modify.\n\nDuring the clustering process, the algorithm first initialises the cluster centroids based on the colours present within the segmented area. By focusing exclusively on the pixels inside the segmentation mask, the algorithm ensures that the palette accurately represents the colour distribution of the object or region of interest, rather than being influenced by surrounding areas. This approach results in a palette that is well-suited for recolouring tasks, as it reflects the true colour diversity within the segment, allowing for more precise and contextually appropriate colour adjustments.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"readme/palettes.jpg\" alt=\"Generated Palettes\" height=\"70\"\u003e\n\u003c/p\u003e\n\n### Recolouring\n\nRecolouring in SegPalette focuses on altering the chromaticity of colours while preserving the luminance, ensuring that the image’s brightness remains consistent for a realistic appearance. This is achieved using the LAB colour space, where the 'L' channel represents luminance, and the 'a' and 'b' channels represent the colour's green-red and blue-yellow components, respectively. This space is ideal for tasks requiring perceptually uniform colour changes, as it allows for precise adjustments without compromising the natural look of the image.\n\nDuring the recolouring process, SegPalette modifies the 'a' and 'b' channels of the LAB representation to match the selected colours from the generated palette. By calculating the difference between the original and target colours and applying it to these channels, the segment’s colour shifts while the 'L' channel remains unchanged. This method ensures that the original shading and depth are preserved, making the recoloured segments appear natural and integrated within the overall image.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"readme/recoloured.jpg\" alt=\"Recoloured Interior\" height=\"300\"\u003e\n\u003c/p\u003e\n\n### Web Application\n\nThe SegPalette web application provides an intuitive interface for real-time interactive recolouring. Users can select and modify colours of individual segments through a responsive colour picker, receiving instant visual feedback to easily experiment with various colour schemes. To enhance precision, the app features a toggle for the segmentation overlay, allowing users to compare the recoloured image with the original and fine-tune adjustments for the desired result.\n\nIn addition to recolouring, the application supports exporting both the final images and the segmentation data in COCO format, facilitating easy sharing, future edits, or further processing. Built with ReactJS for a responsive user experience and powered by Flask on the backend for efficient processing, the app is designed to be both accessible and robust, serving the needs of both casual users and professionals.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"readme/app.jpg\" alt=\"Web Application\" height=\"350\"\u003e\n\u003c/p\u003e\n\n#### MATLAB Recolouring\n\n\u003cdiv align=\"center\"\u003e\n\u003cdiv style=\"position: relative; max-width: 400px;\"\u003e\n  \u003ca href=\"https://youtu.be/3s16IgF_vp0\"\u003e\n    \u003cimg src=\"https://img.youtube.com/vi/3s16IgF_vp0/maxresdefault.jpg\" style=\"width: 100%;\"\u003e\n    \u003cdiv style=\"position: absolute; top: 0; left: 0; width: 100%; height: 100%; background: #FFFFFF99;\"\u003e\u003c/div\u003e\n    \u003cdiv style=\"position: absolute; top: 50%; left: 50%; transform: translate(-50%, -50%);\"\u003e\n      \u003cimg src=\"https://upload.wikimedia.org/wikipedia/commons/4/49/View-playback_Gion_simple.svg\" style=\"width: 75px\"\u003e\n    \u003c/div\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\u003c/div\u003e\n\nThe recolouring feature, initially intended for the SegPalette web app, was not implemented due to unforeseen project challenges. However, to demonstrate the functionality, we provided Python and MATLAB scripts that allow users to generate binary masks from segmentation data and perform recolouring using MATLAB. This alternative approach validates the concept, proving the effectiveness of our recolouring method even without full web app integration.\n\n#### Run\n\nTo generate binary masks, use the decode_rle.py script from the command line:\n\n```bash\ncd matlab\npip install pycocotools-windows # For Windows\npip install pycocotools # Otherwise\npython decode_rle.py SOURCE_IMAGE.jpg SEGMENTATION_FILE.json\n```\n\nOnce you have the binary masks, load the `palette_demo.m` script in MATLAB and follow the comments to generate palettes and perform the recolouring.\n\n\n\n\u003c!-- CONTRIBUTION --\u003e\n## Contributors\n\n- Brendan Bickford ([Github][brendan-github]) - Theoretical framework, MATLAB palette generation and recolouring, web app palette generation, video presentation, project report.\n- Andrei Harbachov ([Github][andrei-github] · [LinkedIn][andrei-linkedin]) - Web app development (excluding palette generation and recolouring), video presentation, project report, video editing.\n- Carolina Partida Bujanda ([Github][carolina-github]) - Web app recolouring.\n\n\n\n\u003c!-- LICENCE --\u003e\n## Licence\n\nBecause SegPalette is MIT-licensed, any developer can essentially do whatever they want with it as long as they include the original copyright and licence notice in any copies of the source code.\n\n\n\n\u003c!-- MARKDOWN LINKS --\u003e\n\u003c!-- Badges and their links --\u003e\n[contributors-badge]: https://img.shields.io/github/contributors/andreihar/segpalette?style=for-the-badge\u0026color=44cc11\n[contributors]: #contributors\n[licence-badge]: https://img.shields.io/github/license/andreihar/segpalette.svg?color=000000\u0026style=for-the-badge\n[licence]: LICENSE\n[react-badge]: https://img.shields.io/badge/React-087EA4?style=for-the-badge\u0026logo=react\u0026logoColor=ffffff\n[react]: https://react.dev/\n[flask-badge]: https://img.shields.io/badge/Flask-000000?style=for-the-badge\u0026logo=flask\u0026logoColor=white\n[flask]: https://flask.palletsprojects.com/\n[pytorch-badge]: https://img.shields.io/badge/PyTorch-EE4C2C?style=for-the-badge\u0026logo=pytorch\u0026logoColor=white\n[pytorch]: https://pytorch.org/\n\n\u003c!-- Socials --\u003e\n[brendan-github]: https://github.com/BrenBick\n[andrei-linkedin]: https://www.linkedin.com/in/andreihar/\n[andrei-github]: https://github.com/andreihar\n[carolina-github]: https://github.com/caropartida","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fandreihar%2Fsegpalette","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fandreihar%2Fsegpalette","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fandreihar%2Fsegpalette/lists"}