{"id":17632832,"url":"https://github.com/shumash/colortriads","last_synced_at":"2025-04-23T21:10:36.834Z","repository":{"id":80025798,"uuid":"302057385","full_name":"shumash/colortriads","owner":"shumash","description":"Code for SIGGRAPH 2020 paper \"Nonlinear Color Triads\" by Shugrina et al.","archived":false,"fork":false,"pushed_at":"2025-03-22T22:57:57.000Z","size":1026,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-30T03:32:02.482Z","etag":null,"topics":["colors","creative","machine-learning","recoloring"],"latest_commit_sha":null,"homepage":"https://colorsandbox.com/research/triads","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shumash.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-10-07T14:11:56.000Z","updated_at":"2025-03-22T23:01:36.000Z","dependencies_parsed_at":null,"dependency_job_id":"403eb4e0-94da-4a9b-be8e-300f1ecc50f7","html_url":"https://github.com/shumash/colortriads","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shumash%2Fcolortriads","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shumash%2Fcolortriads/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shumash%2Fcolortriads/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shumash%2Fcolortriads/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shumash","download_url":"https://codeload.github.com/shumash/colortriads/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250514789,"owners_count":21443209,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["colors","creative","machine-learning","recoloring"],"created_at":"2024-10-23T01:46:04.475Z","updated_at":"2025-04-23T21:10:36.816Z","avatar_url":"https://github.com/shumash.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"teaser.jpg\"\u003e\n\u003c/p\u003e\n\n# Color Triads\n\nThis is the codebase for the SIGGRAPH 2020 paper \"Nonlinear Color Triads for Approximation, Learning and Direct Manipulation of Color Distributions\" by Masha Shugrina, Amlan Kar,\nSanja Fidler and Karan Singh. **TL;DR:** a color triad is a compact color distribution\nrepresentation that can be fit to images or color samples and then used to interactively\ncontrol the entire color distribution, like in the teaser above. \nSee [Project Page](https://colorsandbox.com/research/triads) and interactive demos at [colorsandbox.com](https://colorsandbox.com/recolor). \n\n\n## What this Code Contains\n\nThis code allows fitting color triads to images using two methods: 1) MATLAB-based optimization and 2) legacy tensorflow pre-trained checkpoint, ported to more modern tensorflow. The fitted color triads can be used\nfor recoloring images using interactive interface at [colorsandbox.com](https://colorsandbox.com/). \n\n\n## Fitting Color Triads to Images\n\nThe main script that runs over a whole directory of images is available here:\n```\n./colortriads/learn/tf/color/run_fitting_experiment.sh $TEST_IMG_DIR $OUTPUT_DIR\n```\nThis script should be run twice, with a manual MATLAB step in between. \n\n### Step 0: Environment\n\n:warning: This is a really old codebase written on python2.7 and old tensorflow, and MATLAB. We made a perfunctory\neffort to have the inference run on newer python and tensorflow, but training is not implemented. Inference was tested with CPU only tensorflow running on Mac OS X (the model is tiny, so this is reasonably fast). To construct the environment run:\n```\nconda create --name triads python=3.10\nconda activate triads\npip install -r requirements.txt\n```\n\n### Step 1: Fitting triads with a neural network (optional)\nThis script will first fit triads to images using a pre-trained neural network, which is worse than matlab optimization.\nIf you want to do that, first download and unzip the [checkpoint](https://drive.google.com/file/d/1bJu5Y_U3Jk6rlKbUyiLoeHDTqCi43521/view?usp=sharing) as follows:\n```\nmkdir models\nunzip ~/Downloads/triad_network.zip -d models/.\n```\nNote that the port of the network from earlier version of tensorflow is not very \ncareful and there could be issues. To fit triads with a neural network, run:\n```\n./colortriads/learn/tf/color/run_fitting_experiment.sh $TEST_IMG_DIR $OUTPUT_DIR\n```\nIf you want to skip neural\nnetwork fitting, just run `mkdir $OUTPUT_DIR/palettes_deep` instead.\n\n\n### Step 2: Fitting triads with optimization in MATLAB (best, but slow)\n\nWe tested this code with MATLAB 2024 version; you will require optimization toolbox. To run MATLAB color triad optimization of images in `$TEST_IMG_DIR`, open MATALB and run the following:\n```\n%% Navigate to \"matlab\" subdirectory \ndoIncludes  %% run this\n%% Naviate to matlab/color subdirectory\ncolorSailExperiments_Results(\u003ctest_img_dir\u003e/, \u003coutput_dir\u003e/, true);  %% fit triads\n```\nAfter this, your `$OUTPUT_DIR` will contain two subdirectories for fitted palettes, `palettes_deep` and `palettes`. \n\n### Step 3: Computing triad-image mapping to allow recoloring\n\nThe next step is computing the mapping from image pixels to color triad. To do that, simply re-run the same script:\n\n```\n./colortriads/learn/tf/color/run_fitting_experiment.sh $TEST_IMG_DIR $OUTPUT_DIR\n```\n\nNow, you will have `mappings` subdirectories containing the binary mapping files that can be used on colorsandbox.com.\n\n\n## Recoloring UI\n\nThe result of step 3 above can be uploaded to a special url to allow recoloring the source images in our web GUI. Navigate to [special colorsandbox.com URL](https://colorsandbox.com/dev0a1b2c?bin=yes\u0026no3d) and upload the `*.RES.binary` file from your `mappings` or `mappings_deep` subdirectory. You should now\nbe able to use the recoloring UI.\n\nNote: the recoloring UI uses offset colors by default (i.e. it offsets errors by a per-pixel delta). In order to see only the triad colors\nin the final image, open debug console in your broswer and type:\n```\ncontroller.repainter.use_delta_offset = false\n```\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"gui.jpg\"\u003e\n\u003c/p\u003e\n\n## License and Patents\n\nWhile the codebase is released under MIT License, which does not include a patent grant. Keep in mind that this work is patented by the University of Toronto. Please file an issue if you would like to use color triads as a part of a commercial software application, and we will be happy to set up an agreement with the University of Toronto. \n\n\"Method and System for Color Representation Generation.\" Shugrina et al. US Patent, Issued 2021. \n\nUse of codebase for research and evaluation purposes is encouraged. One-off artistic usage is also encouraged. \n\n## Citation\n\nPlease cite our paper if you find this codebase useful.\n\n```\n@article{shugrina2020nonlinear,\n  title={Nonlinear color triads for approximation, learning and direct manipulation of color distributions},\n  author={Shugrina, Maria and Kar, Amlan and Fidler, Sanja and Singh, Karan},\n  journal={ACM Transactions on Graphics (TOG)},\n  volume={39},\n  number={4},\n  pages={97--1},\n  year={2020},\n  publisher={ACM New York, NY, USA}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshumash%2Fcolortriads","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshumash%2Fcolortriads","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshumash%2Fcolortriads/lists"}