{"id":13737999,"url":"https://github.com/navervision/proxy-synthesis","last_synced_at":"2025-05-08T15:32:13.774Z","repository":{"id":41283607,"uuid":"319590652","full_name":"navervision/proxy-synthesis","owner":"navervision","description":"Official PyTorch implementation of \"Proxy Synthesis: Learning with Synthetic Classes for Deep Metric Learning\" (AAAI 2021)","archived":false,"fork":false,"pushed_at":"2021-12-19T00:45:45.000Z","size":1918,"stargazers_count":34,"open_issues_count":4,"forks_count":7,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-12-16T12:38:53.592Z","etag":null,"topics":["aaai2021","cars196","deep-learning","deep-metric-learning","image-retrieval","metric-learning","norm-softmax","proxy-based","proxy-nca","proxy-synthesis","pytorch","regularizer","synthetic-classes"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/navervision.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-12-08T09:34:53.000Z","updated_at":"2024-06-03T11:44:11.000Z","dependencies_parsed_at":"2022-07-06T16:32:45.587Z","dependency_job_id":null,"html_url":"https://github.com/navervision/proxy-synthesis","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/navervision%2Fproxy-synthesis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/navervision%2Fproxy-synthesis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/navervision%2Fproxy-synthesis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/navervision%2Fproxy-synthesis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/navervision","download_url":"https://codeload.github.com/navervision/proxy-synthesis/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253096309,"owners_count":21853573,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aaai2021","cars196","deep-learning","deep-metric-learning","image-retrieval","metric-learning","norm-softmax","proxy-based","proxy-nca","proxy-synthesis","pytorch","regularizer","synthetic-classes"],"created_at":"2024-08-03T03:02:08.563Z","updated_at":"2025-05-08T15:32:12.506Z","avatar_url":"https://github.com/navervision.png","language":"Python","readme":"# Proxy Synthesis: Learning with Synthetic Classes for Deep Metric Learning\n\nOfficial PyTorch implementation of *\"Proxy Synthesis: Learning with Synthetic Classes for Deep Metric Learning\"* (AAAI 2021)\n\n**Geonmo Gu\\*\u003csup\u003e1\u003c/sup\u003e, Byungsoo Ko\\*\u003csup\u003e1\u003c/sup\u003e, Han-Gyu Kim\u003csup\u003e2\u003c/sup\u003e** (* Authors contributed equally.)\n\n\u003csup\u003e1\u003c/sup\u003e@NAVER/LINE Vision, \u003csup\u003e2\u003c/sup\u003e@NAVER Clova Speech\n\n- [Paper](https://arxiv.org/abs/2103.15454) | [Presentation Video](https://www.youtube.com/watch?v=v_KYo2Crbig) | [PPT](https://www.slideshare.net/ByungSooKo1/aaai2021-proxy-synthesis-learning-with-synthetic-classes-for-deep-metric-learning) | [Poster](https://www.slideshare.net/ByungSooKo1/aaai2021-proxy-synthesis-learning-with-synthetic-classes-for-deep-metric-learning-poster)\n\n## Overview\n\n### Proxy Synthesis\n\n- *Proxy Synthesis (PS)* is a novel regularizer for any softmax variants and proxy-based losses in deep metric learning. \n\n\u003cimg src=\"figures/overview.png\"\u003e\n\n### How it works?\n\n- *Proxy Synthesis* exploits synthetic classes and improves generalization by considering class relations and obtaining smooth decision boundaries.\n- Synthetic classes mimic unseen classes during training phase as described in below Figure.\n\n\u003cimg src=\"figures/tsne.png\"\u003e\n\n### Experimental results\n\n- *Proxy Synthesis* improves performance for every loss and benchmark dataset.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"figures/evaluation.png\" width=\"400\"\u003e\n\u003c/p\u003e\n\n## Getting Started\n\n### Installation\n\n1. Clone the repository locally\n```bash\n$ git clone https://github.com/navervision/proxy-synthesis\n```\n\n2. Create conda virtual environment\n```bash\n$ conda create -n proxy_synthesis python=3.7 anaconda\n$ conda activate proxy_synthesis\n```\n\n3. Install pytorch\n- [Install pytorch](https://pytorch.org/get-started/locally/) according to your CUDA version\n```bash\n$ conda install pytorch torchvision cudatoolkit=\u003cYOUR_CUDA_VERSION\u003e -c pytorch\n```\n\n4. Install faiss\n- We use [faiss](https://github.com/facebookresearch/faiss) library for faster evaluation\n- [Install faiss](https://github.com/facebookresearch/faiss/blob/master/INSTALL.md) according to your CUDA version\n```bash\n$ conda install faiss-gpu cudatoolkit=\u003cYOUR_CUDA_VERSION\u003e -c pytorch\n```\n\n5. Install requirements\n```bash\n$ pip install -r requirements.txt\n```\n\n### Prepare Data\n\n- Download [CARS196](https://ai.stanford.edu/~jkrause/cars/car_dataset.html) dataset and unzip\n\n```bash\n$ wget http://ai.stanford.edu/~jkrause/car196/cars_annos.mat\n$ tar zxvf car_ims.tgz -C ./dataset\n```\n\n- Rearrange CARS196 directory by following structure\n```\n# Dataset structure\n/dataset/carDB/\n  train/\n    class1/\n      img1.jpeg\n    class2/\n      img2.jpeg\n  test/\n    class1/\n      img3.jpeg\n    class2/\n      img4.jpeg\n```\n\n```bash\n# Rearrange dataset structure\n$ python dataset/prepare_cars.py\n```\n\n### Train models\n\n#### Norm-SoftMax loss with CARS196\n\n```bash\n# Norm-SoftMax\n$ python main.py --gpu=0 \\\n--save_path=./logs/CARS196_norm_softmax \\\n--data=./dataset/carDB --data_name=cars196 \\\n--dim=512 --batch_size=128 --epochs=130 \\\n--freeze_BN --loss=Norm_SoftMax \\\n--decay_step=50 --decay_stop=50 --n_instance=1 \\\n--scale=23.0 --check_epoch=5\n```\n\n#### PS + Norm-SoftMax loss with CARS196\n\n```bash\n# PS + Norm-SoftMax\n$ python main.py --gpu=0 \\\n--save_path=./logs/CARS196_PS_norm_softmax \\\n--data=./dataset/carDB --data_name=cars196 \\\n --dim=512 --batch_size=128 --epochs=130 \\\n--freeze_BN --loss=Norm_SoftMax \\\n--decay_step=50 --decay_stop=50 --n_instance=1 \\\n--scale=23.0 --check_epoch=5 \\\n--ps_alpha=0.40 --ps_mu=1.0\n```\n\n#### Proxy-NCA loss with CARS196\n\n```bash\n# Proxy-NCA\n$ python main.py --gpu=0 \\\n--save_path=./logs/CARS196_proxy_nca \\\n--data=./dataset/carDB --data_name=cars196 \\\n--dim=512 --batch_size=128 --epochs=130 \\\n--freeze_BN --loss=Proxy_NCA \\\n--decay_step=50 --decay_stop=50 --n_instance=1 \\\n--scale=12.0 --check_epoch=5\n```\n\n#### PS + Proxy-NCA loss with CARS196\n\n```bash\n# PS + Proxy-NCA\n$ python main.py --gpu=0 \\\n--save_path=./logs/CARS196_PS_proxy_nca \\\n--data=./dataset/carDB --data_name=cars196 \\\n--dim=512 --batch_size=128 --epochs=130 \\\n--freeze_BN --loss=Proxy_NCA \\\n--decay_step=50 --decay_stop=50 --n_instance=1 \\\n--scale=12.0 --check_epoch=5 \\\n--ps_alpha=0.40 --ps_mu=1.0\n```\n\n### Check Test Results\n```\n$ tensorboard --logdir=logs --port=10000\n```\n\n## Experimental results\n- We report Recall@1, RP and MAP performances of each loss, which are trained with CARS196 dataset for 8 runs.\n\n### Recall@1\nLoss | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | Mean ± std\n-- | -- | -- | -- | -- | -- | -- | -- | -- | --\nNorm-SoftMax | 83.38 | 83.25 | 83.25 | 83.18 | 83.05 | 82.90 | 82.83 | 82.79 | 83.08 ± 0.21\nPS + Norm-SoftMax | 84.69 | 84.58 | 84.45 | 84.35 | 84.22 | 83.95 | 83.91 | 83.89 | 84.25 ± 0.31\nProxy-NCA | 83.74 | 83.69 | 83.62 | 83.32 | 83.06 | 83.00 | 82.97 | 82.84 | 83.28 ± 0.36\nPS + Proxy-NCA | 84.52 | 84.39 | 84.32 | 84.29 | 84.22 | 84.12 | 83.94 | 83.88 | 84.21 ± 0.21\n\n### RP\nLoss | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | Mean ± std\n-- | -- | -- | -- | -- | -- | -- | -- | -- | --\nNorm-SoftMax | 35.85 | 35.51 | 35.28 | 35.28 | 35.24 | 34.95 | 34.87 | 34.84 | 35.23 ± 0.34\nPS + Norm-SoftMax | 37.01 | 36.98 | 36.92 | 36.74 | 36.74 | 36.73 | 36.54 | 36.45 | 36.76 ± 0.20\nProxy-NCA | 36.08 | 35.85 | 35.79 | 35.66 | 35.66 | 35.63 | 35.47 | 35.43 | 35.70 ± 0.21\nPS + Proxy-NCA | 36.97 | 36.84 | 36.72 | 36.64 | 36.63 | 36.60 | 36.43 | 36.41 | 36.66 ± 0.18\n\n### MAP\nLoss | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | Mean ± std\n-- | -- | -- | -- | -- | -- | -- | -- | -- | --\nNorm-SoftMax | 25.56 | 25.56 | 25.00 | 24.93 | 24.90 | 24.59 | 24.57 | 24.56 | 24.92 ± 0.35\nPS + Norm-SoftMax | 26.71 | 26.67 | 26.65 | 26.56 | 26.53 | 26.52 | 26.30 | 26.17 | 26.51 ± 0.18\nProxy-NCA | 25.66 | 25.52 | 25.37 | 25.36 | 25.33 | 25.26 | 25.22 | 25.04 | 25.35 ± 0.18\nPS + Proxy-NCA | 26.77 | 26.63 | 26.50 | 26.42 | 26.37 | 26.31 | 26.25 | 26.12 | 26.42 ± 0.20\n\n### Performance Graph\n- Below figure shows performance graph of test set during training.\n\u003cimg src=\"figures/performance.png\"\u003e\n\n## Reference\n- Our code is based on SoftTriple repository ([Arxiv](https://arxiv.org/abs/1909.05235), [Github](https://github.com/idstcv/SoftTriple))\n\n## Citation\nIf you find *Proxy Synthesis* useful in your research, please consider to cite the following paper.\n\n```\n@inproceedings{gu2020proxy,\n    title={Proxy Synthesis: Learning with Synthetic Classes for Deep Metric Learning},\n    author={Geonmo Gu, Byungsoo Ko, and Han-Gyu Kim},\n    booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},\n    year={2021}\n}\n```\n\n## License\n\n```\nCopyright 2021-present NAVER Corp.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n```   \n","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnavervision%2Fproxy-synthesis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnavervision%2Fproxy-synthesis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnavervision%2Fproxy-synthesis/lists"}