{"id":13737922,"url":"https://github.com/ExplainableML/NonIsotropicProxyDML","last_synced_at":"2025-05-08T15:32:03.276Z","repository":{"id":37013015,"uuid":"467543292","full_name":"ExplainableML/NonIsotropicProxyDML","owner":"ExplainableML","description":"This repository contains the code for our CVPR 2022 paper on \"Non-isotropy Regularization for Proxy-based Deep Metric Learning\".","archived":false,"fork":false,"pushed_at":"2023-03-10T21:30:42.000Z","size":432,"stargazers_count":14,"open_issues_count":1,"forks_count":5,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-11-15T06:32:50.780Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ExplainableML.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2022-03-08T14:22:06.000Z","updated_at":"2023-11-11T08:45:25.000Z","dependencies_parsed_at":"2024-04-16T22:03:25.415Z","dependency_job_id":"2e5859ec-0a1b-455a-a26e-d595f6ed4021","html_url":"https://github.com/ExplainableML/NonIsotropicProxyDML","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ExplainableML%2FNonIsotropicProxyDML","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ExplainableML%2FNonIsotropicProxyDML/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ExplainableML%2FNonIsotropicProxyDML/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ExplainableML%2FNonIsotropicProxyDML/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ExplainableML","download_url":"https://codeload.github.com/ExplainableML/NonIsotropicProxyDML/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253096251,"owners_count":21853566,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T03:02:05.936Z","updated_at":"2025-05-08T15:32:02.967Z","avatar_url":"https://github.com/ExplainableML.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Non-isotropy Regularization for Proxy-based Deep Metric Learning\n#### ___Contact___: Karsten Roth (karsten.rh1@gmail.com)\n\n---\n#### Table of Contents:\n   * [Introduction](#usage)\n   * [Requirements](#requirements)\n   * [Quick Guide](#guide)  \n   * [Repository \u0026 Dataset structures](#repo)\n   * [Citation](#citation)   \n\n---\n## Introduction\n![setup](images/setup.png)\n\nThis repository contains the official code for our CVPR 2022 paper on [Non-isotropy Regularization for Proxy-based Deep Metric Learning](#placeholder).\n\nThe goal of the proposed approach is to encourage improved resolution of intraclass structures in proxy-based Deep Metric Learning, and enforce feature-richer representation spaces to be captured __while__ retaining the fast convergence benefits of proxy-based methods and under the absence of sample-to-sample contrastive relations and mining heuristics.\n\nIn particular, the resulting training runs showcase much higher generalization performance, even when applied on top of already very strong proxy-based objectives with little hyperparameter tuning and in parts even __increased__ convergence speeds:\n\n![speed](images/nir_speed.png)\n\n_Note:_ This repository is build on top of preceding work [here](https://github.com/Confusezius/Revisiting_Deep_Metric_Learning_PyTorch).\n\n---\n## Requirements\nThis repository has been built and tested around `Python 3.8` and `PyTorch 1.9+`. The required libraries and their installation process can be found in `requirements.sh`.\n\nFor example, one can create a simple conda environment via\n```\nwget  https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh\nbash Miniconda3-latest-Linux-x86_64.sh # say yes to append path to bashrc.\nsource .bashrc\nconda create -n Env python=3.8\nconda activate Env\n```\n\nand then install the required libraries following `requirements.sh`.\n\n\n\n---\n## Quick Guide\n\n### Rerunning baselines and language-guided variants.\nThe required setup for each benchmark dataset is explained in the subsequent sections. In general, datasets can be downloaded e.g. from\n* [CUB200-2011](http://www.vision.caltech.edu/visipedia/CUB-200.html)\n* [CARS196](https://ai.stanford.edu/~jkrause/cars/car_dataset.html)\n* [Stanford Online Products](http://cvgl.stanford.edu/projects/lifted_struct/)\n\n\nThe main training script is `main.py`, with an exemplary call given here:\n\n```\npython main.py --seed 2 --log_online --group cub200-rn128_proxyanchor_baseline --no_train_metrics --project NIR --gpu $gpu --source_path $datapath \\\n--dataset cub200 --n_epochs 100 --tau 100 --gamma 1 --arch resnet50_frozen_normalize --embed_dim 128 --loss oproxy --bs 90\n```\n\nThis run trains a 128-dimensional (--embed_dim) metric space spanned over ResNet50 features with normalized embeddings and frozen BatchNorm (given via `--arch`) using a ProxyAnchor loss ('--loss', note the use of the `oproxy` objective, which combines both ProxyAnchor and ProxyNCA) for 100 epochs (`--epochs`), no learning rate scheduling (`--tau`, `--gamma`) and without logging metrics on the training data (`--no_train_metrics`).\nThe training metrics are logged to a [Weights \u0026 Biases](https://wandb.ai/) project (`--log_online`, `--project`), with the particular run name given via `--group`. This ensures that changes in `--seed` are assigned to the same overall group.\n\nFor further details and additional parameters, please check out `parameters.py`, which contains all available parameters separated by the general purpose and some additional explanation.\n\nTo now add __non-isotropy regularization__ on top of your proxy-based DML method, simply set `----loss_nir_w_align` to a value bigger `0`, and append e.g. `--warmup 1` to give the network one epoch to adapt the Normalizing Flows network. While this is not always needed, it is usually not detrimental to performance and helps in cases where more proxies are utilized.\n\nOverall, this gives:\n\n```\npython main.py --seed 2 --log_online --group cub200-rn128_proxyanchor_nir --no_train_metrics --project NIR --gpu $gpu --source_path $datapath \\\n--dataset cub200 --n_epochs 100 --tau 100 --gamma 1 --arch resnet50_frozen_normalize --embed_dim 128 --loss multisimilarity --bs 90 \\\n--warmup 1 --loss_nir_w_align 0.0075\n```\n\nFor further non-isotropy regularization hyperparameters, check out `parameters.py \u003e nir_parameters()`.\n\nFinally, various replication runs on all benchmarks are given in `nir_benchmark_runs.sh`.\nAs usual, with changes in software versioning and hardware, convergence may slightly differ, so the scheduling parameters need some adaptation to the local setup. However, even without learning rate scheduling, the extended methods should already closely match the existing state-of-the-art with fast convergence!\n\n\n\n\n---\n## Repository Structure \u0026 Expected Dataset Structures.\n\n### Repository\nThe repository has the following structure:\n\n```\nRepository\n|\n│   README.md # Base Readme.\n│   requirements.sh # List of required libraries.\n│   language_guidance_benchmark_runs.sh # Sample benchmark runs.\n|   main.py # main training script)\n|   parameters.py # collection of pipeline and training parameters.\n│   \n└───criteria # available baseline objectives\n|    │   e.g. margin.py\n│   \n└───batchminer # available batchmining heuristics\n|    │   e.g. distance.py\n│   \n└───datasampler # methods to construct minibatches.\n|    │   e.g. class_random_sampler.py\n│   \n└───metrics # Folder of standard DML evaluation metrics.\n|    │   e.g. mAP.py\n│   \n└───evaluation # Main evaluation protocols.\n|    │   Only __init__.py\n│   \n└───datasets # Main DML benchmarks.\n|    │   e.g. cub200.py\n│   \n└───utilities # Some utility and misc. functions.\n|    │   e.g. logger.py\n|\n└───Training_Results (will be generated during Training)\n|    │   contains folders based on dataset names, e.g. cub200. These folders then contain each run.\n```\n\n### Dataset Structures\nThe benchmarks are expected to follow these setups in order to be directly usable with the provided dataloaders, and path to the respective parent folder should be passed to the main training script via `--datapath \u003cpath_to_parent_folder\u003e`. The dataset folder name is passed via `--dataset \u003cdataset_name\u003e`.\n\n__cub200__\n```\ncub200\n└───images\n|    └───001.Black_footed_Albatross\n|           │   Black_Footed_Albatross_0001_796111\n|           │   ...\n|    ...\n```\n\n__cars196__\n```\ncars196\n└───images\n|    └───Acura Integra Type R 2001\n|           │   00128.jpg\n|           │   ...\n|    ...\n```\n\n__online_products__\n```\nonline_products\n└───images\n|    └───bicycle_final\n|           │   111085122871_0.jpg\n|    ...\n|\n└───Info_Files\n|    │   bicycle.txt\n|    │   ...\n```\n\n\n\n---\n## Citation\nIf you use this code or parts of it for you work, please cite\n```\n@conference{RotVinAka22b,\n  title = {Non-isotropy Regularization for Proxy-based Deep Metric Learning},\n  author = {Roth, K. and Vinyals, O. and Akata, Z.},\n  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},\n  pages = {7420--7430},\n  month = jun,\n  year = {2022},\n  doi = {},\n  url = {https://openaccess.thecvf.com/content/CVPR2022/papers/Roth_Non-Isotropy_Regularization_for_Proxy-Based_Deep_Metric_Learning_CVPR_2022_paper.pdf},\n  month_numeric = {6}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FExplainableML%2FNonIsotropicProxyDML","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FExplainableML%2FNonIsotropicProxyDML","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FExplainableML%2FNonIsotropicProxyDML/lists"}