{"id":15418321,"url":"https://github.com/rahulvigneswaran/tailcalibx","last_synced_at":"2025-10-11T08:31:05.878Z","repository":{"id":54990946,"uuid":"416676725","full_name":"rahulvigneswaran/TailCalibX","owner":"rahulvigneswaran","description":"Pytorch implementation of Feature Generation for Long-Tail Classification by Rahul Vigneswaran, Marc T Law, Vineeth N Balasubramaniam and Makarand Tapaswi","archived":false,"fork":false,"pushed_at":"2022-11-21T13:16:12.000Z","size":1081,"stargazers_count":38,"open_issues_count":2,"forks_count":4,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-01-28T18:38:28.476Z","etag":null,"topics":["cifar100","computer-vision","cvpr","feature-generation","iccv","iclr","icvgip-2021","iiith","iith","long-tail","long-tail-datasets","long-tailed-detection","long-tailed-recognition","longtail","mini-imagenet","nvidia","python","pytorch","pytorch-implementation","representation-learning"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rahulvigneswaran.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-10-13T09:42:20.000Z","updated_at":"2024-06-10T16:54:28.000Z","dependencies_parsed_at":"2023-01-23T14:16:06.567Z","dependency_job_id":null,"html_url":"https://github.com/rahulvigneswaran/TailCalibX","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/rahulvigneswaran/TailCalibX","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rahulvigneswaran%2FTailCalibX","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rahulvigneswaran%2FTailCalibX/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rahulvigneswaran%2FTailCalibX/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rahulvigneswaran%2FTailCalibX/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rahulvigneswaran","download_url":"https://codeload.github.com/rahulvigneswaran/TailCalibX/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rahulvigneswaran%2FTailCalibX/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279006611,"owners_count":26084148,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-11T02:00:06.511Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cifar100","computer-vision","cvpr","feature-generation","iccv","iclr","icvgip-2021","iiith","iith","long-tail","long-tail-datasets","long-tailed-detection","long-tailed-recognition","longtail","mini-imagenet","nvidia","python","pytorch","pytorch-implementation","representation-learning"],"created_at":"2024-10-01T17:20:19.865Z","updated_at":"2025-10-11T08:31:05.096Z","avatar_url":"https://github.com/rahulvigneswaran.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TailCalibX : Feature Generation for Long-tail Classification\nby [Rahul Vigneswaran](https://rahulvigneswaran.github.io/), [Marc T. Law](http://www.cs.toronto.edu/~law/), [Vineeth N. Balasubramanian](https://lab1055.github.io/), [Makarand Tapaswi](https://makarandtapaswi.github.io/)\n\n[[arXiv](https://arxiv.org/abs/2111.05956#:~:text=The%20visual%20world%20naturally%20exhibits,models%20based%20on%20deep%20learning.)] [[Code](https://github.com/rahulvigneswaran/TailCalibX)] [[pip Package](https://pypi.org/project/tailcalib/0.0.1/)] [[Video](https://www.youtube.com/watch?v=BhOtYW2a_pU)] \n![TailCalibX methodology](readme_assets/method.svg \"TailCalibX methodology\")\n\n\n# Table of contents\n  - [🐣 Easy Usage (Recommended way to use our method)](#-easy-usage-recommended-way-to-use-our-method)\n    - [💻 Installation](#-installation)\n    - [👨‍💻 Example Code](#-example-code)\n  - [🧪 Advanced Usage](#-advanced-usage)\n    - [✔ Things to do before you run the code from this repo](#-things-to-do-before-you-run-the-code-from-this-repo)\n    - [📀 How to use?](#-how-to-use)\n    - [📚 How to create the mini-ImageNet-LT dataset?](#-how-to-create-the-mini-imagenet-lt-dataset)\n    - [⚙ Arguments](#-arguments)\n  - [🏋️‍♂️ Trained weights](#%EF%B8%8F%EF%B8%8F-trained-weights)\n  - [🪀 Results on a Toy Dataset](#-results-on-a-toy-dataset)\n  - [🌴 Directory Tree](#-directory-tree)\n  - [📃 Citation](#-citation)\n  - [👁 Contributing](#-contributing)\n  - [❤ About me](#-about-me)\n  - [✨ Extras](#-extras)\n  - [📝 License](#-license)\n  \n## 🐣 Easy Usage (Recommended way to use our method)\n⚠ **Caution:**  TailCalibX is just TailCalib employed multiple times. Specifically, we generate a set of features once every epoch and use them to train the classifier. In order to mimic that, three things must be done at __every epoch__ in the following order:\n1. Collect all the features from your dataloader.\n2. Use the `tailcalib` package to make the features balanced by generating samples.\n3. Train the classifier.\n4. Repeat.\n\n### 💻 Installation\nUse the package manager [pip](https://pip.pypa.io/en/stable/) to install __tailcalib__.\n\n```bash\npip install tailcalib\n```\n\n### 👨‍💻 Example Code\n Check the instruction [here](tailcalib_pip/README.md) for a much more detailed python package information.\n\n```python\n# Import\nfrom tailcalib import tailcalib\n\n# Initialize\na = tailcalib(base_engine=\"numpy\")   # Options: \"numpy\", \"pytorch\"\n\n# Imbalanced random fake data\nimport numpy as np\nX = np.random.rand(200,100)\ny = np.random.randint(0,10, (200,))\n\n# Balancing the data using \"tailcalib\"\nfeat, lab, gen = a.generate(X=X, y=y)\n\n# Output comparison\nprint(f\"Before: {np.unique(y, return_counts=True)}\")\nprint(f\"After: {np.unique(lab, return_counts=True)}\")\n```\n\n## 🧪 Advanced Usage\n\n### ✔ Things to do before you run the code from this repo\n- Change the `data_root` for your dataset in `main.py`.\n- If you are using wandb logging ([Weights \u0026 Biases](https://docs.wandb.ai/quickstart)), make sure to change the `wandb.init` in `main.py` accordingly.\n\n### 📀 How to use?\n- For just the methods proposed in this paper :\n    - For CIFAR100-LT: `run_TailCalibX_CIFAR100-LT.sh`\n    - For mini-ImageNet-LT : `run_TailCalibX_mini-ImageNet-LT.sh`\n- For all the results show in the paper :\n    - For CIFAR100-LT: `run_all_CIFAR100-LT.sh`\n    - For mini-ImageNet-LT : `run_all_mini-ImageNet-LT.sh`\n\n### 📚 How to create the mini-ImageNet-LT dataset?\nCheck `Notebooks/Create_mini-ImageNet-LT.ipynb` for the script that generates the mini-ImageNet-LT dataset with varying imbalance ratios and train-test-val splits.\n### ⚙ Arguments\n- `--seed` : Select seed for fixing it. \n    - Default : `1`\n- `--gpu` : Select the GPUs to be used. \n    - Default : `\"0,1,2,3\"`\n\n- `--experiment`: Experiment number (Check 'libs/utils/experiment_maker.py'). \n    - Default : `0.1`\n- `--dataset` : Dataset number. \n    - Choices : `0 - CIFAR100, 1 - mini-imagenet` \n    - Default : `0`\n- `--imbalance` : Select Imbalance factor. \n    - Choices : `0: 1, 1: 100, 2: 50, 3: 10` \n    - Default : `1`\n- `--type_of_val` : Choose which dataset split to use. \n    - Choices: `\"vt\": val_from_test, \"vtr\": val_from_train, \"vit\": val_is_test`\n    - Default : `\"vit\"`\n\n- `--cv1` to `--cv9` : Custom variable to use in experiments - purpose changes according to the experiment.\n    - Default : `\"1\"`    \n\n- `--train` : Run training sequence\n    - Default : `False`\n- `--generate` : Run generation sequence\n    - Default : `False`\n- `--retraining` : Run retraining sequence\n    - Default : `False`\n- `--resume` : Will resume from the 'latest_model_checkpoint.pth' and wandb if applicable.\n    - Default : `False`\n\n- `--save_features` : Collect feature representations.\n    - Default : `False`\n- `--save_features_phase` : Dataset split of representations to collect.\n    - Choices : `\"train\", \"val\", \"test\"`\n    - Default : `\"train\"`\n\n- `--config` : If you have a yaml file with appropriate config, provide the path here. Will override the 'experiment_maker'.\n    - Default : `None`\n\n## 🏋️‍♂️ Trained weights\n| Experiment       | CIFAR100-LT (ResNet32, seed 1, Imb 100) | mini-ImageNet-LT (ResNeXt50)|\n| -----------      | -----------        | -----------        |\n| TailCalib        | [Git-LFS](https://downgit.github.io/#/home?url=https://github.com/rahulvigneswaran/trained_models/tree/main/TailCalibX/CIFAR100-LT/TailCalib)       | [Git-LFS](https://downgit.github.io/#/home?url=https://github.com/rahulvigneswaran/trained_models/tree/main/TailCalibX/mini-ImageNet-LT/TailCalib)       |\n| TailCalibX       | [Git-LFS](https://downgit.github.io/#/home?url=https://github.com/rahulvigneswaran/trained_models/tree/main/TailCalibX/CIFAR100-LT/TailCalibX)        |[Git-LFS](https://downgit.github.io/#/home?url=https://github.com/rahulvigneswaran/trained_models/tree/main/TailCalibX/mini-ImageNet-LT/TailCalibX)        |\n| CBD + TailCalibX | [Git-LFS](https://downgit.github.io/#/home?url=https://github.com/rahulvigneswaran/trained_models/tree/main/TailCalibX/CIFAR100-LT/CBD%2BTailCalibX)        |[Git-LFS](https://downgit.github.io/#/home?url=https://github.com/rahulvigneswaran/trained_models/tree/main/TailCalibX/mini-ImageNet-LT/CBD%2BTailCalibX)        |\n\n## 🪀 Results on a Toy Dataset \n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1Yj2qymSm3NgCBqvKn5r_cOiEFl9wGp3J?usp=sharing)\n\nThe higher the `Imb ratio`, the more imbalanced the dataset is.\n`Imb ratio = maximum_sample_count / minimum_sample_count`.\n\nCheck [this notebook](https://colab.research.google.com/drive/1Yj2qymSm3NgCBqvKn5r_cOiEFl9wGp3J?usp=sharing) to play with the toy example from which the plot below was generated.\n![](readme_assets/toy_example_output.svg)\n\n## 🌴 Directory Tree\n```bash\nTailCalibX\n├── libs\n│   ├── core\n│   │   ├── ce.py\n│   │   ├── core_base.py\n│   │   ├── ecbd.py\n│   │   ├── modals.py\n│   │   ├── TailCalib.py\n│   │   └── TailCalibX.py\n│   ├── data\n│   │   ├── dataloader.py\n│   │   ├── ImbalanceCIFAR.py\n│   │   └── mini-imagenet\n│   │       ├── 0.01_test.txt\n│   │       ├── 0.01_train.txt\n│   │       └── 0.01_val.txt\n│   ├── loss\n│   │   ├── CosineDistill.py\n│   │   └── SoftmaxLoss.py\n│   ├── models\n│   │   ├── CosineDotProductClassifier.py\n│   │   ├── DotProductClassifier.py\n│   │   ├── ecbd_converter.py\n│   │   ├── ResNet32Feature.py\n│   │   ├── ResNext50Feature.py\n│   │   └── ResNextFeature.py\n│   ├── samplers\n│   │   └── ClassAwareSampler.py\n│   └── utils\n│       ├── Default_config.yaml\n│       ├── experiments_maker.py\n│       ├── globals.py\n│       ├── logger.py\n│       └── utils.py\n├── LICENSE\n├── main.py\n├── Notebooks\n│   ├── Create_mini-ImageNet-LT.ipynb\n│   └── toy_example.ipynb\n├── readme_assets\n│   ├── method.svg\n│   └── toy_example_output.svg\n├── README.md\n├── run_all_CIFAR100-LT.sh\n├── run_all_mini-ImageNet-LT.sh\n├── run_TailCalibX_CIFAR100-LT.sh\n└── run_TailCalibX_mini-imagenet-LT.sh\n```\nIgnored `tailcalib_pip` as it is for the `tailcalib` pip package.\n\n\n## 📃 Citation\n```\n@inproceedings{rahul2021tailcalibX,\n    title   = {{Feature Generation for Long-tail Classification}},\n    author  = {Rahul Vigneswaran and Marc T. Law and Vineeth N. Balasubramanian and Makarand Tapaswi},\n    booktitle = {ICVGIP},\n    year = {2021}\n}\n```\n\n## 👁 Contributing\nPull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.\n\n## ❤ About me\n[Rahul Vigneswaran](https://rahulvigneswaran.github.io/)\n\n## ✨ Extras\n[🐝  Long-tail buzz](https://rahulvigneswaran.github.io/longtail-buzz/) : If you are interested in deep learning research which involves __long-tailed / imbalanced__ dataset, take a look at [Long-tail buzz](https://rahulvigneswaran.github.io/longtail-buzz/) to learn about the recent trending papers in this field.\n\n![](/readme_assets/long_tail-buzz_ss.png)\n\n## 📝 License\n[MIT](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frahulvigneswaran%2Ftailcalibx","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frahulvigneswaran%2Ftailcalibx","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frahulvigneswaran%2Ftailcalibx/lists"}