{"id":18322495,"url":"https://github.com/tencentarc/sfda","last_synced_at":"2025-04-05T23:31:00.801Z","repository":{"id":47054644,"uuid":"503987441","full_name":"TencentARC/SFDA","owner":"TencentARC","description":null,"archived":false,"fork":false,"pushed_at":"2022-07-20T03:48:20.000Z","size":275,"stargazers_count":21,"open_issues_count":3,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-21T13:23:07.209Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TencentARC.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-06-16T02:37:44.000Z","updated_at":"2024-12-01T12:01:38.000Z","dependencies_parsed_at":"2022-09-10T19:01:33.759Z","dependency_job_id":null,"html_url":"https://github.com/TencentARC/SFDA","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FSFDA","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FSFDA/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FSFDA/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FSFDA/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TencentARC","download_url":"https://codeload.github.com/TencentARC/SFDA/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247415783,"owners_count":20935383,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-05T18:24:52.441Z","updated_at":"2025-04-05T23:31:00.022Z","avatar_url":"https://github.com/TencentARC.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Not All Models Are Equal: Predicting Model Transferability in a Self-challenging Fisher Space\n\nThis is the PyTorch implementation of the paper **Not All Models Are Equal: Predicting Model Transferability in a Self-challenging Fisher Space**\n\n\n## Self-challenging Fisher Discriminative Analysis (SFDA)\nWe design a novel model selection method, termed Self-challenging Fisher Discriminative Analysis (SFDA), which is efficient, effective, and robust when measuring the transferability of pre-trained models. Compared with the state-of-the-art method NLEEP, SFDA demonstrates an average of 59.1% gain while bringing 22.5x speedup in wall-clock time.\n\u003cdiv align=center\u003e\u003cimg src=\"SFDA.jpg\" width=\"1080\" height=\"200\"\u003e\u003c/div\u003e\n\n**Comparisons of weighted Kendall's tau** on 11 downstream classification datasets when selecting 11 pretrained supervised models.\n\n|Method |Aircraft|Caltech101|Cars|CIFAR10|CIFAR100|DTD|Flowers|Food|Pets|SUN397|VOC2007|\n| :----:  | :--: |:--: |:--: |:--: |:--: |:--: |:--: |:--: |:--: |:--: |:--: |\n| LEEP| -0.234|0.605|0.367|0.824|0.677|0.486|-0.243|0.491|0.389|0.701|0.446|\n| LogME |0.506|0.435|**0.576**|0.852|0.692| 0.647| 0.111| 0.385| 0.411| 0.511| 0.478|\n| NLEEP | 0.495| 0.661| 0.265| 0.806| 0.823| **0.777**| 0.215| 0.624| 0.599| **0.807**| 0.654|\n|PARC   |-0.182| 0.374 |0.562 |0.845| 0.692| 0.642| -0.082| 0.732| 0.138| 0.698| 0.723|\n| SFDA  | **0.615**| **0.737**| 0.487| **0.949**| **0.866**| 0.597| **0.542**| **0.815**| **0.734**| 0.703|**0.763**|\n\n\n## Getting Started\n* Install [PyTorch](http://pytorch.org/)\n* Clone the repo:\n  ```\n  git clone https://git.woa.com/wenqishao/SFDA.git\n  ```\n\n### Requirements\n\n- Install `PyTorch==1.7.1` and `torchvision==0.8.2` with `CUDA==10.1`:\n\n```bash\nconda install pytorch==1.7.1 torchvision==0.8.2 cudatoolkit=10.1 -c pytorch\n```\n\n- Install `timm==0.4.9`:\n\n```bash\npip install timm==0.4.9\n```\n\n### Data Preparation\n- Download the downstream datasets to ./data/*.\n\n### Pipeline of Model selection using transferability\n- Fine-tune pretrained models with hyper-paramters sweep to obtain ground-truth transferability score\n```\npython finetune_group1.py -m resnet50 -d cifar10\n```\n- Extract features of target data using pretrained models\n```\npython forward_feature_group1.py -m resnet50 -d cifar10\n```\n- Compute transferability scores using SFDA\n```\npython evaluate_metric_group1_cpu.py -me sfda -d cifar10\n```\n- Assess the effectiveness of SFDA\n```\npython tw_group1_cpu.py -me sfda -d cifar10\n```\n\n\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftencentarc%2Fsfda","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftencentarc%2Fsfda","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftencentarc%2Fsfda/lists"}