{"id":28639674,"url":"https://github.com/yaoyao-liu/e3bm","last_synced_at":"2025-06-12T19:41:05.938Z","repository":{"id":43220993,"uuid":"246391597","full_name":"yaoyao-liu/e3bm","owner":"yaoyao-liu","description":"PyTorch implementation of \"An Ensemble of Epoch-wise Empirical Bayes for Few-shot Learning\" (ECCV 2020)","archived":false,"fork":false,"pushed_at":"2022-12-29T02:00:17.000Z","size":971,"stargazers_count":49,"open_issues_count":2,"forks_count":4,"subscribers_count":9,"default_branch":"inductive","last_synced_at":"2023-03-05T08:09:13.175Z","etag":null,"topics":["few-shot-learning","meta-learning"],"latest_commit_sha":null,"homepage":"https://lyy.mpi-inf.mpg.de/e3bm/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yaoyao-liu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-03-10T19:39:25.000Z","updated_at":"2023-02-22T11:07:03.000Z","dependencies_parsed_at":"2023-01-31T07:45:51.812Z","dependency_job_id":null,"html_url":"https://github.com/yaoyao-liu/e3bm","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"purl":"pkg:github/yaoyao-liu/e3bm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yaoyao-liu%2Fe3bm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yaoyao-liu%2Fe3bm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yaoyao-liu%2Fe3bm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yaoyao-liu%2Fe3bm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yaoyao-liu","download_url":"https://codeload.github.com/yaoyao-liu/e3bm/tar.gz/refs/heads/inductive","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yaoyao-liu%2Fe3bm/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259520285,"owners_count":22870415,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["few-shot-learning","meta-learning"],"created_at":"2025-06-12T19:41:03.649Z","updated_at":"2025-06-12T19:41:05.927Z","avatar_url":"https://github.com/yaoyao-liu.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## An Ensemble of Epoch-wise Empirical Bayes for Few-shot Learning\n\n[![LICENSE](https://img.shields.io/github/license/yaoyao-liu/E3BM?style=flat-square)](https://github.com/yaoyao-liu/E3BM/blob/master/LICENSE)\n[![Python](https://img.shields.io/badge/python-3.6-blue.svg?style=flat-square\u0026logo=python\u0026color=3776AB)](https://www.python.org/)\n[![PyTorch](https://img.shields.io/badge/pytorch-1.2.0-%237732a8?style=flat-square\u0026logo=PyTorch\u0026color=EE4C2C)](https://pytorch.org/)\n[![Citations](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/yaoyao-liu/yaoyao-liu.github.io/google-scholar-stats/gs_data_shieldsio_e3bm.json\u0026logo=Google%20Scholar\u0026color=5087ec\u0026style=flat-square\u0026label=citations)](https://scholar.google.com/citations?view_op=view_citation\u0026hl=en\u0026user=Uf9GqRsAAAAJ\u0026citation_for_view=Uf9GqRsAAAAJ:vV6vV6tmYwMC)\n\u003c!--\n[![CodeFactor](https://img.shields.io/codefactor/grade/github/yaoyao-liu/E3BM/inductive?style=flat-square\u0026logo=codefactor\u0026color=F44A6A)](https://www.codefactor.io/repository/github/yaoyao-liu/e3bm)\n--\u003e\n\n[[Paper](https://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123610392.pdf)] [[Project Page](https://e3bm.yyliu.net/)] [[GitLab@MPI](https://gitlab.mpi-klsb.mpg.de/yaoyaoliu/e3bm/)] \n\nThis repository contains the PyTorch implementation for the [ECCV 2020](https://eccv2020.eu/) Paper \"[An Ensemble of Epoch-wise Empirical Bayes for Few-shot Learning](https://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123610392.pdf)\". If you have any questions on this repository or the related paper, feel free to [create an issue](https://github.com/yaoyao-liu/E3BM/issues/new) or [send me an email](mailto:yaoyao.liu+github@mpi-inf.mpg.de).\n\n#### Summary\n\n* [Introduction](#introduction)\n* [Installation](#installation)\n* [Inductive Experiments](#Inductive-Experiments)\n* [Transductive Experiments](#transductive-Experiments)\n* [Citation](#citation)\n* [Acknowledgements](#acknowledgements)\n\n## Introduction\n\nFew-shot learning aims to train efficient predictive models with a few examples. The lack of training data leads to poor models that perform high-variance or low-confidence predictions. In this paper, we propose to meta-learn the ensemble of epoch-wise empirical Bayes models (E\u003csup\u003e3\u003c/sup\u003eBM) to achieve robust predictions. \"Epoch-wise\" means that each training epoch has a Bayes model whose parameters are specifically learned and deployed. \"Empirical\" means that the hyperparameters, e.g., used for learning and ensembling the epoch-wise models, are generated by hyperprior learners conditional on task-specific data. We introduce four kinds of hyperprior learners by considering inductive vs. transductive, and epoch-dependent vs. epoch-independent, in the paradigm of meta-learning. We conduct extensive experiments for five-class few-shot tasks on three challenging benchmarks: miniImageNet, tieredImageNet, and FC100, and achieve top performance using the epoch-dependent transductive hyperprior learner, which captures the richest information. Our ablation study shows that both \"epoch-wise ensemble\" and \"empirical\" encourage high efficiency and robustness in the model performance.\n\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"https://yyliu.net/images/misc/e3bm.png\" width=\"800\"/\u003e\n\u003c/p\u003e\n\n\u003e Figure: Conceptual illustrations of the model adaptation on the blue, red and yellow tasks. (a) MAML is the classical inductive method that meta-learns a network initialization θ that is used to learn a single base-learner on each task. (b) SIB is a transductive method that formulates a variational posterior as a function of both labeled training data T(tr) and unlabeled test data x(te). It also uses a single base-learner and optimizes the learner by running several synthetic gradient steps on x(te). (c) Our E\u003csup\u003e3\u003c/sup\u003eBM is a generic method that learns to combine the epoch-wise base-learners, and to generate task-specific learningcrates α and combination weights v that encourage robust adaptation.\n\n### Installation\n\nIn order to run this repository, we advise you to install python 3.6 and PyTorch 1.2.0 with Anaconda.\nYou may download Anaconda and read the installation instruction on their official website:\n\u003chttps://www.anaconda.com/download/\u003e\n\nCreate a new environment and install PyTorch and torchvision on it:\n```bash\nconda create --name e3bm-pytorch python=3.6\nconda activate e3bm-pytorch\nconda install pytorch=1.2.0 \nconda install torchvision -c pytorch\n```\n\nInstall other requirements:\n```bash\npip install -r requirements.txt\n```\n\n### Inductive Experiments\n\n#### Performance (ResNet-12)\n\nExperiment results (%) for 5-way few-shot classification on ResNet-12 (same as [this repository](https://github.com/kjunelee/MetaOptNet)).\n\n\n| Method          | Backbone  |𝑚𝑖𝑛𝑖 1-shot   | 𝑚𝑖𝑛𝑖 5-shot  | 𝒕𝒊𝒆𝒓𝒆𝒅 1-shot | 𝒕𝒊𝒆𝒓𝒆𝒅 5-shot |\n| --------------  |---------- | ----------  | ----------   |------------ | ------------ |\n| [ProtoNet](https://github.com/orobix/Prototypical-Networks-for-Few-shot-Learning-PyTorch)    | ResNet-12 |`60.37 ± 0.83` | `78.02 ± 0.57` | `65.65 ± 0.92` | `83.40 ± 0.65` |\n| [MatchNet](https://github.com/gitabcworld/MatchingNetworks)    | ResNet-12 |`63.08 ± 0.80` | `75.99 ± 0.60` | `68.50 ± 0.92` | `80.60 ± 0.71` |\n| [MetaOptNet](https://github.com/kjunelee/MetaOptNet)    | ResNet-12 |`62.64 ± 0.61` | `78.63 ± 0.46` | `65.99 ± 0.72` | `81.56 ± 0.53` |\n| [Meta-Baseline](https://github.com/cyvius96/few-shot-meta-baseline)    | ResNet-12 |`63.17 ± 0.23` | `79.26 ± 0.17` | `68.62 ± 0.27` | `83.29 ± 0.18` | \n| [CAN](https://github.com/blue-blue272/fewshot-CAN)    | ResNet-12 |`63.85 ± 0.48` | `79.44 ± 0.34` | `69.89 ± 0.51` | `84.93 ± 0.38` | \n| E\u003csup\u003e3\u003c/sup\u003eBM (Ours)   | ResNet-12 |`64.09 ± 0.37` | `80.29 ± 0.25` | `71.34 ± 0.41` | `85.82 ± 0.29` | \n\n#### Running experiments\n\nRun meta-training with default settings:\n```bash\npython main.py -backbone resnet12 -shot 1 -way 5 -mode meta_train -dataset miniimagenet\npython main.py -backbone resnet12 -shot 5 -way 5 -mode meta_train -dataset miniimagenet\npython main.py -backbone resnet12 -shot 1 -way 5 -mode meta_train -dataset tieredimagenet\npython main.py -backbone resnet12 -shot 5 -way 5 -mode meta_train -dataset tieredimagenet\n```\n\nRun pre-training with default settings:\n```bash\npython main.py -backbone resnet12 -mode pre_train -dataset miniimagenet\npython main.py -backbone resnet12 -mode pre_train -dataset tieredimagenet\n```\n\n#### Download resources\nAll the datasets and pre-trained models will be downloaded automatically.\n\nYou may also download the resources on Google Drive/百度网盘 using the following links:\n\u003cbr\u003e\nDataset 1 - miniImageNet: \\[[Google Drive](https://drive.google.com/file/d/1vv3m14kusJcRpCsG-brG_Xk9MnetY9Bt/view?usp=sharing)\\] \\[[百度网盘](https://pan.baidu.com/s/1148LuaebBDic0DJNPFL0_w)\\] 提取码: p6w4\n\u003cbr\u003e\nDataset 2 - tieredImageNet: \\[[Google Drive](https://drive.google.com/file/d/1T-4NVTSa5T6CXKSRbymYLnWp_OrtF-mo/view?usp=sharing)\\] \\[[百度网盘](https://pan.baidu.com/s/19hjvbAcr-53b8nPj7Cws-A)\\] 提取码: 729f\n\u003cbr\u003e\nPre-trained models: \\[[Google Drive](https://drive.google.com/file/d/13pzlvn9s4psbZlGpIsYCi9fwQnWeSIkP/view?usp=sharing)\\] \\[[百度网盘](https://pan.baidu.com/s/1bjbPKmhztHrofWlkFKCJPg)\\] 提取码: 2e7p\n\u003cbr\u003e\nMeta-trained checkpoints: \\[[Google Drive](https://drive.google.com/drive/folders/17qTMpovfgEV6mRi8M4FkMYLIfBm3smgc?usp=sharing)\\] \\[[百度网盘](https://pan.baidu.com/s/1POKnhd-EmfiI7388eb7yVA)\\] 提取码: wc7g\n\u003cbr\u003e\n\n### Transductive Experiments\n\nSee the transductive setting experiments in this branch: \u003chttps://github.com/yaoyao-liu/E3BM/tree/transductive\u003e.\n\n\n\n### Citation\n\nPlease cite our paper if it is helpful to your work:\n\n```bibtex\n@inproceedings{Liu2020E3BM,\n  author    = {Liu, Yaoyao and\n               Schiele, Bernt and\n               Sun, Qianru},\n  title     = {An Ensemble of Epoch-wise Empirical Bayes for Few-shot Learning},\n  booktitle = {European Conference on Computer Vision (ECCV)},\n  year      = {2020}\n}\n```\n\n### Acknowledgements\n\nOur implementations use the source code from the following repositories:\n\n* [Learning Embedding Adaptation for Few-Shot Learning](https://github.com/Sha-Lab/FEAT)\n\n* [Empirical Bayes Transductive Meta-Learning with Synthetic Gradients](https://github.com/hushell/sib_meta_learn)\n\n* [DeepEMD: Differentiable Earth Mover's Distance for Few-Shot Learning](https://github.com/icoz69/DeepEMD)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyaoyao-liu%2Fe3bm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyaoyao-liu%2Fe3bm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyaoyao-liu%2Fe3bm/lists"}