{"id":13564424,"url":"https://github.com/oscarknagg/few-shot","last_synced_at":"2025-05-16T17:04:09.471Z","repository":{"id":57992844,"uuid":"155469554","full_name":"oscarknagg/few-shot","owner":"oscarknagg","description":"Repository for few-shot learning machine learning projects","archived":false,"fork":false,"pushed_at":"2019-11-25T04:18:31.000Z","size":16934,"stargazers_count":1240,"open_issues_count":36,"forks_count":248,"subscribers_count":14,"default_branch":"master","last_synced_at":"2025-04-12T15:58:27.731Z","etag":null,"topics":["few-shot-learning","machine-learning","maml","meta-learning","miniimagenet","omniglot","pytorch","research"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oscarknagg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-10-30T23:24:51.000Z","updated_at":"2025-04-04T19:51:26.000Z","dependencies_parsed_at":"2022-09-22T03:00:52.638Z","dependency_job_id":null,"html_url":"https://github.com/oscarknagg/few-shot","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oscarknagg%2Ffew-shot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oscarknagg%2Ffew-shot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oscarknagg%2Ffew-shot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oscarknagg%2Ffew-shot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oscarknagg","download_url":"https://codeload.github.com/oscarknagg/few-shot/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254573589,"owners_count":22093731,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["few-shot-learning","machine-learning","maml","meta-learning","miniimagenet","omniglot","pytorch","research"],"created_at":"2024-08-01T13:01:31.124Z","updated_at":"2025-05-16T17:04:09.420Z","avatar_url":"https://github.com/oscarknagg.png","language":"Python","funding_links":[],"categories":["Python","Table of Contents"],"sub_categories":[],"readme":"# Few-shot learning\n\nThe aim for this repository is to contain clean, readable and tested\ncode to reproduce few-shot learning research.\n\nThis project is written in python 3.6 and Pytorch and assumes you have\na GPU.\n\nSee these Medium articles for some more information\n1. [Theory and concepts](https://towardsdatascience.com/advances-in-few-shot-learning-a-guided-tour-36bc10a68b77)\n2. [Discussion of implementation details](https://towardsdatascience.com/advances-in-few-shot-learning-reproducing-results-in-pytorch-aba70dee541d)\n\n# Setup\n### Requirements\n\nListed in `requirements.txt`. Install with `pip install -r\nrequirements.txt` preferably in a virtualenv.\n\n### Data\nEdit the `DATA_PATH` variable in `config.py` to the location where\nyou store the Omniglot and miniImagenet datasets.\n\nAfter acquiring the\ndata and running the setup scripts your folder structure should look\nlike\n```\nDATA_PATH/\n    Omniglot/\n        images_background/\n        images_evaluation/\n    miniImageNet/\n        images_background/\n        images_evaluation/\n```\n\n**Omniglot** dataset. Download from https://github.com/brendenlake/omniglot/tree/master/python,\nplace the extracted files into `DATA_PATH/Omniglot_Raw` and run\n`scripts/prepare_omniglot.py`\n\n**miniImageNet** dataset. Download files from\nhttps://drive.google.com/file/d/0B3Irx3uQNoBMQ1FlNXJsZUdYWEE/view,\nplace in `data/miniImageNet/images` and run `scripts/prepare_mini_imagenet.py`\n\n### Tests (optional)\n\nAfter adding the datasets run `pytest` in the root directory to run\nall tests.\n\n# Results\n\nThe file `experiments/experiments.txt` contains the hyperparameters I\nused to obtain the results given below.\n\n### Prototypical Networks\n\n![Prototypical Networks](https://github.com/oscarknagg/few-shot/blob/master/assets/proto_nets_diagram.png)\n\n\nRun `experiments/proto_nets.py` to reproduce results from [Prototpyical\nNetworks for Few-shot Learning](https://arxiv.org/pdf/1703.05175.pdf)\n(Snell et al).\n\n**Arguments**\n- dataset: {'omniglot', 'miniImageNet'}. Whether to use the Omniglot\n    or miniImagenet dataset\n- distance: {'l2', 'cosine'}. Which distance metric to use\n- n-train: Support samples per class for training tasks\n- n-test: Support samples per class for validation tasks\n- k-train: Number of classes in training tasks\n- k-test: Number of classes in validation tasks\n- q-train: Query samples per class for training tasks\n- q-test: Query samples per class for validation tasks\n\n\n|                  | Omniglot |     |      |      |\n|------------------|----------|-----|------|------|\n| **k-way**        | **5**    |**5**|**20**|**20**|\n| **n-shot**       | **1**    |**5**|**1** |**5** |\n| Published        | 98.8     |99.7 |96.0  |98.9  |\n| This Repo        | 98.2     |99.4 |95.8  |98.6  |\n\n|                  | miniImageNet|     |\n|------------------|-------------|-----|\n| **k-way**        | **5**       |**5**|\n| **n-shot**       | **1**       |**5**|\n| Published        | 49.4        |68.2 |\n| This Repo        | 48.0        |66.2 |\n\n### Matching Networks\n\nA differentiable nearest neighbours classifier.\n\n![Matching Networks](https://github.com/oscarknagg/few-shot/blob/master/assets/matching_nets_diagram.png)\n\nRun `experiments/matching_nets.py` to reproduce results from [Matching\nNetworks for One Shot Learning](https://arxiv.org/pdf/1606.04080.pdf)\n(Vinyals et al).\n\n**Arguments**\n- dataset: {'omniglot', 'miniImageNet'}. Whether to use the Omniglot\n    or miniImagenet dataset\n- distance: {'l2', 'cosine'}. Which distance metric to use\n- n-train: Support samples per class for training tasks\n- n-test: Support samples per class for validation tasks\n- k-train: Number of classes in training tasks\n- k-test: Number of classes in validation tasks\n- q-train: Query samples per class for training tasks\n- q-test: Query samples per class for validation tasks\n- fce: Whether (True) or not (False) to use full context embeddings (FCE)\n- lstm-layers: Number of LSTM layers to use in the support set\n    FCE\n- unrolling-steps: Number of unrolling steps to use when calculating FCE\n    of the query sample\n\nI had trouble reproducing the results of this paper using the cosine\ndistance metric as I found the converge to be slow and final performance\ndependent on the random initialisation. However I was able to reproduce\n(and slightly exceed) the results of this paper using the l2 distance\nmetric.\n\n|                     | Omniglot|     |      |      |\n|---------------------|---------|-----|------|------|\n| **k-way**           | **5**   |**5**|**20**|**20**|\n| **n-shot**          | **1**   |**5**|**1** |**5** |\n| Published (cosine)  | 98.1    |98.9 |93.8  |98.5  |\n| This Repo (cosine)  | 92.0    |93.2 |75.6  |77.8  |\n| This Repo (l2)      | 98.3    |99.8 |92.8  |97.8   |\n\n|                        | miniImageNet|     |\n|------------------------|-------------|-----|\n| **k-way**              | **5**       |**5**|\n| **n-shot**             | **1**       |**5**|\n| Published (cosine, FCE)| 44.2        |57.0 |\n| This Repo (cosine, FCE)| 42.8        |53.6 |\n| This Repo (l2)         | 46.0        |58.4 |\n\n### Model-Agnostic Meta-Learning (MAML)\n\n![MAML](https://github.com/oscarknagg/few-shot/blob/master/assets/maml_diagram.png)\n\nI used max pooling instead of strided convolutions in order to be\nconsistent with the other papers. The miniImageNet experiments using\n2nd order MAML took me over a day to run.\n\nRun `experiments/maml.py` to reproduce results from [Model-Agnostic\nMeta-Learning](https://arxiv.org/pdf/1703.03400.pdf)\n(Finn et al).\n\n**Arguments**\n- dataset: {'omniglot', 'miniImageNet'}. Whether to use the Omniglot\n    or miniImagenet dataset\n- distance: {'l2', 'cosine'}. Which distance metric to use\n- n: Support samples per class for few-shot tasks\n- k: Number of classes in training tasks\n- q: Query samples per class for training tasks\n- inner-train-steps: Number of inner-loop updates to perform on training\n    tasks\n- inner-val-steps: Number of inner-loop updates to perform on validation\n    tasks\n- inner-lr: Learning rate to use for inner-loop updates\n- meta-lr: Learning rate to use when updating the meta-learner weights\n- meta-batch-size: Number of tasks per meta-batch\n- order: Whether to use 1st or 2nd order MAML\n- epochs: Number of training epochs\n- epoch-len: Meta-batches per epoch\n- eval-batches: Number of meta-batches to use when evaluating the model\n    after each epoch\n\n\nNB: For MAML n, k and q are fixed between train and test. You may need\nto adjust meta-batch-size to fit your GPU. 2nd order MAML uses a _lot_\nmore memory.\n\n|                  | Omniglot |     |      |      |\n|------------------|----------|-----|------|------|\n| **k-way**        | **5**    |**5**|**20**|**20**|\n| **n-shot**       | **1**    |**5**|**1** |**5** |\n| Published        | 98.7     |99.9 |95.8  |98.9  |\n| This Repo (1)    | 95.5     |99.5 |92.2  |97.7  |\n| This Repo (2)    | 98.1     |99.8 |91.6  |95.9  |\n\n|                  | miniImageNet|     |\n|------------------|-------------|-----|\n| **k-way**        | **5**       |**5**|\n| **n-shot**       | **1**       |**5**|\n| Published        | 48.1        |63.2 |\n| This Repo (1)    | 46.4        |63.3 |\n| This Repo (2)    | 47.5        |64.7 |\n\nNumber in brackets indicates 1st or 2nd order MAML.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foscarknagg%2Ffew-shot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foscarknagg%2Ffew-shot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foscarknagg%2Ffew-shot/lists"}