{"id":20063261,"url":"https://github.com/markdtw/meta-learning-lstm-pytorch","last_synced_at":"2025-08-20T18:33:15.289Z","repository":{"id":93067255,"uuid":"171411621","full_name":"markdtw/meta-learning-lstm-pytorch","owner":"markdtw","description":"pytorch implementation of Optimization as a Model for Few-shot Learning","archived":false,"fork":false,"pushed_at":"2022-12-26T08:05:02.000Z","size":46,"stargazers_count":177,"open_issues_count":3,"forks_count":35,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-12-06T18:52:51.968Z","etag":null,"topics":["few-shot-learning","learning-to-learn","meta-learning","pytorch"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/markdtw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-02-19T05:34:39.000Z","updated_at":"2024-11-25T12:32:41.000Z","dependencies_parsed_at":"2023-06-04T15:15:07.121Z","dependency_job_id":null,"html_url":"https://github.com/markdtw/meta-learning-lstm-pytorch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/markdtw%2Fmeta-learning-lstm-pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/markdtw%2Fmeta-learning-lstm-pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/markdtw%2Fmeta-learning-lstm-pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/markdtw%2Fmeta-learning-lstm-pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/markdtw","download_url":"https://codeload.github.com/markdtw/meta-learning-lstm-pytorch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230445926,"owners_count":18227060,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["few-shot-learning","learning-to-learn","meta-learning","pytorch"],"created_at":"2024-11-13T13:41:25.225Z","updated_at":"2024-12-19T14:07:30.925Z","avatar_url":"https://github.com/markdtw.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Optimization as a Model for Few-shot Learning\nPytorch implementation of [Optimization as a Model for Few-shot Learning](https://openreview.net/forum?id=rJY0-Kcll) in ICLR 2017 (Oral)\n\n![Model Architecture](https://i.imgur.com/lydKeUc.png)\n\n## Prerequisites\n- python 3+\n- pytorch 0.4+ (developed on 1.0.1 with cuda 9.0)\n- [pillow](https://pillow.readthedocs.io/en/stable/installation.html)\n- [tqdm](https://tqdm.github.io/) (a nice progress bar)\n\n## Data\n- Mini-Imagenet as described [here](https://github.com/twitter/meta-learning-lstm/tree/master/data/miniImagenet)\n  - You can download it from [here](https://drive.google.com/file/d/1rV3aj_hgfNTfCakffpPm7Vhpr1in87CR/view?usp=sharing) (~2.7GB, google drive link)\n\n## Preparation\n- Make sure Mini-Imagenet is split properly. For example:\n  ```\n  - data/\n    - miniImagenet/\n      - train/\n        - n01532829/\n          - n0153282900000005.jpg\n          - ...\n        - n01558993/\n        - ...\n      - val/\n        - n01855672/\n        - ...\n      - test/\n        - ...\n  - main.py\n  - ...\n  ```\n  - It'd be set if you download and extract Mini-Imagenet from the link above\n- Check out `scripts/train_5s_5c.sh`, make sure `--data-root` is properly set\n\n## Run\nFor 5-shot, 5-class training, run\n```bash\nbash scripts/train_5s_5c.sh\n```\nHyper-parameters are referred to the [author's repo](https://github.com/twitter/meta-learning-lstm).\n\nFor 5-shot, 5-class evaluation, run *(remember to change `--resume` and `--seed` arguments)*\n```bash\nbash scripts/eval_5s_5c.sh\n```\n\n## Notes\n- Results (This repo is developed following the [pytorch reproducibility guideline](https://pytorch.org/docs/stable/notes/randomness.html)):\n\n|seed|train episodes|val episodes|val acc mean|val acc std|test episodes|test acc mean|test acc std|\n|-|-|-|-|-|-|-|-|\n|719|41000|100|59.08|9.9|100|56.59|8.4|\n|  -|    -|  -|    -|  -|250|57.85|8.6|\n|  -|    -|  -|    -|  -|600|57.76|8.6|\n| 53|44000|100|58.04|9.1|100|57.85|7.7|\n|  -|    -|  -|    -|  -|250|57.83|8.3|\n|  -|    -|  -|    -|  -|600|58.14|8.5|\n\n- The results I get from directly running the author's repo can be found [here](https://i.imgur.com/rtagm2c.png), I have slightly better performance (~5%) but neither results match the number in the paper (60%) *(Discussion and help are welcome!)*.\n- Training with the default settings takes ~2.5 hours on a single Titan Xp while occupying ~2GB GPU memory.\n- The implementation replicates two learners similar to the author's repo:\n  - `learner_w_grad` functions as a regular model, get gradients and loss as inputs to meta learner.\n  - `learner_wo_grad` constructs the graph for meta learner:\n    - All the parameters in `learner_wo_grad` are replaced by `cI` output by meta learner.\n    - `nn.Parameters` in this model are casted to `torch.Tensor` to connect the graph to meta learner.\n- Several ways to **copy** a parameters from meta learner to learner depends on the scenario:\n  - `copy_flat_params`: we only need the parameter values and keep the original `grad_fn`.\n  - `transfer_params`: we want the values as well as the `grad_fn` (from `cI` to `learner_wo_grad`).\n    - `.data.copy_` v.s. `clone()` -\u003e the latter retains all the properties of a tensor including `grad_fn`.\n    - To maintain the batch statistics, `load_state_dict` is used (from `learner_w_grad` to `learner_wo_grad`).\n\n## References\n- [CloserLookFewShot](https://github.com/wyharveychen/CloserLookFewShot) (Data loader)\n- [pytorch-meta-optimizer](https://github.com/ikostrikov/pytorch-meta-optimizer) (Casting `nn.Parameters` to `torch.Tensor` inspired from here)\n- [meta-learning-lstm](https://github.com/twitter/meta-learning-lstm) (Author's repo in Lua Torch)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarkdtw%2Fmeta-learning-lstm-pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarkdtw%2Fmeta-learning-lstm-pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarkdtw%2Fmeta-learning-lstm-pytorch/lists"}