{"id":15136345,"url":"https://github.com/openai/supervised-reptile","last_synced_at":"2025-05-16T15:08:08.742Z","repository":{"id":45698473,"uuid":"113241584","full_name":"openai/supervised-reptile","owner":"openai","description":"Code for the paper \"On First-Order Meta-Learning Algorithms\"","archived":false,"fork":false,"pushed_at":"2023-05-20T19:59:33.000Z","size":1221,"stargazers_count":1007,"open_issues_count":14,"forks_count":213,"subscribers_count":156,"default_branch":"master","last_synced_at":"2025-04-12T11:58:17.594Z","etag":null,"topics":["paper"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/1803.02999","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-12-05T22:50:22.000Z","updated_at":"2025-04-04T15:39:24.000Z","dependencies_parsed_at":"2024-12-22T22:04:37.962Z","dependency_job_id":"43dfef44-45c1-4673-9e3b-6247b59f6f91","html_url":"https://github.com/openai/supervised-reptile","commit_stats":{"total_commits":88,"total_committers":3,"mean_commits":"29.333333333333332","dds":0.03409090909090906,"last_synced_commit":"8f2b71c67a31c1a605ced0cecb76db876b607a7a"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openai%2Fsupervised-reptile","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openai%2Fsupervised-reptile/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openai%2Fsupervised-reptile/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openai%2Fsupervised-reptile/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openai","download_url":"https://codeload.github.com/openai/supervised-reptile/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254553958,"owners_count":22090417,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["paper"],"created_at":"2024-09-26T06:21:04.166Z","updated_at":"2025-05-16T15:08:03.734Z","avatar_url":"https://github.com/openai.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"**Status:** Archive (code is provided as-is, no updates expected)\n\n# supervised-reptile\n\n[Reptile](https://arxiv.org/abs/1803.02999) training code for [Omniglot](https://github.com/brendenlake/omniglot) and [Mini-ImageNet](https://openreview.net/pdf?id=rJY0-Kcll).\n\nReptile is a meta-learning algorithm that finds a good initialization. It works by sampling a task, training on the sampled task, and then updating the initialization towards the new weights for the task.\n\n# Getting the data\n\nThe [fetch_data.sh](fetch_data.sh) script creates a `data/` directory and downloads Omniglot and Mini-ImageNet into it. The data is on the order of 5GB, so the download takes 10-20 minutes on a reasonably fast internet connection.\n\n```\n$ ./fetch_data.sh\nFetching omniglot/images_background ...\nExtracting omniglot/images_background ...\nFetching omniglot/images_evaluation ...\nExtracting omniglot/images_evaluation ...\nFetching Mini-ImageNet train set ...\nFetching wnid: n01532829\nFetching wnid: n01558993\nFetching wnid: n01704323\nFetching wnid: n01749939\n...\n```\n\nIf you want to download Omniglot but not Mini-ImageNet, you can simply kill the script after it starts downloading Mini-ImageNet. The script automatically deletes partially-downloaded data when it is killed early.\n\n# Reproducing training runs\n\nYou can train models with the `run_omniglot.py` and `run_miniimagenet.py` scripts. Hyper-parameters are specified as flags (see `--help` for a detailed list). Here are the commands used for the paper:\n\n```shell\n# transductive 1-shot 5-way Omniglot.\npython -u run_omniglot.py --shots 1 --inner-batch 10 --inner-iters 5 --meta-step 1 --meta-batch 5 --meta-iters 100000 --eval-batch 5 --eval-iters 50 --learning-rate 0.001 --meta-step-final 0 --train-shots 10 --checkpoint ckpt_o15t --transductive\n\n# transductive 1-shot 5-way Mini-ImageNet.\npython -u run_miniimagenet.py --shots 1 --inner-batch 10 --inner-iters 8 --meta-step 1 --meta-batch 5 --meta-iters 100000 --eval-batch 5 --eval-iters 50 --learning-rate 0.001 --meta-step-final 0 --train-shots 15 --checkpoint ckpt_m15t --transductive\n\n# 5-shot 5-way Mini-ImageNet.\npython -u run_miniimagenet.py --inner-batch 10 --inner-iters 8 --meta-step 1 --meta-batch 5 --meta-iters 100000 --eval-batch 15 --eval-iters 50 --learning-rate 0.001 --meta-step-final 0 --train-shots 15 --checkpoint ckpt_m55\n\n# 1-shot 5-way Mini-ImageNet.\npython -u run_miniimagenet.py --shots 1 --inner-batch 10 --inner-iters 8 --meta-step 1 --meta-batch 5 --meta-iters 100000 --eval-batch 5 --eval-iters 50 --learning-rate 0.001 --meta-step-final 0 --train-shots 15 --checkpoint ckpt_m15\n\n# 5-shot 5-way Omniglot.\npython -u run_omniglot.py --train-shots 10 --inner-batch 10 --inner-iters 5 --learning-rate 0.001 --meta-step 1 --meta-step-final 0 --meta-batch 5 --meta-iters 100000 --eval-batch 5 --eval-iters 50 --checkpoint ckpt_o55\n\n# 1-shot 5-way Omniglot.\npython -u run_omniglot.py --shots 1 --inner-batch 10 --inner-iters 5 --meta-step 1 --meta-batch 5 --meta-iters 100000 --eval-batch 5 --eval-iters 50 --learning-rate 0.001 --meta-step-final 0 --train-shots 10 --checkpoint ckpt_o15\n\n# 1-shot 20-way Omniglot.\npython -u run_omniglot.py --shots 1 --classes 20 --inner-batch 20 --inner-iters 10 --meta-step 1 --meta-batch 5 --meta-iters 200000 --eval-batch 10 --eval-iters 50 --learning-rate 0.0005 --meta-step-final 0 --train-shots 10 --checkpoint ckpt_o120\n\n# 5-shot 20-way Omniglot.\npython -u run_omniglot.py --classes 20 --inner-batch 20 --inner-iters 10 --meta-step 1 --meta-batch 5 --meta-iters 200000 --eval-batch 10 --eval-iters 50 --learning-rate 0.0005 --meta-step-final 0 --train-shots 10 --checkpoint ckpt_o520\n```\n\nTraining creates checkpoints. Currently, you cannot resume training from a checkpoint, but you can re-run evaluation from a checkpoint by passing `--pretrained`. You can use TensorBoard on the checkpoint directories to see approximate learning curves during training and testing.\n\nTo evaluate with transduction, pass the `--transductive` flag. In this implementation, transductive evaluation is faster than non-transductive evaluation since it makes better use of batches.\n\n# Comparing different inner-loop gradient combinations\n\nHere are the commands for comparing different gradient combinations. The `--foml` flag indicates that only the final gradient should be used.\n\n```shell\n# Shared hyper-parameters for all experiments.\nshared=\"--sgd --seed 0 --inner-batch 25 --learning-rate 0.003 --meta-step-final 0 --meta-iters 40000 --eval-batch 25 --eval-iters 5 --eval-interval 1\"\n\npython run_omniglot.py --inner-iters 1 --train-shots 5 --meta-step 0.25 --checkpoint g1_ckpt $shared | tee g1.txt\n\npython run_omniglot.py --inner-iters 2 --train-shots 10 --meta-step 0.25 --checkpoint g1_g2_ckpt $shared | tee g1_g2.txt\npython run_omniglot.py --inner-iters 2 --train-shots 10 --meta-step 0.125 --checkpoint half_g1_g2_ckpt $shared | tee half_g1_g2.txt\npython run_omniglot.py --foml --inner-iters 2 --train-shots 10 --meta-step 0.25 --checkpoint g2_ckpt $shared | tee g2.txt\n\npython run_omniglot.py --inner-iters 3 --train-shots 15 --meta-step 0.25 --checkpoint g1_g2_g3_ckpt $shared | tee g1_g2_g3.txt\npython run_omniglot.py --inner-iters 3 --train-shots 15 --meta-step 0.08325 --checkpoint third_g1_g2_g3_ckpt $shared | tee third_g1_g2_g3.txt\npython run_omniglot.py --foml --inner-iters 3 --train-shots 15 --meta-step 0.25 --checkpoint g3_ckpt $shared | tee g3.txt\n\npython run_omniglot.py --foml --inner-iters 4 --train-shots 20 --meta-step 0.25 --checkpoint g4_ckpt $shared | tee g4.txt\npython run_omniglot.py --inner-iters 4 --train-shots 20 --meta-step 0.25 --checkpoint g1_g2_g3_g4_ckpt $shared | tee g1_g2_g3_g4.txt\npython run_omniglot.py --inner-iters 4 --train-shots 20 --meta-step 0.0625 --checkpoint fourth_g1_g2_g3_g4_ckpt $shared | tee fourth_g1_g2_g3_g4.txt\n\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenai%2Fsupervised-reptile","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenai%2Fsupervised-reptile","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenai%2Fsupervised-reptile/lists"}