{"id":20337920,"url":"https://github.com/gt-ripl/continual-learning-benchmark","last_synced_at":"2025-04-05T19:11:40.606Z","repository":{"id":45697720,"uuid":"159851413","full_name":"GT-RIPL/Continual-Learning-Benchmark","owner":"GT-RIPL","description":"Evaluate three types of task shifting with popular continual learning algorithms.","archived":false,"fork":false,"pushed_at":"2021-04-26T10:03:40.000Z","size":481,"stargazers_count":519,"open_issues_count":3,"forks_count":90,"subscribers_count":14,"default_branch":"master","last_synced_at":"2025-03-29T18:06:36.703Z","etag":null,"topics":["artificial-neural-networks","continual-learning","continuous-learning","deep-learning","incremental-learning","lifelong-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/GT-RIPL.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-11-30T16:40:23.000Z","updated_at":"2025-03-02T09:35:50.000Z","dependencies_parsed_at":"2022-08-24T22:50:32.677Z","dependency_job_id":null,"html_url":"https://github.com/GT-RIPL/Continual-Learning-Benchmark","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GT-RIPL%2FContinual-Learning-Benchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GT-RIPL%2FContinual-Learning-Benchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GT-RIPL%2FContinual-Learning-Benchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GT-RIPL%2FContinual-Learning-Benchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/GT-RIPL","download_url":"https://codeload.github.com/GT-RIPL/Continual-Learning-Benchmark/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247386265,"owners_count":20930619,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-neural-networks","continual-learning","continuous-learning","deep-learning","incremental-learning","lifelong-learning"],"created_at":"2024-11-14T21:10:52.172Z","updated_at":"2025-04-05T19:11:40.584Z","avatar_url":"https://github.com/GT-RIPL.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Continual-Learning-Benchmark\nEvaluate three types of task shifting with popular continual learning algorithms.\n\nThis repository implemented and modularized following algorithms with PyTorch:\n- EWC: [code](https://github.com/GT-RIPL/Continual-Learning-Benchmark/blob/master/agents/regularization.py), [paper](https://arxiv.org/abs/1612.00796) (Overcoming catastrophic forgetting in neural networks)\n- Online EWC: [code](https://github.com/GT-RIPL/Continual-Learning-Benchmark/blob/master/agents/regularization.py), [paper](https://arxiv.org/abs/1805.06370) \n- SI: [code](https://github.com/GT-RIPL/Continual-Learning-Benchmark/blob/master/agents/regularization.py), [paper](https://arxiv.org/abs/1703.04200) (Continual Learning Through Synaptic Intelligence)\n- MAS: [code](https://github.com/GT-RIPL/Continual-Learning-Benchmark/blob/master/agents/regularization.py), [paper](https://eccv2018.org/openaccess/content_ECCV_2018/papers/Rahaf_Aljundi_Memory_Aware_Synapses_ECCV_2018_paper.pdf) (Memory Aware Synapses: Learning what (not) to forget)\n- GEM: [code](https://github.com/GT-RIPL/Continual-Learning-Benchmark/blob/master/agents/exp_replay.py), [paper](https://arxiv.org/abs/1706.08840) (Gradient Episodic Memory for Continual Learning)\n- (More are coming)\n\nAll the above algorithms are compared to following baselines with **the same static memory overhead**:\n- Naive rehearsal: [code](https://github.com/GT-RIPL/Continual-Learning-Benchmark/blob/master/agents/exp_replay.py)\n- L2: [code](https://github.com/GT-RIPL/Continual-Learning-Benchmark/blob/master/agents/regularization.py), [paper](https://arxiv.org/abs/1612.00796)\n\nKey tables:\n\n\u003cimg src=\"fig/task_shifts.png\" width=\"640\"\u003e\u003c/a\u003e\n\u003cimg src=\"fig/results_split_mnist.png\" width=\"640\"\u003e\u003c/a\u003e\n\nIf this repository helps your work, please cite:\n```\n@inproceedings{Hsu18_EvalCL,\n  title={Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines},\n  author={Yen-Chang Hsu and Yen-Cheng Liu and Anita Ramasamy and Zsolt Kira},\n  booktitle={NeurIPS Continual learning Workshop },\n  year={2018},\n  url={https://arxiv.org/abs/1810.12488}\n}\n```\n\n## Preparation\nThis repository was tested with Python 3.6 and PyTorch 1.0.1.post2. Part of the cases is tested with PyTorch 1.5.1 and gives the same results.\n\n```bash\npip install -r requirements.txt\n```\n\n## Demo\nThe scripts for reproducing the results of this paper are under the scripts folder.\n\n- Example: Run all algorithms in the incremental domain scenario with split MNIST.\n```bash\n./scripts/split_MNIST_incremental_domain.sh 0\n# The last number is gpuid\n# Outputs will be saved in ./outputs\n```\n\n- Eaxmple outputs: Summary of repeats\n```text\n===Summary of experiment repeats: 3 / 3 ===\nThe regularization coefficient: 400.0\nThe last avg acc of all repeats: [90.517 90.648 91.069]\nmean: 90.74466666666666 std: 0.23549144829955856\n```\n\n- Eaxmple outputs: The grid search for regularization coefficient\n```text\nreg_coef: 0.1 mean: 76.08566666666667 std: 1.097717733400629\nreg_coef: 1.0 mean: 77.59100000000001 std: 2.100847606721314\nreg_coef: 10.0 mean: 84.33933333333334 std: 0.3592671553160509\nreg_coef: 100.0 mean: 90.83800000000001 std: 0.6913701372395712\nreg_coef: 1000.0 mean: 87.48566666666666 std: 0.5440161353816179\nreg_coef: 5000.0 mean: 68.99133333333333 std: 1.6824762174313899\n\n```\n\n## Usage\n- Enable the grid search for the regularization coefficient: Use the option with a list of values, ex: -reg_coef 0.1 1 10 100 ...\n- Repeat the experiment N times: Use the option -repeat N\n\nLookup available options:\n```bash\npython iBatchLearn.py -h\n```\n\n## Other results\n\nBelow are CIFAR100 results. Please refer to the [scripts](https://github.com/GT-RIPL/Continual-Learning-Benchmark/blob/master/scripts/split_CIFAR100_incremental_class.sh) for details.\n\n\u003cimg src=\"fig/results_split_cifar100.png\" width=\"640\"\u003e\u003c/a\u003e","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgt-ripl%2Fcontinual-learning-benchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgt-ripl%2Fcontinual-learning-benchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgt-ripl%2Fcontinual-learning-benchmark/lists"}