{"id":13738596,"url":"https://github.com/Mattdl/CLsurvey","last_synced_at":"2025-05-08T16:34:48.262Z","repository":{"id":43921145,"uuid":"253443185","full_name":"Mattdl/CLsurvey","owner":"Mattdl","description":"Continual Hyperparameter Selection Framework. Compares 11 state-of-the-art Lifelong Learning methods and  4 baselines. Official Codebase of \"A continual learning survey: Defying forgetting in classification tasks.\" in  IEEE TPAMI.","archived":false,"fork":false,"pushed_at":"2021-06-03T09:23:07.000Z","size":169,"stargazers_count":197,"open_issues_count":4,"forks_count":24,"subscribers_count":9,"default_branch":"master","last_synced_at":"2024-11-07T19:50:58.655Z","etag":null,"topics":["classification-tasks","continual-learning","deep-learning","defy-forgetting","elastic-weight-consolidation","framework","gradient-episodic-memory","hat","hyperparameter-tuning","icarl","inaturalist","incremental-learning","incremental-moment-matching","memory-aware-synapses","neural-networks","packnet","synaptic-intelligence","tinyimagenet","tpami"],"latest_commit_sha":null,"homepage":"https://ieeexplore.ieee.org/abstract/document/9349197","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Mattdl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-04-06T08:49:34.000Z","updated_at":"2024-10-13T06:48:54.000Z","dependencies_parsed_at":"2022-09-06T08:10:28.718Z","dependency_job_id":null,"html_url":"https://github.com/Mattdl/CLsurvey","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mattdl%2FCLsurvey","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mattdl%2FCLsurvey/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mattdl%2FCLsurvey/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mattdl%2FCLsurvey/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Mattdl","download_url":"https://codeload.github.com/Mattdl/CLsurvey/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224746937,"owners_count":17363153,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["classification-tasks","continual-learning","deep-learning","defy-forgetting","elastic-weight-consolidation","framework","gradient-episodic-memory","hat","hyperparameter-tuning","icarl","inaturalist","incremental-learning","incremental-moment-matching","memory-aware-synapses","neural-networks","packnet","synaptic-intelligence","tinyimagenet","tpami"],"created_at":"2024-08-03T03:02:28.672Z","updated_at":"2024-11-15T07:31:30.722Z","avatar_url":"https://github.com/Mattdl.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# A continual learning survey: Defying forgetting in classification tasks\n\nThis is the original source code for the Continual Learning survey paper _\"A continual learning survey: Defying forgetting in classification tasks\"_ published at TPAMI [[TPAMI paper]](https://ieeexplore.ieee.org/abstract/document/9349197) [[Open-Access paper]](https://arxiv.org/abs/1909.08383).\n\nThis work allows comparing the state-of-the-art in a fair fashion using the **Continual Hyperparameter Framework**, which sets the hyperparameters dynamically based on the stability-plasticity dilemma.\nThis addresses the longstanding problem in literature to set hyperparameters for different methods in a fair fashion, using ONLY the current task data (hence without using iid validation data, which is not available in continual learning).\n\nThe code contains a generalizing framework for 11 SOTA methods and 4 baselines in Pytorch. \u003c/br\u003e\nImplemented task-incremental methods are\n\u003cdiv align=\"center\"\u003e\n\u003cp align=\"center\"\u003e\u003cb\u003e\n  SI | EWC | MAS | mean/mode-IMM | LWF | EBLL | PackNet |  HAT | GEM | iCaRL \n\u003c/b\u003e\u003c/p\u003e\n\u003c/div\u003e\n\nThese are compared with 4 baselines:\n\u003cdiv align=\"center\"\u003e\n\u003cp align=\"center\"\u003e\u003cb\u003e\n  Joint | Finetuning | Finetuning-FM | Finetuning-PM\n\u003c/b\u003e\u003c/p\u003e\n\u003c/div\u003e\n\n- **Joint**: Learn from all task data at once with a single head (multi-task learning baseline).\n- **Finetuning**: standard SGD\n- **Finetuning with Full Memory replay**: Allocate memory dynamically to incoming tasks.\n- **Finetuning with Partial Memory replay**: Divide memory a priori over all tasks.\n\n\nThis source code is released under a Attribution-NonCommercial 4.0 International\nlicense, find out more about it in the [LICENSE file](LICENSE).\n\n\n\n\n## Pipeline\n**Reproducibility**: Results from the paper can be obtained from [src/main_'dataset'.sh](src/main_tinyimagenet.sh). \nFull pipeline example in [src/main_tinyimagenet.sh](src/main_tinyimagenet.sh) .\n\n**Pipeline**: Constructing a custom pipeline typically requires the following steps.\n1. Project Setup\n    1. For all requirements see [requirements.txt](requirements.txt).\n    Main packages can be installed as in\n        ```\n        conda create --name \u003cENV-NAME\u003e python=3.7\n        conda activate \u003cENV-NAME\u003e\n       \n        # Main packages\n        conda install -c conda-forge matplotlib tqdm\n        conda install pytorch torchvision cudatoolkit=10.2 -c pytorch\n\n       # For GEM QP\n        conda install -c omnia quadprog\n       \n       # For PackNet: torchnet \n       pip install git+https://github.com/pytorch/tnt.git@master\n        ```\n    1. Set paths in '[config.init](src/config.init)' (or leave default)\n        1. '{tr,test}_results_root_path': where to save training/testing results.\n        1. 'models_root_path': where to store initial models (to ensure same initial model)\n        1. 'ds_root_path': root path of your datasets\n    1. Prepare dataset: see [src/data](src/data)/\"dataset\"_dataprep.py (e.g. [src/data/tinyimgnet_dataprep.py](src/data/tinyimgnet_dataprep.py))\n1. **Train** any out of the 11 SOTA methods or 4 baselines\n    1. **Regularization-based/replay methods:** We run a *first task model dump*, for Synaptic Intelligence (SI) as it acquires importance weights during training. \n    Other methods start from this same initial model. \n    1. **Baselines/parameter isolation methods**: Start training sequence from scratch\n1. **Evaluate** performance, sequence for testing on a task is saved in dictionary format under *test_results_root_path* defined in [config.init](src/config.init).\n1. **Plot** the evaluation results, using one of the configuration files in [utilities/plot_configs](src/utilities/plot_configs)\n\n## Implement Your Method\n1. Find class \"YourMethod\" in [methods/method.py](src/methods/method.py). Implement the framework phases (documented in code).\n1. Implement your task-based training script in [methods](src/methods): methods/\"YourMethodDir\". \nThe class \"YourMethod\" will call this code for training/eval/processing of a single task. \n\n        \n## Project structure\n- [src/data](src/data): datasets and automated preparation scripts for Tiny Imagenet and iNaturalist.\n- [src/framework](src/framework): the novel task incremental continual learning framework. \n**main.py** starts training pipeline, specify *--test* argument to perform evaluation with **eval.py**. \n- [src/methods](src/methods): all methods source code and **method.py** wrapper.\n- [src/models](src/models): **net.py** all model preprocessing.\n- [src/utilities](src/utilities): utils used across all modules and plotting.\n- Config:\n    - [src/data](src/data)/{datasets/models}: default datasets and models directory (see [config.init](src/config.init))\n    - [src/results](src/results)/{train/test}: default training and testing results directory (see [config.init](src/config.init))\n\n\n## Credits\n- Consider citing our work upon using this repo.\n  ```\n  @ARTICLE{delange2021clsurvey,\n    author={M. {Delange} and R. {Aljundi} and M. {Masana} and S. {Parisot} and X. {Jia} and A. {Leonardis} and G. {Slabaugh} and T. {Tuytelaars}},\n    journal={IEEE Transactions on Pattern Analysis and Machine Intelligence}, \n    title={A continual learning survey: Defying forgetting in classification tasks}, \n    year={2021},volume={},number={},pages={1-1},\n    doi={10.1109/TPAMI.2021.3057446}}\n  ```\n- Thanks to Huawei for funding this project.\n- Checkout the CL [Avalanche project](https://github.com/ContinualAI/avalanche) for benchmark setups of TinyImagenet and iNaturalist2018.\n- Thanks to the following repositories:\n    - https://github.com/rahafaljundi/MAS-Memory-Aware-Synapses\n    - https://github.com/facebookresearch/GradientEpisodicMemory\n    - https://github.com/arunmallya/packnet\n    - https://github.com/joansj/hat\n* If you want to join the Continual Learning community, checkout https://www.continualai.org\n\n## Support\n* If you have troubles, please open a Git issue.\n* Have you defined your method in the framework and want to share it with the community? Send a pull request!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FMattdl%2FCLsurvey","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FMattdl%2FCLsurvey","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FMattdl%2FCLsurvey/lists"}