{"id":17312959,"url":"https://github.com/vlomonaco/ar1-pytorch","last_synced_at":"2025-04-14T14:22:30.038Z","repository":{"id":62683676,"uuid":"239764197","full_name":"vlomonaco/ar1-pytorch","owner":"vlomonaco","description":"AR1* with Latent Replay, implemented in PyTorch","archived":false,"fork":false,"pushed_at":"2020-05-08T07:26:28.000Z","size":16481,"stargazers_count":30,"open_issues_count":4,"forks_count":9,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-14T14:22:26.201Z","etag":null,"topics":["computer-vision","continual-learning","continualai","core50","deep-learning","incremental-learning","lifelong-learning","pytorch"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/1912.01100","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vlomonaco.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-02-11T13:07:46.000Z","updated_at":"2024-12-26T07:24:40.000Z","dependencies_parsed_at":"2022-11-04T11:15:53.494Z","dependency_job_id":null,"html_url":"https://github.com/vlomonaco/ar1-pytorch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vlomonaco%2Far1-pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vlomonaco%2Far1-pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vlomonaco%2Far1-pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vlomonaco%2Far1-pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vlomonaco","download_url":"https://codeload.github.com/vlomonaco/ar1-pytorch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248894977,"owners_count":21179159,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","continual-learning","continualai","core50","deep-learning","incremental-learning","lifelong-learning","pytorch"],"created_at":"2024-10-15T12:45:12.346Z","updated_at":"2025-04-14T14:22:30.008Z","avatar_url":"https://github.com/vlomonaco.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AR1* with Latent Replay\n\n[![License: CC BY 4.0](https://img.shields.io/badge/License-CC%20BY%204.0-lightgrey.svg)](http://creativecommons.org/licenses/by/4.0/)\n[![built with Python3.6](https://img.shields.io/badge/build%20with-python%203.6-red.svg)](https://www.python.org/)\n[![built with PyTorch1.4](https://img.shields.io/badge/build%20with-pytorch%201.4-brightgreen.svg)](https://pytorch.org/)\n\n\u003cimg src=\"https://repository-images.githubusercontent.com/239764197/2c621f00-8f13-11ea-8250-162421cbd36b\" width=\"400\"/\u003e\n\n### Introduction\n\nIn this repository you will find a pytorch re-implementation of **AR1\\* with\n Latent Replay**. AR1* was shown to be very effective and efficient for\n  continual learning with real-world images. \n  \nPlease consider citing the following paper if you want to use our algorithm in\n your research project or application:\n\t\n\t@article{pellegrini2019,\n\t   title = {Latent Replay for Real-Time Continual Learning},\n\t   author = {Lorenzo Pellegrini and Gabriele Graffieti and Vincenzo Lomonaco\n\t    and Davide Maltoni,\n\t   journal = {Arxiv preprint arXiv:1912.01100},\n\t   url = \"https://arxiv.org/abs/1912.01100\",\n\t   year = {2019}\n\t}\n\t\nThe **original Caffe implementation** can be found [here](https://github.com/lrzpellegrini/Latent-Replay). \nFor more details about other variations or past versions of AR1 you can refer\n to these papers:\n\n\t@InProceedings{lomonaco2019nicv2,\n\t   title = {Rehearsal-Free Continual Learning over Small Non-I.I.D. Batches},\n\t   author = {Vincenzo Lomonaco and Davide Maltoni and Lorenzo Pellegrini},\n\t   journal = {1st Workshop on Continual Learning in Computer Vision\n\t    at CVPR2020},\n\t   url = \"https://arxiv.org/abs/1907.03799\",\n\t   year = {2019}\n\t}\n\t\n\t@article{MALTONI201956,\n        title = \"Continuous learning in single-incremental-task scenarios\",\n        journal = \"Neural Networks\",\n        volume = \"116\",\n        pages = \"56 - 73\",\n        year = \"2019\",\n        issn = \"0893-6080\",\n        doi = \"https://doi.org/10.1016/j.neunet.2019.03.010\",\n        url = \"http://www.sciencedirect.com/science/article/pii/S0893608019300838\",\n        author = \"Davide Maltoni and Vincenzo Lomonaco\"\n    }\n    \n### Project Structure\nThe project is structured as follows:\n\n- [`models/`](models): In this folder the main MobileNetV1 model is defined\n along with the custom Batch Renormalization Pytorch layer.\n- [`ar1star_lat_replay.py`](ar1star_lat_replay.py): Main AR1* with Latent Replay\n algorithm.\n- [`data_loader.py`](data_loader.py): CORe50 data loader.\n- [`LICENSE`](LICENSE): CC BY 4.0 Licence file.\n- [`params.cfg`](params.cfg): Hyperparameters that will be used in the main\n experiment on CORe50 NICv2-391.\n- [`README.md`](README.md): This instructions file.\n- [`utils.py`](utils.py): Utility functions used in the rest of the code.\n\n### Getting Started\n\nWhen using anaconda virtual environment all you need to do is run the following \ncommand and conda will install everything for you. \nSee [environment.yml](./environment.yml):\n\n    conda env create --file environment.yml\n    conda activate ar1-env\n    \nThen to reproduce the results on the CORe50 NICv2-391 benchmark you just\n need to run the following code:\n \n ```bash\npython ar1star_lat_replay.py\n```\n\nThe results will be logged on tensorboard, you can run it with:\n\n \n ```bash\ntensorboard --logdir logs\n```\n\nThen open your browser at `http://localhost:6006`. If everything is setup you\n should reach ~77% of accuracy at the end of the entire training procedure\n  (~24m on a single TitanX GPU). \n  \nThis results is a few percentage point better than the one \n  suggested in the original paper. Keep in mind that this implementation\n   is *slightly different* from the original one in Caffe for a number of\n    reasons:\n   \n   - the ImageNet pre-trained model is different.\n   - the pytorch SGD optimizer is different.\n   - the Batch Renormalization Layers are different.\n   - we did not find any advantage in keeping the BRN layers below the latent\n    reply layer free to learn, so we freeze them from the first batch.\n\n### Use AR1* in Your Project\n\nYou are free to take this code and use it in your own project! However, take\n in mind that the hyper-parameters used in the experiment have been chosen to \n replicate the results shown in the paper for the CORe50 NICv2-391 scenario\n  and may result suboptimal in different settings.\n\nWe suggest to take a look at the papers linked above to have a better idea\n on how to parametrize AR1* on different benchmarks. In particular we\n  underline the importance of BN / BRN parameters, which may be fundamental\n   to tune appropriately. \n   \nWe are working to release AR1* hyper-parameter settings for other\n common Continual Learning benchmarks. Send an email to vincenzo.lomonaco@unibo.it\n  in case you're interested!","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvlomonaco%2Far1-pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvlomonaco%2Far1-pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvlomonaco%2Far1-pytorch/lists"}