{"id":37105809,"url":"https://github.com/bjzhb666/gs-lora","last_synced_at":"2026-01-14T12:45:35.020Z","repository":{"id":224934190,"uuid":"666625959","full_name":"bjzhb666/GS-LoRA","owner":"bjzhb666","description":"Continual Forgetting for Pre-trained Vision Models (CVPR 2024)","archived":false,"fork":false,"pushed_at":"2025-12-23T11:47:45.000Z","size":1821,"stargazers_count":68,"open_issues_count":1,"forks_count":4,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-12-24T22:49:55.688Z","etag":null,"topics":["casia-webface","continual-forgetting","continual-learning","editing","face-transformer","forgetting","machine-unlearning","model-editing","unlearning"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2403.11530","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bjzhb666.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-07-15T03:27:35.000Z","updated_at":"2025-12-23T11:47:49.000Z","dependencies_parsed_at":"2024-05-11T09:27:06.123Z","dependency_job_id":"41f8d966-b38a-46e1-8545-35dea44e25f9","html_url":"https://github.com/bjzhb666/GS-LoRA","commit_stats":null,"previous_names":["bjzhb666/gs-lora"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/bjzhb666/GS-LoRA","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bjzhb666%2FGS-LoRA","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bjzhb666%2FGS-LoRA/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bjzhb666%2FGS-LoRA/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bjzhb666%2FGS-LoRA/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bjzhb666","download_url":"https://codeload.github.com/bjzhb666/GS-LoRA/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bjzhb666%2FGS-LoRA/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28420814,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T10:47:48.104Z","status":"ssl_error","status_checked_at":"2026-01-14T10:46:19.031Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["casia-webface","continual-forgetting","continual-learning","editing","face-transformer","forgetting","machine-unlearning","model-editing","unlearning"],"created_at":"2026-01-14T12:45:34.214Z","updated_at":"2026-01-14T12:45:35.007Z","avatar_url":"https://github.com/bjzhb666.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Practical Continual Forgetting for Pre-trained Vision Models (CVPR 2024; T-PAMI 2026)\n\n![1735645758635](image/README/1735645758635.png)\n\nThis is the official implementation of ***GS-LoRA*** (CVPR 2024). GS-LoRA is effective, parameter-efficient, data-efficient, and easy to implement continual forgetting, where selective information is expected to be *continuously* removed from a pre-trained model while maintaining the rest. The core idea is to use LoRA combining *group* *Lasso* to realize fast model editing. For more details, please refer to:\n\n**Continual Forgetting for Pre-trained Vision Models [[paper](https://arxiv.org/abs/2403.11530)] [[video](https://www.youtube.com/watch?v=yigUY5v1Rgc)] [[video in bilibili](https://www.bilibili.com/video/BV1wi421C7PQ/?share_source=copy_web\u0026vd_source=7bb418d0ef24f46374712edc865ed254)]**\n\n[‪Hongbo Zhao](https://scholar.google.com/citations?user=Gs22F0UAAAAJ\u0026hl=zh-CN), Bolin Ni, [‪Junsong Fan‬](https://scholar.google.com/citations?user=AfK4UcUAAAAJ\u0026hl=zh-CN\u0026oi=sra), [‪Yuxi Wang‬](https://scholar.google.com/citations?user=waLCodcAAAAJ\u0026hl=zh-CN\u0026oi=sra), [‪Yuntao Chen‬](https://scholar.google.com/citations?hl=zh-CN\u0026user=iLOoUqIAAAAJ), [‪Gaofeng Meng](https://scholar.google.com/citations?hl=zh-CN\u0026user=5hti_r0AAAAJ), [‪Zhaoxiang Zhang‬](https://scholar.google.com/citations?hl=zh-CN\u0026user=qxWfV6cAAAAJ)\n\n**Practical Continual Forgetting for Pre-trained Vision Models (**T-PAMI 2026**, where we consider more practical scenarios including few-shot settings and missing class settings) [[paper](https://arxiv.org/pdf/2501.09705)]**\n\n[‪Hongbo Zhao](https://scholar.google.com/citations?user=Gs22F0UAAAAJ\u0026hl=zh-CN), [Fei Zhu](https://impression2805.github.io/), Bolin Ni, [Feng Zhu](https://zhufengx.github.io/), [‪Gaofeng Meng](https://scholar.google.com/citations?hl=zh-CN\u0026user=5hti_r0AAAAJ), [‪Zhaoxiang Zhang‬](https://scholar.google.com/citations?hl=zh-CN\u0026user=qxWfV6cAAAAJ)\n\n## Method\n\n![1735646397186](image/README/1735646397186.png)\n\nWe incorporate a set of LoRA modules in each continual forgetting task and propose a sparse structure selection strategy and prototype regularization to achieve accurate and few modifications. (Left) All LoRA modules are added in the Linear layers of FFN in the Transformer blocks and we regard the LoRA modules in a\nTransformer block as one group. We use group sparse regularization ( $\\mathcal{L}_{structure}$ ) to automatically select LoRA groups. The purple groups are selected to modify and the white groups are neglected. The pre-trained model (including Transformer blocks and other parts) is frozen and only LoRA groups are trainable. (Right) To achieve selective forgetting, we utilize selective forgetting and knowledge retention ( $\\mathcal{L}_{data}$ ). To further extend our method to more practical scenarios, we introduce\nprototype regularization $\\mathcal{L}_{pro}$. We use the original model to calculate the prototype of each class and pull away logits from its original prototype for each forgotten class and pull in logits from its own prototype for the remaining classes.\n\n\n## Getting Started\n\n### Installation\n\n#### a. Clone this repository\n\n```bash\ngit clone https://github.com/bjzhb666/GS-LoRA.git\ncd GS-LoRA\n```\n\n#### b. Install the environment\n\n```bash\nconda create -n GSlora python=3.9\npip install -r requirements.txt\n```\n\n#### c. Prepare the datasets\n\n##### c.1 CASIA-100 dataset\n\n```bash\nmkdir data\ncd data\nunzip data.zip\n```\n\nYou can get our CASIA-100 in [https://drive.google.com/file/d/16CaYf45UsHPff1smxkCXaHE8tZo4wTPv/view?usp=sharing](https://drive.google.com/file/d/16CaYf45UsHPff1smxkCXaHE8tZo4wTPv/view?usp=sharing), and put it in the data folder.\n\nNote: CASIA-100 is a subdataset from  [CASIA-WebFace](https://paperswithcode.com/dataset/casia-webface). We have already split the train/test dataset in our google drive.\n\n##### c.2 ImageNet100 dataset\n\nWe get ImageNet100 dataset from [ImageNet100](https://www.kaggle.com/datasets/ambityga/imagenet100). \nThen we should change the folders' names. Please refer to this post for details: [https://github.com/bjzhb666/GS-LoRA/issues/4#issuecomment-2621800559](https://github.com/bjzhb666/GS-LoRA/issues/4#issuecomment-2621800559).\n\nWe get imagenet_classes.txt from [pytorch](https://github.com/pytorch/hub/blob/master/imagenet_classes.txt).\n\nFor `imagenet_folder_names.txt`,  it contains the folder names of all 1k classes. You can download it from our [huggingface link]() or [modelscope link](https://www.modelscope.cn/datasets/zhaohongbo/GS-LoRA-imagenet100). **We highly recommend you to use the link because we have already fix small bugs of the meta data.**\n\n##### c.3 Final File structure\n\n```\n.\n├── faces_webface_112x112_sub100_train_test\n│   ├── test\n│   └── train\n├── imagenet100\n│  ├── convert.py\n│  ├── imagenet_classes.txt\n│  ├── imagenet_class_index.json\n│  ├── imagenet_folder_names.txt\n│  ├── test\n│  └── train\n```\n\n## Pretrain a Face Transformer\n\n```bash\nmkdir result\ncd result\n```\n\nYou can use our [pre-trained Face Transformer](https://drive.google.com/file/d/1Qqeb8fx2_kLeGA1uuLzutpauvyqY3CSV/view?usp=drive_link) directly. Download the pre-trained weight and put it into the result folder\n\n**Or** you can train your own pre-trained models.\n\nYour result folder should be like this:\n\n```bash\nresult\n└── ViT-P8S8_casia100_cosface_s1-1200-150de-depth6new\n    ├── Backbone_VIT_Epoch_1185_Batch_45020_Time_2024-09-26-03-26_checkpoint.pth\n    └── config.txt\n```\n\nCode is in `run_sub.sh`\n\n```bash\nbash scripts/run_sub.sh\n```\n\n`test_sub.sh` is the test code for our Face Transformer. You can test the pre-trained model use it.\n\n```bash\nbash scripts/test_sub.sh\n```\n\n## Continual Forgetting and Single Step Forgetting\n\n### a. Continual Forgetting and Few-shot Continual Forgetting\n\n**We provide the code of all baselines and our methods GS-LoRA, GS-LoRA++ metioned in our paper.**\n\nIn `run_cl_forget.sh`, **LIRF, SCRUB, SCRUB-S, EWC, MAS, L2, DER, DER++, FDR, LwF, Retrain and GS-LoRA, GS-LoRA++** (**Main Table**)\n\n### b. Single-step forgetting and Few-shot single-step forgetting\n\nFor baseline methods, you can still use `run_cl_forget.sh` and change `--num_tasks` to 1. (There are examples in `run_cl_forget.sh`.)\n\n**For single step GS-LoRA, we recommend you to use `run_forget.sh`.**\n\n### c. Continual Forgetting on ImageNet100\n\nFor continual forgetting on ImageNet100 dataset, please use `run_cl_forget_image.sh`.\n\n### d. Missing class forgetting\n\nPlease use `run_cl_forget_open.sh`.\n\n## Citation\n\nIf you find this project useful in your research, please consider citing:\n\n```\n@article{zhao2024continual,\n  title={Continual Forgetting for Pre-trained Vision Models},\n  author={Zhao, Hongbo and Ni, Bolin and Wang, Haochen and Fan, Junsong and Zhu, Fei and Wang, Yuxi and Chen, Yuntao and Meng, Gaofeng and Zhang, Zhaoxiang},\n  journal={arXiv preprint arXiv:2403.11530},\n  year={2024}\n}\n\n@article{zhao2025practical,\n  title={Practical Continual Forgetting for Pre-trained Vision Models},\n  author={Zhao, Hongbo and Zhu, Fei and Ni, Bolin and Zhu, Feng and Meng, Gaofeng and Zhang, Zhaoxiang},\n  journal={arXiv preprint arXiv:2501.09705},\n  year={2025}\n}\n\n```\n\n## Contact\n\nPlease contact us or post an issue if you have any questions.\n\n## Acknowledgement\n\nThis work is built upon the [zhongyy/Face-Transformer: Face Transformer for Recognition (github.com)](https://github.com/zhongyy/Face-Transformer)\n\n## License\n\nThis project is released under the [MIT License](https://github.com/bjzhb666/GS-LoRA/blob/master/LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbjzhb666%2Fgs-lora","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbjzhb666%2Fgs-lora","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbjzhb666%2Fgs-lora/lists"}