{"id":19457853,"url":"https://github.com/grypesc/adagauss","last_synced_at":"2025-04-25T05:31:27.854Z","repository":{"id":261763085,"uuid":"863560378","full_name":"grypesc/AdaGauss","owner":"grypesc","description":"2024 Neurips paper on Continual Learning and Class Incremental Learning","archived":false,"fork":false,"pushed_at":"2025-01-30T19:30:00.000Z","size":1067,"stargazers_count":7,"open_issues_count":1,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-03T17:22:21.594Z","etag":null,"topics":["catastrophic-forgetting","class-incremental","class-incremental-learning","continual-learning","facil","lifelong-learning","lifelong-machine-learning","machine-learning","machinelearning"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2409.18265","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/grypesc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-26T14:00:43.000Z","updated_at":"2025-01-30T19:30:04.000Z","dependencies_parsed_at":"2024-11-08T10:36:10.198Z","dependency_job_id":null,"html_url":"https://github.com/grypesc/AdaGauss","commit_stats":null,"previous_names":["grypesc/adagauss"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grypesc%2FAdaGauss","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grypesc%2FAdaGauss/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grypesc%2FAdaGauss/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grypesc%2FAdaGauss/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/grypesc","download_url":"https://codeload.github.com/grypesc/AdaGauss/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250760834,"owners_count":21482876,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["catastrophic-forgetting","class-incremental","class-incremental-learning","continual-learning","facil","lifelong-learning","lifelong-machine-learning","machine-learning","machinelearning"],"created_at":"2024-11-10T17:24:18.539Z","updated_at":"2025-04-25T05:31:27.074Z","avatar_url":"https://github.com/grypesc.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AdaGauss repository\n\nThis repostory contains code for NeurIPS 2024 paper on Continual Learning: **Task-recency bias strikes back: Adapting covariances in Exemplar-Free Class Incremental Learning** (\u003chttps://arxiv.org/abs/2409.18265\u003e). The repository is based on FACIL benchmark \u003chttps://github.com/mmasana/FACIL\u003e.\n\nWe consider exemplar free class incremental scenario, where we revisit the task-recency bias. Unlike previous works, that focused on the biased classification head, we look at the latent space. We show that old class representations have lower ranks than new classes and this is the core of the problem. We solve this issue with anti-collapse loss. Additionally, we are first to adapt covariances on classes from old tasks to the new one.\n\nIn our method we train feature extractor on all tasks using: cross-entropy, feature distillation through a neural projector and anti-collapse loss functions. We represent each class as Gaussian distribution in the latent space. After each task we transform these distributions from the old model's latent space to the new using an auxilary neural network (to alleviate semantic drift problem).\n\n![image](images/method.png?raw=true \"Adagauss\")\n\n### Setup\nCreate virtual environment and install dependencies:\n```bash\npython3 -m venv venv \u0026\u0026 source venv/bin/activate\npip install torch==2.2.0 torchvision==0.17.0 torchaudio==2.2.0 --index-url https://download.pytorch.org/whl/cu118\npip install requirements.txt\n```\n\nReproduce experiments using scripts in ```scripts``` directory:\n```bash\nbash scripts/cifar-10x10.sh\n```\n\nFeel free to contact me on LinkedIn if you have any questions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgrypesc%2Fadagauss","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgrypesc%2Fadagauss","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgrypesc%2Fadagauss/lists"}