https://github.com/freya135/optimizing-catastrophic-forgetting

This project investigates various continual learning methods to mitigate catastrophic forgetting
https://github.com/freya135/optimizing-catastrophic-forgetting

adam-optimizer catastrophic-forgetting dataset elastic-weight-consolidation f1score learning-without-forgetting mnist replay-based-approach split-cifar100 synaptic-intelligence visualization

Last synced: 8 months ago
JSON representation

This project investigates various continual learning methods to mitigate catastrophic forgetting

• Host: GitHub
• URL: https://github.com/freya135/optimizing-catastrophic-forgetting
• Owner: Freya135
• Created: 2025-05-05T16:34:10.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2025-05-05T16:34:27.000Z (about 1 year ago)
• Last Synced: 2025-05-31T10:50:42.177Z (about 1 year ago)
• Topics: adam-optimizer, catastrophic-forgetting, dataset, elastic-weight-consolidation, f1score, learning-without-forgetting, mnist, replay-based-approach, split-cifar100, synaptic-intelligence, visualization
• Language: Python
• Homepage:
• Size: 994 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# Continual Learning on Split CIFAR-100

This project investigates various continual learning methods to mitigate catastrophic forgetting using the Split CIFAR-100 benchmark. A custom multi-head CNN model is used, with each head handling a 10-class task (total of 10 tasks). Methods include:

- ADAM optimizer with frozen heads

- Replay-based approach

- Elastic Weight Consolidation (EWC)

- Learning Without Forgetting (LwF)

- Synaptic Intelligence (SI)

---

##  File Descriptions

| File                              | Description                                                                                      |

|-----------------------------------|--------------------------------------------------------------------------------------------------|

| `adam_optimizer+freezing.py`      | Baseline training using ADAM optimizer with frozen output heads for each task.                   |

| `catastrophic_forgetting_demo.py` | Demonstration of catastrophic forgetting using naive sequential training without any mitigation. |

| `elastic_weight_consolidation.py` | Implementation of EWC for continual learning to preserve important weights.                      |

| `learning_without_forgetting.py`  | Implements LwF using distillation losses to retain knowledge of past tasks.                      |

| `replay_based_approach.py`        | Implements replay by storing and mixing a memory buffer of past samples during training.         |

| `si.py`                           | Synaptic Intelligence implementation to regularize changes in important parameters.              | 

| `vis.py`                          | Visualizes a sample image from the CIFAR-100 dataset with its label.                             |

| `F1score/`                        | Stores F1 score plots for each task and approach.                                                |

| `Graphs/`                         | Stores plots for accuracy, AUC, and confusion matrices.                                          |

| `data/`                           | Directory automatically created by torchvision for CIFAR-100 dataset.                            |

---

## Setup Instructions

1. **Install Requirements** (Python 3.7+ recommended):

    ```bash

    pip install torch torchvision scikit-learn matplotlib seaborn

    ```

2. **Run Training Scripts**

Each script runs training and evaluation on Split CIFAR-100:

```bash

python adam_optimizer+freezing.py

python replay_based_approach.py

python elastic_weight_consolidation.py

python learning_without_forgetting.py

python si.py