https://github.com/elliottower/abstraction
This repo provides a scalable way of investigating the layer-by-layer evolution of abstraction in deep neural networks.
https://github.com/elliottower/abstraction
Last synced: 3 months ago
JSON representation
This repo provides a scalable way of investigating the layer-by-layer evolution of abstraction in deep neural networks.
- Host: GitHub
- URL: https://github.com/elliottower/abstraction
- Owner: elliottower
- License: bsd-3-clause
- Created: 2022-04-07T21:20:31.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2022-04-08T18:16:21.000Z (about 3 years ago)
- Last Synced: 2025-01-26T03:26:28.528Z (5 months ago)
- Language: Python
- Homepage:
- Size: 36.1 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Abstraction Project
This repo provides a scalable way of investigating the layer-by-layer evolution of abstraction in deep neural networks.
(Extension of *"Evolution of Abstraction Across Layers in Deep Learning Neural Networks"* ([Kozma, 2018](https://www.sciencedirect.com/science/article/pii/S1877050918322294)))
Dataloading code adapted from [PyTorch Imagenet Training Example](https://github.com/pytorch/examples/tree/main/imagenet)
## Requirements
- Install PyTorch ([pytorch.org](http://pytorch.org))
- `pip install -r requirements.txt`
- Download the ImageNet dataset from http://www.image-net.org/
- Then, move and extract the training and validation images to labeled subfolders, using [the following shell script](extract_ILSVRC.sh): `bash extract_ILSVRC.sh`
## Experiments
To run an experiment, run `main.py` with the desired model architecture and the path to the ImageNet dataset.
```bash
python main.py -a resnet18 --data [imagenet-folder with train and val folders]
```
### Example: ResNet18
```bash
python main.py --arch resnet18 \
--workers 2 \
--batch-size 50 \
--evaluate \
--pretrained \
--sample_percent 0.4 \
--output_name "" \
--theta 0.75 \
--data imagenet/
```
## Usage
```
usage: main.py [-h] [--arch ARCH] [-j N] [--epochs N] [--start-epoch N] [-b N]
[--lr LR] [--momentum M] [--weight-decay W] [--print-freq N]
[--resume PATH] [-e] [--pretrained] [--seed SEED] [--gpu GPU]
[--sample_percent P] [--output_name NAME] [--theta T] [--theta-list []]
DIR
PyTorch ImageNet Training
required arguments:
--data DIR path to imagenet dataset (with train and val folders)
optional arguments:
-h, --help show this help message and exit
--arch ARCH, -a ARCH model architecture: alexnet | densenet121 |
densenet161 | densenet169 | densenet201 |
resnet101 | resnet152 | resnet18 | resnet34 |
resnet50 | squeezenet1_0 | squeezenet1_1 | vgg11 |
vgg11_bn | vgg13 | vgg13_bn | vgg16 | vgg16_bn | vgg19
| vgg19_bn (default: resnet18)
-j N, --workers N number of data loading workers (default: 4)
--epochs N number of total epochs to run
--start-epoch N manual epoch number (useful on restarts)
-b N, --batch-size N mini-batch size (default: 256), this is the total
batch size of all GPUs on the current node when using
Data Parallel or Distributed Data Parallel
--lr LR, --learning-rate LR
initial learning rate
--momentum M momentum
--weight-decay W, --wd W
weight decay (default: 1e-4)
--print-freq N, -p N print frequency (default: 10)
--resume PATH path to latest checkpoint (default: none)
-e, --evaluate evaluate model on validation set
--pretrained use pre-trained model
--seed SEED seed for initializing training.
--gpu GPU GPU id to use.
--sample_percent P Percentage of neurons to sample from each layer for abstraction calculation (defaut: 0.1)
--output_name NAME Custom name for output folder and Q folder, allows for saving of cached values (default: "")
--theta T Theta value for Q matrix: correlation value which neurons in the next layer must have towards
the output in order to be kept nonzero (default 0.75)
--theta_list [] Allows for for passing in variable length list of theta values, starting from the last layer
and going backwards. Unspecified values will default to --theta argument.
```