An open API service indexing awesome lists of open source software.

https://github.com/elliottower/abstraction

This repo provides a scalable way of investigating the layer-by-layer evolution of abstraction in deep neural networks.
https://github.com/elliottower/abstraction

Last synced: 3 months ago
JSON representation

This repo provides a scalable way of investigating the layer-by-layer evolution of abstraction in deep neural networks.

Awesome Lists containing this project

README

        

# Abstraction Project

This repo provides a scalable way of investigating the layer-by-layer evolution of abstraction in deep neural networks.

(Extension of *"Evolution of Abstraction Across Layers in Deep Learning Neural Networks"* ([Kozma, 2018](https://www.sciencedirect.com/science/article/pii/S1877050918322294)))

Dataloading code adapted from [PyTorch Imagenet Training Example](https://github.com/pytorch/examples/tree/main/imagenet)

## Requirements

- Install PyTorch ([pytorch.org](http://pytorch.org))
- `pip install -r requirements.txt`
- Download the ImageNet dataset from http://www.image-net.org/
- Then, move and extract the training and validation images to labeled subfolders, using [the following shell script](extract_ILSVRC.sh): `bash extract_ILSVRC.sh`

## Experiments

To run an experiment, run `main.py` with the desired model architecture and the path to the ImageNet dataset.

```bash
python main.py -a resnet18 --data [imagenet-folder with train and val folders]
```

### Example: ResNet18

```bash
python main.py --arch resnet18 \
--workers 2 \
--batch-size 50 \
--evaluate \
--pretrained \
--sample_percent 0.4 \
--output_name "" \
--theta 0.75 \
--data imagenet/
```

## Usage

```
usage: main.py [-h] [--arch ARCH] [-j N] [--epochs N] [--start-epoch N] [-b N]
[--lr LR] [--momentum M] [--weight-decay W] [--print-freq N]
[--resume PATH] [-e] [--pretrained] [--seed SEED] [--gpu GPU]
[--sample_percent P] [--output_name NAME] [--theta T] [--theta-list []]
DIR

PyTorch ImageNet Training

required arguments:
--data DIR path to imagenet dataset (with train and val folders)

optional arguments:
-h, --help show this help message and exit
--arch ARCH, -a ARCH model architecture: alexnet | densenet121 |
densenet161 | densenet169 | densenet201 |
resnet101 | resnet152 | resnet18 | resnet34 |
resnet50 | squeezenet1_0 | squeezenet1_1 | vgg11 |
vgg11_bn | vgg13 | vgg13_bn | vgg16 | vgg16_bn | vgg19
| vgg19_bn (default: resnet18)
-j N, --workers N number of data loading workers (default: 4)
--epochs N number of total epochs to run
--start-epoch N manual epoch number (useful on restarts)
-b N, --batch-size N mini-batch size (default: 256), this is the total
batch size of all GPUs on the current node when using
Data Parallel or Distributed Data Parallel
--lr LR, --learning-rate LR
initial learning rate
--momentum M momentum
--weight-decay W, --wd W
weight decay (default: 1e-4)
--print-freq N, -p N print frequency (default: 10)
--resume PATH path to latest checkpoint (default: none)
-e, --evaluate evaluate model on validation set
--pretrained use pre-trained model
--seed SEED seed for initializing training.
--gpu GPU GPU id to use.
--sample_percent P Percentage of neurons to sample from each layer for abstraction calculation (defaut: 0.1)
--output_name NAME Custom name for output folder and Q folder, allows for saving of cached values (default: "")
--theta T Theta value for Q matrix: correlation value which neurons in the next layer must have towards
the output in order to be kept nonzero (default 0.75)
--theta_list [] Allows for for passing in variable length list of theta values, starting from the last layer
and going backwards. Unspecified values will default to --theta argument.

```