https://github.com/kahsolt/nn-prune-svd

Neural network weights prune in a static LoRA–like way
https://github.com/kahsolt/nn-prune-svd

svd tutorial weight-pruning

Last synced: 10 months ago
JSON representation

Neural network weights prune in a static LoRA–like way

• Host: GitHub
• URL: https://github.com/kahsolt/nn-prune-svd
• Owner: Kahsolt
• License: mit
• Created: 2023-09-04T12:40:06.000Z (almost 3 years ago)
• Default Branch: main
• Last Pushed: 2023-12-29T12:37:09.000Z (over 2 years ago)
• Last Synced: 2024-11-09T07:47:16.476Z (over 1 year ago)
• Topics: svd, tutorial, weight-pruning
• Language: Python
• Homepage:
• Size: 131 KB
• Stars: 4
• Watchers: 3
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# nn-prune-svd

    Neural network weights prune in a static LoRA–like way -- via SVD.

----

### results

⚠ Not correct, need more investigation

![](img/run_grid.png)

### dataset

The test dataset is 1000 images picked from the validset of ImageNet-1k. 

- download it from [https://pan.quark.cn/s/cf8688f5bca1](https://pan.quark.cn/s/cf8688f5bca1)

- unzip under `/data` folder; the password is `]@vD4S%ojjH6UC#I!b37`, just to avoid content censorship :(

```

data

└── imagenet-1k

    ├── image_name_to_class_id_and_name.json

    └── val

        └── ILSVRC2012_val_*.png

```

### reference

- torch.svd [https://pytorch.org/docs/stable/generated/torch.svd.html](https://pytorch.org/docs/stable/generated/torch.svd.html)

- LoRA: [https://arxiv.org/abs/2106.09685](https://arxiv.org/abs/2106.09685)

----

by Armit

2023/09/04