https://github.com/erogol/net2net

Net2Net implementation on PyTorch for any possible vision layers.
https://github.com/erogol/net2net

computer-vision deep-learning net2net pytorch

Last synced: 24 days ago
JSON representation

Net2Net implementation on PyTorch for any possible vision layers.

• Host: GitHub
• URL: https://github.com/erogol/net2net
• Owner: erogol
• Created: 2017-09-21T13:33:27.000Z (about 7 years ago)
• Default Branch: master
• Last Pushed: 2017-10-09T07:59:24.000Z (about 7 years ago)
• Last Synced: 2024-10-03T12:35:15.181Z (about 1 month ago)
• Topics: computer-vision, deep-learning, net2net, pytorch
• Language: Python
• Size: 11 MB
• Stars: 38
• Watchers: 4
• Forks: 10
• Open Issues: 4
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Net2Net

Net2Net implementation on PyTorch for any possible vision layers (nn.Linear, nn.Conv2d, nn.Conv3D, even wider operator btw nn.ConvX to nn.Linear).

Checkout the [paper](https://arxiv.org/abs/1511.05641:) for more detail

## Observations:

- Using BatchNorm between layers, improves the competence of Net2Net. Otherwise, Net2Net approach is not able to get

comparable results to a network trained from scratch.

- Inducing noise to new units and connections prelude to better networks. The effect is more evident without BathNorm layer.

- Normalizing layer weights before any Net2Net operation increases the speed of learning and gives better convergence. Even so, it worths to investgate better normalization methods than L2 norm.