https://github.com/pseudotensor/geoacc

https://github.com/pseudotensor/geoacc

Last synced: 5 months ago
JSON representation

• Host: GitHub
• URL: https://github.com/pseudotensor/geoacc
• Owner: pseudotensor
• Created: 2017-02-06T21:29:29.000Z (over 9 years ago)
• Default Branch: master
• Last Pushed: 2017-02-13T23:03:46.000Z (over 9 years ago)
• Last Synced: 2025-02-08T20:18:02.525Z (over 1 year ago)
• Language: Matlab
• Size: 344 KB
• Stars: 0
• Watchers: 3
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# What: Geodesic Acceleration being implemented with base of Martens et al. code.

# Why: Faster optimization on difficult deep networks

# Used parts of (or inspired by) the following papers/repos:

http://www.cs.toronto.edu/~jmartens/docs/HF_book_chapter.pdf (associated with HFdemo.zip)

https://arxiv.org/pdf/1010.1449v1.pdf (Geodesic acceleration applied to non-linear least squares)

https://arxiv.org/pdf/1503.05671.pdf (updated KFAC method by Martens)

http://www.cs.toronto.edu/~jmartens/research.html (Codes HFDemo.zip and KFAC3-MATLAB.zip)

# Requirements

* Matlab

# To run:

* Download data: wget www.cs.toronto.edu/~jmartens/digs3pts_1.mat

* Open in Matlab: nnet_demo_2.m file

* Hit Run

# Expected results

* Geodesic acceleration helps reach smaller training error faster than natural gradient on deep networks

# Issues

* May have to set jacket to 0 or compMode to 'cpu_single' to work.