Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/facebookresearch/luckmatters
Understanding Training Dynamics of Deep ReLU Networks
https://github.com/facebookresearch/luckmatters
Last synced: 1 day ago
JSON representation
Understanding Training Dynamics of Deep ReLU Networks
- Host: GitHub
- URL: https://github.com/facebookresearch/luckmatters
- Owner: facebookresearch
- License: other
- Created: 2019-06-05T22:42:56.000Z (over 5 years ago)
- Default Branch: main
- Last Pushed: 2024-10-24T00:46:17.000Z (22 days ago)
- Last Synced: 2024-11-06T12:12:10.428Z (8 days ago)
- Language: Python
- Size: 580 KB
- Stars: 278
- Watchers: 14
- Forks: 31
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
This repo contains the code of the following 6 papers:
## Analysis of Self-supervised learning (`./ssl`)
**Understanding the Role of Nonlinearity in Training Dynamics of Contrastive Learning**
Yuandong Tian
[arXiv](https://arxiv.org/abs/2206.01342)
**Understanding Deep Contrastive Learning via Coordinate-wise Optimization**
Yuandong Tian
[NeurIPS'22](https://arxiv.org/abs/2201.12680) Oral
**Understanding self-supervised Learning Dynamics without Contrastive Pairs**
Yuandong Tian, Xinlei Chen, Surya Ganguli
ICML 2021 [link](https://arxiv.org/abs/2102.06810), *Outstanding Paper Honorable Mention*
**Understanding Self-supervised Learning with Dual Deep Networks**
Yuandong Tian, Lantao Yu, Xinlei Chen, Surya Ganguli
arXiv [link](https://arxiv.org/abs/2010.00578)
## Teacher-student setting in supervised learning
**Student Specialization in Deep ReLU Networks With Finite Width and Input Dimension** (`./student_specialization`)
Yuandong Tian
ICML 2020 [link](https://arxiv.org/abs/1909.13458)
**Luck Matters: Luck Matters: Understanding Training Dynamics of Deep ReLU Networks** (`./luckmatter`)
Yuandong Tian, Tina Jiang, Qucheng Gong, Ari Morcos
arxiv [link](https://arxiv.org/abs/1905.13405)