https://github.com/ffiirree/dl_notes

Deep Learning Notes
https://github.com/ffiirree/dl_notes

Last synced: 9 months ago
JSON representation

Deep Learning Notes

• Host: GitHub
• URL: https://github.com/ffiirree/dl_notes
• Owner: ffiirree
• Created: 2020-09-12T08:01:02.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2021-11-26T02:47:17.000Z (about 4 years ago)
• Last Synced: 2025-02-13T07:41:47.955Z (11 months ago)
• Language: TeX
• Size: 16.2 MB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# Papers

## Basic

### Activation Function & Initialization

- [x] [2010] `Xavier` - [Understanding the difficulty of training deep feedforward neural networks](http://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf) [AISTATS]

- [x] [2015] `PReLU,Kaiming` - [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://arxiv.org/abs/1502.01852) [ICCV]

### PCA & Whitening & Smoothness of the Optimization Landscape

- [ ] [UFLDL Tutorial] [PCA Whitening](https://arxiv.org/abs/1312.4400)

- [ ] [1997] [Edges are the 'independent components' of natural scenes](https://papers.nips.cc/paper/1321-edges-are-the-independent-components-of-natural-scenes.pdf)

- [x] [2015] [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://arxiv.org/abs/1502.03167) [arXiv]

- [x] [2018] Smoothness of the Optimization Landscape - [How Does Batch Normalization Help Optimization?](https://arxiv.org/abs/1805.11604) [NeurIPS]

- [x] [2019] [Rethinking the Usage of Batch Normalization and Dropout in the Training of Deep Neural Networks](https://arxiv.org/abs/1905.05928) [arXiv]

### Deep supervision

- [x] [2014] [Deeply-Supervised Nets](https://arxiv.org/abs/1409.5185)

- [x] [2015] [Training Deeper Convolutional Networks with Deep Supervision](https://arxiv.org/abs/1505.02496)

### Data Augmentation

## Optimization Algorithms

- [ ] [An overview of gradient descent optimization algorithms](https://ruder.io/optimizing-gradient-descent/)

## Models

- [x] [1998] `LeNet` - [GradientBased Learning Applied to Document Recognition](http://vision.stanford.edu/cs598_spring07/papers/Lecun98.pdf)

- [x] [2012] `AlexNet` - [ImageNet Classification with Deep Convolutional Neural Networks](https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf)

- [x] [2013] `NIN, Global Average Pooling, 1 x 1 convolution` - [Network In Network](https://arxiv.org/abs/1312.4400) [arXiv]

- [x] [2014] `VGGNet` - [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://arxiv.org/abs/1409.1556) [arXiv]

- [x] [2021] [`RepVGG`: Making VGG-style ConvNets Great Again](https://arxiv.org/abs/2101.03697) [CVPR]

### Inception

- [ ] [2014] `Inception V1, GooLeNet` - [Going deeper with convolutions](https://arxiv.org/abs/1409.4842) [CVPR]

- [ ] [2015] `Inception V2, Inception V3` - [Rethinking the Inception Architecture for Computer Vision](https://arxiv.org/pdf/1512.00567v3.pdf) [CVPR]

- [ ] [2016] [`Inception-v4`, `Inception-ResNet` and the Impact of Residual Connections on Learning](https://arxiv.org/abs/1602.07261) [arXiv]

## Skip connections

- [x] [2015] `ResNet`- [Deep Residual Learning for Image Recognition](https://arxiv.org/abs/1512.03385) [CVPR]

- [x] [2016] [Identity Mappings in Deep Residual Networks.](https://arxiv.org/abs/1603.05027) [CVPR]

- [ ] [2017] [The Shattered Gradients Problem: If resnets are the answer, then what is the question?](https://arxiv.org/abs/1702.08591) [arXiv]

- [ ] [2017] `DenseNet` - [Densely Connected Convolutional Networks](https://arxiv.org/abs/1608.06993) [CVPR]

## Interpretability

### Visualization

- [ ] [2013] [Visualizing and Understanding Convolutional Networks](https://arxiv.org/abs/1311.2901)

- [ ] [2013] [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://arxiv.org/abs/1312.6034)

- [ ] [2015] [Understanding Neural Networks Through Deep Visualization](https://arxiv.org/abs/1506.06579)

- [x] [2015] `DeepDream` - [Inceptionism: Going Deeper into Neural Networks](https://ai.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html)

## Attention Mechanism

- [ ] [2017] `Transformer` - [Attention is all you need](https://arxiv.org/abs/1706.03762) [NeurIPS]

- [ ] [2018] [Non-local Neural Networks](https://arxiv.org/abs/1711.07971) [CVPR]

- [ ] [2018] `SENet` - [Squeeze-and-Excitation Networks](https://arxiv.org/abs/1709.01507) [CVPR]

- [ ] [2018] `CBAM` - [Convolutional Block Attention Module](https://arxiv.org/abs/1807.06521) [CVPR]

- [ ] [2019] `DANet` - [Dual Attention Network for Scene Segmentation](https://arxiv.org/abs/1809.02983) [CVPR]

## Object Detection & Semantic Segmentation

- [ ] [2013] `R-CNN` - [Rich feature hierarchies for accurate object detection and semantic segmentation](https://arxiv.org/abs/1311.2524) [CVPR]

- [ ] [2014] `SPPNet` - [Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition](https://arxiv.org/abs/1406.4729) [TPAMI]

- [x] [2014] `FCN` - [Fully Convolutional Networks for Semantic Segmentation](https://arxiv.org/abs/1411.4038) [CVPR]

- [ ] [2015] [`Fast R-CNN`](https://arxiv.org/abs/1504.08083) [ICCV]

- [ ] [2015] [`Faster R-CNN`: Towards Real-Time Object Detection with Region Proposal Networks](https://arxiv.org/abs/1506.01497) [NeurIPS]

- [ ] [2015] `YOLOv1` - [You Only Look Once: Unified, Real-Time Object Detection](https://arxiv.org/abs/1506.02640) [CVPR]

- [ ] [2015] [`SSD`: Single Shot MultiBox Detector](https://arxiv.org/abs/1512.02325) [ECCV]

- [ ] [2016] `YOLOv2` - [YOLO9000: Better, Faster, Stronger](https://arxiv.org/abs/1612.08242) [CVPR]

- [ ] [2017] `FPN` - [Feature Pyramid Networks for Object Detection](https://arxiv.org/abs/1612.03144) [CVPR]

- [ ] [2017] [`Mask R-CNN`](https://arxiv.org/abs/1703.06870) [ICCV]

- [ ] [2018] [`YOLOv3`: An Incremental Improvement](https://arxiv.org/abs/1804.02767) [CVPR]

- [ ] [2019] [`Mask Scoring R-CNN`](https://arxiv.org/abs/1903.00241) [CVPR]

- [ ] [2019] [`CenterMask` : Real-Time Anchor-Free Instance Segmentation](https://arxiv.org/abs/1911.06667) [CVPR]

- [ ] [2019] [`EfficientDet`: Scalable and Efficient Object Detection](https://arxiv.org/abs/1911.09070) [CVPR]

- [ ] [2019] [`SOLO`: Segmenting Objects by Locations](https://arxiv.org/abs/1912.04488) [arXiv]

- [ ] [2019] [`PointRend`: Image Segmentation as Rendering](https://arxiv.org/abs/1912.08193) [CVPR]

- [ ] [2020] [`SOLOv2`: Dynamic and Fast Instance Segmentation](https://arxiv.org/abs/2003.10152) [NeurIPS]

- [ ] [2020] [`BlendMask`: Top-Down Meets Bottom-Up for Instance Segmentation](https://arxiv.org/abs/2001.00309) [CVPR]

### UNet family

- [x] [2015] [`U-Net`: Convolutional Networks for Biomedical Image Segmentation](https://arxiv.org/abs/1505.04597)

- [ ] [2016] [`V-Net`: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://arxiv.org/abs/1606.04797)

- [x] [2018] [`UNet++`: A Nested U-Net Architecture for Medical Image Segmentation](https://arxiv.org/abs/1807.10165)

- [x] [2020] [`UNet 3+`: A Full-Scale Connected UNet for Medical Image Segmentation](https://arxiv.org/abs/2004.08790)

## Neural Style Transfer

- [ ] [2018] [Neural Style Transfer: A Review](https://arxiv.org/abs/1705.04058)

- [ ] [2015] [A Neural Algorithm of Artistic Style](https://arxiv.org/abs/1508.06576)

- [ ] [2016] [Image Style Transfer Using Convolutional Neural Networks](https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Gatys_Image_Style_Transfer_CVPR_2016_paper.pdf) [CVPR]

- [ ] [2016] [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://arxiv.org/abs/1603.08155)

- [ ] [2016] [Texture Networks: Feed-forward Synthesis of Textures and Stylized Images](https://arxiv.org/abs/1603.03417)

- [ ] [2016] [Instance Normalization: The Missing Ingredient for Fast Stylization](https://arxiv.org/abs/1607.08022)

- [ ] [2017] [StyleBank: An Explicit Representation for Neural Image Style Transfer](https://arxiv.org/abs/1703.09210)

- [ ] [2017] [A Learned Representation For Artistic Style](https://arxiv.org/abs/1610.07629)

- [ ] [2017] [Exploring the structure of a real-time, arbitrary neural artistic stylization network](https://arxiv.org/abs/1705.06830)

- [ ] [2017] [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://arxiv.org/abs/1703.06868)

## Super Resolution

## Pose Estimation

## Deep Generative Models

### Auto Regressive Models

- [ ] [2016] `PixelRNN` - [Pixel Recurrent Neural Networks](https://arxiv.org/abs/1601.06759) [arXiv]

- [ ] [2016] `PixelCNN` - [Conditional Image Generation with PixelCNN Decoders](https://arxiv.org/abs/1606.05328) [NeurIPS]

### VAE

- [x] [2013] `VAE` - [Auto-Encoding Variational Bayes](https://arxiv.org/abs/1312.6114)

- [ ] [2014] [Stochastic Backpropagation and Approximate Inference in Deep Generative Models](https://arxiv.org/abs/1401.4082)

- [ ] [2015] `CVAE` - [Learning Structured Output Representation using Deep Conditional Generative Models](https://papers.nips.cc/paper/2015/file/8d55a249e6baa5c06772297520da2051-Paper.pdf) [NeurIPS]

- [ ] [2016] [Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders](https://arxiv.org/abs/1611.02648) [arXiv]

- [x] [2016] [Tutorial on Variational Autoencoders](https://arxiv.org/abs/1606.05908)

- [x] [2018] `VI` - [Variational Inference: A Review for Statisticians](https://arxiv.org/abs/1601.00670)

### GAN

- [ ] [2019] [The Six Fronts of the Generative Adversarial Networks](https://arxiv.org/abs/1910.13076)

- [ ] [2019] [How Generative Adversarial Networks and Their Variants Work: An Overview](https://arxiv.org/abs/1711.05914)

- [x] [2014] `GAN` - [Generative Adversarial Nets](https://arxiv.org/abs/1406.2661) [NeurIPS]

- [x] [2014] `CGAN` - [Conditional Generative Adversarial Nets](https://arxiv.org/abs/1411.1784) [arXiv]

- [x] [2014] `LAPGAN` - [Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks](https://arxiv.org/abs/1506.05751) [NeurIPS]

- [x] [2015] `DCGAN` - [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://arxiv.org/abs/1511.06434) [ICCV]

- [x] [2016] [Pixel-Level Domain Transfer](https://arxiv.org/abs/1603.07442) [ECCV]

- [ ] [2016] [Generative Adversarial Text to Image Synthesis](https://arxiv.org/abs/1605.05396) [arXiv]

- [x] [2016] [`InfoGAN`: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://arxiv.org/abs/1606.03657) [NeurIPS]

- [x] [2016] `IS` - [Improved Techniques for Training GANs](https://arxiv.org/abs/1606.03498) [NeurIPS]

- [ ] [2016] [`f-GAN`: Training Generative Neural Samplers using Variational Divergence Minimization](https://arxiv.org/abs/1606.00709) [NeurIPS]

- [x] [2016] [Semi-Supervised Learning with Generative Adversarial Networks](https://arxiv.org/abs/1606.01583) [arXiv]

- [ ] [2016] EBGAN - [Energy-based Generative Adversarial Network](https://arxiv.org/abs/1609.03126) [arXiv]

- [ ] [2016] `GAWWN` - [Learning What and Where to Draw](https://arxiv.org/abs/1610.02454)

- [x] [2016] `Pix2Pix` - [Image-to-Image Translation with Conditional Adversarial Networks](https://arxiv.org/abs/1611.07004) [CVPR]

- [x] [2016] `ACGAN` - [Conditional Image Synthesis with Auxiliary Classifier GANs](https://arxiv.org/abs/1610.09585) [ICML]

- [x] [2016] [`StackGAN`: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks](https://arxiv.org/abs/1612.03242) [ICCV]

- [x] [2016] `ALI` - [Adversarially Learned Inference](https://arxiv.org/abs/1606.00704) [arXiv] / `BiGANs` - [Adversarial Feature Learning](https://arxiv.org/abs/1605.09782) [arXiv]

- [ ] [2017] [Towards Principled Methods for Training Generative Adversarial Networks](https://arxiv.org/abs/1701.04862) [arXiv]

- [ ] [2017] `WGAN` - [Wasserstein GAN](https://arxiv.org/abs/1701.07875) [arXiv]

- [x] [2017] `LSGAN` - [Least Squares Generative Adversarial Networks](https://arxiv.org/abs/1611.04076) [ICCV]

- [x] [2017] `CycleGAN` - [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://arxiv.org/abs/1703.10593) [ICCV]

- [x] [2017] `TripleGAN` - [Triple Generative Adversarial Nets](https://arxiv.org/abs/1703.02291) [NeurIPS]

- [ ] [2017] `WGAN-GP` - [Improved Training of Wasserstein GANs](https://arxiv.org/abs/1704.00028) [NeurIPS]

- [ ] [2017] `FID` - [GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium](https://arxiv.org/abs/1706.08500) [NeurIPS]

- [x] [2017] `ProGAN` - [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://arxiv.org/abs/1710.10196) [arXiv]

- [ ] [2018] `BSGAN` - [Boundary-Seeking Generative Adversarial Networks](https://arxiv.org/abs/1702.08431) [arXiv]

- [ ] [2018] `SNGAN` - [Spectral Normalization for Generative Adversarial Networks](https://arxiv.org/abs/1802.05957) [arXiv]

- [ ] [2018] `WGAN-div` - [Wasserstein Divergence for GANs](https://arxiv.org/abs/1712.01026) [ECCV]

- [ ] [2018] `SAGAN` - [Self-Attention Generative Adversarial Networks](https://arxiv.org/abs/1805.08318) [ICML]

- [ ] [2018] [cGANs with Projection Discriminator](https://arxiv.org/abs/1802.05637) [arXiv]

- [ ] [2018] [How good is my GAN?](http://thoth.inrialpes.fr/research/ganeval/) [ECCV]

- [ ] [2018] `MUNIT` - [Multimodal Unsupervised Image-to-Image Translation](https://arxiv.org/abs/1804.04732) [ECCV]

- [ ] [2018] `Pix2PixHD` - [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://arxiv.org/abs/1711.11585) [CVPR]

- [x] [2018] `BigGAN` - [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://arxiv.org/abs/1809.11096) [arXiv]

- [x] [2018] [`StarGAN`: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://arxiv.org/abs/1711.09020) [CVPR]

- [ ] [2018] [`PacGAN`: The power of two samples in generative adversarial networks](https://arxiv.org/abs/1712.04086) [NeurIPS]

- [ ] [2018] `StyleGAN` - [A Style-Based Generator Architecture for Generative Adversarial Networks](https://arxiv.org/abs/1812.04948) [CVPR]

- [ ] [2019] `FUNIT` - [Few-Shot Unsupervised Image-to-Image Translation](https://arxiv.org/abs/1905.01723) [ICCV]

- [ ] [2019] `SPAGAN` - [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://arxiv.org/abs/1903.07291) [CVPR]

- [ ] [2019] `StyleGAN2` - [Analyzing and Improving the Image Quality of StyleGAN](https://arxiv.org/abs/1912.04958) [CVPR]