Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/hurshd0/must-read-papers-for-ml

Collection of must read papers for Data Science, or Machine Learning / Deep Learning Engineer
https://github.com/hurshd0/must-read-papers-for-ml

convolutional-networks data-analysis data-science deep-learning exploratory-data-analysis generalized-additive-models machine-learning neural-networks papers recommender-system recurrent-neural-networks rnn-lstm

Last synced: 6 days ago
JSON representation

Collection of must read papers for Data Science, or Machine Learning / Deep Learning Engineer

Awesome Lists containing this project

README 

        
# Must Read Papers for Data Science, ML, and DL

### Curated collection of Data Science, Machine Learning and Deep Learning papers, reviews and articles that are on must read list.



---



> NOTE: :construction: in process of updating, let me know what additional papers, articles, blogs to add I will add them here. 



### How to use  

> :point_right: :star: this repo



## Contributing

- :point_right: :arrows_clockwise: Please feel free to [Submit Pull Request](https://github.com/hurshd0/must-read-papers-for-ml/pulls), if links are broken, or I am missing any important papers, blogs or articles.



[![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-green.svg)](https://github.com/hurshd0/must-read-papers-for-ml/graphs/commit-activity)



### :point_down: READ THIS :point_down:



- :point_right: Reading paper with heavy math is hard, it takes time and effort to understand, most of it is dedication and motivation to not quit, don't be discouraged, read once, read twice, read thrice,... until it clicks and blows you away.



:1st_place_medal: - Read it first



:2nd_place_medal: - Read it second 



:3rd_place_medal: - Read it third



---



## Data Science



### :bar_chart: Pre-processing & EDA



:1st_place_medal: :page_facing_up:[Data preprocessing - Tidy data - by Hadley Wickham](https://vita.had.co.nz/papers/tidy-data.pdf)



### :notebook: General DS



:1st_place_medal: :page_facing_up: [Statistical Modeling: The Two Cultures - by Leo Breiman](https://projecteuclid.org/download/pdf_1/euclid.ss/1009213726)



:2nd_place_medal: :page_facing_up: [A study in Rashomon curves and volumes: A new perspective on

generalization and model simplicity in machine learning](https://arxiv.org/pdf/1908.01755.pdf)



- :video_camera: [KDD 2019 Cynthia Rudin's Keynote](https://youtu.be/wL4X4lG20sM)



:1st_place_medal: :page_facing_up: [Frequentism and Bayesianism: A Python-driven Primer by Jake VanderPlas](https://arxiv.org/pdf/1411.5018.pdf)



---



## Machine Learning



### :dart: General ML



:1st_place_medal: :page_facing_up: [Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning - by Sebastian Raschka](https://arxiv.org/pdf/1811.12808.pdf)



:1st_place_medal: :page_facing_up: [A Brief Introduction into Machine Learning - by Gunnar Ratsch](https://events.ccc.de/congress/2004/fahrplan/files/105-machine-learning-paper.pdf)



:3rd_place_medal: :page_facing_up: [An Introduction to the Conjugate Gradient Method Without the Agonizing Pain - by Jonathan Richard Shewchuk](http://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradient.pdf)



:3rd_place_medal: :page_facing_up: [On Model Stability as a Function of Random Seed](https://arxiv.org/pdf/1909.10447)



### :mag: Outlier/Anomaly detection



:1st_place_medal: :newspaper: [Outlier Detection : A Survey](https://pdfs.semanticscholar.org/912b/0b7879ca99bf654a26bbb0d50d4b8e0ed6c0.pdf)



### :rocket: Boosting



:2nd_place_medal: :page_facing_up: [XGBoost: A Scalable Tree Boosting System](https://arxiv.org/pdf/1603.02754.pdf)



:2nd_place_medal: :page_facing_up: [LightGBM: A Highly Efficient Gradient BoostingDecision Tree](https://papers.nips.cc/paper/6907-lightgbm-a-highly-efficient-gradient-boosting-decision-tree.pdf)



:2nd_place_medal: :page_facing_up: [AdaBoost and the Super Bowl of Classifiers - A Tutorial Introduction to Adaptive Boosting](http://www.inf.fu-berlin.de/inst/ag-ki/adaboost4.pdf)



:3rd_place_medal: :page_facing_up: [Greedy Function Approximation: A Gradient Boosting Machine](https://projecteuclid.org/download/pdf_1/euclid.aos/1013203451)



### :book: Unraveling Blackbox ML



:3rd_place_medal: :page_facing_up: [Peeking Inside the Black Box: Visualizing Statistical Learning with Plots of Individual Conditional Expectation](https://arxiv.org/pdf/1309.6392.pdf)



:3rd_place_medal: :page_facing_up: [Data Shapley: Equitable Valuation of Data for Machine Learning](https://arxiv.org/pdf/1904.02868.pdf)



### :scissors: Dimensionality Reduction 



:1st_place_medal: :page_facing_up: [A Tutorial on Principal Component Analysis](https://arxiv.org/pdf/1404.1100.pdf)



:2nd_place_medal: :page_facing_up: [How to Use t-SNE Effectively](https://distill.pub/2016/misread-tsne/)



:3rd_place_medal: :page_facing_up: [Visualizing Data using t-SNE](https://lvdmaaten.github.io/publications/papers/JMLR_2008.pdf)



### :chart_with_upwards_trend: Optimization



:1st_place_medal: :page_facing_up: [A Tutorial on Bayesian Optimization](https://arxiv.org/abs/1807.02811)



:2nd_place_medal: :page_facing_up: [Taking the Human Out of the Loop: A review of Bayesian Optimization](https://www.cs.ox.ac.uk/people/nando.defreitas/publications/BayesOptLoop.pdf)



---



### Famous Blogs



[Sebastian Raschka](https://sebastianraschka.com/blog/index.html)

[Chip Huyen](https://huyenchip.com/blog/)



---



### :8ball: :crystal_ball: Recommenders



#### Surveys



:1st_place_medal: :page_facing_up: [A Survey of Collaborative Filtering Techniques](http://downloads.hindawi.com/archive/2009/421425.pdf)



:1st_place_medal: :page_facing_up: [Collaborative Filtering Recommender Systems](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.130.4520&rep=rep1&type=pdf)



:1st_place_medal: :page_facing_up: [Deep Learning Based Recommender System: A Survey and New Perspectives](https://sci-hub.tw/10.1145/3285029)



:1st_place_medal: :page_facing_up: :thinking: :star: [Explainable Recommendation: A Survey and New Perspectives](https://arxiv.org/abs/1804.11192) :star:



#### Case Studies



:2nd_place_medal: :page_facing_up: [The Netflix Recommender System: Algorithms, Business Value,and Innovation](http://delivery.acm.org/10.1145/2850000/2843948/a13-gomez-uribe.pdf)



- :globe_with_meridians: Netflix Medium Blog

  - [Netflix Recommendations: Beyond the 5 stars Part 1](https://medium.com/netflix-techblog/netflix-recommendations-beyond-the-5-stars-part-2-d9b96aa399f5)

  - [Netflix Recommendations: Beyond the 5 stars Part 2](https://medium.com/netflix-techblog/netflix-recommendations-beyond-the-5-stars-part-2-d9b96aa399f5)



:2nd_place_medal: :page_facing_up: [Two Decades of Recommender Systems at Amazon.com](https://pdfs.semanticscholar.org/0f06/d328f6deb44e5e67408e0c16a8c7356330d1.pdf)



:2nd_place_medal: :globe_with_meridians: [How Does Spotify Know You So Well?](https://medium.com/s/story/spotifys-discover-weekly-how-machine-learning-finds-your-new-music-19a41ab76efe)



:point_right: More In-Depth study, :closed_book: [Recommender Systems Handbook](https://www.amazon.com/Recommender-Systems-Handbook-Francesco-Ricci/dp/1489976361) 



---



### Famous Deep Learning Blogs :cowboy_hat_face:



:globe_with_meridians: [Stanford UFLDL Deep Learning Tutorial](http://ufldl.stanford.edu/tutorial/)



:globe_with_meridians: [Distill.pub](https://distill.pub/)



:globe_with_meridians: [Colah's Blog](http://colah.github.io/)



:globe_with_meridians: [Andrej Karpathy](https://karpathy.github.io/)



:globe_with_meridians: [Zack Lipton](http://zacklipton.com/articles/)



:globe_with_meridians: [Sebastian Ruder](https://ruder.io/)



:globe_with_meridians: [Jay Alammar](http://jalammar.github.io/)



---



## :books: Neural Networks and Deep Learning Neural Networks



:star: :1st_place_medal: :newspaper: [The Matrix Calculus You Need For Deep Learning - Terence Parr and Jeremy Howard](https://arxiv.org/pdf/1802.01528.pdf) :star:



:1st_place_medal: :newspaper: [Deep learning -Yann LeCun, Yoshua Bengio & Geoffrey Hinton](https://www.cs.toronto.edu/~hinton/absps/NatureDeepReview.pdf)



:1st_place_medal: :page_facing_up: [Generalization in Deep Learning](https://arxiv.org/pdf/1710.05468.pdf)



:1st_place_medal: :page_facing_up: [Topology of Learning in Artificial Neural Networks](https://arxiv.org/pdf/1902.08160v1.pdf)



:1st_place_medal: :page_facing_up: [Dropout: A Simple Way to Prevent Neural Networks from Overfitting](https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf)



:2nd_place_medal: :page_facing_up: [Polynomial Regression As an Alternative to Neural Nets](https://arxiv.org/pdf/1806.06850)



:2nd_place_medal: :globe_with_meridians: [The Neural Network Zoo](https://www.asimovinstitute.org/neural-network-zoo/?utm_source=mybridge&utm_medium=blog&utm_campaign=read_more)



:2nd_place_medal: :globe_with_meridians: [Image Completion with Deep Learning in TensorFlow](http://bamos.github.io/2016/08/09/deep-completion/?utm_source=mybridge&utm_medium=blog&utm_campaign=read_more)



:2nd_place_medal: :page_facing_up: [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://arxiv.org/pdf/1502.03167v3)



:3rd_place_medal: :page_facing_up: [A systematic study of the class imbalance problem in convolutional neural networks](https://arxiv.org/pdf/1710.05381)



:3rd_place_medal: :page_facing_up: [All Neural Networks are Created Equal](https://arxiv.org/pdf/1905.10854)



:3rd_place_medal: :page_facing_up: [Adam: A Method for Stochastic Optimization](https://arxiv.org/pdf/1412.6980)



:3rd_place_medal: :page_facing_up: [AutoML: A Survey of the State-of-the-Art](https://arxiv.org/pdf/1908.00709v1)



### :framed_picture: CNNs



:1st_place_medal: :page_facing_up: [Visualizing and Understanding Convolutional Networks -by Andrej Karpathy Justin Johnson Li Fei-Fei](https://arxiv.org/pdf/1311.2901.pdf)



:2nd_place_medal: :page_facing_up: [Deep Residual Learning for Image Recognition](https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/He_Deep_Residual_Learning_CVPR_2016_paper.pdf)



:2nd_place_medal: :page_facing_up:[AlexNet-ImageNet Classification with Deep Convolutional

Neural Networks](https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf)



:2nd_place_medal: :page_facing_up:[VGG Net-VERY DEEP CONVOLUTIONAL NETWORKS FOR LARGE-SCALE IMAGE RECOGNITION](https://arxiv.org/pdf/1409.1556v6.pdf)



:3rd_place_medal: :page_facing_up: [A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extraction](https://arxiv.org/pdf/1512.06293)



:3rd_place_medal: :page_facing_up: [Large-scale Video Classification with Convolutional Neural Networks](https://www.cv-foundation.org/openaccess/content_cvpr_2014/papers/Karpathy_Large-scale_Video_Classification_2014_CVPR_paper.pdf)



:3rd_place_medal: :page_facing_up: [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://arxiv.org/pdf/1707.07998.pdf)



### :black_circle: CapsNet :trident:	



:1st_place_medal: :page_facing_up: [Dynamic Routing Between Capsules](https://arxiv.org/pdf/1710.09829.pdf)



- Blog explaning, ["What are CapsNet, or Capsule Networks?"](https://medium.com/ai%C2%B3-theory-practice-business/understanding-hintons-capsule-networks-part-i-intuition-b4b559d1159b)



- [Capsule Networks Tutorial by Aureline Geron](https://www.youtube.com/watch?v=pPN8d0E3900&t=1199s)



###  :national_park: :speech_balloon: Image Captioning



:1st_place_medal: :page_facing_up: [Show and Tell: A Neural Image Caption Generator](https://arxiv.org/abs/1411.4555)



:2nd_place_medal: :page_facing_up: [Neural Machine Translation by Jointly Learning to Align and Translate](https://arxiv.org/pdf/1409.0473v7)



:2nd_place_medal: :page_facing_up: [StyleNet: Generating Attractive Visual Captions with Styles](https://www.microsoft.com/en-us/research/uploads/prod/2017/06/Generating-Attractive-Visual-Captions-with-Styles)



:2nd_place_medal: :page_facing_up: [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://arxiv.org/abs/1502.03044)



:2nd_place_medal: :page_facing_up: [Where to put the Image in an Image Caption Generator](https://arxiv.org/abs/1703.09137)



:2nd_place_medal: :page_facing_up: [Dank Learning: Generating Memes Using Deep Neural Networks](https://arxiv.org/abs/1806.04510)



### :car: :walking_man: Object Detection :eagle: :football:



:2nd_place_medal: :page_facing_up:[ResNet-Deep Residual Learning for Image Recognition](https://arxiv.org/pdf/1512.03385)



:2nd_place_medal: :page_facing_up: [YOLO-You Only Look Once: Unified, Real-Time Object Detection](https://arxiv.org/pdf/1506.02640)



:2nd_place_medal: :page_facing_up: [Microsoft COCO: Common Objects in Context](https://arxiv.org/pdf/1405.0312)



- [COCO dataset](http://cocodataset.org/#home)



:2nd_place_medal: :page_facing_up:  [(R-CNN) Rich feature hierarchies for accurate object detection and semantic segmentation](https://arxiv.org/pdf/1311.2524.pdf)



:2nd_place_medal: :page_facing_up: [Fast R-CNN](https://arxiv.org/pdf/1504.08083.pdf)



- :computer: [Papers with Code](https://www.paperswithcode.com/paper/fast-r-cnn)



:2nd_place_medal: :page_facing_up: [Faster R-CNN](https://arxiv.org/pdf/1506.01497v3.pdf)



- :computer: [Papers with Code](https://www.paperswithcode.com/paper/mask-r-cnn)



:2nd_place_medal: :page_facing_up: [Mask R-CNN](https://arxiv.org/pdf/1703.06870.pdf)



- :computer: [Papers with Code](https://www.paperswithcode.com/paper/mask-r-cnn)



### :car: :walking_man: :couple: Pose Detection :runner: :dancer:



:2nd_place_medal: :page_facing_up: [DensePose: Dense Human Pose Estimation In The Wild](https://arxiv.org/pdf/1802.00434v1.pdf)



- :computer: [Papers with Code](https://www.paperswithcode.com/paper/densepose-dense-human-pose-estimation-in-the)



:2nd_place_medal: :page_facing_up:  [Parsing R-CNN for Instance-Level Human Analysis](https://arxiv.org/pdf/1811.12596v1.pdf)



- :computer: [Papers with Code](https://www.paperswithcode.com/paper/parsing-r-cnn-for-instance-level-human)



### :abcd: :symbols: Deep NLP :currency_exchange: :1234:



:1st_place_medal: :page_facing_up: [A Primer on Neural Network Models for Natural Language Processing](https://arxiv.org/pdf/1510.00726.pdf)



:1st_place_medal: :page_facing_up: [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://arxiv.org/pdf/1412.3555.pdf)



:1st_place_medal: :page_facing_up: [On the Properties of Neural Machine Translation: Encoder–Decoder Approaches](https://arxiv.org/pdf/1409.1259.pdf)



:1st_place_medal: :page_facing_up: [LSTM: A Search Space Odyssey - by Klaus Greff et al.](https://arxiv.org/pdf/1503.04069.pdf)



:1st_place_medal: :page_facing_up: [A Critical Review of Recurrent Neural Networksfor Sequence Learning](https://arxiv.org/pdf/1506.00019.pdf)



:1st_place_medal: :page_facing_up: [Visualizing and Understanding Recurrent Networks](https://arxiv.org/pdf/1506.02078.pdf)



:star: :1st_place_medal: :page_facing_up: [Attention Is All You Need](https://arxiv.org/pdf/1706.03762) :star:



:1st_place_medal: :page_facing_up: [An Empirical Exploration of Recurrent Network Architectures](http://proceedings.mlr.press/v37/jozefowicz15.pdf)



:1st_place_medal: :page_facing_up: [Open AI (GPT-2) Language Models are Unsupervised Multitask Learners](https://paperswithcode.com/paper/language-models-are-unsupervised-multitask)



:1st_place_medal: :page_facing_up: [BERT: Pre-training of Deep Bidirectional Transformers forLanguage Understanding](https://arxiv.org/pdf/1810.04805)



- [Google BERT Annoucement](https://ai.googleblog.com/2018/11/open-sourcing-bert-state-of-art-pre.html)



:3rd_place_medal: :page_facing_up: [Parameter-Efficient Transfer Learning for NLP](https://arxiv.org/pdf/1902.00751)



:3rd_place_medal: :page_facing_up: [A Sensitivity Analysis of (and Practitioners’ Guide to) ConvolutionalNeural Networks for Sentence Classification](https://arxiv.org/pdf/1510.03820v4.pdf)



:3rd_place_medal: :page_facing_up: [A Survey on Recent Advances in Named Entity Recognition from Deep Learning models](https://arxiv.org/pdf/1910.11470v1)



:3rd_place_medal: :page_facing_up: [Convolutional Neural Networks for Sentence Classification](https://arxiv.org/pdf/1408.5882v2)



:3rd_place_medal: :page_facing_up: [Pervasive Attention: 2D Convolutional Neural Networks for Sequence-to-Sequence Prediction](https://arxiv.org/abs/1808.03867)



:3rd_place_medal: :page_facing_up: [Single Headed Attention RNN: Stop Thinking With Your Head](https://arxiv.org/pdf/1911.11423.pdf)



### :alien: GANs



:1st_place_medal: :page_facing_up: [Generative Adversarial Nets - Goodfellow et al.](https://arxiv.org/pdf/1406.2661v1.pdf)



:books: GAN Rabbit Hole -> [GAN Papers](https://github.com/zhangqianhui/AdversarialNetsPapers)



### :o::heavy_minus_sign::o: GNNs (Graph Neural Networks)



:3rd_place_medal: :page_facing_up: [A Comprehensive Survey on Graph Neural Networks](https://arxiv.org/pdf/1901.00596.pdf)



---



### :man_health_worker: :syringe: Medical AI :pill: :microscope:



[Machine learning classifiers and fMRI: a tutorial overview - by Francisco et al.](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2892746/pdf/nihms100405.pdf)



---



## :point_down: Cool Stuff :point_down:



:loud_sound: :page_facing_up: [SoundNet: Learning Sound

Representations from Unlabeled Video](http://soundnet.csail.mit.edu/)



:art: :page_facing_up: [CAN: Creative Adversarial NetworksGenerating “Art” by Learning About Styles andDeviating from Style Norms](https://arxiv.org/pdf/1706.07068)



:art: :page_facing_up: [Deep Painterly Harmonization](https://arxiv.org/pdf/1804.03189)



- [Github Code](https://github.com/luanfujun/deep-painterly-harmonization)



:man_dancing: :dancer: :page_facing_up: [Everybody Dance Now](https://arxiv.org/pdf/1808.07371)



- [Everybody Dance Now - Youtube Video](https://www.youtube.com/watch?v=PCBTZh41Ris)



:soccer: [Soccer on Your Tabletop](https://arxiv.org/pdf/1806.00890)



:blonde_woman: :haircut_woman: :page_facing_up: [SC-FEGAN: Face Editing Generative Adversarial Network with User's Sketch and Color](https://arxiv.org/abs/1902.06838)



- [Github Code](https://github.com/run-youngjoo/SC-FEGAN)



:camera_flash: :page_facing_up: [Handheld Mobile Photography in Very Low Light](https://arxiv.org/pdf/1910.11336v1)



:japanese_castle: :mosque: :page_facing_up: [Learning Deep Features for Scene Recognitionusing Places Database](http://places.csail.mit.edu/places_NIPS14.pdf)



:bullettrain_front: :bullettrain_side: :page_facing_up: [High-Speed Tracking withKernelized Correlation Filters](https://arxiv.org/pdf/1404.7584)



:clapper: :page_facing_up: [Recent progress in semantic image segmentation](https://arxiv.org/ftp/arxiv/papers/1809/1809.10198)



Rabbit hole -> :loud_sound: :globe_with_meridians: [Analytics Vidhya Top 10 Audio Processing Tasks and their papers](https://www.analyticsvidhya.com/blog/2018/01/10-audio-processing-projects-applications/)



:blonde_man: -> :older_man: :page_facing_up: :page_facing_up: [Face Aging With Condintional GANS](https://arxiv.org/pdf/1702.01983)



:blonde_man: -> :older_man: :page_facing_up: :page_facing_up: [Dual Conditional GANs for Face Aging and Rejuvenation](https://www.ijcai.org/proceedings/2018/0125.pdf)



:balance_scale: :page_facing_up: [BAGAN: Data Augmentation with Balancing GAN](https://arxiv.org/pdf/1803.09655)



[labml.ai Annotated PyTorch Paper Implementations](https://nn.labml.ai/)



---



## :newspaper: Cap Stone Projects :newspaper:



[8 Awesome Data Science Capstone Projects](https://www.analyticsvidhya.com/blog/2019/04/8-awesome-data-science-capstone-projects-from-praxis-business-school/)



[10 Powerful Applications of Linear Algebra in Data Science](https://www.analyticsvidhya.com/blog/2019/07/10-applications-linear-algebra-data-science/)



[Top 5 Interesting Applications of GANs](https://www.analyticsvidhya.com/blog/2019/04/top-5-interesting-applications-gans-deep-learning/)



[Deep Learning Applications a beginner can build in minutes ](https://www.analyticsvidhya.com/blog/2017/02/5-deep-learning-applications-beginner-python/)



---



#### CHANGELOG



2019-10-28 Started `must-read-papers-for-ml` repo



2019-10-29 Added analytics vidhya use case studies article links



2019-10-30 Added Outlier/Anomaly detection paper, separated Boosting, CNN, Object Detection, NLP papers, and added Image captioning papers 



2019-10-31 Added Famous Blogs from Deep and Machine Learning Researchers



2019-11-1 Fixed markdown issues, added contribution guideline



2019-11-20 Added Recommender Surveys, and Papers



2019-12-12 Added R-CNN variants, PoseNets, GNNs



2020-02-23 Added GRU paper