Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/caselles/paper_notes

Paper notes for my PhD on Machine Learning (mostly focused on Reinforcement Learning)
https://github.com/caselles/paper_notes

Last synced: over 1 year ago
JSON representation

Paper notes for my PhD on Machine Learning (mostly focused on Reinforcement Learning)

Awesome Lists containing this project

README 

          
# Summaries of Machine Learning (mostly Reinforcement Learning) papers



Inspired by [Adrian Colyer][1], [Denny Britz][2] and [Daniel Seita][3] 



This contains my notes for research papers that are relevant for my PhD on Machine Learning. First, I list papers that I've read and papers that I want to read. Then, read papers are numbered on a (1) to (5) scale where a (1) means I have only barely skimmed it, while a (5) means I feel

confident that I understand almost everything about the paper. The links

here go to my paper summaries if I have them, otherwise those papers are on my

**TODO** list. 



Contents:



- [Machine Learning for Robotics](#machine-learning-for-robotics)

    - [Papers I have read](#papers-i-have-read)

    - [Papers I want to read](#papers-i-want-to-read)

- [Questions for which I need the answer](#questions-for-which-i-need-the-answer)



# Machine Learning papers



## Papers I've read



- [Learning to Act by Predicting the Future](https://github.com/Caselles/paper_notes/blob/master/read_papers/learning_to_act_by_predicting_the_future.md) (5)

- [Universal Value Function Approximators](https://github.com/Caselles/paper_notes/blob/master/read_papers/universal_value_function_approximators.md) (4)

- [Horde: A Scalable Real-time Architecture for Learning Knowledge from Unsupervised Sensorimotor Interaction](https://github.com/Caselles/paper_notes/blob/master/read_papers/horde_a_scalable_real_time.md) (5)

- [Learning by Playing – Solving Sparse Reward Tasks from Scratch](https://github.com/Caselles/paper_notes/blob/master/read_papers/learning_by_playing_solving_sparse_reward_tasks_from_scratch.md) (3)

- [Reinforcement Learning with Unsupervised Auxiliary Tasks](https://github.com/Caselles/paper_notes/blob/master/read_papers/reinforcement_learning_with_unsupervised_auxiliary_tasks.md) (4)

- [Successor Features for Transfer in Reinforcement Learning

](https://github.com/Caselles/paper_notes/blob/master/read_papers/successor_features_for_transfer_in_reinforcemen_learning.md) (4)

- [Hindsight Experience Replay](https://github.com/Caselles/paper_notes/blob/master/read_papers/hindsight_experience_replay.md) (4)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://github.com/Caselles/paper_notes/blob/master/read_papers/model_agnostic_meta_learning_for_fast_adaptation_of_deep_networks.md) (4)

- [Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm](https://github.com/Caselles/paper_notes/blob/master/read_papers/meta_learning_and_universality_deep_representations_and_gradient_descent_can_approximate_any_learning_algorithm.md) (2)

- [A Distributional Perspective on Reinforcement Learning](https://github.com/Caselles/paper_notes/blob/master/read_papers/a_distributional_perspective_on_reinforcement_learning.md) (4)

- [Transfer for Reinforcement Learning Domains: A Survey](https://github.com/Caselles/paper_notes/blob/master/read_papers/transfer_for_reinforcement_learning_domains_a_survey.md) (3)

- [World Models](https://github.com/Caselles/paper_notes/blob/master/read_papers/world_models.md) (4)

- [Simple random search provides a competitive approach to reinforcement learning](https://github.com/Caselles/paper_notes/blob/master/read_papers/simple_random_search_provides_a_competitive_approach%20to_reinforcement_learning.md) (5)

- [Levine's lecture on "what do to when you have a forward model ?"](https://github.com/Caselles/paper_notes/blob/master/read_papers/levine_lecture_what_to_do_when_you_have_a_forward_model.md) (5)

- [Trust Region Policy Optimization](https://github.com/Caselles/paper_notes/blob/master/read_papers/trust_region_policy_optimization.md) (4)

- [Proximal Policy Optimization](https://github.com/Caselles/paper_notes/blob/master/read_papers/proximal_policy_optimization.md) (4)

- [Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images](https://github.com/Caselles/paper_notes/blob/master/read_papers/embed_to_control.md) (3)

- [Imagination-Augmented Agents for Deep Reinforcement Learning](https://github.com/Caselles/paper_notes/blob/master/read_papers/imagination_augmented_agents_for_deep_reinforcement_learning.md) (3)

- [Deep Successor Reinforcement Learning](https://github.com/Caselles/paper_notes/blob/master/read_papers/deep_successor_reinforcement_learning.md) (4)

- [Learning to Navigate in Complex Environments](https://github.com/Caselles/paper_notes/blob/master/read_papers/learning_to_navigate_in_complex_environments.md) (4)

- [Universal Option Models](https://github.com/Caselles/paper_notes/blob/master/read_papers/universal_options_model.md) (3)

- [Options: Temporal abstraction in Reinforcement Learning](https://github.com/Caselles/paper_notes/blob/master/read_papers/options_precup.md) (4)

- [A Laplacian Framework for Option Discovery in Reinforcement Learning](https://github.com/Caselles/paper_notes/blob/master/read_papers/a_laplacian_framework_for_option_discovery_in_reinforcement_learning.md) (3) 

- [Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation](https://github.com/Caselles/paper_notes/blob/master/read_papers/hierarchical_deep_reinforcement_learning_integrating_temporal_abstraction_and_intrinsic_motivation.md) (2) 

- [Temporal Difference Models: Model-Free Deep RL for Model-Based Control](https://github.com/Caselles/paper_notes/blob/master/read_papers/temporal_difference_models_model_free_deep_rl_for_model_based_control.md) (3)

- [The Predictron: End-To-End Learning and Planning](https://github.com/Caselles/paper_notes/blob/master/read_papers/the_predictron_end_to_end_learning_and_planning.md) (2)

- [Action-Conditional Video Prediction using Deep Networks in Atari Games](https://github.com/Caselles/paper_notes/blob/master/read_papers/action_conditional_video_prediction_using_deep_networks_in_atari_games.md) (3)

- [Decoupling Dynamics and Reward for Transfer Learning](https://github.com/Caselles/paper_notes/blob/master/read_papers/decoupling_dynamic_and_reward_for_transfer_learning.md) (5)

- [Diversity is All You Need: Learning Skills without a Reward Function](https://github.com/Caselles/paper_notes/blob/master/read_papers/diversity_is_all_you_need.md) (2)

- [Generative Temporal Models with Spatial Memory for Partially Observed Environments](https://github.com/Caselles/paper_notes/blob/master/read_papers/generative_temporal_models_with_spatial_memory_for_partially_observed_environments.md) (?)

- [Learning and Querying Fast Generative Models for Reinforcement Learning](https://github.com/Caselles/paper_notes/blob/master/read_papers/learning_and_querying_generative_models_for_rl.md) (2)

- [Model-Based Value Expansion for Efficient Model-Free Reinforcement Learning](https://github.com/Caselles/paper_notes/blob/master/read_papers/model_based_value_expansion_for_efficient_model_free_reinforcement_learning.md) (4)

- [A Deep Hierarchical Approach to Lifelong Learning in Minecraft](https://github.com/Caselles/paper_notes/blob/master/read_papers/a_deep_hierarchical_approach_to_lifelong_learning_in_minecraft.md) (4)

- [Deep Reinforcement Learning that Matters](https://github.com/Caselles/paper_notes/blob/master/read_papers/deep_reinforcement_learning_that_matters.md) (5)

- [Continuous Deep Q-Learning with Model-based Acceleration](https://github.com/Caselles/paper_notes/blob/master/read_papers/continous_deep_q_learning_with_model_based_acceleration.md) (3) 

- [Overcoming catastrophic forgetting in neural networks](https://github.com/Caselles/paper_notes/blob/master/read_papers/overcoming_catastrophic_forgetting_in_neural_networks.md) (4) 

- [Progressive Neural Networks](https://github.com/Caselles/paper_notes/blob/master/read_papers/progressive_neural_networks.md) (4)

- [Learning without Forgetting](https://github.com/Caselles/paper_notes/blob/master/read_papers/learning_without_forgetting.md) (3)

- [Distilling the Knowledge in a Neural Network](https://github.com/Caselles/paper_notes/blob/master/read_papers/distilling_the_knowledge_in_a_neural_network.md) (3)

- [Policy Distillation](https://github.com/Caselles/paper_notes/blob/master/read_papers/policy_distillation.md) (4)

- [Progress & Compress: a scalable framework for continual learning](https://github.com/Caselles/paper_notes/blob/master/read_papers/progress_compress_a_scalable_framework_for_continual_learning.md) (4)

- [Progressive Reinforcement Learning with Distillation for Multi-Skilled Motion Control](https://github.com/Caselles/paper_notes/blob/master/read_papers/progressive_reinforcement_learning_with_dwistillation_for_multi-skilled_motion_control.md) (3)

- [Residual Loss Prediction: Reinforcement Learning With No Incremental Feedback](https://github.com/Caselles/paper_notes/blob/master/read_papers/residual%20loss_prediction_reinforcement_learning_with_no_incremental_feedback.md) (3)

- [Distral: Robust Multitask Reinforcement Learning](https://github.com/Caselles/paper_notes/blob/master/read_papers/distral_robust_multitask_reinforcement_learning.md) (?)

- [Mixture Density Networks](https://github.com/Caselles/paper_notes/blob/master/read_papers/mixture_density_networks.md) (5)

- [DARLA: Improving Zero-Shot Transfer in Reinforcement Learning](https://github.com/Caselles/paper_notes/blob/master/read_papers/darla_Improving_zero-shot_transfer_in_reinforcement_learning.md) (4)

- [Divide-and-Conquer Reinforcement Learning]() (?)

- [Mix & Match – Agent Curricula for Reinforcement Learning]() (?)

- [Generative Temporal Models with Spatial Memory for Partially Observed Environments]() (?)

- [Been There, Done That: Meta-Learning with Episodic Recall]() (?)

- [Continual Reinforcement Learning with Complex Synapses]() (?)

- [Importance Weighted Transfer of Samples in Reinforcement Learning]() (?)

- [Self-Imitation Learning]() (?)

- [Efficient Model-Based Deep Reinforcement Learning with Variational State Tabulation]() (?)

- [A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning]() (?)

- [Hybrid computing using a neural network with dynamic external memory]() (?)

- [Transfer in variable-reward hierarchical reinforcement learning]() (?)

- [Learning Actionable Representations with Goal-Conditioned Policies]() (?)

- [Disentangling Controllable and Uncontrollable Factors by Interacting with the World]() (?)

- [The Laplacian in RL: Learning Representations with Efficient Approximations]() (?)

- [Associative Compression Networks]() (?)

- [Online Structured Laplace Approximations for Overcoming Catastrophic Forgetting]() (?)

- [Learning Latent Subspaces with Variational Autoencoders]() (?)

- [APIAE: Representation Learning and Planning for Dynamical Systems]() (?)

- [Isolating Sources of Disentanglement in Variational Autoencoders]() (?)

- [Learning Latent Dynamics for Planning from Pixels]() (?)

- [Learning by watching Youtube]() (?)

- [Policy Optimization via Importance Sampling]() (?)

- [Learning Attractor Dynamics for Generative Memory]() (?)

- [End-to-end Differentiable Physics for Learning and Control]() (?)

- [HIRO: Data-Efficient Hierarchical Reinforcement Learning]() (?)

- [Evolved Policy Gradients]() (?)

- [Variational Memory Autoencoder]() (?)

- [Emergence of Invariance and Disentanglement in Deep Representations]() (?)

- [Adaptative Path-Integral Autoencoder: Representation Learning and Planning for Dynamical Systems]() (?)

- [Discrete Variational Autoencoders]() (?)

- [Towards a Definition of Disentangled Representations]() (5, Reading group presentation)

- [Learning Latent Dynamics for Planning from Pixels]() (?)

- [Model Based Reinforcement Learning for Atari]() (?)

- [How do Mixture Density RNNs Predict the Future?]() (?

- [CURIOUS: Intrinsically Motivated Multi-Task Multi-Goal Reinforcement Learning](https://github.com/Caselles/paper_notes/blob/master/read_papers/CURIOUS_intrinsically_motivated_multi_task_multi_goal_reinforcement_learning.md) (4)

- [Universal Successor Features Approximators]() (2)

- [A new dog learns an old trick: RL learns classical algorithms]() (?)

- [Near-Optimal Representation Learning for Hierarchical RL]() (?)

- [Recurrent experience replay in Distributed RL]() (3)

- [Large Memory Layers with Product Keys](https://github.com/Caselles/paper_notes/blob/master/read_papers/large_memory_layers_with_product_keys.md) (4)

- [VQ-VAE: Neural Discrete Representation Learning]() (4)

- [StyleGAN: A Style-Based Generator Architecture for Generative Adversarial Networks]() (5)

- [BigGAN: Large Scale GAN Training for High Fidelity Natural Image Synthesis]() (5)

- []() (?)

- []() (?)

- []() (?)

- []() (?)



### Special: [Notes and thoughts about VAEs and how to make them work!](https://github.com/Caselles/paper_notes/blob/master/read_papers/notes_and_thoughts_about_VAEs.md) Based on the following papers:



- [β-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework (β-VAE)]() (4)

- [Understanding disentangling in β-VAE (CCI-VAE)]() (4)

- [Disentangling by Factorising (FactorVAE)]() (4)

- [Isolating Sources of Disentanglement in VAEs (β-TCVAE]() (3)

- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations]() (4)

- [Fixing a Broken ELBO (Fixed-rate objective function for VAEs)]() (4)

- [Taming VAEs (GECO VAEs)]() (3)



## Papers I want to read



- FeUdal Networks for Hierarchical Reinforcement Learning

- Diversity is All You Need: Learning Skills without a Reward Function

- Learning to Search Better than Your Teacher

- Transfer in Variable-Reward Hierarchical Reinforcement Learning

- Curriculum Learning

- Theoretical TL papers from TL survey



# Questions for which I need the answer



[I need the answers to these questions.](https://github.com/Caselles/paper_notes/blob/master/read_papers/questions_unanswered.md) Any help is welcome ! Reach me at casellesdupre.hugo@gmail.com



[1]:https://blog.acolyer.org/about/

[2]:https://github.com/dennybritz/deeplearning-papernotes

[3]:https://github.com/DanielTakeshi/Paper_Notes#2018-rlil-papers