Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/schatty/awesome-memory-rl

A curated list of awesome memory in reinforcement learning research materials
https://github.com/schatty/awesome-memory-rl

List: awesome-memory-rl

awesome-list deep-learning memory memory-deep-learning memory-mechanisms memory-reinfocement-learning memory-rl reinforcement-learning

Last synced: 3 months ago
JSON representation

A curated list of awesome memory in reinforcement learning research materials

Awesome Lists containing this project

README 

        
# Awesome Memory in RL



[![Awesome](https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/sindresorhus/awesome#readme)



A curated list of conference papers studying memory mechanisms for reinforcement learning. Also check [awesome-offline-rl](https://github.com/hanjuku-kaso/awesome-offline-rl), [awesome-ebm](https://github.com/yataobian/awesome-ebm), [awesome-model-mbrl](https://github.com/hejia-zhang/awesome-model-based-reinforcement-learning). Forks and PRs are welcome.



## Reinforcement Learning

### 2021

* [End-to-End Egospheric Spatial Memory](https://arxiv.org/abs/2102.07764)

  * Daniel Lenton, Stephen James, Ronald Clark, Andrew J. Davison [ICLR]

* [Learning Associative Inference Using Fast Weight Memory](https://arxiv.org/abs/2011.07831)

  * Imanol Schlag, Tsendsuren Munkhdalai, Jürgen Schmidhuber [ICLR]

* [Solving Continuous Control with Episodic Memory](https://arxiv.org/abs/2106.08832)

  * Igor Kuznetsov, Andrey Filchenkov [IJCAI]

* [Generalizable Episodic Memory for Deep Reinforcement Learning](https://arxiv.org/abs/2103.06469)

  * Hao Hu, Jianing Ye, Guangxiang Zhu, Zhizhou Ren, Chongjie Zhang [ICML]

### 2020

* [Episodic Reinforcement Learning with Associative Memory](https://openreview.net/forum?id=HkxjqxBYDB)

  * Guangxiang Zhu, Zichuan Lin, Guangwen Yang, Chongjie Zhang [ICLR]

* [AMRL: Aggregated Memory For Reinforcement Learning](https://openreview.net/forum?id=Bkl7bREtDr)

  * Jacob Beck, Kamil Ciosek, Sam Devlin, Sebastian Tschiatschek, Cheng Zhang, Katja Hofmann [ICLR]

* [Sparse Graphical Memory for Robust Planning](https://arxiv.org/abs/2003.06417)

  * Scott Emmons, Ajay Jain, Michael Laskin, Thanard Kurutach, Pieter Abbeel, Deepak Pathak [NeurIPS]

* [Memory Based Trajectory-conditioned Policies for Learning from Sparse Rewards](https://arxiv.org/abs/1907.10247)

  * Yijie Guo, Jongwook Choi, Marcin Moczulski, Shengyu Feng, Samy Bengio, Mohammad Norouzi, Honglak Lee [NeurIPS]

* [Working Memory Graphs](https://arxiv.org/abs/1911.07141)

  * Ricky Loynd, Roland Fernandez, Asli Celikyilmaz, Adith Swaminathan, Matthew Hausknecht [ICML]

* [Hallucinative Topological Memory for Zero-Shot Visual Planning](https://arxiv.org/abs/2002.12336)

  * Kara Liu, Thanard Kurutach, Christine Tung, Pieter Abbeel, Aviv Tamar [ICML]

### 2019

* [Episodic Curiosity through Reachability](https://arxiv.org/abs/1810.02274)

  * Nikolay Savinov, Anton Raichuk, Raphaël Marinier, Damien Vincent, Marc Pollefeys, Timothy Lillicrap, Sylvain Gelly [ICLR]

* [Generalization of Reinforcement Learners with Working and Episodic Memory](https://arxiv.org/abs/1910.13406)

  * Meire Fortunato, Melissa Tan, Ryan Faulknel et. al [NeurIPS]

* [Policy Consolidation for Continual Reinforcement Learning](https://arxiv.org/abs/1902.00255)

  * Christos Kaplanis, Murray Shanahan, Claudia Clopath [ICML]

* [Remember and Forget for Experience Replay](https://arxiv.org/abs/1807.05827)

  * Guido Novati, Petros Koumoutsakos [ICML]

* [Reinforcement Learning, Fast and Slow](https://www.cell.com/trends/cognitive-sciences/fulltext/S1364-6613(19)30061-0)

  * Matthew Botvinick, Sam Ritter, Jane X. Wang, Zeb Kurth-Nelson, Charles Blundell et. al [Trends in Cognitive Sciences]



### 2018

* [Memory Augmented Control Networks](https://arxiv.org/abs/1709.05706)

  * Arbaaz Khan, Clark Zhang, Nikolay Atanasov, Konstantinos Karydis, Vijay Kumar, Daniel D. Lee [ICLR]

* [Neural Map: Structured Memory for Deep Reinforcement Learning](https://arxiv.org/abs/1702.08360)

  * Emilio Parisotto, Ruslan Salakhutdinov [ICLR]

* [Memory Augmented Policy Optimization for Program Synthesis and Semantic Parsing](https://arxiv.org/abs/1807.02322)

  * Chen Liang, Mohammad Norouzi, Jonathan Berant, Quoc Le, Ni Lao [NeurIPS]

* [Fast deep reinforcement learning using online adjustments from the past](https://arxiv.org/abs/1810.08163)

  * Steven Hansen, Pablo Sprechmann, Alexander Pritzel, André Barreto, Charles Blundell [NeurIPS]

* [Continual Reinforcement Learning with Complex Synapses](https://arxiv.org/abs/1802.07239)

  * Christos Kaplanis, Murray Shanahan, Claudia Clopath [ICML]

* [Been There, Done That: Meta-Learning with Episodic Recall](https://arxiv.org/abs/1805.09692)

  * Samuel Ritter, Jane X. Wang, Zeb Kurth-Nelson, Siddhant M. Jayakumar, Charles Blundell et. al [ICML]

* [Episodic Memory Deep Q-Networks](https://arxiv.org/abs/1805.07603)

  * Zichuan Lin, Tianqi Zhao, Guangwen Yang, Lintao Zhang [IJCAI]

* [Unsupervised Predictive Memory in a Goal-Directed Agent ](https://arxiv.org/abs/1803.10760)

  * Greg Wayne, Chia-Chun Hung, David Amos et. al



### 2017

* [Fast Reinforcement Learning via Slow Reinforcement Learning](https://arxiv.org/abs/1611.02779)

  * Yan Duan, John Schulman, Xi Chen, Peter L. Bartlett, Ilya Sutskever, Pieter Abbeel [ICLR]

* [Neural Episodic Control](https://arxiv.org/abs/1703.01988)

  * Alexander Pritzel, Benigno Uria, Sriram Srinivasan, Adrià Puigdomènech, Oriol Vinyals, Demis Hassabil et. al [ICML]



### ... - 2016

* [Using Fast Weights to Attend to the Recent Past](https://arxiv.org/abs/1610.06258)

  * Jimmy Ba, Geoffrey Hinton, Volodymyr Mnih, Joel Z. Leibo, Catalin Ionescu [NIPS-2016]

* [Control of Memory, Active Perception, and Action in Minecraft](https://arxiv.org/abs/1605.09128)

  * Junhyuk Oh, Valliappa Chockalingam, Satinder Singh, Honglak Lee [ICLR-2016]

* [Model-Free Episodic Control](https://arxiv.org/abs/1606.04460)

  * Charles Blundell, Benigno Uria, Alexander Pritzel et. al

* [Hippocampal Contributions to Control: The Third Way](https://proceedings.neurips.cc/paper/2007/hash/1f4477bad7af3616c1f933a02bfabe4e-Abstract.html)

  * Máté Lengyel, Peter Dayan [NIPS-2007]



## Deep Learning



### 2021

* [Remembering for the Right Reasons: Explanations Reduce Catastrophic Forgetting](https://arxiv.org/abs/2010.01528)

  * Sayna Ebrahimi, Suzanne Petryk, Akash Gokul, William Gan, Joseph E. Gonzalez, Marcus Rohrbach, Trevor Darrell [ICLR]

* [Gradient Projection Memory for Continual Learning](https://arxiv.org/abs/2103.09762)

  * Gobinda Saha, Isha Garg, Kaushik Roy [ICLR]

* [Learn from Concepts: Towards the Purified Memory for Few-shot Learning](https://www.ijcai.org/proceedings/2021/123)

  * Xuncheng Liu, Xudong Tian, Shaohui Lin, Yanyun Qu, Lizhuang Ma, Wang Yuan, Zhizhong Zhang, Yuan Xi [IJCAI]

* [Not All Memories are Created Equal: Learning to Forget by Expiring](https://arxiv.org/abs/2105.06548)

  * Sainbayar Sukhbaatar, Da Ju, Spencer Poff, Stephen Roller, Arthur Szlam, Jason Weston, Angela Fan [ICML]



### 2020

* [Memory-Based Graph Networks](https://arxiv.org/abs/2002.09518)

  * Amir Hosein Khasahmadi, Kaveh Hassani, Parsa Moradi, Leo Lee, Quaid Morris [ICLR]

* [Meta-Learning Deep Energy-Based Memory Models](https://arxiv.org/abs/1910.02720)

  * Sergey Bartunov, Jack W Rae, Simon Osindero, Timothy P Lillicrap [ICLR]

* [MEMO: A Deep Network for Flexible Combination of Episodic Memories](https://arxiv.org/abs/2001.10913)

  * Andrea Banino, Adrià Puigdomènech Badia, Raphael Köster et. al [ICLR]

* [Progressive Memory Banks for Incremental Domain Adaptation](https://arxiv.org/abs/1811.00239)

  * Nabiha Asghar, Lili Mou, Kira A. Selby, Kevin D. Pantasdo, Pascal Poupart, Xin Jiang [ICLR]

* [Neural Stored-program Memory](https://arxiv.org/abs/1906.08862)

  * Hung Le, Truyen Tran, Svetha Venkatesh [ICLR]

* [H-Mem: Harnessing synaptic plasticity with Hebbian Memory Networks](https://proceedings.neurips.cc/paper/2020/file/f6876a9f998f6472cc26708e27444456-Paper.pdf)

  * Thomas Limbacher and Robert Legenstein [NeurIPS]

* [Online Multitask Learning with Long-Term Memory](https://arxiv.org/abs/2008.07055)

  * Mark Herbster, Stephen Pasteris, Lisa Tse [NeurIPS]

* [HiPPO: Recurrent Memory with Optimal Polynomial Projections](https://arxiv.org/abs/2008.07669)

  * Albert Gu, Tri Dao, Stefano Ermon, Atri Rudra, Christopher Re [NeurIPS]

* [Learning to Learn Variational Semantic Memory](https://arxiv.org/abs/2010.10341)

  * Xiantong Zhen, Yingjun Du, Huan Xiong, Qiang Qiu, Cees G. M. Snoek, Ling Shao [NeurIPS]

* [Improved Schemes for Episodic Memory-based Lifelong Learning](https://arxiv.org/abs/1909.11763)

  * Yunhui Guo, Mingrui Liu, Tianbao Yang, Tajana Rosing [NeurIPS]

* [Self-Attentive Associative Memory](https://arxiv.org/abs/2002.03519)

  * Hung Le, Truyen Tran, Svetha Venkatesh [ICML]

* [Associative Memory in Iterated Overparameterized Sigmoid Autoencoders](https://arxiv.org/abs/2006.16540)

  * Yibo Jiang, Cengiz Pehlevan [ICML]

* [Multigrid Neural Memory](https://arxiv.org/abs/1906.05948)

  * Tri Huynh, Michael Maire, Matthew R. Walter [ICML]



### 2019



* [Learning to Remember More with Less Memorization](https://arxiv.org/abs/1901.01347)

  * Hung Le, Truyen Tran, Svetha Venkatesh [ICLR]

* [Adaptive Posterior Learning: few-shot learning with a surprise-based memory module](https://arxiv.org/abs/1902.02527)

  * Tiago Ramalho, Marta Garnelo [ICLR]

* [Large Memory Layers with Product Keys](https://arxiv.org/abs/1907.05242)

  * Guillaume Lample, Alexandre Sablayrolles, Marc'Aurelio Ranzato, Ludovic Denoyer, Hervé Jégou [NeurIPS]

* [Episodic Memory in Lifelong Language Learning](https://arxiv.org/abs/1906.01076)

  * Cyprien de Masson d'Autume, Sebastian Ruder, Lingpeng Kong, Dani Yogatama [NeurIPS]

* [Metalearned Neural Memory](https://arxiv.org/abs/1907.09720)

  * Tsendsuren Munkhdalai, Alessandro Sordoni, Tong Wang, Adam Trischler [NeurIPS]

* [Ordered Memory](https://arxiv.org/abs/1910.13466)

  * Yikang Shen, Shawn Tan, Arian Hosseini, Zhouhan Lin, Alessandro Sordoni, Aaron Courville [NeurIPS]

* [Legendre Memory Units: Continuous-Time Representation in Recurrent Neural Networks](https://papers.nips.cc/paper/2019/file/952285b9b7e7a1be5aa7849f32ffff05-Paper.pdf)

  * Aaron R. Voelker, Ivana Kajic ́, Chris Eliasmith [NeurIPS]



### 2018



* [Semi-parametric Topological Memory for Navigation](https://arxiv.org/abs/1803.00653)

  * Nikolay Savinov, Alexey Dosovitskiy, Vladlen Koltun [ICLR]

* [Memory-based Parameter Adaptation](https://arxiv.org/abs/1802.10542)

  * Pablo Sprechmann, Siddhant M. Jayakumar, Jack W. Rae, Alexander Pritzel et. al [ICLR]

* [Convolutional Memory Blocks for Depth Data Representation Learning](https://www.ijcai.org/proceedings/2018/0387.pdf)

  * Keze Wang, Liang Lin, Chuangjie Ren, Wei Zhang, Wenxiu Sun [IJCAI]

* [Visual Memory for Robust Path Following](https://arxiv.org/abs/1812.00940)

  * Ashish Kumar, Saurabh Gupta, David Fouhey, Sergey Levine, Jitendra Malik [NeurIPS]

* [A Simple Cache Model for Image Recognition](https://arxiv.org/abs/1805.08709)

  * A. Emin Orhan [NeurIPS]

* [Variational Memory Encoder-Decoder](https://arxiv.org/abs/1807.09950)

  * Hung Le, Truyen Tran, Thin Nguyen, Svetha Venkatesh [NeurIPS]

* [Fast Parametric Learning with Activation Memorization](https://arxiv.org/abs/1803.10049)

  * Jack W Rae, Chris Dyer, Peter Dayan, Timothy P Lillicrap [ICML]

* [Learning and Memorization](http://proceedings.mlr.press/v80/chatterjee18a.html)

  * Satrajit Chatterjee [ICML]



### 2017



* [Reasoning with Memory Augmented Neural Networks for Language Comprehension](https://arxiv.org/abs/1610.06454)

  * Tsendsuren Munkhdalai, Hong Yu [ICLR]

* [Learning to Remember Rare Events](https://arxiv.org/abs/1703.03129)

  * Łukasz Kaiser, Ofir Nachum, Aurko Roy, Samy Bengio [ICLR]

* [Variational Memory Addressing in Generative Models](https://arxiv.org/abs/1709.07116)

  * Jörg Bornschein, Andriy Mnih, Daniel Zoran, Danilo J. Rezende [NIPS]

* [A simple model of recognition and recall memory](https://papers.nips.cc/paper/2017/hash/57aeee35c98205091e18d1140e9f38cf-Abstract.html)

  * Nisheeth Srivastava, Edward Vul [NIPS]

* [Gradient Episodic Memory for Continual Learning](https://arxiv.org/abs/1706.08840)

  * David Lopez-Paz, Marc'Aurelio Ranzato [NIPS]



### ... - 2016



* [End-To-End Memory Networks](https://arxiv.org/abs/1503.08895) [NIPS-205]



## Cognition and Neuroscience



* [Large Associative Memory Problem in Neurobiology and Machine Learning](https://arxiv.org/abs/2008.06996)

  * Dmitry Krotov, John Hopfield [ICLR-2021]

* [Compositional Explanations of Neurons](https://arxiv.org/abs/2006.14032)

  * Jesse Mu, Jacob Andreas [NeurIPS-2020]

* [Coordinated hippocampal-entorhinal replay as structural inference](https://proceedings.neurips.cc/paper/2019/hash/aa68c75c4a77c87f97fb686b2f068676-Abstract.html)

  * Talfan Evans, Neil Burgess [NeurIPS-2019]

* [Generalisation of structural knowledge in the hippocampal-entorhinal system](https://arxiv.org/abs/1805.09042)

  * James C. R. Whittington, Timothy H. Muller, Shirley Mark, Caswell Barry, Timothy E. J. Behrens [NeurIPS-2018]

* [Dendritic cortical microcircuits approximate the backpropagation algorithm](https://arxiv.org/abs/1810.11393)

  * João Sacramento, Rui Ponte Costa, Yoshua Bengio, Walter Senn [NeurIPS-2018]