https://github.com/borealisai/constant-memory-anp
https://github.com/borealisai/constant-memory-anp
Last synced: 8 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/borealisai/constant-memory-anp
- Owner: BorealisAI
- License: other
- Created: 2024-05-27T17:51:36.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2024-07-08T17:46:46.000Z (almost 2 years ago)
- Last Synced: 2025-02-16T03:35:58.296Z (over 1 year ago)
- Language: Python
- Size: 1.13 MB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Memory Efficient Neural Processes via Constant Memory Attention Block
This is the official implementation of the paper [Memory Efficient Neural Processes via Constant Memory Attention Block](https://openreview.net/forum?id=xtwCf7iAs2).
This repository includes our implementation of Constant Memory Attention Block (CMAB) and Constant Memory Attentive Neural Processes (CMANPs).
CMAB is a novel attention block that (i) is permutation invariant, (ii) computes its output in constant memory, and (iii) performs constant computation updates.
CMANP is a NP variant that builds on CMAB, requiring only constant memory unlike previous existing attention-based NPs. Empirically, CMANPs achieve results competitive with other SOTA NPs on popular benchmarks while requiring only constant memory.
## Install
Create and activate a conda environment. Install the dependencies as listed in `requirements.txt`:
```
conda create --name cmanp python=3.7
conda activate cmanp
pip install -r requirements.txt
```
## Usage
Please check the `regression` directory for specific usage.
## Reference
For technical details, please check the conference version of our paper.
```
@inproceedings{
feng2024memory,
title={Memory Efficient Neural Processes via Constant Memory Attention Block},
author={Leo Feng and Frederick Tung and Hossein Hajimirsadeghi and Yoshua Bengio and Mohamed Osama Ahmed},
booktitle={International Conference on Machine Learning},
year={2024},
url={https://openreview.net/forum?id=xtwCf7iAs2}
}
```
## Acknowledgement
This code is based on the official code base of [Latent Bottlenecked Attentive Neural Processes](https://github.com/BorealisAI/latent-bottlenecked-anp).