https://github.com/borhanmorphy/etc
Encoding Long and Structured Inputs in Transformers
https://github.com/borhanmorphy/etc
Last synced: over 1 year ago
JSON representation
Encoding Long and Structured Inputs in Transformers
- Host: GitHub
- URL: https://github.com/borhanmorphy/etc
- Owner: borhanMorphy
- Created: 2024-05-27T19:25:03.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2024-05-27T19:38:26.000Z (about 2 years ago)
- Last Synced: 2025-01-23T16:19:27.405Z (over 1 year ago)
- Language: Python
- Size: 528 KB
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Extended Transformer Construction (ETC)
## Contents
- [Installation](#installation)
- [References](#references)
- [Citations](#citations)
## Installation
```
git clone https://github.com/borhanMorphy/ETC.git
cd ETC
pip install .
```
## References
- [Official Implementation](https://github.com/google-research/google-research/tree/master/etcmodel)
- [Paper](https://arxiv.org/pdf/2004.08483)
## Citations
```bibtex
@inproceedings{ainslie2020etc,
title={ETC: Encoding Long and Structured Data in Transformers},
author={Joshua Ainslie and Santiago Onta{\~n}{\'o}n and Chris Alberti and Vaclav Cvicek and Zachary Fisher and Philip Pham and Anirudh Ravula and Sumit Sanghai and Qifan Wang and Li Yang},
booktitle={Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020)},
year={2020}
}
```