https://github.com/borealisai/dt-fixup
Optimizing Deeper Transformers on Small Datasets https://arxiv.org/abs/2012.15355
https://github.com/borealisai/dt-fixup
Last synced: about 1 year ago
JSON representation
Optimizing Deeper Transformers on Small Datasets https://arxiv.org/abs/2012.15355
- Host: GitHub
- URL: https://github.com/borealisai/dt-fixup
- Owner: BorealisAI
- Created: 2021-05-31T15:03:00.000Z (about 5 years ago)
- Default Branch: main
- Last Pushed: 2022-11-02T06:04:14.000Z (over 3 years ago)
- Last Synced: 2025-03-20T15:41:01.593Z (about 1 year ago)
- Language: Python
- Homepage:
- Size: 99.6 KB
- Stars: 15
- Watchers: 5
- Forks: 10
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# DT-Fixup
Optimizing Deeper Transformers on Small Datasets
Paper published in ACL 2021: [arXiv](https://arxiv.org/abs/2012.15355)
Detailed instructions to replicate our results in the paper can be found in the folders `spider` and `reclor`.
### Cite
If you found this codebase or our work useful, please cite:
```
@InProceedings{xu2021optimizing,
author = {Xu, Peng and Kumar, Dhruv and Yang, Wei and Zi, Wenjie and Tang, Keyi and Huang, Chenyang and Cheung, Jackie Chi Kit and Prince, Simon J.D. and Cao, Yanshuai},
title = {Optimizing Deeper Transformers on Small Datasets}
booktitle = {The 59th Annual Meeting of the Association for Computational Linguistics (ACL 2021)},
month = {August},
year = {2021},
publisher = {ACL}
}
```