https://github.com/BorealisAI/DT-Fixup

Optimizing Deeper Transformers on Small Datasets https://arxiv.org/abs/2012.15355
https://github.com/BorealisAI/DT-Fixup

Last synced: about 2 months ago
JSON representation

Optimizing Deeper Transformers on Small Datasets https://arxiv.org/abs/2012.15355

• Host: GitHub
• URL: https://github.com/BorealisAI/DT-Fixup
• Owner: BorealisAI
• Created: 2021-05-31T15:03:00.000Z (over 3 years ago)
• Default Branch: main
• Last Pushed: 2022-11-02T06:04:14.000Z (almost 2 years ago)
• Last Synced: 2024-07-21T20:46:14.138Z (2 months ago)
• Language: Python
• Homepage:
• Size: 99.6 KB
• Stars: 15
• Watchers: 6
• Forks: 10
• Open Issues: 4
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# DT-Fixup

Optimizing Deeper Transformers on Small Datasets

Paper published in ACL 2021: [arXiv](https://arxiv.org/abs/2012.15355)

Detailed instructions to replicate our results in the paper can be found in the folders `spider` and `reclor`.

### Cite

If you found this codebase or our work useful, please cite:

```

@InProceedings{xu2021optimizing,

  author = {Xu, Peng and Kumar, Dhruv and Yang, Wei and Zi, Wenjie and Tang, Keyi and Huang, Chenyang and Cheung, Jackie Chi Kit and Prince, Simon J.D. and Cao, Yanshuai},

  title = {Optimizing Deeper Transformers on Small Datasets}

  booktitle = {The 59th Annual Meeting of the Association for Computational Linguistics (ACL 2021)},

  month = {August},

  year = {2021},

  publisher = {ACL}

}

```