https://github.com/xingyaoww/leti
Official repo for NAACL 2024 Findings paper "LeTI: Learning to Generate from Textual Interactions."
https://github.com/xingyaoww/leti
Last synced: 19 days ago
JSON representation
Official repo for NAACL 2024 Findings paper "LeTI: Learning to Generate from Textual Interactions."
- Host: GitHub
- URL: https://github.com/xingyaoww/leti
- Owner: xingyaoww
- License: apache-2.0
- Created: 2023-05-16T18:35:20.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2023-06-29T22:34:03.000Z (almost 2 years ago)
- Last Synced: 2025-04-18T21:31:19.519Z (about 1 month ago)
- Language: Python
- Homepage:
- Size: 210 KB
- Stars: 64
- Watchers: 1
- Forks: 7
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# LeTI: Learning to Generate from Textual Interactions
Official repo for paper [LeTI: Learning to Generate from Textual Interactions](https://arxiv.org/abs/2305.10314).
This repo contains code that can be used to reproduce the experiments in the paper. Train/evaluation code are written using Jax/Flax to train on Google Cloud's TPU VM instances.
Research is supported with Cloud TPUs from [Google's TPU Research Cloud (TRC)](https://sites.research.google/trc/about/).
**WARNING**
Training and evaluation of LeTI requires executing untrusted model-generated code. Users are strongly encouraged to sandbox the code execution so that it does not perform destructive actions on their host or network.
## Setup
You can setup your Google Cloud TPU and Storage following [docs/SETUP.md](docs/SETUP.md). Alternatively, you may also adapt the released code to your specific computing setup.
## Dataset
You can prepare datasets for training and evaluation following instructions in [docs/DATA.md](docs/DATA.md).
## Model
Since the training and evaluation code is implemented using Jax/Flax, you will need to convert huggingface model checkpoints (pytorch) into [T5X](https://github.com/google-research/t5x) format, following instructions in [docs/MODEL.md](docs/MODEL.md). We release the 350M and 2B model checkpoints [here](https://huggingface.co/xingyaoww/LeTI).
## Training and Evaluation
You can follow [docs/TRAIN.md](docs/TRAIN.md) and [docs/EVAL.md](docs/EVAL.md) to train or evaluate a specific model.
## Citation
```
@article{Wang2023LeTI,
title={LeTI: Learning to Generate from Textual Interactions},
author={Xingyao Wang and Hao Peng and Reyhaneh Jabbarvand and Heng Ji},
journal={ArXiv},
year={2023},
volume={abs/2305.10314},
}
```