https://github.com/unbabel/wmt21-qe-task
MBART Quality Estimator proposed in IST-Unbabel 2021 Submission for the Quality Estimation Shared Task
https://github.com/unbabel/wmt21-qe-task
machine-translation quality-estimation uncertainty
Last synced: 6 months ago
JSON representation
MBART Quality Estimator proposed in IST-Unbabel 2021 Submission for the Quality Estimation Shared Task
- Host: GitHub
- URL: https://github.com/unbabel/wmt21-qe-task
- Owner: Unbabel
- License: apache-2.0
- Created: 2021-10-21T13:53:12.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2022-04-06T13:17:22.000Z (about 4 years ago)
- Last Synced: 2023-10-06T20:32:50.115Z (over 2 years ago)
- Topics: machine-translation, quality-estimation, uncertainty
- Language: Python
- Homepage:
- Size: 190 KB
- Stars: 9
- Watchers: 10
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# WMT21 QE Shared Task
Code for the MBART QE Estimator (M2) from [IST-Unbabel 2021 Submission for the Quality Estimation Shared Task](http://statmt.org/wmt21/pdf/2021.wmt-1.102.pdf)
For XLMR Estimators (M1) please take a look at [OpenKiwi](https://github.com/Unbabel/OpenKiwi)
## Citation:
```
@inproceedings{Zerva-etal-2021-ist,
title = {{IST-Unbabel 2021 Submission for the Quality Estimation Shared Task}},
author = {Zerva, Chrysoula and van Stigt, Daan and Rei, Ricardo and C Farinha, Ana and Souza, José G. C. de and Glushkova, Taisiya and Vera, Miguel and Kepler, Fabio and Martins, André},
year = 2021,
month = nov,
booktitle = {Proceedings of the Sixth Conference on Machine Translation},
publisher = {Association for Computational Linguistics},
address = {Online},
}
```
### Installation:
```bash
pip install -r requirements.txt
pip install -e .
```
### Data:
Download the MLQE-PE with Glass-Box features:
```bash
cd data
wget https://unbabel-experimental-data-sets.s3.eu-west-1.amazonaws.com/wmt21/glassbox-MLQE-PE.tar.gz
tar -xf glassbox-MLQE-PE.tar.gz
```
**NOTE:**
If you use this data please cite the [original MLQE-PE corpus](https://github.com/sheffieldnlp/mlqe-pe)!
### Training:
```bash
python cli.py train -f configs/mbart50-m2m.yaml
```
### Hyperparameter search_
```bash
python cli.py search -f configs/mbart50-m2m.yaml
```
### Testing:
```bash
python evaluate.py --checkpoint {path/to/checkpoint}.ckpt
```