https://github.com/facebookresearch/SimulEval

SimulEval: A General Evaluation Toolkit for Simultaneous Translation
https://github.com/facebookresearch/SimulEval

Last synced: 29 days ago
JSON representation

SimulEval: A General Evaluation Toolkit for Simultaneous Translation

• Host: GitHub
• URL: https://github.com/facebookresearch/SimulEval
• Owner: facebookresearch
• License: cc-by-sa-4.0
• Created: 2020-08-11T17:42:10.000Z (over 4 years ago)
• Default Branch: main
• Last Pushed: 2024-09-13T14:11:46.000Z (3 months ago)
• Last Synced: 2024-11-06T12:12:10.652Z (about 1 month ago)
• Language: Python
• Homepage:
• Size: 8.76 MB
• Stars: 102
• Watchers: 16
• Forks: 36
• Open Issues: 26
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE
  • Code of conduct: CODE_OF_CONDUCT.md

Awesome Lists containing this project

• Awesome-Simultaneous-Translation - SimulEval

README

        
# SimulEval

[![](https://github.com/facebookresearch/SimulEval/workflows/build/badge.svg)](https://github.com/facebookresearch/SimulEval/actions)

SimulEval is a general evaluation framework for simultaneous translation on text and speech. Full documentation can be found [here](https://simuleval.readthedocs.io/en/v1.1.0/).

## Installation

```

git clone https://github.com/facebookresearch/SimulEval.git

cd SimulEval

pip install -e .

```

## Quick Start

Following is the evaluation of a [dummy agent](examples/quick_start) which operates wait-k (k = 3) policy and generates random words until the length of the generated words is the same as the number of all the source words.

```shell

cd examples/quick_start

simuleval --source source.txt --target target.txt --agent first_agent.py

```

# License

SimulEval is licensed under Creative Commons BY-SA 4.0.

# Citation

Please cite as:

```bibtex

@inproceedings{simuleval2020,

  title = {Simuleval: An evaluation toolkit for simultaneous translation},

  author = {Xutai Ma, Mohammad Javad Dousti, Changhan Wang, Jiatao Gu, Juan Pino},

  booktitle = {Proceedings of the EMNLP},

  year = {2020},

}

```