https://github.com/apertium/apertium-eval-translator
Evaluate machine translation output against reference: WER, PER, TER, BLEU
https://github.com/apertium/apertium-eval-translator
apertium evaluation mt
Last synced: 10 months ago
JSON representation
Evaluate machine translation output against reference: WER, PER, TER, BLEU
- Host: GitHub
- URL: https://github.com/apertium/apertium-eval-translator
- Owner: apertium
- License: gpl-3.0
- Created: 2018-11-16T08:35:46.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2021-01-26T09:45:52.000Z (over 5 years ago)
- Last Synced: 2025-08-05T01:33:34.778Z (11 months ago)
- Topics: apertium, evaluation, mt
- Language: Perl
- Homepage: http://wiki.apertium.org/wiki/Evaluation
- Size: 45.9 KB
- Stars: 1
- Watchers: 12
- Forks: 4
- Open Issues: 3
-
Metadata Files:
- Readme: README
- License: COPYING
Awesome Lists containing this project
README
apertium-eval-translator.pl
-------------------------------------------------------------------
Simple Perl script to evaluate Apertium-based machine translation
systems.
This software calculates (at document level) the word error rate (WER)
and the position-independent word error rate (PER) between a
translation performed by the Apertium MT system and a reference
translation obtained by post-editing the system output.
For more information execute the script (after installation)
without arguments
$ apertium-eval-translator
apertium-eval-translator-line.pl
-------------------------------------------------------------------
Similar to apertium-eval-translator but it assumes that test and reference
files has the same number of sentences and are in the one-sentence-per-line format
bootstrap_resampling.pl
-------------------------------------------------------------------
Computes confidence intervals for MT evaluation. See paper
http://acl.ldc.upenn.edu/acl2004/emnlp/pdf/Koehn.pdf
Scripts wer.sh, ter.sh and bleu.sh ares provided as examples of
the script that needs to be supplied to
bootstrap_resampling.pl to evaluate MT performance.
Note: ter.sh and bleu.sh need software not distributed in this package.
-------------------------------------------------------------------
Suggestions are very welcome. Please write to Felipe Sánchez Martínez
at fsanchez@dlsi.ua.es .
===================================================================
===================================================================
apertium-eval
-------------------------------------------------------------------
This is not intended to be a substitute for Felipe Sanchez's
apertium-eval-translator. It just provides an alternative way
to approximate the word error rate by processing the output of "diff -d".
Run apertium-eval without arguments for a usage message.