{"id":18887139,"url":"https://github.com/thunlp-mt/template-nmt","last_synced_at":"2026-02-26T08:21:46.509Z","repository":{"id":61677558,"uuid":"553493316","full_name":"THUNLP-MT/Template-NMT","owner":"THUNLP-MT","description":null,"archived":false,"fork":false,"pushed_at":"2022-11-15T14:37:12.000Z","size":6080,"stargazers_count":23,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-28T10:54:19.626Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/THUNLP-MT.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-10-18T09:52:12.000Z","updated_at":"2024-10-31T02:05:10.000Z","dependencies_parsed_at":"2023-01-22T03:03:40.159Z","dependency_job_id":null,"html_url":"https://github.com/THUNLP-MT/Template-NMT","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUNLP-MT%2FTemplate-NMT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUNLP-MT%2FTemplate-NMT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUNLP-MT%2FTemplate-NMT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUNLP-MT%2FTemplate-NMT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/THUNLP-MT","download_url":"https://codeload.github.com/THUNLP-MT/Template-NMT/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248973030,"owners_count":21191900,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T07:34:41.931Z","updated_at":"2026-02-26T08:21:41.474Z","avatar_url":"https://github.com/THUNLP-MT.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# A Template-based Method for Constrained Neural Machine Translation (EMNLP 2022)\n\n## Introduction\n\nWe propose a template-based framework that can cope with both lexical and structural constraints for neural machine translation. We use [fairseq](https://github.com/facebookresearch/fairseq) to implement our method, and the commit code is `d5f7b50` (Mon Sep 21 13:45:35 2020 -0700). Our major modification is that we update the [generate.py](https://github.com/shuo-git/Template-NMT/blob/main/fairseq-pro/fairseq_cli/generate.py) to achieve prefix-based inference.\n\n## Lexically Constrained Translation\n\n### Data\n\nWe use [WMT17 En-Zh](https://www.statmt.org/wmt17/translation-task.html) and [WMT20 En-De](https://www.statmt.org/wmt20/translation-task.html) as our training sets. We use [En-Zh](http://nlp.csai.tsinghua.edu.cn/~ly/systems/TsinghuaAligner/TsinghuaAligner.html) and [En-De](https://github.com/lilt/alignment-scripts) alignment test sets for evaluation. We follow [Chen et al., (2021)](https://ojs.aaai.org/index.php/AAAI/article/view/17496) to extract the constraints.\n\n### Build the templates\n\nWe use the script [prepare_template.py](https://github.com/shuo-git/Template-NMT/blob/main/lexical_scripts/prepare_template.py) to convert natural language sentences into template-based ones.\n\nPrepare the template-based training data:\n```\npython prepare_template.py en zh train.bpe train.bpe.const.en-zh t2s\n```\nPrepare the template-based inference data:\n```\npython prepare_template.py en zh test.bpe test.bpe.const.en-zh t2s.infer\n```\n\nHere is an example:\n\nFilename | Example\n---|---\nen | this approach , also adopted by italy , contributes to reducing costs and increasing the overall volume of funds for beneficiaries .\nzh | 这种 方式 也 被 意大利 所 采用 。 它 有助于 降低成本 , 增加 受益人 收到 的 汇款 总额 。\nconst.en-zh | italy \\|\\|\\| 意大利 \\|\\|\\| approach \\|\\|\\| 方式 \\|\\|\\| increasing \\|\\|\\| 增加 \\|\\|\\|\nt2s.en | [C0] italy [C1] approach [C2] increasing **[SEP]** [S0] [C1] [S1] [C0] [S2] [C2] [S3] **[SEP]** [S0] this [S1] , also adopted by [S2] , contributes to reducing costs and [S3] the overall volume of funds for beneficiaries .\nt2s.zh | [C0] 意大利 [C1] 方式 [C2] 增加 **[SEP]** [T0] [C1] [T1] [C0] [T2] [C2] [T3] **[SEP]** [T0] 这种 [T1] 也 被 [T2] 所 采用 。 它 有助于 降低成本 , [T3] 受益人 收到 的 汇款 总额 。\nt2s.infer.en | [C0] italy [C1] approach [C2] increasing **[SEP]** [S0] [C1] [S1] [C0] [S2] [C2] [S3] **[SEP]** [S0] this [S1] , also adopted by [S2] , contributes to reducing costs and [S3] the overall volume of funds for beneficiaries .\nt2s.infer.zh | [C0] 意大利 [C1] 方式 [C2] 增加 **[SEP]**\n\n### Training\n\n#### Step 1: Binarize the training corpus\n```\nfairseq-preprocess \\\n    -s $src -t $tgt \\\n    --joined-dictionary --srcdict dict.${src}-${tgt}.txt \\\n    --trainpref train.bpe.t2s \\\n    --validpref valid.bpe.t2s \\\n    --destdir train-const-bin \\\n    --workers 128\n```\n\n#### Step 2: Model training\n##### En-Zh \\\u0026 Zh-En\n```\nCUDA_VISIBLE_DEVICES=0,1,2,3 fairseq-train train-const-bin \\\n    --fp16 --seed 32 --ddp-backend no_c10d \\\n    -s $src -t $tgt \\\n    --lr-scheduler inverse_sqrt --lr 0.0007 \\\n    --warmup-init-lr 1e-07 --warmup-updates 4000 \\\n    --max-update 200000 \\\n    --weight-decay 0.0 --clip-norm 0.0 --dropout 0.1 \\\n    --max-tokens 8192 --update-freq 1 \\\n    --arch transformer --share-all-embeddings \\\n    --optimizer adam --adam-betas '(0.9, 0.98)' \\\n    --save-dir ckpts \\\n    --tensorboard-logdir logs \\\n    --criterion label_smoothed_cross_entropy \\\n    --label-smoothing 0.1 \\\n    --no-progress-bar --log-format simple --log-interval 10 \\\n    --no-epoch-checkpoints \\\n    --save-interval-updates 1000 --keep-interval-updates 5 \\\n    |\u0026 tee -a logs/train.log\n```\n##### En-De \\\u0026 De-En\n```\nCUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 fairseq-train train-const-bin \\\n    --fp16 --seed 32 --ddp-backend no_c10d \\\n    -s $src -t $tgt \\\n    --lr-scheduler inverse_sqrt --lr 0.0005 \\\n    --warmup-init-lr 1e-07 --warmup-updates 4000 \\\n    --max-update 300000 \\\n    --weight-decay 0.0 --clip-norm 0.0 --dropout 0.1 \\\n    --max-tokens 4096 --update-freq 1 \\\n    --arch transformer_vaswani_wmt_en_de_big --share-all-embeddings \\\n    --optimizer adam --adam-betas '(0.9, 0.98)' \\\n    --save-dir ckpts \\\n    --tensorboard-logdir logs \\\n    --criterion label_smoothed_cross_entropy \\\n    --label-smoothing 0.1 \\\n    --no-progress-bar --log-format simple --log-interval 10 \\\n    --no-epoch-checkpoints \\\n    --save-interval-updates 1000 --keep-interval-updates 5 \\\n    |\u0026 tee -a logs/train.log\n```\n\n### Inference\n\n#### Step 1: Binarize the inference data\n```\nfairseq-preprocess \\\n    -s $src -t $tgt \\\n    --joined-dictionary --srcdict dict.${src}-${tgt}.txt \\\n    --testpref test.bpe.t2s.infer \\\n    --destdir infer-const-bin\n```\n#### Step 2: Model inference\n```\ngenout=test\nCUDA_VISIBLE_DEVICES=0 fairseq-generate infer-const-bin \\\n    -s $src -t $tgt \\\n    --gen-subset test \\\n    --path ${src}-${tgt}.pt \\\n    --lenpen 1.0 --beam 4 \\\n    --batch-size 128 \\\n    --prefix-size 1024 \\\n    \u003e $genout\n```\n\n#### Step 3: Post-process\nWe use the script [restore_template.py](https://github.com/shuo-git/Template-NMT/blob/main/lexical_scripts/restore_template.py) to convert the template-based model output into natural language sentences.\n```\ngrep ^H $genout |\\\n    sed 's/H-//g' | sort -k 1 -n -t ' ' | awk -F'\\t' '{print $3}' \u003e $genout.sort\n\npython restore_template.py test.bpe.t2s.infer.$tgt $genout.sort $genout.bpe\n\ncat $genout.bpe |\\\n    sed -r 's/(@@ )|(@@ ?$)//g' | tee $genout.tok |\\\n    perl mosesdecoder/scripts/tokenizer/detokenizer.perl -l $tgt \u003e $genout.detok\n```\n#### Step 4: Evaluate\nCalculate BLEU:\n```\n# English and German\ncat $genout.detok | sacrebleu test.detok.$tgt | tee $genout.bleu\n# Chinese\ncat $genout.detok | sacrebleu -tok zh test.detok.$tgt | tee $genout.bleu\n```\nCalculate Exact Match, Window Overlap, and 1-TERm using [evaluate_term_plain.py](https://github.com/shuo-git/Template-NMT/blob/main/lexical_scripts/terminology_evaluation/evaluate_term_plain.py), which is adapted from [this repository](https://github.com/mahfuzibnalam/terminology_evaluation):\n```\npython terminology_evaluation/evaluate_term_plain.py \\\n    --language $tgt \\\n    --source test.$src \\\n    --target_reference test.$tgt \\\n    --const test.const.${src}-${tgt} \\\n    --hypothesis $genout.tok\n```\n\n## Structurally Constrained Translation\n\n### Data\n\nWe conduct structurally constrained translation experiments on the dataset provided by [Hashimoto et al., (2019)](https://aclanthology.org/W19-5212).\n\n### Build the templates\n\nWe use the script [prepare_template.py](https://github.com/shuo-git/Template-NMT/blob/main/structural_scripts/prepare_template.py) to convert natural language sentences into template-based ones.\n\nPrepare the template-based training data:\n```\npython prepare_template.py en zh train.tag.spm t2s\n```\nPrepare the template-based inference data:\n```\npython prepare_template.py en zh dev.tag.spm t2s.infer\n```\n\nHere is an example:\nFilename | Example\n---|---\nen | Let’s look at how faceting works with the number and chart widgets that we added to our \\\u003cph\\\u003e classic designer \\\u003c/ph\\\u003e \\\u003cph\\\u003e dashboard \\\u003c/ph\\\u003e .\nzh | 让 我们 查看 多 面 化 如何 使用 添加到 \\\u003cph\\\u003e 经 典 设计 器 \\\u003c/ph\\\u003e \\\u003cph\\\u003e 仪表板 \\\u003c/ph\\\u003e 的 数字 和 图表 小部件 。\nt2s.en | [S0] \\\u003cph\\\u003e [S1] \\\u003c/ph\\\u003e [S2] \\\u003cph\\\u003e [S3] \\\u003c/ph\\\u003e [S4] **[SEP]** [S0] Let’s look at how faceting works with the number and chart widgets that we added to our [S1] classic designer [S2] [S3] dashboard [S4] .\nt2s.zh | [T0] \\\u003cph\\\u003e [T1] \\\u003c/ph\\\u003e [T2] \\\u003cph\\\u003e [T3] \\\u003c/ph\\\u003e [T4] **[SEP]** [T0] 让 我们 查看 多 面 化 如何 使用 添加到 [T1] 经 典 设计 器 [T2] [T3] 仪表板 [T4] 的 数字 和 图表 小部件 。\nt2s.infer.en | [S0] \\\u003cph\\\u003e [S1] \\\u003c/ph\\\u003e [S2] \\\u003cph\\\u003e [S3] \\\u003c/ph\\\u003e [S4] **[SEP]** [S0] Let ’ s look at how faceting works with the number and chart widgets that we added to our [S1] classic designer [S2] [S3] dashboard [S4] .\nt2s.infer.zh | [T0]\n\n### Training\n\n#### Step 1: Binarize the training corpus\n\n```\nfairseq-preprocess \\\n    -s $src -t $tgt \\\n    --joined-dictionary --srcdict dict.${src}-${tgt}.txt \\\n    --trainpref train.tag.spm.t2s \\\n    --destdir train-bin \\\n    --workers 128\n```\n\n#### Step 2: Model training\n\n```\nCUDA_VISIBLE_DEVICES=0,1,2,3 fairseq-train train-bin \\\n    --fp16 --seed 32 --ddp-backend no_c10d \\\n    -s $src -t $tgt \\\n    --lr-scheduler cosine \\\n    --lr 1e-07 --max-lr 7e-4 \\\n    --warmup-init-lr 1e-07 --warmup-updates 8000 \\\n    --lr-shrink 1 --lr-period-updates 32000 \\\n    --max-update 40000 \\\n    --weight-decay 0.001 --clip-norm 0.0 \\\n    --dropout 0.2 --attention-dropout 0.2 --activation-dropout 0.2 \\\n    --max-tokens 8192 --update-freq 1 \\\n    --arch transformer --share-all-embeddings \\\n    --encoder-embed-dim 256 --decoder-embed-dim 256 \\\n    --encoder-ffn-embed-dim 1024 --decoder-ffn-embed-dim 1024 \\\n    --encoder-attention-heads 4 --decoder-attention-heads 4 \\\n    --encoder-layers 6 --decoder-layers 6 \\\n    --optimizer adam --adam-betas '(0.9, 0.98)' \\\n    --save-dir ckpts \\\n    --tensorboard-logdir logs \\\n    --criterion label_smoothed_cross_entropy \\\n    --label-smoothing 0.2 \\\n    --no-progress-bar --log-format simple --log-interval 10 \\\n    --no-epoch-checkpoints \\\n    --save-interval-updates 500 --keep-interval-updates 5 \\\n    |\u0026 tee -a $logs/train.log\n```\n\n### Inference\n\n#### Step 1: Binarize the inference data\n\n```\nfairseq-preprocess \\\n    -s $src -t $tgt \\\n    --joined-dictionary --srcdict dict.${src}-${tgt}.txt \\\n    --testpref dev.tag.spm.t2s.infer \\\n    --destdir infer-bin\n```\n\n#### Step 2: Model inference\n```\ngenout=output\nCUDA_VISIBLE_DEVICES=0 fairseq-generate infer-bin \\\n    -s $src -t $tgt \\\n    --gen-subset test \\\n    --path ${src}-${tgt}.pt \\\n    --lenpen 1.0 --beam 4 \\\n    --prefix-size 1024 \\\n    --batch-size 128 \u003e $genout\n```\n\n#### Step 3: Post-process\n\nWe use the script [restore_template.py](https://github.com/shuo-git/Template-NMT/blob/main/structural_scripts/restore_template.py) to convert the template-based model output into natural language sentences.\n```\ngrep ^H $genout |\\\n    sed 's/H-//g' | sort -k 1 -n -t ' ' | awk -F'\\t' '{print $3}' \u003e $genout.sort\npython restore_template.py $genout.sort $genout.spm\ncat $genout.spm | spm_decode --model tag.spm.model \u003e $genout.txt\n```\n\n#### Step 4: Evaluate\n\nPrepare the json file using [convert2json.py](https://github.com/shuo-git/Template-NMT/blob/main/structural_scripts/convert2json.py), which is released by [this repository](https://github.com/salesforce/localization-xml-mt):\n```\npython convert2json.py \\\n    --input $genout.txt --output $genout.json \\\n    --lang $tgt --type translation --split dev \n```\nPerform the evaluation using [evaluate.py](https://github.com/shuo-git/Template-NMT/blob/main/structural_scripts/evaluate.py), which is released by [this repository](https://github.com/salesforce/localization-xml-mt):\n```\npython evaluate.py --target ${src}${tgt}_${tgt}_dev.json \\\n    --translation $genout.json | tee $genout.res\n```\n\n## Contact\n\nPlease email [wangshuo.thu@gmail.com](mailto:wangshuo.thu@gmail.com) if you have any questions, suggestions, or bug reports :)\n\n## Citation\n\nPlease cite as:\n```\n@inproceedings{Wang:2022:TemplateNMT,\n  title = {A Template-based Method for Constrained Neural Machine Translation},\n  author = {Wang, Shuo and Li, Peng and Tan, Zhixing and Tu, Zhaopeng and Sun, Maosong and Liu, Yang},\n  booktitle = {Proceedings of EMNLP 2022},\n  year = {2022},\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthunlp-mt%2Ftemplate-nmt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthunlp-mt%2Ftemplate-nmt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthunlp-mt%2Ftemplate-nmt/lists"}