{"id":18887136,"url":"https://github.com/thunlp-mt/uce4bt","last_synced_at":"2026-03-03T07:43:35.098Z","repository":{"id":86099786,"uuid":"219900346","full_name":"THUNLP-MT/UCE4BT","owner":"THUNLP-MT","description":null,"archived":false,"fork":false,"pushed_at":"2020-04-25T14:02:14.000Z","size":305,"stargazers_count":19,"open_issues_count":0,"forks_count":4,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-28T10:54:19.565Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/THUNLP-MT.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-11-06T03:03:09.000Z","updated_at":"2024-02-22T18:14:57.000Z","dependencies_parsed_at":null,"dependency_job_id":"91d5006c-e239-4dab-ae15-115a7bd76b39","html_url":"https://github.com/THUNLP-MT/UCE4BT","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUNLP-MT%2FUCE4BT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUNLP-MT%2FUCE4BT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUNLP-MT%2FUCE4BT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUNLP-MT%2FUCE4BT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/THUNLP-MT","download_url":"https://codeload.github.com/THUNLP-MT/UCE4BT/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248973023,"owners_count":21191900,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T07:34:39.646Z","updated_at":"2026-03-03T07:43:30.076Z","avatar_url":"https://github.com/THUNLP-MT.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Improving Back-Translation with Uncertainty-based Confidence Estimation\n## Contents\n* [Introduction](#introduction)\n* [Prerequisites](#prerequisites)\n* [Usage](#usage)\n* [Contact](#contact)\n\n## Introduction\n\nThis is the implementation of our work \u003ca href=\"https://arxiv.org/abs/1909.00157\"\u003eImproving Back-Translation with Uncertainty-based Confidence Estimation\u003c/a\u003e. \n\n\u003cpre\u003e\u003ccode\u003e@inproceedings{Wang:2019:EMNLP,\n    title = \"Improving Back-Translation with Uncertainty-based Confidence Estimation\",\n    author = \"Wang, Shuo and Liu, Yang and Wang, Chao and Luan, Huanbo and Sun, Maosong\",\n    booktitle = \"EMNLP\",\n    year = \"2019\"\n}\n\u003c/code\u003e\u003c/pre\u003e\n\nThe implementation is on top of [THUMT](https://github.com/thumt/THUMT).\n\n## Prerequisites\nThis repository runs in the same environment as THUMT, please refer to \u003ca href=\"https://github.com/THUNLP-MT/THUMT/blob/master/UserManual.pdf\"\u003ethe user manual of THUMT\u003c/a\u003e to config the environment.\n\n## Usage\nNote: The usage is not user-friendly. May improve later.\n\u003cbr\u003e\nSuppose the local path to this repository is CODE_DIR.\n\n1. Standard training:\n\u003cpre\u003e\u003ccode\u003epython [CODE_DIR]/thumt/bin/trainer.py \\\n\t--input [source corpus] [target corpus] \\\n\t--side none \\\n\t--vocabulary [source vocabulary] [target vocabulary] \\\n\t--model transformer \\\n\t--parameters=train_steps=60000,constant_batch_size=false,batch_size=6250,device_list=[0,1,2,3]\n\u003c/code\u003e\u003c/pre\u003e\nYou can train a target-source translation model by simply exchanging source corpus and target corpus, source vocabulary and target vocabulary.\n\n2. Translate target-side monolingual corpus:\n\u003cpre\u003e\u003ccode\u003epython [CODE_DIR]/thumt/bin/translator.py \\\n\t--input [monolingual corpus] \\\n\t--output [translated corpus] \\\n\t--vocabulary [target vocabulary] [source vocabulary] \\\n\t--model transformer \\\n\t--checkpoint [path to the target-source model] \\\n\t--parameters=device_list=[0]\n\u003c/code\u003e\u003c/pre\u003e\nWe recommand splitting the entire monolingual corpus into small corpora before translation if the monolingual corpus is too big.\n\n3. Uncertainty estimation for the translated corpus:\n\u003cpre\u003e\u003ccode\u003epython [CODE_DIR]/thumt/bin/scorer.py \\\n\t--input [monolingual corpus] [translated corpus] \\\n\t--vocabulary [target vocabulary] [source vocabulary] \\\n\t--mean_file [word-level mean] \\\n\t--var_file [word-level var] \\\n\t--rv_file [word-level var/mean] \\\n\t--sen_mean [sentence-level mean] \\\n\t--sen_var [sentence-level var] \\\n\t--sen_rv [sentence-level var/mean] \\\n\t--model transformer \\\n\t--checkpoint [path to the target-source model] \\\n\t--parameters=model_uncertainty=true,device_list=[0]\n\u003c/code\u003e\u003c/pre\u003e\n\n4. Confidence-aware training:\n\u003cpre\u003e\u003ccode\u003epython [CODE_DIR]/thumt/bin/trainer.py \\\n\t--input [source corpus] [target corpus] \\\n\t--word_confidence [word-level uncertainty file] \\\n\t--sen_confidence [sentence-level uncertainty file] \\\n\t--side source_sentence_source_word \\\n\t--vocabulary [source vocabulary] [target vocabulary] \\\n\t--model transformer \\\n\t--checkpoint [path to the source-target checkpoint] \\\n\t--parameters=train_steps=60000,constant_batch_size=false,batch_size=6250,device_list=[0,1,2,3]\n\u003c/code\u003e\u003c/pre\u003e\n\n## Contact\n\nIf you have questions, suggestions and bug reports, please email [wangshuo18@mails.tsinghua.edu.cn](mailto:wangshuo18@mails.tsinghua.edu.cn).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthunlp-mt%2Fuce4bt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthunlp-mt%2Fuce4bt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthunlp-mt%2Fuce4bt/lists"}