{"id":18887160,"url":"https://github.com/thunlp-mt/ubilexemd","last_synced_at":"2025-07-18T00:05:59.336Z","repository":{"id":86099771,"uuid":"160185530","full_name":"THUNLP-MT/UBiLexEMD","owner":"THUNLP-MT","description":"An Unsupervised Bilingual Lexicon Inducer From Non-Parallel Data by Earth Mover's Distance Minimization","archived":false,"fork":false,"pushed_at":"2018-12-03T12:13:17.000Z","size":4037,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-14T22:52:38.693Z","etag":null,"topics":["bilingual-lexicon-extraction","bilingual-word-embedding"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/THUNLP-MT.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-12-03T12:12:07.000Z","updated_at":"2021-05-15T03:16:46.000Z","dependencies_parsed_at":null,"dependency_job_id":"5ac9fb88-5364-4151-912b-4a287e0d404e","html_url":"https://github.com/THUNLP-MT/UBiLexEMD","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/THUNLP-MT/UBiLexEMD","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUNLP-MT%2FUBiLexEMD","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUNLP-MT%2FUBiLexEMD/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUNLP-MT%2FUBiLexEMD/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUNLP-MT%2FUBiLexEMD/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/THUNLP-MT","download_url":"https://codeload.github.com/THUNLP-MT/UBiLexEMD/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUNLP-MT%2FUBiLexEMD/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265683172,"owners_count":23810807,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bilingual-lexicon-extraction","bilingual-word-embedding"],"created_at":"2024-11-08T07:34:53.279Z","updated_at":"2025-07-18T00:05:59.328Z","avatar_url":"https://github.com/THUNLP-MT.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# UBiLexEMD: An Unsupervised Bilingual Lexicon Inducer From Non-Parallel Data by Earth Mover's Distance Minimization #\r\n\r\nThis software can produce a bilingual lexicon from non-parallel data without any cross-lingual supervision. It does so by learning a transformation between source word embeddings and target ones by earth mover's distance minimization. The technique is described in the following paper:\r\n\r\n\u003e Meng Zhang, Yang Liu, Huanbo Luan, and Maosong Sun. Earth Mover's Distance Minimization for Unsupervised Bilingual Lexicon Induction. In Proceedings of EMNLP, 2017.\r\n\r\n## Runtime Environment ##\r\n\r\nThis software has been tested in the following environment, but should work in a compatible one.\r\n\r\n- 64-bit Linux\r\n- Python 2.7 (for WGAN code in the `src` folder)\r\n\t- Theano\r\n\t- Lasagne ([bleeding-edge version](http://lasagne.readthedocs.io/en/latest/user/installation.html#bleeding-edge-version) needed as of April, 2017)\r\n\t- scikit-learn\r\n- Python 3.4 (for the code in the `scripts` folder)\r\n- Matlab R2010b (for EMDOT code)\r\n\r\n## Usage ##\r\n\r\n### Preparation ###\r\n\r\n1\\. Specify the variables in the `config` file. For example, if `config` contains the following lines:\r\n\r\n\tconfig=zh-en\r\n\tlang1=zh\r\n\tlang2=en\r\n\r\nthen the data should be located in `data/zh-en` with file extensions `zh` and `en`. Prepare the `matlab.config` file accordingly.\r\n\r\n2\\. Prepare the following data in the folder specified in Step 1:\r\n\r\n- word2vec.zh/en: Word embeddings, which can be obtained by running word2vec on monolingual data.\r\n- vocab-freq.zh/en: Space-separated word-frequency pairs.\r\n\r\nBesides, prepare vocab.zh/en, vec.zh/en, count.zh/en from the above data.\r\n\r\n### WGAN ###\r\n\r\nExecute `./runWGAN.sh` to obtain a file named `W` that stores the transformation matrix.\r\n\r\n### EMDOT ###\r\n\r\n1\\. Copy the `W` file produced by WGAN into `data/zh-en` (the folder specified in `config`). One such file is provided in this release.\r\n\r\n2\\. Launch Matlab. In the console, execute:\r\n\r\n\tloadData\r\n\tSinkhornWithTransformationInitFromData(X_s, X_t, weight_s, weight_t, length(weight_s), length(weight_t));\r\n\texit\r\n\r\n3\\. Process the transport scheme to obtain the translations.\r\n\r\n`./processFlow.sh 10`\r\n\r\n4\\. In `data/zh-en`, result.10 will contain translations of vocab.zh. For each source word, there will be multiple translations after the tab character, separated by space.\r\n\r\n## Known Issue ##\r\n\r\nIt is recommended that the bleeding-edge version of Lasagne works with the latest development version of Theano. It has been tested on Lasagne version 0.2.dev1 and Theano version 0.9.0dev4.dev-RELEASE, but NaN may appear on Theano version 0.8.2.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthunlp-mt%2Fubilexemd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthunlp-mt%2Fubilexemd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthunlp-mt%2Fubilexemd/lists"}