{"id":33207848,"url":"https://github.com/fbenites/TRANSLIT","last_synced_at":"2025-11-16T15:01:59.023Z","repository":{"id":39729935,"uuid":"241075632","full_name":"fbenites/TRANSLIT","owner":"fbenites","description":null,"archived":false,"fork":false,"pushed_at":"2022-12-08T07:25:48.000Z","size":61,"stargazers_count":6,"open_issues_count":7,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-07-23T09:59:57.106Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc0-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fbenites.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-02-17T10:07:30.000Z","updated_at":"2023-01-27T21:55:00.000Z","dependencies_parsed_at":"2023-01-25T08:45:49.044Z","dependency_job_id":null,"html_url":"https://github.com/fbenites/TRANSLIT","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/fbenites/TRANSLIT","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fbenites%2FTRANSLIT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fbenites%2FTRANSLIT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fbenites%2FTRANSLIT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fbenites%2FTRANSLIT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fbenites","download_url":"https://codeload.github.com/fbenites/TRANSLIT/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fbenites%2FTRANSLIT/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":284725595,"owners_count":27053255,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-11-16T02:00:05.974Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-11-16T10:00:27.931Z","updated_at":"2025-11-16T15:01:59.018Z","avatar_url":"https://github.com/fbenites.png","language":"Python","funding_links":[],"categories":["**Datasets**","Urdu Datasets"],"sub_categories":["Transliteration","Urdu Transliteration Datasets"],"readme":"# TRANSLIT: A Large Name Transliteration Resource\n\n**TRANSLIT** is A Large Name Transliteration Resource. If you find this code useful in your research, please consider citing:\n\n\n    @inproceedings{benitesLREC2020,\n\tAuthor = {Fernando Benites, Gilbert François Duivesteijn, Pius von Däniken, Mark Cieliebak}\n\tTitle = {Large Name Transliteration Resource},\n\tbooktitle = {Proceedings of the Thirteenth International Conference on Language Resources and Evaluation (LREC 2020)},\n\tYear = {2020},\n    }\n\n## We merged together sources that now encompasses 3 Millions surfaces (names) of around 1.6 Million entities\n\nWe merged four data sources:\n1. [JRC named entities](https://ec.europa.eu/jrc/en/language-technologies/jrc-names)\n2. [Amazon Wiki-Names](https://github.com/steveash/NETransliteration-COLING2018)\n3. [Google En-Ar transliterations](https://github.com/google/transliteration)\n4. [Geonames](https://download.geonames.org/export/dump/alternateNamesV2.zip)\n\nWe also searched for lang tags of wikipedia for transliterations (wiki-all).\n\nWe merged multiple names of an entity and assigned a UUID to it. We saved all the gathered names/entities in the file [TRANSLIT.json](https://github.com/fbenites/TRANSLIT/blob/master/artefacts/TRANSLIT.json), in the artefacts directory.\n\n|Dataset       |# entities|# name variations| mean length of chars per name|\n|--------------|----------|-----------------|------------------------------|   \n|JRC           |819'209   |1'338'463        |14.3                          |\n| Geonames       | 139'549    | 758'274           | 10.6                           |\n| SubWikiLang    | 609'420    | 1'376'446         | 10.3                           |\n| En-Ar          | 15'858     | 31'716            | 4.4                            |\n| Wiki-lang-all  | 122'180    | 144'588           | 17.0                           |\n| TRANSLIT (all) | 1'655'972  | 3'008'239         | 11.8                           |\n     \n     \n## Experiments\n\nThe experiments of the paper can be retraced with the use of the scripts abalation_study.py, classification_experiments.py and cnn_classification.py in the code directory. For their use, the data in artefact is used. To recreate this data, you need to download the original data (17G zipped) with download_data.sh. Afterward you should run run_preprare_data.sh.\n\n## Troubleshooting\n\nthe artefacts are quite large, so git lfs needs to be installed:\n$ sudo apt install git-lfs\n$ git lfs install --local\n$ git lfs fetch\n\n\n     \n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffbenites%2FTRANSLIT","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffbenites%2FTRANSLIT","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffbenites%2FTRANSLIT/lists"}