{"id":13752368,"url":"https://github.com/thomas-chauvet/names_transliteration","last_synced_at":"2026-02-06T17:20:39.714Z","repository":{"id":56223452,"uuid":"288742265","full_name":"thomas-chauvet/names_transliteration","owner":"thomas-chauvet","description":"Neural Machine Translation (NMT) applied to transliterate names in arabic characters to latin characters (romanization).","archived":false,"fork":false,"pushed_at":"2021-06-03T09:38:20.000Z","size":5039,"stargazers_count":7,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-11-16T05:31:56.366Z","etag":null,"topics":["arabic","characters","cli","data","dataset","deep-learning","latin","neural-network","nlp","nmt","romanization","seq2seq","translation","transliteration","typer-cli"],"latest_commit_sha":null,"homepage":"https://colab.research.google.com/github/thomas-chauvet/names_transliteration/blob/master/arabic_to_english_names_transliteration_with_nmt_and_attention.ipynb","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thomas-chauvet.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-08-19T13:41:01.000Z","updated_at":"2024-08-17T16:36:38.000Z","dependencies_parsed_at":"2022-08-15T15:00:37.276Z","dependency_job_id":null,"html_url":"https://github.com/thomas-chauvet/names_transliteration","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thomas-chauvet%2Fnames_transliteration","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thomas-chauvet%2Fnames_transliteration/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thomas-chauvet%2Fnames_transliteration/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thomas-chauvet%2Fnames_transliteration/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thomas-chauvet","download_url":"https://codeload.github.com/thomas-chauvet/names_transliteration/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253312296,"owners_count":21888615,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arabic","characters","cli","data","dataset","deep-learning","latin","neural-network","nlp","nmt","romanization","seq2seq","translation","transliteration","typer-cli"],"created_at":"2024-08-03T09:01:04.702Z","updated_at":"2026-02-06T17:20:39.675Z","avatar_url":"https://github.com/thomas-chauvet.png","language":"Jupyter Notebook","funding_links":[],"categories":["Uncategorized"],"sub_categories":["Uncategorized"],"readme":"[![Streamlit App](https://static.streamlit.io/badges/streamlit_badge_black_white.svg)](https://share.streamlit.io/thomas-chauvet/names_transliteration/app.py) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/thomas-chauvet/names_transliteration/blob/master/arabic_to_english_names_transliteration_with_nmt_and_attention.ipynb)\n\n# Names transliteration\n\nIn this [repository](https://github.com/thomas-chauvet/names_transliteration) you will find:\n- a [dataset](https://raw.githubusercontent.com/thomas-chauvet/names_transliteration/master/data/clean/arabic_english.csv) \n(and associated code to build it) containing \nnames in arabic characters and associated names in latin \ncharacters (english),\n- a (google colab) notebook to train a \n[Neural Machine Translation](https://en.wikipedia.org/wiki/Neural_machine_translation) (NMT) model\nbased on [seq2seq](https://en.wikipedia.org/wiki/Seq2seq). The objective\nof this model is to [transliterate](https://en.wikipedia.org/wiki/Transliteration) names\nin arabic alphabet to latin alphabet. This task is also called \n[romanization](https://en.wikipedia.org/wiki/Romanization).\n\nThe model is trained thanks to Google Colab providing (free) GPU.\n\nThe model is based on Tensorflow tutorial \n[NMT with attention](https://www.tensorflow.org/tutorials/text/nmt_with_attention).\n\n## Data\n\nWe use 3 datasets:\n*   [Google transliteration data](https://github.com/google/transliteration/blob/master/ar2en.txt).\nExample: *عادل; adel*\n*   [ANETAC dataset](https://github.com/MohamedHadjAmeur/ANETAC/blob/master/EN-AR%20NE/EN-AR%20Named-entities.txt). \nExample: *PERSON; Adel; اديل*. For this file we'll filter on *PERSON* only,\n*   [NETranliteration COLING 2018](https://github.com/steveash/NETransliteration-COLING2018/blob/master/data/wd_arabic.normalized.aligned.tokens).\n\nThese 3 datasets will give us a clean dataset containing names in arabic and \ncorresponding names in latin alphabet (english).\n\n## Pre-trained models\n\nA pre-trained model (arabic to latin characters) is stored on \n[dropbox](https://www.dropbox.com/s/leqc4k9c4hzfvi3/names-translation-model-2020-10-02.zip?dl=1).\n\n## Colab notebook\n\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/thomas-chauvet/names_transliteration/blob/master/arabic_to_english_names_transliteration_with_nmt_and_attention.ipynb)\n\nA jupyter notebook is provided to train the model used for transliteration.\n\n## Web application - Streamlit\n\nA streamlit is provided. You can find a deployed version [here](https://share.streamlit.io/thomas-chauvet/names_transliteration/app.py).\n\n[![Streamlit App](https://static.streamlit.io/badges/streamlit_badge_black_white.svg)](https://share.streamlit.io/thomas-chauvet/names_transliteration/app.py)\n\n## Library\n\nInstall library:\n```bash\npython setup.py install\n```\n\n## CLI\n\n- `get-data`: Get data from 3 sources to get a training dataset.\n- `get-pretrained-model`: Download pre-trained model for the task.\n- `train-nmt-model`: Train an NMT model.\n- `transliterate-name`: Transliterate a name in arabic in latin character.\n\n## Python environment\n\nPlease refer to the `environment.yml` file for conda environment.\n\nTo create the environment with conda:\n```bash\nconda env create -f environment.yml\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthomas-chauvet%2Fnames_transliteration","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthomas-chauvet%2Fnames_transliteration","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthomas-chauvet%2Fnames_transliteration/lists"}