awesome-urdu

📖 A curated list of resources dedicated to Urdu language.
https://github.com/urduhack/awesome-urdu

Last synced: 9 days ago
JSON representation

Online Resources/Services
- Dictionaries
  - ur.oxforddictionaries.com - Oxford Dictionary
  - English Urdu Dictionary - English Urdu Dictionary
  - Urdu English Dictionary 2 - Urdu English Dictionary 2
  - English Urdu Dictionary - English Urdu Dictionary
- Transliteration Libraries
  - Pakistani Center for Language Engineering - Online Services
- Urdu News websites
Urdu Datasets
- Cross-lingual Datasets
  - Cross-lingual Natural Language Inference (XNLI) Corpus, 2020
  - Google XTrEME Benchmark, 2020 - Evaluation of cross-lingual generalization of multilingual models
  - Urdu-Punjabi Pairs, Apertium
- General NLP Datasets
  - Web news Data - Urdu Web news Data
  - Urdu Paraphrase Plagiarism Corpus, 2016
  - COrpus of Urdu News TExt Reuse (CoUNTeR), 2016
  - Urdu Short Text Reuse Corpus (USTRC), 2018
  - TaPaCo: A Corpus of Sentential Paraphrases for 73 Languages
  - Roman Urdu Dataset - Data for sentiment analysis, along with misc compiled data for Roman Urdu
  - Collection of Urdu Datasets - Datasets for POS, NER and NLP tasks
  - Urdu Universal Dependency Treebank
  - UrduSummary Corpus Benchmark, 2016
  - Rekhta Ghazals
  - Urdu Paraphrase Plagiarism Corpus, 2016
  - COrpus of Urdu News TExt Reuse (CoUNTeR), 2016
  - Urdu Short Text Reuse Corpus (USTRC), 2018
  - Flickr8k Urdu Image-Caption Generation Dataset, 2020
  - mLAMA: multilingual LAnguage Model Analysis, 2021
  - Urdu Word Segmentation using CRF, 2018
  - Apertium linguistic data for Urdu
  - Web news Data - Urdu Web news Data
- Urdu Lexical Resources
  - CLE Urdu WordNet
  - Verb List
  - MTurks-10k Multilingual Dictionary, 2014
  - Microsoft IT Terminology
  - Urdu N-grams, 2020 - Uni-Gram, Bi-Gram, Tri-Gram and Tetra-Gram
  - CLE Urdu Books N-Grams
  - Offline Eng-Urd Dictionary DB
  - UrduHack Words-List - Includes N-grams, NER Labels
  - IndoWordnet Parallel Corpus - [pyiwn](https://github.com/riteshpanjwani/pyiwn), [Demo](https://www.cfilt.iitb.ac.in/indowordnet/))
  - Roman Urdu Lexical Normalization, 2019
- Urdu Monolingual Corpora
  - UFAL Corpus, 2014 - 5.4M sentences (with POS tags)
  - iNLTK Wiki Articles, 2020 - NLP/Tatoeba-Challenge/blob/master/data/Backtranslations.md), [2016 UrduWikiCorpus](http://urdu-corpus.blogspot.com/p/published-packages.html)
  - Leipzig Corpora
  - UrduWaC-2010 and urTenTen-2018, SketchEngine
  - A Gold Standard Urdu Raw Text Corpus, LDCIL
  - WiToKit
  - Maḵẖzan
  - OSCAr Corpus, 2020
  - CC-100 Corpus, 2019 - CC crawls from Jan-Dec 2018
- Urdu Named-Entity Recognition
  - MK-PUCIT-NER, 2019
  - WikiAnn: Cross-lingual Name Tagging & Linking, 2017
- Urdu OCR Datasets
  - U-HAT - Urdu Hand-Written Text Dataset
  - IIIT-Hyderabad: Unconstrained OCR for Urdu using Deep CNN-RNN Hybrid Networks, 2017
  - CLE Pakistan Urdu Image Corpora
  - Cursive-Text: A Benchmark for Urdu Text Recognition in Natural Scene Images, 2020 - 2500 images, email for dataset
  - Qaida - Synthetic datasets and pre-trained models
  - 45K+ Clean-Background-Urdu-Ligatures-Dataset, 2019
- Urdu Parallel Corpora for Machine Translation
  - OPUS Corpora - >ur)
  - CC-Aligned - 1310/), [OpenSubtitles](https://www.aclweb.org/anthology/L16-1147/), [TED](https://www.ted.com/participate/translate), [QED](https://www.aclweb.org/anthology/L14-1675/), etc.
  - IIIT-Hyderabad MT Bhasha
  - English-Urdu Religious Parallel Corpus
  - Urdu-Nepali-English Parallel Corpus
  - Cross-Language English-Urdu (CLEU) Corpus, 2018
  - Flickr 8k Benchmark - 2.7k sentences
  - Universal Declaration of Human Rights (benchmark)
  - EMILLE/CIIL Corpus - Contains monolingual data as well
  - Technology Development for Indian Languages
  - National Platform for Language Technology
  - Anuvaad Parallel Corpora
  - MechanicalTurks 2012 Parallel Corpora
  - Cross-Language English-Urdu (CLEU) Corpus, 2018
  - EMILLE/CIIL Corpus - Contains monolingual data as well
  - Technology Development for Indian Languages
- Urdu Sentiment Datasets
  - Urdu IMDb Movie Reviews - IMDB Movie Reviews data in Urdu
  - 2010 Disaster Response Messages
  - Urdu Sentiment Lexicon
  - UCI Roman-Urdu Sentiment Classification, 2018 - 20k records
  - Did You Offend Me? Classification of Offensive Tweets, 2018 - 3k tweets
  - Urdu Sentiment Benchmark, 2020
  - Hate Speech & Offensive Language Detection, 2020 - 10k tweets
  - UCI Roman-Urdu Sentiment Classification, 2018 - 20k records
- Urdu Speech Datasets
  - Urdu 250 Isolated Words, 2018
  - CLE Phonetically Rich Urdu Speech Corpus
  - CMU Wilderness Speech Dataset, 2019
  - FCBH Recordings
  - LibriVox AudioBooks
  - CLE Pakistan Urdu Speech Corpus
  - LDC UPenn Datasets - Filter search by selecting language
  - Urdu Raw Speech Corpus, LDCIL
  - LDCIL ASR Corpus
  - Urdu-Sindhi Speech Emotion Corpus, 2020 - Introducing_the_Urdu_Sindhi_Speech_Emotion_Corpus.pdf))
  - Speech Emotion Recognition Benchmark, 2018
  - CMU Wilderness Speech Dataset, 2019
  - CLE Pakistan Urdu Speech Corpus
  - LibriVox AudioBooks
- Urdu Text Classification
  - iNLTK Urdu News Headlines Classification Benchmark, 2020
  - Fake News Classification, 2020 - News-Augmented-Dataset))
  - Express News Headlines+Summary, 2019
- Urdu Transliteration Datasets
Urdu NLP Tools, Libraries and Models
- Cross-lingual Datasets
  - UrduHack
  - Urdu Morphological Analyzer, IIIT Hyderabad
  - PronouncUR - Urdu words to pronouniciations format
  - Indic PoS/NER Tagger
  - Urdu Morphological Analyzer, IIIT Hyderabad
  - EasyOCR
- Language Models
- Translation Models
  - Facebook M2M-100, 2020
  - IL-Multi, 2020
  - Python Translators Services - Library to use Google, Bing, etc. translators for free
  - Facebook M2M-100, 2020
- Transliteration Libraries
  - PolyGlot
  - AksharaMukhi - Devanagari (Hindi) to Urdu script converter
  - Google Transliterate API - Roman Urdu to Perso-Arabic
  - LibIndicTrans - Transliterate Roman/Hindi to Urdu and vice-versa
  - AksharaMukhi - Devanagari (Hindi) to Urdu script converter
- Word Embeddings
  - UrduHack Word-Vectors, 2019 - Word2Vec and FastText models
  - Wiki-2016 - 2017](https://fasttext.cc/docs/en/crawl-vectors.html), [Multilingual Aligned, 2017](https://github.com/babylonhealth/fastText_multilingual)
  - Polyglot Embeddings, 2013
  - ConceptNet Embeddings, 2017
  - BPEmb: Subword Embeddings, 2017 - its.org/bpemb/multi/))

Programming Languages

Python 11 XML 1 Jupyter Notebook 1

Categories

Urdu Datasets 93 Online Resources/Services 55 Urdu NLP Tools, Libraries and Models 26

Sub Categories

Urdu News websites 50 General NLP Datasets 18 Urdu Parallel Corpora for Machine Translation 16 Urdu Speech Datasets 14 Urdu Lexical Resources 10 Urdu Monolingual Corpora 9 Cross-lingual Datasets 9 Urdu Sentiment Datasets 8 Transliteration Libraries 6 Urdu OCR Datasets 6 Language Models 6 Word Embeddings 5 Urdu Transliteration Datasets 4 Dictionaries 4 Translation Models 4 Urdu Text Classification 3 Urdu Named-Entity Recognition 2

Keywords

nlp 4 machine-learning 3 dataset 2 pytorch 2 urdu-language 2 urdu 2 baidu 1 bing 1 caiyun 1 deepl 1 google 1 iciba 1 iflytek 1 itranslate 1 lingvanex 1 modernmt 1 mymemory 1 niutrans 1 papago 1 reverso 1 argos 1 alibaba 1 scene-text-recognition 1 scene-text 1 python 1 optical-character-recognition 1 ocr 1 lstm 1 information-retrieval 1 image-processing 1 easyocr 1 deep-learning 1 data-mining 1 crnn 1 cnn 1 apertium-languages 1 wikipedia-dump 1 wikipedia 1 tokenize 1 multilingual 1 dump 1 rekhta 1 parser 1 language-model 1 data 1 urdu-nlp 1 natural-language-processing 1 hindi-language 1 hindi 1 data-science 1