Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

indicnlp_catalog

A collaborative catalog of NLP resources for Indic languages
https://github.com/AI4Bharat/indicnlp_catalog

Last synced: 28 days ago
JSON representation

<a name='Benchmarks'></a>Evaluation Benchmarks

<a name='LanguageSpecificCatalogs'></a>Language Specific Catalogs

<a name='NER'></a>NER

<a name='Libraries'></a>Libraries and Tools

<a name='MajorIndicLanguageNLPRepositories'></a>Major Indic Language NLP Repositories

<a name='Models'></a>Models

<a name='MultilingualWordEmbeddings'></a>Multilingual Word Embeddings

<a name='NER'></a>NER

<a name='PreTrainedLanguageModels'></a>Pre-trained Language Models

<a name='SpeechModels'></a>Speech Models

<a name='TranslationModels'></a>Translation Models

<a name='TransliterationModels'></a>Transliteration Models

  • AI4Bharat IndicXlit - based multilingual transliteration model with 11M parameters for Roman to native script conversion and vice versa that supports 21 Indic languages. Described in this paper.

<a name='WordEmbeddings'></a>Word Embeddings

<a name='MultimodalCorpora'></a>Multimodal Corpora

<a name='NER'></a>NER

<a name='OCRCorpora'></a>OCR Corpora

<a name='NER'></a>NER

<a name='SpeechCorpora'></a>Speech Corpora

<a name='NER'></a>NER

<a name='Standards'></a>Standards

<a name='TextCorpora'></a>Text Corpora

<a name='ChunkCorpus'></a>Chunk Corpus

<a name='CoreferenceCorpus'></a>Coreference Corpus

<a name='DatatoText'></a>Data to Text

  • XAlign - lingual Fact-to-Text Alignment and Generation for Low-Resource Languages comprising of a high quality XF2T dataset in 7 languages: Hindi, Marathi, Gujarati, Telugu, Tamil, Kannada, Bengali, and monolingual dataset in English. The dataset is available upon request. Described in this paper.

<a name='DependencyParseCorpus'></a>Dependency Parse Corpus

<a name='Dialog'></a>Dialog

<a name='Discourse'></a>Discourse

<a name='HateSpeech'></a>Hate Speech and Offensive Comments

<a name='InformationExtraction'></a>Information Extraction

  • EventXtract-IL - ws.org/Vol-2266/T5-1.pdf).
  • EDNIL-FIRE2020
  • Facebook - MTOP Benchmark - Oriented Semantic Parsing Benchmark with a dataset comprising of 100k annotated utterances in 6 languages(including Indic language: Hindi) across 11 domains. Described in this paper.
  • Amazon MASSIVE - Assistant Evaluation containing one million realistic, parallel, labeled virtual-assistant text utterances spanning 51 languages, 18 domains, 60 intents, and 55 slots. Described in this paper.
  • Facebook - MTOP Benchmark - Oriented Semantic Parsing Benchmark with a dataset comprising of 100k annotated utterances in 6 languages(including Indic language: Hindi) across 11 domains. Described in this paper.

<a name='LanguageIdentification'></a>Language Identification

<a name='LexicalResources'></a>Lexical Resources and Semantic Similarity

<a name='MonolingualCorpus'></a>Monolingual Corpus

<a name='MTEvaluation'></a>MT Evaluation

  • WMT23 QE task - edits are also available as are word-level annotations error annotations are also available. 26k training sentences for Marathi, 7k for the others. report
  • AI4Bharat IndicMT-Eval
  • WMT23 QE task - edits are also available as are word-level annotations error annotations are also available. 26k training sentences for Marathi, 7k for the others. report

<a name='NERCorpora'></a>NER Corpora

<a name='ParallelTranslationCorpus'></a>Parallel Translation Corpus