{"id":13676605,"url":"https://github.com/Hironsan/awesome-embedding-models","last_synced_at":"2025-04-29T07:32:44.266Z","repository":{"id":51163776,"uuid":"75587993","full_name":"Hironsan/awesome-embedding-models","owner":"Hironsan","description":"A curated list of awesome embedding models tutorials, projects and communities.","archived":false,"fork":false,"pushed_at":"2019-04-07T22:56:01.000Z","size":49,"stargazers_count":1720,"open_issues_count":2,"forks_count":248,"subscribers_count":106,"default_branch":"master","last_synced_at":"2024-05-20T00:21:55.194Z","etag":null,"topics":["awesome","embedding-models","embeddings","machine-learning","natural-language-processing","papers","word2vec"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Hironsan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-12-05T04:16:20.000Z","updated_at":"2024-05-19T10:49:06.000Z","dependencies_parsed_at":"2022-09-01T19:13:08.601Z","dependency_job_id":null,"html_url":"https://github.com/Hironsan/awesome-embedding-models","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Hironsan%2Fawesome-embedding-models","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Hironsan%2Fawesome-embedding-models/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Hironsan%2Fawesome-embedding-models/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Hironsan%2Fawesome-embedding-models/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Hironsan","download_url":"https://codeload.github.com/Hironsan/awesome-embedding-models/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223806487,"owners_count":17205982,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["awesome","embedding-models","embeddings","machine-learning","natural-language-processing","papers","word2vec"],"created_at":"2024-08-02T13:00:30.245Z","updated_at":"2024-11-11T18:31:11.546Z","avatar_url":"https://github.com/Hironsan.png","language":"Jupyter Notebook","readme":"# awesome-embedding-models[![Awesome](https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/sindresorhus/awesome)\nA curated list of awesome embedding models tutorials, projects and communities.\nPlease feel free to pull requests to add links.\n\n## Table of Contents\n\n\n* **[Papers](#papers)**\n* **[Researchers](#researchers)**\n* **[Courses and Lectures](#courses-and-lectures)**\n* **[Datasets](#datasets)**\n* **[Implementations and Tools](#implementations-and-tools)**\n\u003c!--* **[Articles](#articles)**--\u003e\n\n## Papers\n### Word Embeddings\n\n**Word2vec, GloVe, FastText**\n\n* Efficient Estimation of Word Representations in Vector Space (2013), T. Mikolov et al. [[pdf]](https://arxiv.org/pdf/1301.3781.pdf)\n* Distributed Representations of Words and Phrases and their Compositionality (2013), T. Mikolov et al. [[pdf]](https://arxiv.org/pdf/1310.4546.pdf)\n* word2vec Parameter Learning Explained (2014), Xin Rong [[pdf]](https://arxiv.org/pdf/1411.2738.pdf)\n* word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method (2014), Yoav Goldberg, Omer Levy [[pdf]](https://arxiv.org/pdf/1402.3722.pdf)\n* GloVe: Global Vectors for Word Representation (2014), J. Pennington et al. [[pdf]](http://nlp.stanford.edu/pubs/glove.pdf)\n* Improving Word Representations via Global Context and Multiple Word Prototypes (2012), EH Huang et al. [[pdf]](http://www.aclweb.org/anthology/P12-1092)\n* Enriching Word Vectors with Subword Information (2016), P. Bojanowski et al. [[pdf]](https://arxiv.org/pdf/1607.04606v1.pdf)\n* Bag of Tricks for Efficient Text Classification (2016), A. Joulin et al. [[pdf]](https://arxiv.org/pdf/1607.01759.pdf)\n\n**Language Model**\n\n* Semi-supervised sequence tagging with bidirectional language models (2017), Peters, Matthew E., et al. [[pdf]](https://arxiv.org/abs/1705.00108)\n* Deep contextualized word representations (2018), Peters, Matthew E., et al. [[pdf]](https://arxiv.org/abs/1802.05365)\n* Contextual String Embeddings for Sequence Labeling (2018), Akbik, Alan, Duncan Blythe, and Roland Vollgraf. [[pdf]](http://alanakbik.github.io/papers/coling2018.pdf)\n* BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2018), [[pdf]](https://arxiv.org/abs/1810.04805)\n\n\n\n**Embedding Enhancement**\n\n* Sentence Embedding:Learning Semantic Sentence Embeddings using Pair-wise Discriminator(2018),Patro et al.[[Project Page]](https://badripatro.github.io/Question-Paraphrases/) [[Paper]](https://www.aclweb.org/anthology/C18-1230)\n* Retrofitting Word Vectors to Semantic Lexicons (2014), M. Faruqui et al. [[pdf]](https://arxiv.org/pdf/1411.4166.pdf)\n* Better Word Representations with Recursive Neural Networks for Morphology (2013), T.Luong et al. [[pdf]](http://www.aclweb.org/website/old_anthology/W/W13/W13-35.pdf#page=116)\n* Dependency-Based Word Embeddings (2014), Omer Levy, Yoav Goldberg [[pdf]](https://levyomer.files.wordpress.com/2014/04/dependency-based-word-embeddings-acl-2014.pdf)\n* Not All Neural Embeddings are Born Equal (2014), F. Hill et al. [[pdf]](https://arxiv.org/pdf/1410.0718.pdf)\n* Two/Too Simple Adaptations of Word2Vec for Syntax Problems (2015), W. Ling[[pdf]](http://www.cs.cmu.edu/~lingwang/papers/naacl2015.pdf)\n\n\n**Comparing count-based vs predict-based method**\n\n* Linguistic Regularities in Sparse and Explicit Word Representations (2014), Omer Levy, Yoav Goldberg[[pdf]](https://www.cs.bgu.ac.il/~yoavg/publications/conll2014analogies.pdf)\n* Don’t count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors (2014), M. Baroni [[pdf]](http://www.aclweb.org/anthology/P14-1023)\n* Improving Distributional Similarity with Lessons Learned from Word Embeddings (2015), Omer Levy [[pdf]](http://www.aclweb.org/anthology/Q15-1016)\n\n\n**Evaluation, Analysis**\n\n* Evaluation methods for unsupervised word embeddings (2015), T. Schnabel [[pdf]](http://www.aclweb.org/anthology/D15-1036)\n* Intrinsic Evaluation of Word Vectors Fails to Predict Extrinsic Performance (2016), B. Chiu [[pdf]](https://www.aclweb.org/anthology/W/W16/W16-2501.pdf)\n* Problems With Evaluation of Word Embeddings Using Word Similarity Tasks (2016), M. Faruqui [[pdf]](https://arxiv.org/pdf/1605.02276.pdf)\n* Improving Reliability of Word Similarity Evaluation by Redesigning Annotation Task and Performance Measure (2016), Oded Avraham, Yoav Goldberg [[pdf]](https://arxiv.org/pdf/1611.03641.pdf)\n* Evaluating Word Embeddings Using a Representative Suite of Practical Tasks (2016), N. Nayak [[pdf]](https://cs.stanford.edu/~angeli/papers/2016-acl-veceval.pdf)\n\n### Phrase, Sentence and Document Embeddings\n\n**Sentence**\n\n* [Skip-Thought Vectors](https://arxiv.org/abs/1506.06726)\n* [A Simple but Tough-to-Beat Baseline for Sentence Embeddings](https://openreview.net/forum?id=SyK00v5xx)\n* [An efficient framework for learning sentence representations](https://arxiv.org/abs/1803.02893)\n* [Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning](https://arxiv.org/abs/1804.00079)\n* [Universal Sentence Encoder](https://arxiv.org/abs/1803.11175)\n\n**Document**\n\n* [Distributed Representations of Sentences and Documents](https://arxiv.org/abs/1405.4053)\n\n### Sense Embeddings\n\n* [SENSEMBED: Learning Sense Embeddings for Word and Relational Similarity](http://wwwusers.di.uniroma1.it/~navigli/pubs/ACL_2015_Iacobaccietal.pdf)\n* [Multi-Prototype Vector-Space Models of Word Meaning](http://www.cs.utexas.edu/~ml/papers/reisinger.naacl-2010.pdf)\n\n### Neural Language Models\n\n* [Recurrent neural network based language model](http://www.fit.vutbr.cz/research/groups/speech/publi/2010/mikolov_interspeech2010_IS100722.pdf)\n* [A Neural Probabilistic Language Model](http://www.jmlr.org/papers/volume3/bengio03a/bengio03a.pdf)\n* [Linguistic Regularities in Continuous Space Word Representations](http://www.aclweb.org/anthology/N13-1090)\n\n## Researchers\n\n* [Tomas Mikolov](https://scholar.google.co.jp/citations?user=oBu8kMMAAAAJ\u0026hl=en)\n* [Yoshua Bengio](https://scholar.google.co.jp/citations?user=kukA0LcAAAAJ\u0026hl=en)\n* [Yoav Goldberg](https://scholar.google.co.jp/citations?user=0rskDKgAAAAJ\u0026hl=en)\n* [Omer Levy](https://scholar.google.co.jp/citations?user=PZVd2h8AAAAJ\u0026hl=en)\n* [Kai Chen](https://scholar.google.co.jp/citations?user=TKvd_Z4AAAAJ\u0026hl=en)\n\n## Courses and Lectures\n\n* [CS224d: Deep Learning for Natural Language Processing](http://cs224d.stanford.edu/index.html)\n* [Udacity Deep Learning](https://www.udacity.com/course/deep-learning--ud730)\n\n## Datasets\n### Training\n\n* [Wikipedia](https://dumps.wikimedia.org/enwiki/)\n* [WestburyLab.wikicorp.201004](http://www.socher.org/index.php/Main/ImprovingWordRepresentationsViaGlobalContextAndMultipleWordPrototypes)\n\n### Evaluation\n\n* [SemEval-2012 Task 2](https://www.cs.york.ac.uk/semeval-2012/task2.html)\n* [WordSimilarity-353](http://www.cs.technion.ac.il/~gabr/resources/data/wordsim353/)\n* [Stanford's Contextual Word Similarities (SCWS)](http://www.socher.org/index.php/Main/ImprovingWordRepresentationsViaGlobalContextAndMultipleWordPrototypes)\n* [Stanford Rare Word (RW) Similarity Dataset](http://stanford.edu/~lmthang/morphoNLM/)\n\n### Pre-Trained Language Models\n\nBelow is pre-trained [ELMo](https://arxiv.org/abs/1802.05365) models. Adding ELMo to existing NLP systems significantly improves the state-of-the-art for every considered task.\n\n* [ELMo by AllenNLP](https://allennlp.org/elmo)\n* [ELMo by TensorFlow Hub](https://alpha.tfhub.dev/google/elmo/2)\n\nBelow is pre-trained [sent2vec](https://github.com/epfml/sent2vec) models.\n* [BioSentVec: sent2vec pretrained vector for biomedical text](https://github.com/ncbi-nlp/BioSentVec)\n\n### Pre-Trained Word Vectors\nConvenient downloader for pre-trained word vectors:\n* [chakin](https://github.com/chakki-works/chakin)\n\n\nLinks for pre-trained word vectors:\n* [Word2vec pretrained vector(English Only)](https://code.google.com/archive/p/word2vec/)\n* [Word2vec pretrained vectors for 30+ languages](https://github.com/Kyubyong/wordvectors)\n* [FastText pretrained vectors for 157 languages](https://github.com/facebookresearch/fastText/blob/master/docs/crawl-vectors.md)\n* [FastText pretrained vector for Japanese with NEologd](https://drive.google.com/open?id=0ByFQ96A4DgSPUm9wVWRLdm5qbmc)\n* [word vectors trained by GloVe](http://nlp.stanford.edu/projects/glove/)\n* [Dependency-Based Word Embeddings](https://levyomer.wordpress.com/2014/04/25/dependency-based-word-embeddings/)\n* [Meta-Embeddings](http://cistern.cis.lmu.de/meta-emb/)\n* [Lex-Vec](https://github.com/alexandres/lexvec)\n* [Huang et al. (2012)'s embeddings (HSMN+csmRNN)](http://stanford.edu/~lmthang/morphoNLM/)\n* [Collobert et al. (2011)'s embeddings (CW+csmRNN)](http://stanford.edu/~lmthang/morphoNLM/)\n* [BPEmb: subword embeddings for 275 languages](https://github.com/bheinzerling/bpemb)\n* [Wikipedia2Vec: pretrained word and entity embeddings for 12 languages](https://wikipedia2vec.github.io/wikipedia2vec/pretrained/)\n* [word2vec-slim](https://github.com/eyaler/word2vec-slim)\n* [BioWordVec: fastText pretrained vector for biomedical text](https://github.com/ncbi-nlp/BioSentVec)\n\n\u003c!--\n## Articles\n--\u003e\n\n## Implementations and Tools\n### Word2vec\n\n* [Original](https://code.google.com/archive/p/word2vec/)\n* [gensim](https://radimrehurek.com/gensim/models/word2vec.html)\n* [TensorFlow](https://www.tensorflow.org/versions/r0.12/tutorials/word2vec/index.html)\n\n### GloVe\n\n* [Original](https://github.com/stanfordnlp/GloVe)\n* [GloVe as an optimized TensorFlow GPU Layer using chakin](https://github.com/guillaume-chevalier/GloVe-as-a-TensorFlow-Embedding-Layer)\n\n","funding_links":[],"categories":["Jupyter Notebook","Alternative Awesome","Table of Contents","Other Lists","Awesome lists"],"sub_categories":["no code","TeX Lists","NLP"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHironsan%2Fawesome-embedding-models","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FHironsan%2Fawesome-embedding-models","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHironsan%2Fawesome-embedding-models/lists"}