{"id":26758212,"url":"https://github.com/ksm26/embedding-models-from-architecture-to-implementation","last_synced_at":"2025-07-25T20:34:46.027Z","repository":{"id":252947037,"uuid":"841993855","full_name":"ksm26/Embedding-Models-From-Architecture-to-Implementation","owner":"ksm26","description":"Understand and build embedding models, focusing on word and sentence embeddings, dual encoder architectures. Learn to train embedding models using contrastive loss, implement them in semantic search and RAG systems.","archived":false,"fork":false,"pushed_at":"2024-08-21T12:17:03.000Z","size":2092,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-28T16:18:51.888Z","etag":null,"topics":["ai-applications","ai-architecture","bert","bert-embeddings","bert-fine-tuning","bert-model","contrastive-loss","dual-encoder","embedding-models","machine-learning","model-training","question-answer-retrieval","rag-systems","semantic-search","sentence-embeddings","transformer-models","word-embeddings","word2vec"],"latest_commit_sha":null,"homepage":"https://www.deeplearning.ai/short-courses/embedding-models-from-architecture-to-implementation/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ksm26.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-13T13:08:01.000Z","updated_at":"2024-11-11T09:19:30.000Z","dependencies_parsed_at":"2024-08-19T13:45:19.687Z","dependency_job_id":null,"html_url":"https://github.com/ksm26/Embedding-Models-From-Architecture-to-Implementation","commit_stats":null,"previous_names":["ksm26/embedding-models-from-architecture-to-implementation"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ksm26/Embedding-Models-From-Architecture-to-Implementation","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ksm26%2FEmbedding-Models-From-Architecture-to-Implementation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ksm26%2FEmbedding-Models-From-Architecture-to-Implementation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ksm26%2FEmbedding-Models-From-Architecture-to-Implementation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ksm26%2FEmbedding-Models-From-Architecture-to-Implementation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ksm26","download_url":"https://codeload.github.com/ksm26/Embedding-Models-From-Architecture-to-Implementation/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ksm26%2FEmbedding-Models-From-Architecture-to-Implementation/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267057131,"owners_count":24028784,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-25T02:00:09.625Z","response_time":70,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-applications","ai-architecture","bert","bert-embeddings","bert-fine-tuning","bert-model","contrastive-loss","dual-encoder","embedding-models","machine-learning","model-training","question-answer-retrieval","rag-systems","semantic-search","sentence-embeddings","transformer-models","word-embeddings","word2vec"],"created_at":"2025-03-28T16:18:54.462Z","updated_at":"2025-07-25T20:34:45.973Z","avatar_url":"https://github.com/ksm26.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🧠 [Embedding Models: From Architecture to Implementation](https://www.deeplearning.ai/short-courses/embedding-models-from-architecture-to-implementation/)\n\nWelcome to the \"Embedding Models: From Architecture to Implementation\" course! 🧑‍🏫 The course delves deep into the architecture and capabilities of embedding models, widely used in AI applications to capture the meaning of words and sentences.\n\n## 📘 Course Summary\nIn this course, you’ll explore the evolution of embedding models, from word to sentence embeddings, and build and train a simple dual encoder model. 🧠 The hands-on approach will enable you to grasp the technical concepts behind embedding models and how to effectively use them.\n\n**Detailed Learning Outcomes:**\n1. 🧩 **Embedding Models**: Learn about word embedding, sentence embedding, and cross-encoder models, and how they are utilized in Retrieval-Augmented Generation (RAG) systems.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"images/l1_1.png\" height=\"300\"\u003e \n\u003c/p\u003e\n\n2. 🧠 **Transformer Models**: Understand how transformer models, specifically BERT (Bi-directional Encoder Representations from Transformers), are trained and used in semantic search systems.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"images/l3_1.png\" height=\"300\"\u003e \n\u003cimg src=\"images/l3_2.png\" height=\"300\"\u003e \n\u003c/p\u003e\n\n3. 🏗️ **Dual Encoder Architecture**: Gain knowledge of the evolution of sentence embedding and understand the formation of the dual encoder architecture.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"images/l3_4.png\" height=\"350\"\u003e \n\u003cimg src=\"images/l4_1.png\" height=\"350\"\u003e \n\u003c/p\u003e\n\n4. 🔧 **Training with Contrastive Loss**: Use contrastive loss to train a dual encoder model, with one encoder trained for questions and another for responses.\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"images/l5_1.png\" height=\"200\"\u003e \n\u003cimg src=\"images/l5_2.png\" height=\"200\"\u003e \n\u003c/p\u003e\n\n5. 🔍 **RAG Pipeline**: Utilize separate encoders for questions and answers in a RAG pipeline and observe the differences in retrieval effectiveness compared to a single encoder model.\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"images/l5_3.png\" height=\"450\"\u003e \n\u003cimg src=\"images/l5_4.png\" height=\"450\"\u003e \n\u003c/p\u003e\n\n## 🔑 Key Points\n- 🏛️ **In-depth Understanding**: Gain a deep understanding of embedding model architecture and learn how to train and use them effectively in AI applications.\n- 🧩 **Embedding Models in Practice**: Learn how to apply different embedding models such as Word2Vec and BERT in various semantic search systems.\n- 🏋️ **Dual Encoder Training**: Build and train dual encoder models using contrastive loss to enhance the accuracy of question-answer retrieval applications.\n\n## 👩‍🏫 About the Instructor\n- 👨‍🏫 **Ofer Mendelevitch**: Head of Developer Relations at Vectara, Ofer brings extensive experience in embedding models and their implementation in real-world AI applications.\n\n🔗 To enroll in the course or for further information, visit 📚 [deeplearning.ai](https://www.deeplearning.ai/short-courses/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fksm26%2Fembedding-models-from-architecture-to-implementation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fksm26%2Fembedding-models-from-architecture-to-implementation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fksm26%2Fembedding-models-from-architecture-to-implementation/lists"}