https://github.com/nihar3453/llm-transformers-and-rag

A hands-on suite for exploring and fine-tuning foundation models (Transformers, BERT, GPT-2, BART) and end-to-end RAG pipelines with attention visualizations, semantic search (ChromaDB/Weaviate), LangChain workflow demos.
https://github.com/nihar3453/llm-transformers-and-rag

bart bert chromadb chunking generative-ai hnsw huggingface-transformers langchain llms minigpt rag transformers vector-database weav

Last synced: 2 months ago
JSON representation

A hands-on suite for exploring and fine-tuning foundation models (Transformers, BERT, GPT-2, BART) and end-to-end RAG pipelines with attention visualizations, semantic search (ChromaDB/Weaviate), LangChain workflow demos.

• Host: GitHub
• URL: https://github.com/nihar3453/llm-transformers-and-rag
• Owner: Nihar3453
• Created: 2025-10-07T09:22:05.000Z (8 months ago)
• Default Branch: main
• Last Pushed: 2025-10-07T09:49:41.000Z (8 months ago)
• Last Synced: 2025-10-07T11:42:20.483Z (8 months ago)
• Topics: bart, bert, chromadb, chunking, generative-ai, hnsw, huggingface-transformers, langchain, llms, minigpt, rag, transformers, vector-database, weav
• Language: Jupyter Notebook
• Homepage:
• Size: 290 KB
• Stars: 0
• Watchers: 0
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# Foundation Models and RAG Labs

A collection of end-to-end notebooks exploring decoder-only Transformers from scratch, BERT fine-tuning across frameworks, multi-model fine-tuning with Hugging Face, and practical semantic search/RAG pipelines with ChromaDB and Weaviate.

What’s inside:- 

-mini-transformer-decoder.ipynb

Builds a GPT-style, decoder-only Transformer from scratch: scaled dot-product attention, multi-head attention, sinusoidal positional encodings, masked self-attention blocks, a minimal training loop on toy text, and attention map visualizations.

-BERT_TensorFlow_vs_HuggingFace_Comparison.ipynb

Implements BERT text classification with both TensorFlow Hub and Hugging Face, using the same dataset/splits/epochs, then compares accuracy, macro-F1, confusion matrices, runtime, GPU memory, sequence lengths, and implementation effort.

-Finetune_BERT_GPT2_BART.ipynb

Loads and fine-tunes three families with Hugging Face: BERT for sentiment classification (IMDB), GPT-2 for language modeling and generation (Wikitext-2), and BART for summarization (CNN/DailyMail); includes short demo training and inference pipelines.

-LangChain_Primitives_and_JSON_Parsing.ipynb

Sets up an LLM API key flow, demonstrates LangChain model primitives and PromptTemplates, builds prompt→model chains, parses plain text and structured JSON (JsonOutputParser, PydanticOutputParser, OutputFixingParser), and contrasts temperature 0.1 vs 0.9 outputs.

-chromadb_weaviate_semantic_search.ipynb

Two semantic search tracks over the same mini corpus: ChromaDB local collection with sentence-transformer embeddings and cosine distance; Weaviate Cloud collection with manual vectorization, HNSW indexing, and semantic queries; includes multi-query evaluation.

-text_chunking_hnsw_rag_pipeline.ipynb

End-to-end RAG lab featuring multiple chunking strategies (fixed, sentence, paragraph, sliding window, heading-based, semantic, hybrid), FAISS HNSW retrieval, a small embedding model, a simple LLM client, and an interactive RAG loop with qualitative comparisons.