{"id":51102780,"url":"https://github.com/holasoymalva/llm-glossary","last_synced_at":"2026-06-24T12:03:22.535Z","repository":{"id":355412462,"uuid":"1076321962","full_name":"holasoymalva/llm-glossary","owner":"holasoymalva","description":"Glossary of key concepts in Large Language Models (LLMs), Artificial Intelligence (AI), Natural Language Processing (NLP), and Machine Learning (ML). Clear definitions and resources for developers, researchers, and AI enthusiasts exploring generative language technologies.","archived":false,"fork":false,"pushed_at":"2025-10-15T15:15:15.000Z","size":8,"stargazers_count":6,"open_issues_count":1,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-03T14:39:04.999Z","etag":null,"topics":["ai","artificial-intelligence","artificial-intelligence-algorithms","artificial-intelligence-framework","artificial-intelligence-models","generative-ai","generative-ai-model","glossary","hacktoberfest","large-language-models","llm","llm-agents","llm-evaluation","llm-framework","llm-inference","llm-tools","llm-training","llms","machine-learning-models"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/holasoymalva.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-14T17:39:49.000Z","updated_at":"2026-05-02T22:27:37.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/holasoymalva/llm-glossary","commit_stats":null,"previous_names":["holasoymalva/llm-glossary"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/holasoymalva/llm-glossary","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/holasoymalva%2Fllm-glossary","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/holasoymalva%2Fllm-glossary/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/holasoymalva%2Fllm-glossary/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/holasoymalva%2Fllm-glossary/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/holasoymalva","download_url":"https://codeload.github.com/holasoymalva/llm-glossary/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/holasoymalva%2Fllm-glossary/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34731275,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-24T02:00:07.484Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","artificial-intelligence","artificial-intelligence-algorithms","artificial-intelligence-framework","artificial-intelligence-models","generative-ai","generative-ai-model","glossary","hacktoberfest","large-language-models","llm","llm-agents","llm-evaluation","llm-framework","llm-inference","llm-tools","llm-training","llms","machine-learning-models"],"created_at":"2026-06-24T12:03:21.752Z","updated_at":"2026-06-24T12:03:22.530Z","avatar_url":"https://github.com/holasoymalva.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# LLM Glossary\n\n\u003cdiv align=\"center\"\u003e\n\n**Your essential reference for Large Language Models, AI, NLP, and Machine Learning terminology**\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)\n[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](CONTRIBUTING.md)\n[![Contributions](https://img.shields.io/github/contributors/holasoymalva/llm-glossary.svg)](https://github.com/holasoymalva/llm-glossary/graphs/contributors)\n\n[Explore Terms](#glossary) • [Contributing](#contributing) • [Resources](#additional-resources)\n\n\u003c/div\u003e\n\n---\n\n## Overview\n\nThe LLM Glossary is a comprehensive, community-driven reference guide for understanding the rapidly evolving landscape of Large Language Models and artificial intelligence. Whether you're a developer building AI applications, a researcher exploring cutting-edge techniques, or an enthusiast learning about generative AI, this glossary provides clear, concise definitions for the concepts that matter.\n\n### Why This Glossary?\n\n- **Always Current**: Community-maintained to keep pace with the fast-moving AI field\n- **Practical Focus**: Definitions written for practitioners, not just academics\n- **Cross-Referenced**: Terms link to related concepts for deeper understanding\n- **Resource-Rich**: Each entry includes links to papers, tutorials, and implementations\n\n## Quick Start\n\nBrowse the glossary by category:\n\n- [Core Concepts](#core-concepts) - Foundation terms everyone should know\n- [Model Architectures](#model-architectures) - Transformer variants and neural network designs\n- [Training \u0026 Fine-tuning](#training--fine-tuning) - Methods for optimizing models\n- [Inference \u0026 Deployment](#inference--deployment) - Production considerations\n- [Evaluation \u0026 Benchmarks](#evaluation--benchmarks) - Measuring model performance\n- [Applications \u0026 Use Cases](#applications--use-cases) - Real-world implementations\n\n## Glossary\n\n### Core Concepts\n\n#### Large Language Model (LLM)\nA neural network trained on massive text datasets to understand and generate human-like text. LLMs use deep learning architectures (typically Transformers) with billions to trillions of parameters to capture patterns in language.\n\n**Key characteristics:**\n- Trained on diverse internet-scale data\n- Capable of few-shot and zero-shot learning\n- General-purpose language understanding\n\n**Examples**: GPT-4, Claude, Gemini, LLaMA\n\n**Resources**:\n- [Attention Is All You Need](https://arxiv.org/abs/1706.03762) (Original Transformer paper)\n- [Language Models are Few-Shot Learners](https://arxiv.org/abs/2005.14165) (GPT-3 paper)\n\n---\n\n#### Token\nThe fundamental unit of text processed by language models. Tokens are typically subword pieces that represent common character sequences.\n\n**Common tokenization methods:**\n- **Byte-Pair Encoding (BPE)**: Merges frequently occurring character pairs\n- **WordPiece**: Used by BERT and similar models\n- **SentencePiece**: Language-agnostic tokenization\n\n**Example**:\n```\nInput: \"Tokenization is important\"\nTokens: [\"Token\", \"ization\", \" is\", \" important\"]\n```\n\n**Related**: Context Window, Vocabulary\n\n---\n\n#### Prompt Engineering\nThe practice of designing inputs (prompts) to effectively communicate with and guide LLMs toward desired outputs.\n\n**Key techniques:**\n- **Zero-shot**: Task description without examples\n- **Few-shot**: Including example input-output pairs\n- **Chain-of-Thought (CoT)**: Encouraging step-by-step reasoning\n- **System Prompts**: Setting model behavior and constraints\n\n**Example**:\n```\n# Zero-shot\n\"Translate this to French: Hello, world!\"\n\n# Few-shot\n\"Translate to French:\nEnglish: Hello → French: Bonjour\nEnglish: Thank you → French: Merci\nEnglish: Good morning → French: ?\"\n```\n\n**Resources**:\n- [Prompt Engineering Guide](https://www.promptingguide.ai/)\n\n---\n\n#### Context Window\nThe maximum number of tokens an LLM can process in a single interaction, including both input and output.\n\n**Considerations:**\n- Larger windows enable processing longer documents\n- Computational cost scales quadratically with window size\n- Recent models support 128K+ tokens (≈100K words)\n\n**Examples**:\n- GPT-4 Turbo: 128K tokens\n- Claude 3: 200K tokens\n- Gemini 1.5 Pro: 1M tokens\n\n---\n\n### Model Architectures\n\n#### Transformer\nThe foundational neural network architecture for modern LLMs, introduced in 2017. Uses self-attention mechanisms to process sequences in parallel.\n\n**Key components:**\n- **Self-Attention**: Weighs importance of different tokens\n- **Feed-Forward Networks**: Transforms representations\n- **Positional Encoding**: Captures token position information\n\n**Variants**:\n- **Encoder-only**: BERT, RoBERTa (classification tasks)\n- **Decoder-only**: GPT, LLaMA (text generation)\n- **Encoder-Decoder**: T5, BART (translation, summarization)\n\n---\n\n#### Attention Mechanism\nA technique that allows models to focus on relevant parts of the input when processing each token.\n\n**Types**:\n- **Self-Attention**: Tokens attend to other tokens in same sequence\n- **Cross-Attention**: Tokens attend to separate sequence (e.g., encoder outputs)\n- **Multi-Head Attention**: Parallel attention computations\n\n**Formula**:\n```\nAttention(Q, K, V) = softmax(QK^T / √d_k)V\n```\n\nWhere Q=Query, K=Key, V=Value, d_k=dimension\n\n---\n\n#### Mixture of Experts (MoE)\nAn architecture that uses multiple specialized \"expert\" networks, activating only relevant experts for each input.\n\n**Benefits**:\n- Increases model capacity without proportional compute cost\n- Each expert can specialize in different domains\n- More efficient than dense models at scale\n\n**Examples**: GPT-4, Mixtral, Switch Transformers\n\n---\n\n### Training \u0026 Fine-tuning\n\n#### Pre-training\nThe initial phase where models learn general language understanding from large unlabeled datasets.\n\n**Objectives**:\n- **Causal Language Modeling**: Predict next token (GPT-style)\n- **Masked Language Modeling**: Predict masked tokens (BERT-style)\n- **Denoising**: Reconstruct corrupted text (T5-style)\n\n**Scale**: Trillions of tokens, thousands of GPU-hours\n\n---\n\n#### Fine-tuning\nAdapting a pre-trained model to specific tasks or domains using smaller, task-specific datasets.\n\n**Approaches**:\n- **Full Fine-tuning**: Update all model parameters\n- **Parameter-Efficient Fine-tuning (PEFT)**: Update subset of parameters\n- **Instruction Tuning**: Train on task instructions\n- **RLHF**: Reinforcement Learning from Human Feedback\n\n---\n\n#### Low-Rank Adaptation (LoRA)\nAn efficient fine-tuning technique that adds small, trainable rank decomposition matrices to model layers while keeping original weights frozen.\n\n**Benefits**:\n- Reduces trainable parameters by 10,000x\n- Memory-efficient: multiple adapters can share base model\n- Fast training and switching between tasks\n\n**Formula**:\n```\nW' = W + BA\n```\nWhere W is frozen, B and A are low-rank trainable matrices\n\n**Resources**:\n- [LoRA Paper](https://arxiv.org/abs/2106.09685)\n\n---\n\n#### Reinforcement Learning from Human Feedback (RLHF)\nTraining approach that uses human preferences to align model outputs with desired behaviors.\n\n**Process**:\n1. Collect human comparisons of model outputs\n2. Train reward model to predict human preferences\n3. Use PPO/DPO to optimize policy against reward model\n\n**Impact**: Critical for models like ChatGPT, Claude, Gemini\n\n---\n\n### Inference \u0026 Deployment\n\n#### Quantization\nReducing model precision (e.g., from 32-bit to 8-bit or 4-bit) to decrease memory usage and increase inference speed.\n\n**Methods**:\n- **Post-Training Quantization (PTQ)**: Quantize after training\n- **Quantization-Aware Training (QAT)**: Train with quantization in mind\n- **GPTQ**: Optimal quantization for generative models\n- **GGUF**: Efficient format for local inference\n\n**Trade-offs**: Lower precision can reduce quality but enables deployment on consumer hardware\n\n---\n\n#### Temperature\nA hyperparameter controlling randomness in text generation.\n\n**Scale**:\n- **Low (0.1-0.5)**: Deterministic, focused outputs\n- **Medium (0.7-0.8)**: Balanced creativity and coherence\n- **High (1.0+)**: More random and creative outputs\n\n**Implementation**: Divides logits before softmax\n\n---\n\n#### Top-k and Top-p Sampling\nTechniques to constrain token selection during generation.\n\n**Top-k**: Sample from k most probable tokens\n**Top-p (Nucleus)**: Sample from tokens comprising top p probability mass\n\n**Best practice**: Often use together with temperature for quality control\n\n---\n\n#### Model Serving\nInfrastructure and techniques for deploying LLMs in production.\n\n**Frameworks**:\n- **vLLM**: High-throughput serving with PagedAttention\n- **Text Generation Inference (TGI)**: HuggingFace's serving solution\n- **TensorRT-LLM**: NVIDIA's optimized serving\n- **Ollama**: Local model serving\n\n**Optimizations**: Batching, KV-cache, speculative decoding\n\n---\n\n### Evaluation \u0026 Benchmarks\n\n#### Perplexity\nA measurement of how well a probability model predicts a sample. Lower perplexity indicates better model performance.\n\n**Formula**:\n```\nPPL = exp(-1/N Σ log P(token_i))\n```\n\n**Note**: Best for comparing models, not absolute quality assessment\n\n---\n\n#### MMLU (Massive Multitask Language Understanding)\nBenchmark measuring knowledge across 57 subjects including STEM, humanities, and social sciences.\n\n**Evaluation**: Multiple-choice questions, reports accuracy percentage\n\n---\n\n#### HumanEval\nCoding benchmark measuring ability to generate functionally correct Python code from docstrings.\n\n**Metric**: pass@k - percentage of problems solved with k attempts\n\n---\n\n### Applications \u0026 Use Cases\n\n#### Retrieval-Augmented Generation (RAG)\nArchitecture combining LLMs with external knowledge retrieval to ground responses in factual information.\n\n**Architecture**:\n1. **Retrieval**: Find relevant documents using vector search\n2. **Augmentation**: Add retrieved context to prompt\n3. **Generation**: LLM generates response using context\n\n**Benefits**: Reduces hallucinations, enables up-to-date information, domain-specific knowledge\n\n**Tools**: LangChain, LlamaIndex, Haystack\n\n---\n\n#### Function Calling / Tool Use\nCapability allowing LLMs to invoke external functions or APIs with structured parameters.\n\n**Use cases**:\n- Database queries\n- API integrations\n- Calculator functions\n- Web searches\n\n**Example**:\n```json\n{\n  \"name\": \"get_weather\",\n  \"parameters\": {\n    \"location\": \"San Francisco\",\n    \"unit\": \"celsius\"\n  }\n}\n```\n\n---\n\n#### Agents\nAutonomous systems that use LLMs for reasoning and decision-making to accomplish complex tasks.\n\n**Components**:\n- **Planning**: Break down goals into steps\n- **Memory**: Maintain context across interactions\n- **Tools**: Access to external capabilities\n- **Reflection**: Self-evaluation and improvement\n\n**Frameworks**: AutoGPT, BabyAGI, LangGraph, CrewAI\n\n---\n\n#### Embeddings\nDense vector representations of text that capture semantic meaning.\n\n**Applications**:\n- Semantic search\n- Clustering and classification\n- Recommendation systems\n- RAG retrieval\n\n**Models**: OpenAI text-embedding-ada-002, Cohere Embed, Sentence-BERT\n\n---\n\n## Contributing\n\nWe welcome contributions from the community! Here's how you can help:\n\n### Adding New Terms\n\n1. Fork the repository\n2. Create a new branch (`git checkout -b add-new-term`)\n3. Add your term following the established format:\n   ```markdown\n   #### Term Name\n   Clear, concise definition (1-2 sentences)\n   \n   **Key points**:\n   - Important detail 1\n   - Important detail 2\n   \n   **Examples/Resources**: Links to papers or implementations\n   ```\n4. Submit a pull request\n\n### Guidelines\n\n- **Clarity First**: Definitions should be understandable to practitioners\n- **Cite Sources**: Link to original papers or authoritative resources\n- **Stay Current**: Update outdated information\n- **Be Concise**: Respect readers' time\n- **Cross-Reference**: Link related terms\n\n## Additional Resources\n\n### Learning Paths\n\n**For Beginners**:\n- [Fast.ai Practical Deep Learning](https://course.fast.ai/)\n- [Andrej Karpathy's Neural Networks: Zero to Hero](https://karpathy.ai/zero-to-hero.html)\n\n**For Practitioners**:\n- [Hugging Face Course](https://huggingface.co/learn)\n- [Full Stack LLM Bootcamp](https://fullstackdeeplearning.com/)\n\n**For Researchers**:\n- [Papers with Code](https://paperswithcode.com/)\n- [Arxiv Sanity](https://arxiv-sanity-lite.com/)\n\n### Community\n\n- [Hugging Face Discord](https://discord.gg/huggingface)\n- [r/LocalLLaMA](https://reddit.com/r/LocalLLaMA)\n- [EleutherAI Discord](https://discord.gg/eleutherai)\n\n### Tools \u0026 Frameworks\n\n- **Training**: PyTorch, JAX, DeepSpeed, Megatron-LM\n- **Inference**: vLLM, TGI, llama.cpp, Ollama\n- **Applications**: LangChain, LlamaIndex, Semantic Kernel\n- **Evaluation**: lm-evaluation-harness, HELM\n\n## Roadmap\n\n- [ ] Add interactive search functionality\n- [ ] Create visual diagrams for complex concepts\n- [ ] Develop multilingual versions\n- [ ] Build companion API for programmatic access\n- [ ] Add video explanations for key terms\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## Acknowledgments\n\nBuilt with contributions from developers, researchers, and AI enthusiasts worldwide. Special thanks to:\n\n- The open-source AI community\n- Papers with Code for inspiration\n- All our contributors\n\n---\n\n\u003cdiv align=\"center\"\u003e\n\n**Star this repo if you find it helpful! ⭐**\n\nMade with ❤️ by the AI community\n\n[Report Issue](https://github.com/holasoymalva/llm-glossary/issues) • [Request Feature](https://github.com/holasoymalva/llm-glossary/issues) • [Discuss](https://github.com/holasoymalva/llm-glossary/discussions)\n\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fholasoymalva%2Fllm-glossary","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fholasoymalva%2Fllm-glossary","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fholasoymalva%2Fllm-glossary/lists"}