awesome-parametric-knowledge-in-llms
Must-read papers and blogs about parametric knowledge mechanism in LLMs.
https://github.com/trae1oung/awesome-parametric-knowledge-in-llms
Last synced: 13 days ago
JSON representation
-
Knowledge in Transformer-based Model——Analysis🧠
-
2024
- Knowledge entropy decay during language model pretraining hinders new knowledge acquisition
- When Context Leads but Parametric Memory Follows in Large Language Models
- Understanding the Interplay between Parametric and Contextual Knowledge for Large Language Models
- Evaluating the External and Parametric Knowledge Fusion of Large Language Models
- Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts - NLP-Group/LLM-Knowledge-Conflict)](https://github.com/OSU-NLP-Group/LLM-Knowledge-Conflict)
- What does the knowledge neuron thesis have to do with knowledge?
- Neuron-level knowledge attribution in large language models - attribution)](https://github.com/zepingyu0512/neuron-attribution)
- Dissecting recall of factual associations in auto-regressive language models - research/google-research/tree/master/dissecting_factual_predictions)]
- Knowledge Mechanisms in Large Language Models: A Survey and Perspective
- Disentangling Memory and Reasoning Ability in Large Language Models - Memory-and-Reasoning)](https://github.com/MingyuJ666/Disentangling-Memory-and-Reasoning)
- Linguistic collapse: Neural collapse in (large) language models - collapse)]( https://github.com/rhubarbwu/linguistic-collapse)
-
2021
-
2025
-
-
Knowledge in Transformer-based Model——Gradient Attribution👀
-
2024
- Does Large Language Model contain Task-Specific Neurons.
- Identifying query-relevant neurons in large language models for long-form texts
- Journey to the center of the knowledge neurons: Discoveries of language-independent knowledge neurons and degenerate knowledge neurons
- Revealing the parametric knowledge of language models: A unified framework for attribution methods
-
2022
- Knowledge Neurons in Pretrained Transformers - DDM/knowledge-neurons)](https://github.com/Hunter-DDM/knowledge-neurons)
-
-
Knowledge Editing 🧑⚕️
-
2024
- Forgetting before learning: Utilizing parametric arithmetic for knowledge updating in large language models
- A Comprehensive Study of Knowledge Editing for Large Language Models
- FAME: Towards Factual Multi-Task Model Editing
- To Forget or Not? Towards Practical Knowledge Unlearning for Large Language Models
- Understanding the Collapse of LLMs in Model Editing
- Is it possible to edit large language models robustly?
- Retrieval-enhanced Knowledge Editing in Language Models for Multi-Hop Question Answering
- A Comprehensive Study of Knowledge Editing for Large Language Models
- FAME: Towards Factual Multi-Task Model Editing
- To Forget or Not? Towards Practical Knowledge Unlearning for Large Language Models
- Understanding the Collapse of LLMs in Model Editing
- Is it possible to edit large language models robustly?
- Retrieval-enhanced Knowledge Editing in Language Models for Multi-Hop Question Answering
- Latent paraphrasing: Perturbation on layers improves knowledge injection in language models
- Learning to edit: Aligning LLMs with knowledge editing
- Inspecting and Editing Knowledge Representations in Language Models
-
2022
-
2023
-
2021
-
2020
-
-
Knowledge Transfer🧚♀️
-
2024
- Initializing models with larger ones - selection)](https://github.com/OscarXZQ/weight-selection)
- Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective
- Cross-model Control: Improving Multiple Large Language Models in One-time Training
- Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective
- Cross-model Control: Improving Multiple Large Language Models in One-time Training
- Knowledge fusion of large language models
- Tuning language models by proxy - tuning)](https://github.com/alisawuffles/proxy-tuning)
- Initializing models with larger ones - selection)](https://github.com/OscarXZQ/weight-selection)
-
2023
- Learning to grow pretrained models for efficient transformer training - Group/LiGO)](https://github.com/VITA-Group/LiGO)
- Mutual enhancement of large and small language models with cross-silo knowledge transfer
- Retrieval-based knowledge transfer: An effective approach for extreme large language model compression
- Mutual enhancement of large and small language models with cross-silo knowledge transfer
- Retrieval-based knowledge transfer: An effective approach for extreme large language model compression
- Learning to grow pretrained models for efficient transformer training - Group/LiGO)](https://github.com/VITA-Group/LiGO)
-
2021
- Weight distillation: Transferring the knowledge in neural network parameters - distillation)](https://github.com/Lollipop321/weight-distillation)
- Weight distillation: Transferring the knowledge in neural network parameters - distillation)](https://github.com/Lollipop321/weight-distillation)
-
-
Knowledge Distillation
-
2024
- PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuning - ai/PromptKD)](https://github.com/gmkim-ai/PromptKD)(Note: not parametric)
- From Instance Training to Instruction Learning: Task Adapters Generation from Instructions
- From Instance Training to Instruction Learning: Task Adapters Generation from Instructions
- PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuning - ai/PromptKD)](https://github.com/gmkim-ai/PromptKD)(Note: not parametric)
- When babies teach babies: Can student knowledge sharing outperform teacher-guided distillation on small datasets?
-
-
Pramatric Quantization
-
Knowledge in Transformer-based Model——Activation🫀
-
2024
- Language-specific neurons: The key to multilingual capabilities in large language models.
- Separating tongue from thought: Activation patching reveals language-agnostic concept representations in transformers - lang-agnostic)](https://github.com/Butanium/llm-lang-agnostic)
- From yes-men to truth-tellers Addressing sycophancy in large language models with pinpoint tuning
-
-
Knowledge in Transformer-based Model🧠
-
2024
-
2021
-
2022
- Knowledge Neurons in Pretrained Transformers - DDM/knowledge-neurons)](https://github.com/Hunter-DDM/knowledge-neurons)
-
-
Different Type Neurons in LLMs👀
-
Knowledge Injection
-
2023
-
2022
-
-
Star History
-
2022
- 
-
-
2023
Categories
Knowledge Editing 🧑⚕️
26
Knowledge Transfer🧚♀️
16
Knowledge in Transformer-based Model——Analysis🧠
13
Knowledge in Transformer-based Model——Gradient Attribution👀
5
Knowledge Distillation
5
Knowledge in Transformer-based Model🧠
4
Knowledge Injection
4
Knowledge in Transformer-based Model——Activation🫀
3
Pramatric Quantization
2
2023
1
Different Type Neurons in LLMs👀
1
Knowledge in Transformer-based Model——Causal Tracing🦾
1
Star History
1
2024
1