Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
Awesome-MLLM-Hallucination
📖 A curated list of resources dedicated to hallucination of multimodal large language models (MLLM).
https://github.com/showlab/Awesome-MLLM-Hallucination
Last synced: 3 days ago
JSON representation
-
Table of Contents <!-- omit in toc -->
-
Hallucination Evaluation & Analysis
- ![Star - group/moh)
- BEAF: Observing BEfore-AFter Changes to Evaluate Hallucination in Vision-language Models
- ![Star - ami/BEAF)
- Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Models
- Hallu-PI: Evaluating Hallucination in Multi-modal Large Language Models within Perturbed Inputs
- ![Star - PI)
- HaloQuest: A Visual Hallucination Dataset for Advancing Multimodal Reasoning
- Multi-Object Hallucination in Vision-Language Models
- Understanding Multimodal Hallucination with Parameter-Free Representation Alignment
- ![Star - binary-tree/Pfram)
- Pre-Training Multimodal Hallucination Detectors with Corrupted Grounding Data
- MetaToken: Detecting Hallucination in Image Descriptions by Meta Classification
- THRONE: An Object-based Hallucination Benchmark for the Free-form Generations of Large Vision-Language Models
- VALOR-EVAL: Holistic Coverage and Faithfulness Evaluation of Large Vision-Language Models
- ALOHa: A New Measure for Hallucination in Captioning Models
- Unsolvable Problem Detection: Evaluating Trustworthiness of Vision Language Models
- IllusionVQA: A Challenging Optical Illusion Dataset for Vision Language Models
- PhD: A Prompted Visual Hallucination Evaluation Dataset
- Visual Hallucinations of Multi-modal Large Language Models
- Behind the Magic, MERLIM: Multi-modal Evaluation Benchmark for Large Image-Language Models
- Negative Object Presence Evaluation (NOPE) to Measure Object Hallucination in Vision-Language Models
- CIEM: Contrastive Instruction Evaluation Method for Better Instruction Tuning
- Diagnosing Event Hallucinations in Video LLMs
- ![Star
- Autonomous Hallucination Evaluation in Vision-Language Models with Davidson Scene Graphs
- Explore the Hallucination on Low-level Perception for MLLMs
- Open-Set Evaluation of Hallucinations in Multimodal Large Language Models
- VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models
- ![Star - tssn/VideoHallucer)
- Evaluating the Quality of Hallucination Benchmarks for Large Vision-Language Models
- ![Star
- Does Object Grounding Really Reduce Hallucination of Large Vision-Language Models?
- Detecting Hallucination in Large Vision-Language Model via Uncertainty Estimation
- Hierarchical Polling-based Probing Evaluation of Hallucinations in Large Vision-Language Models
- Unified Triplet-Level Hallucination Evaluation for Large Vision-Language Models
- ![Star - HE)
- A Cross-Modal Hallucination Benchmark for Audio-Visual Large Language Models
- The Curse of Multi-Modalities: Evaluating Hallucinations of Large Multimodal Models across Language, Visual, and Audio
- ![Star - NLP-SG/CMM)
- Automatically Generating Visual Hallucination Test Cases for Multimodal Large Language Models
- ![Star
- Long-Context Hallucination Evaluation for MultiModal Large Language Models
- ![Star - hq/LongHalQA)
- AUTOHALLUSION: Automatic Generation of Hallucination Benchmarks for Vision-Language Models
- Detecting and Evaluating Medical Hallucinations in Large Vision Language Models
- Do CLIPs Always Generalize Better than ImageNet Models ?
- AIGCs Confuse AI Too: Investigating and Explaining Synthetic Image-induced Hallucinations in Large Vision-Language Models
- Hal-Eval: A Universal and Fine-grained Hallucination Evaluation Framework for Large Vision Language Models
- How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Prompts
- Unified Hallucination Detection for Multimodal Large Language Models
- The Instinctive Bias: Spurious Images lead to Hallucination in MLLMs
- Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs
- HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination & Visual Illusion in Large Vision-Language Models
- An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation
- Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference Challenges
- FAITHSCORE: Evaluating Hallucinations in Large Vision-Language Models
- Evaluation and Analysis of Hallucination in Large Vision-Language Models
- Evaluating Object Hallucination in Large Vision-Language Models
- Object Hallucination in Image Captioning
- Who Brings the Frisbee: Probing Hidden Hallucination Factors in Large Vision-Language Model via Causality Analysis
- Beyond Logit Lens: Contextual Embeddings for Robust Hallucination Detection & Grounding in VLMs
- DHCP: Detecting Hallucinations by Cross-modal Attention Pattern in Large Vision-Language Models
- VidHalluc: Evaluating Temporal Hallucinations in Multimodal Large Language Models for Video Understanding
- Benchmarking Temporal Hallucinations in Vision LLMs
- ![Star
- Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens
- ![Star - 061X/VL-Uncertainty)
- VGA: Vision GUI Assistant -- Minimizing Hallucinations through Image-Centric Fine-Tuning
- Do More Details Always Introduce More Hallucinations in LVLM-based Image Captioning?
- MFC-Bench: Benchmarking Multimodal Fact-Checking with Large Vision-Language Models
-
Hallucination Mitigation
- Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models
- ![Star
- Alleviating Hallucination in Large Vision-Language Models with Active Retrieval Augmentation
- Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs
- ![Star
- BACON: Supercharge Your VLM with Bag-of-Concept Graph to Mitigate Hallucinations
- ![Star
- Pelican: Correcting Hallucination in Vision-LLMs via Claim Decomposition and Program of Thought Verification
- Interpreting and Mitigating Hallucination in MLLMs through Multi-agent Debate
- ![Star
- VACoDe: Visual Augmented Contrastive Decoding
- Reflective Instruction Tuning: Mitigating Hallucinations in Large Vision-Language Models
- Investigating and Mitigating the Multimodal Hallucination Snowballing in Large Vision-Language Models
- ![Star
- Mitigating Hallucination in Visual-Language Models via Re-Balancing Contrastive Decoding
- Reminding Multimodal Large Language Models of Object-aware Knowledge with Retrieved Tags
- ![arXiv
- Look, Compare, Decide: Alleviating Hallucination in Large Vision-Language Models via Multi-View Multi-Path Reasoning
- ![Star
- Contrastive Decoding with Hallucination Visualization for Mitigating Hallucinations in Multimodal Large Language Models
- ![Star - m/ConVis)
- Vision-Language Models as a Source of Preference for Fixing Hallucinations in LVLMs
- Mitigating Hallucination in Visual Language Models with Visual Supervision
- Visual Fact Checker: Enabling High-Fidelity Detailed Caption Generation
- CODE: Contrasting Self-generated Description to Combat Hallucination in Large Multi-modal Models
- NoiseBoost: Alleviating Hallucination with Noise Perturbation for Multimodal Large Language Models
- Mitigating Object Hallucination via Data Augmented Contrastive Tuning
- Don't Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Models
- ![Star
- RLAIF-V: Aligning MLLMs through Open-Source AI Feedback for Super GPT-4V Trustworthiness
- ![Star - V/RLAIF-V)
- Alleviating Hallucinations in Large Vision-Language Models through Hallucination-Induced Optimization
- VDGD: Mitigating LVLM Hallucinations in Cognitive Prompts by Bridging the Visual Perception Gap
- List Items One by One: A New Data Source and Learning Paradigm for Multimodal LLMs
- ![Star - LLaVA)
- Cantor: Inspiring Multimodal Chain-of-Thought of MLLM
- Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback
- Fact :Teaching MLLMs with Faithful, Concise and Transferable Rationales
- Self-Supervised Visual Preference Alignment
- Prescribing the Right Remedy: Mitigating Hallucinations in Large Vision-Language Models via Targeted Instruction Tuning
- FGAIF: Aligning Large Vision-Language Models with Fine-grained AI Feedback
- Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding
- Exploiting Semantic Reconstruction to Mitigate Hallucinations in Vision-Language Models
- Pensieve: Retrospect-then-Compare Mitigates Visual Hallucination
- Multi-Modal Hallucination Control by Visual Information Grounding
- What if...?: Counterfactual Inception to Mitigate Hallucination Effects in Large Multimodal Models
- Mitigating Dialogue Hallucination for Large Multi-modal Models via Adversarial Instruction Tuning
- HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding
- IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding
- Seeing is Believing: Mitigating Hallucination in Large Vision-Language Models via CLIP-Guided Decoding
- Less is More: Mitigating Multimodal Hallucination from an EOS Decision Perspective
- Logical Closed Loop: Uncovering Object Hallucinations in Large Vision-Language Models
- Aligning Modalities in Vision Large Language Models via Preference Fine-tuning
- ![Star
- EFUF: Efficient Fine-grained Unlearning Framework for Mitigating Hallucinations in Multimodal Large Language Models
- Visually Dehallucinative Instruction Generation: Know What You Don't Know
- Mitigating Object Hallucination in Large Vision-Language Models via Classifier-Free Guidance
- Skip \n: A Simple Method to Reduce Hallucination in Large Vision-Language Models
- ViGoR: Improving Visual Grounding of Large Vision Language Models with Fine-Grained Reward Modeling
- Enhancing Multimodal Large Language Models with Vision Detection Models: An Empirical Study
- Silkie: Preference Distillation for Large Visual Language Models
- Hallucination Augmented Contrastive Learning for Multimodal Large Language Model
- Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewrites
- RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback
- OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation
- Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding
- Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization
- HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data
- Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision
- ![Star
- Woodpecker: Hallucination Correction for Multimodal Large Language Models
- Analyzing and Mitigating Object Hallucination in Large Vision-Language Models
- Aligning Large Multimodal Models with Factually Augmented RLHF
- VIGC: Visual Instruction Generation and Correction
- Detecting and Preventing Hallucinations in Large Vision Language Models
- Mitigating Hallucination of LVLMs by Hierarchical Feedback Learning with Vision-enhanced Penalty Decoding
- A Unified Hallucination Mitigation Framework for Large Vision-Language Models
- Effectively Enhancing Vision Language Large Models by Prompt Augmentation and Caption Utilization
- HallE-Switch: Controlling Object Hallucination in Large Vision Language Models
- Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models
- ![Star - Xu-666/visual_inference_chain)
- Seeing Clearly by Layer Two: Enhancing Attention Heads to Alleviate Hallucination in LVLMs
- Mitigating Hallucination in Multimodal Large Language Model via Hallucination-targeted Direct Preference Optimization
- Mitigating Hallucination in Large Vision Language Models via Vision-Guided Direct Preference Optimization
- ![Star - DPO)
- Reducing Hallucinations in Vision-Language Models via Latent Space Steering
- Mitigating Object Hallucination via Concentric Causal Attention
- ![Star - llava)
- Mitigating Hallucinations in Large Vision-Language Models via Summary-Guided Decoding
- Magnifier Prompt: Tackling Multimodal Hallucination via Extremely Simple Instructions
- MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation
- ![Star
- Norm Voting off Hallucinations with Attention Heads in Large Language Models
- From Pixels to Tokens: Revisiting Object Hallucinations in Large Vision-Language Models
- Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality
- Looking Beyond Text: Reducing Language bias in Large Vision-Language Models via Multimodal Dual-Attention and Soft-Image Guidance
- Complementary Adaptive Token-level Contrastive Decoding to Mitigate Hallucinations in LVLMs
- Thinking Before Looking: Improving Multimodal LLM Reasoning via Mitigating Visual Hallucination
- ![Star - Martyr/CausalMM)
- Dive into the Attention Mechanism of LVLM to Reduce Object Hallucination
- Investigating and Mitigating Object Hallucinations in Pretrained Vision-Language (CLIP) Models
- ![Star - Liu/clip_hallucination)
- Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations
- MedThink: Inducing Medical Large-scale Visual Language Models to Hallucinate Less by Thinking More
- AGLA: Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention
- RITUAL: Random Image Transformations as a Universal Anti-hallucination Lever in LVLMs
- ![Star
- MOCHa: Multi-Objective Reinforcement Mitigating Caption Hallucinations
- Mitigating the Hallucination of Large Vision Language Models by Visual Layer Fusion Contrastive Decoding
-
Hallucination Survey
-
Hallucination Benchmarks
-