• [Paper - upenn/llm_token_bias)], 2024.6
• [Project
• [Paper
• [Paper
• [Project
• [Paper
• [Project - synergizing-reasoning-and-acting.html)], 2022.10
• [Paper
• [Project - machines/pal)], 2022.11
• [Paper
• [Paper
• [Paper
• [Paper
• [Paper
• [Paper
• [Paper
• [Paper
• [Paper - NLP-SG/contrastive-cot)], 2023.11
• [Paper
• [Paper

<h3 id="lm">Scaling Smaller Language Models to Reason<h3/>

• [Paper
• [Paper
• [Paper
• [Project
• [Project
• [Blog
• [Project - CAIR/MiniGPT-4)], 2023.4
• [Paper
• [Paper
• [Paper
• [Paper - gpt.github.io/)], 2023.7
• [Paper
• [Paper - stanford/med-flamingo)], 2023.7
• [Paper - VL)], 2023.8
• [Paper
• [Paper
• [Paper
• [Project - iep)], 2017.5
• [Paper
• [Paper
• [Project - vid.github.io/#video-demos)], 2023.10
• ARO
• OK-VQA
• A-OKVQA - based visual question answering benchmark.
• NExT-QA
• GQA - world images.
• VQA
• VQAv2
• TAG
• Bongard-HOI - object interactions (HOIs) from natural images.
• ARC - like form of general fluid intelligence.
• [Project - liu/LLaVA)], 2023.10

2023

• Multimodal Chain-of-Thought Reasoning in Language Models. - science/mm-cot)]
• Specializing Smaller Language Models towards Multi-Step Reasoning.
• Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them. - Bench-Hard)]
• Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters. - osu/Understanding-CoT)]
• A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity.
• Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting. - unfaithfulness)]
• Self-consistency improves chain of thought reasoning in language models.
• Ask Me Anything: A simple strategy for prompting language models.
• Language Models are Multilingual Chain-of-Thought Reasoners.
• Mind's Eye: Grounded language model reasoning through simulation.
• Unsupervised Explanation Generation via Correct Instantiations.
• Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks. - of-thoughts)]
• Complementary Explanations for Effective In-Context Learning.
• Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions.
• Rethinking with Retrieval: Faithful Large Language Model Inference.
• Faithful Chain-of-Thought Reasoning.
• Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data. - cot)]
• Tree of Thoughts: Deliberate Problem Solving with Large Language Models. - of-thought-llm)]
• Reasoning Implicit Sentiment with Chain-of-Thought Prompting. - ISA)]
• Reasoning with Language Model is Planning with World Model.
• Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning. - LLM)]
• Boosting LLM Reasoning: Push the Limits of Few-shot Learning with Reinforced In-Context Pruning.
• SatLM: Satisfiability-Aided Language Models Using Declarative Prompting. - lm)]
• Learning Deductive Reasoning from Synthetic Corpus based on Formal Logic. - nlp/FLD)]
• Solving Math Word Problems via Cooperative Reasoning induced Language Models.
• Measuring Faithfulness in Chain-of-Thought Reasoning.
• Faith and Fate: Limits of Transformers on Compositionality.
• Large Language Models Can Be Easily Distracted by Irrelevant Context.
• On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning.
• Recursion of Thought: A Divide and Conquer Approach to Multi-Context Reasoning with Language Models. - lee/RoT)] [[poster](https://soochanlee.com/img/rot/rot_poster.pdf)]
• ART: Automatic multi-step reasoning and tool-use for large language models.
• Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Models.
• LAMBADA: Backward Chaining for Automated Reasoning in Natural Language.
• Large Language Models are Reasoners with Self-Verification. - Verification)]
• Can Retriever-Augmented Language Models Reason? The Blame Game Between the Retriever and the Language Model. - NLP/retriever-lm-reasoning)]
• PAL: Program-aided Language Models. - machines/pal)]
• Large Language Models Can Self-Improve.
• Automatic Chain of Thought Prompting in Large Language Models. - research/auto-cot)]
• Making Large Language Models Better Reasoners with Step-Aware Verifier.
• Least-to-most prompting enables complex reasoning in large language models.
• Distilling Multi-Step Reasoning Capabilities of Large Language Models into Smaller Models via Semantic Decompositions.
• Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models. - llm.github.io/)] [[code](https://github.com/lupantech/chameleon-llm)]
• MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action. - react.github.io/)] [[code](https://github.com/microsoft/MM-REACT)] [[demo](https://huggingface.co/spaces/microsoft-cognitive-service/mm-react)]
• ViperGPT: Visual Inference via Python Execution for Reasoning. - columbia/viper)]
• Visual Programming: Compositional Visual Reasoning without Training.
• Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning.
• Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language. - research/google-research/tree/master/socraticmodels)]
• G-LLaVA: Solving Geometric Problems with Multi-Modal Large Language Model.
• Gemini in Reasoning: Unveiling Commonsense in Multimodal Large Language Models.
• Teaching Small Language Models to Reason.
• Large Language Models Are Reasoning Teachers. - teacher)]
• Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Step.
• Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models. - chatgpt)]

2024

• Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding.
• A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasoners. - upenn/llm_token_bias)]
• Training Language Models to Self-Correct via Reinforcement Learning.
• Large Language Models Cannot Self-Correct Reasoning Yet.
• Language Models as Inductive Reasoners.
• Active Prompting with Chain-of-Thought for Large Language Models. - cot)]
• Question Decomposition Improves the Faithfulness of Model-Generated Reasoning.
• Skeleton-of-Thought: Large Language Models Can Do Parallel Decoding.
• Chain-of-Verification Reduces Hallucination in Large Language Models.
• Teaching Language Models to Self-Improve through Interactive Demonstrations.
• Efficient Tool Use with Chain-of-Abstraction Reasoning.
• Iteration Head: A Mechanistic Study of Chain-of-Thought
• SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities. - vlm.github.io/)]
• The Impact of Reasoning Step Length on Large Language Models.
• REFINER: Reasoning Feedback on Intermediate Representations.
• Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models. - Hu/VisualSketchpad)]
• Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers.
• Let's Verify Step by Step.
• V-STaR: Training Verifiers for Self-Taught Reasoners.
• Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking.
• Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Models. - of-thought-llm)]
• DotaMath: Decomposition of Thought with Code Assistance and Self-correction for Mathematical Reasoning.
• Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents.
• OpenAI o1.
• Enhancing Zero-Shot Chain-of-Thought Reasoning in Large Language Models through Logic. - zhao/LoT)]
• Self-playing Adversarial Language Game Enhances LLM Reasoning.
• At Which Training Stage Does Code Data Help LLM Reasoning?
• To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning.
• Link-Context Learning for Multimodal LLMs. - portal/Link-Context-Learning)]
• Evaluating Mathematical Reasoning Beyond Accuracy.
• Q\*: Improving Multi-step Reasoning for LLMs with Deliberative Planning.
• LLM-ARC: Enhancing LLMs with an Automated Reasoning Critic.
• Premise Order Matters in Reasoning with Large Language Models.
• Do Large Language Models Latently Perform Multi-Hop Reasoning?
• K-Level Reasoning with Large Language Models.
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models.
• Self-Discover: Large Language Models Self-Compose Reasoning Structures.
• InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoning.
• Chain-of-Thought Reasoning Without Prompting.
• GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinements.
• Advancing LLM Reasoning Generalists with Preference Trees.
• Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing.
• MathScale: Scaling Instruction Tuning for Mathematical Reasoning.
• Chart-based Reasoning: Transferring Capabilities from LLMs to VLMs.
• LLM3: Large Language Model-based Task and Motion Planning with Motion Failure Reasoning. - TAMP)]

2022

• Can language models learn from explanations in context?
• Large Language Models are Zero-Shot Reasoners.
• Large Language Models Still Can't Plan. - plan-benchmark)]
• Solving Quantitative Reasoning Problems with Language Models. - solving-quantitative-reasoning.html)]
• Language Models of Code are Few-Shot Commonsense Learners.
• Retrieval Augmentation for Commonsense Reasoning: A Unified Approach.
• Emergent Abilities of Large Language Models. - emergent-phenomena-in.html)]
• Iteratively Prompt Pre-trained Language Models for Chain of Thought. - osu/iterprompt)]
• Scaling Instruction-Finetuned Language Models.
• Chain of Thought Prompting Elicits Reasoning in Large Language Models. - models-perform-reasoning-via.html)]

<h3 id="mllm">🧠 Multimodal Reasoning in Large Language Models<h3/>

• ARB - bench](https://doi.org/10.48550/arXiv.2206.04615) / [AGIEval](https://arxiv.org/abs/2304.06364) / [ALERT](https://arxiv.org/abs/2212.08286) / [CONDAQA](https://arxiv.org/abs/2211.00295) / [SCAN](https://arxiv.org/abs/1711.00350) / [WikiWhy](https://arxiv.org/abs/2210.12152) |
• [Project
• [Project
• here
• [Project - Hu/VisualSketchpad)], 2024.6
• [Project - REACT)] [[Demo](https://huggingface.co/spaces/microsoft-cognitive-service/mm-react)], 2023.3
• [Project - llm)], 2023.4
• [Paper
• GSM8K - main.168) / [ASDiv](https://aclanthology.org/2020.acl-main.92/) / [AQuA](https://aclanthology.org/P17-1015/) / [MAWPS](https://aclanthology.org/N16-1136/) / [AddSub](https://aclanthology.org/D14-1058/) / [MultiArith](https://aclanthology.org/D15-1202/) / [SingleEq](https://aclanthology.org/Q15-1042/) / [SingleOp]( https://doi.org/10.1162/tacl_a_00118) / [Lila](https://arxiv.org/abs/2210.17517) |
• CommonsenseQA
• ReClor
• SCIENCEQA - VQA](https://arxiv.org/abs/1906.00067) / [A-OKVQA](https://arxiv.org/abs/2206.01718) / [NExT-QA](https://arxiv.org/abs/2105.08276) / [GQA](https://arxiv.org/abs/1902.09506) / [VQA](https://arxiv.org/abs/1505.00468) / [VQAv2](https://arxiv.org/abs/1612.00837) / [TAG](https://arxiv.org/abs/2208.01813) / [Bongard-HOI](https://arxiv.org/abs/2205.13803) / [ARC](https://arxiv.org/abs/1911.01547) |
• [Project - research/google-research/tree/master/socraticmodels)], 2022.4
• [Project - columbia/viper)], 2023.3

<h3 id="lm">🤏 Scaling Smaller Language Models to Reason<h3/>

• [Paper

2023

• ThoughtSource - of-thought reasoning in large language models.
• AgentChain
• google/Cascades - inference, and more.
• LogiTorch - based library for logical reasoning on natural language.
• salesforce/LAVIS - stop Library for Language-Vision Intelligence.
• Chain-of-Thought Hub - of-thought prompting.
• facebookresearch/RAM
• LLM Reasoners

<h3 id="mllm">🧠 Multimodal Reasoning in Large Language Models<h3/>

• CoTEVer
• Promptify
• MiniChain
• EasyInstruct - 3 in research experiments.

2023

• Chain-of-ThoughtsPapers - of-Thought Prompting Elicits Reasoning in Large Language Models".
• LM-reasoning
• Prompt4ReasoningPapers
• ReasoningNLP
• Awesome-LLM
• Awesome LLM Self-Consistency - consistency in Large Language Models.
• Deep-Reasoning-Papers - Symbolic Reasoning, Logical Reasoning, and Visual Reasoning.

2022

• Reasoning with Language Model Prompting: A Survey.
• Towards Reasoning in Large Language Models: A Survey. - reasoning)]

2024

• Attention Heads of Large Language Models: A Survey. - Shanghai/Awesome-Attention-Heads)]
• Internal Consistency and Self-Feedback in Large Language Models: A Survey. - Shanghai/ICSFSurvey)]
• Puzzle Solving using Reasoning of Large Language Models: A Survey.
• Large Language Models for Mathematical Reasoning: Progresses and Challenges.

Contributors

• ![Star History Chart - history.com/#atfortes/LLM-Reasoning-Papers&Date)

<h3 id="lm">Scaling Smaller Language Models to Reason<h3/>

• ![Star History Chart - history.com/#atfortes/Awesome-LLM-Reasoning&Timeline)