Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome-LLMs-Evaluation-Papers

The papers are organized according to our survey: Evaluating Large Language Models: A Comprehensive Survey.
https://github.com/tjunlp-lab/Awesome-LLMs-Evaluation-Papers

[Paper - DT-Code/Core-Competency-Test-for-the-Evaluation-of-LLMs)]
[Paper - eval-survey)]
[Paper
[Paper - deepmind/narrativeqa)]
[Paper
[Paper
[Paper - research-datasets/natural-questions)]
[Paper
[Paper
[Paper
[Paper
- orange)
[Paper
[Paper
[Paper - solvers)]
[Paper
[Paper
[Paper
[Paper - research-datasets/timedial)]
[Paper
[Paper
[Paper - Semantic-Plausibility-NAACL18/)]
[Paper
[Paper
[Paper
[Paper - evaluation)]
[Paper
[Paper
[Paper - mll/multiNLI/)]
[Paper
[Paper - dataset)]
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper - dataset)]
[Paper
[Paper
[Paper - research/google-research/tree/master/logic_inference_dataset)]
[Paper - LILY/FOLIO)]
- green)
[Paper
[Paper - evaluation)]
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper - evaluation)]
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper - school-math)]
[Paper - lab/M3KE)]
[Paper
[Paper
[Paper - iitd/jeebench)]
[Paper - llm)]
[Paper
[Paper
[Paper
- orange)
[Paper - Song793/RestGPT)]
- orange)
[Paper - can.github.io/)]
- orange)
[Paper
[Paper - nlp/WebShop)] [[Source](https://webshop-pnlp.github.io/)]
- orange)
[Paper
[Paper
[Paper - chen/ToolQA)]
[Paper - pytorch)]
- orange)
[Paper
[Paper
- orange)
[Paper
[Paper
[Paper - llm-benchmark)]
[Paper
- green)
[Paper
[Paper
[Paper - rlhf-pytorch)]
[Paper
[Paper - research/google-research/tree/master/code_as_policies)]
[Paper
- orange)
[Paper
[Paper - inf.mpg.de/)]
- green)
[Paper - zh15/NeQA)]
- green)
[Paper
[Paper
- green)
[Paper
- green)
[Paper - zentroa/This-is-not-a-Dataset)][[Source](https://huggingface.co/datasets/HiTZ/This-is-not-a-dataset)]
[Paper
[Paper - chemistry-101)]
[Paper
[Paper
[Paper
[Paper - NLP/mic)]
[Paper
[Paper - dialog)]
[Paper
[Paper - KEG/EvaluationPapers4ChatGPT)]
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper - SA.html)]
[Paper - Understanding-Gender-Bias-in-Neural-Relation-Extraction)]
[Paper - ml-bias-analysis)]
[Paper
[Paper
[Paper
[Paper - salt/implicit-hate)]
[Paper - Generated-Hate-Speech-Dataset)]
[Paper - data)]
[Paper - zhou/CDial-Bias)]
[Paper - PM)]
[Paper - rottger/hatecheck-data)]
[Paper
[Paper - mll/crows-pairs)] [[Source](https://huggingface.co/datasets/crows_pairs)]
- orange)
[Paper
- orange)
[Paper - science/bold)] [[Source](https://huggingface.co/datasets/AlexaAI/bold)]
- orange)
[Paper
[Paper
[Paper
[Paper - mll/bbq)]
[Paper
[Paper - LIT/MultilingualBias)]
[Paper
[Paper
[Paper
[Paper
[Paper - biases)]
[Paper
[Paper
- green)
[Paper
[Paper
[Paper
[Paper - br)]
[Paper
[Paper
[Paper - NLP/chain-of-thought-bias)]
[Paper
- green)
[Paper
[Paper
[Paper
[Paper - bench/tree/main/bigbench/benchmark_tasks/known_unknowns)]
[Paper
[Paper
[Paper
[Paper
- orange)
[Paper - squared)]
[Paper - dataset)]
- orange)
[Paper
- orange)
[Paper - research-datasets/xsum_hallucination_annotations)]
- orange)
[Paper - zh15/FactCCX)]
[Paper - LILY/SummEval)]
[Paper
- orange)
[Paper
- orange)
[Paper
[Paper - datasets)]
[Paper
[Paper
[Paper
[Paper
[Paper - three/fib)]
[Paper - NLP/factool)]
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper - NLP/FaithDial)]
[Paper - LieDetector)]
[Paper - Instruction)]
- green)
[Paper - lab/HallusionBench)]
[Paper
[Paper
[Paper - coai/Safety-Prompts)] [[Source](http://coai.cs.tsinghua.edu.cn/leaderboard/)]
[Paper
- orange)![](https://img.shields.io/badge/Rearch-green)
[Paper - sys/FastChat/tree/main/fastchat/llm_judge)]
[GitHub
[Paper
- orange)
[Github
[Paper
- green)
[Paper - Eval/FairEval)]
[Paper
[Paper
[Paper
[Paper - instruct)]
[Paper
[Paper
[Paper
[Paper
[Paper - science/recode)]
[Paper - formatted-problems)]
[Paper - math)]
[Paper
[Paper
[Paper
- green)
[Paper
[Paper
[Paper
[Paper - Waver-In-Judgements)]
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper
[Paper
- blue)
[Paper - bench)] [[Source](https://xingyaoww.github.io/mint-bench/)]
[Paper
- blue)
[Paper - nlp/intercode)] [[Source](https://intercode-benchmark.github.io/)]
- orange)![](https://img.shields.io/badge/Rearch-green)
[Paper
[Paper
[Paper
[Paper
- green)
[Paper
- green)
[Paper
[Paper
- green)
[Paper
[Paper - reasoning)]
[Paper
[Paper
[Paper
- green)
[Paper - teacher-test)]
- green)
[Paper - shot-teacher-feedback)]
- green)
[Paper - LLM-Studies/#/)]
- green)
[Paper
- orange)![](https://img.shields.io/badge/Rearch-green)
[Paper - passes-the-bar)]
- green)
[Paper
- green)
[Paper - takes-the-bar-exam)]
- green)
[Paper
- green)
[Paper
- green)
[Paper
[Paper
[Paper - statutes)]
- green)
[Paper - compass/LawBench)]
- green)
[Paper - CompEval-Legal)]
[Paper - LMs)]
[Paper
[Paper
- blue)
[Paper - nlp/swe-bench)] [[Source](https://swe-bench.github.io/)]
- blue)
[Paper - nlp/intercode)] [[Source](https://intercode-benchmark.github.io/)]
[Paper - benchmark.github.io/)]
[Paper
[Paper
[Paper
[Paper
- yellow)
[Paper
- yellow)
[Paper
[Paper
[Paper
[Paper
- yellow)
[Paper
[Paper - lab/M3KE)]
[Paper - li/CMMLU)]
[Paper
[Paper - NLP-SG/M3Exam)]
- yellow)
[Paper - Eval)]
[Paper - bench)]
[GitHub
- yellow)
[Paper - crfm/helm)]
[GitHub - blue)
- green)
[Paper - Fathom/GPT-Fathom)]
[Paper - lab/InstructEvalImpact)] [[GitHub](https://github.com/declare-lab/instruct-eval)]
[Source - yellow)
- orange)
[Paper - sys/FastChat/tree/main/fastchat/llm_judge)]
[Source - yellow)
[Source - compass/opencompass)]
- yellow)
[Paper - Lab/CLEVA)]
[Source - yellow)
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source
[Source

Programming Languages

Python 2 Jupyter Notebook 2

Keywords

rlhf 2 nlp 2 leaderboard 2 large-language-models 2 instruction-following 2 foundation-models 2 evaluation 2 deep-learning 2 llm 1 hallucinations 1 generative-ai 1 transformer 1 language-model 1 evaluation-framework 1