{"id":18109121,"url":"https://open-compass.github.io/CriticEval/","last_synced_at":"2025-03-29T15:30:39.199Z","repository":{"id":224031739,"uuid":"761535311","full_name":"open-compass/CriticEval","owner":"open-compass","description":"[NeurIPS 2024] A comprehensive benchmark for evaluating critique ability of LLMs","archived":false,"fork":false,"pushed_at":"2024-11-29T02:38:26.000Z","size":51529,"stargazers_count":39,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-13T16:11:51.409Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2402.13764","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/open-compass.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-22T02:24:06.000Z","updated_at":"2025-02-23T19:40:04.000Z","dependencies_parsed_at":null,"dependency_job_id":"bd071a53-c288-416e-baf7-52594ffa5e83","html_url":"https://github.com/open-compass/CriticEval","commit_stats":null,"previous_names":["open-compass/criticbench","open-compass/criticeval"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/open-compass%2FCriticEval","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/open-compass%2FCriticEval/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/open-compass%2FCriticEval/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/open-compass%2FCriticEval/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/open-compass","download_url":"https://codeload.github.com/open-compass/CriticEval/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246204412,"owners_count":20740304,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-01T00:01:34.857Z","updated_at":"2025-03-29T15:30:39.190Z","avatar_url":"https://github.com/open-compass.png","language":"Python","funding_links":[],"categories":["Model Ranking"],"sub_categories":["Text"],"readme":"# CriticEval: Evaluating Large Language Models as Critic\n\nThis repository is the official implementation of [CriticEval](https://arxiv.org/abs/2402.13764), a comprehensive benchmark for evaluating critique ability of LLMs.\n\n## Introduction\n\n**[CriticEval: Evaluating Large Language Models as Critic](https://arxiv.org/abs/2402.13764)** \n\u003c/br\u003e\nTian Lan\u003csup\u003e1*\u003c/sup\u003e,\nWenwei Zhang\u003csup\u003e2*\u003c/sup\u003e,\nChen Xu\u003csup\u003e1\u003c/sup\u003e,\nHeyan Huang\u003csup\u003e1\u003c/sup\u003e,\nDahua Lin\u003csup\u003e2\u003c/sup\u003e,\nKai Chen\u003csup\u003e2†\u003c/sup\u003e,\nXian-ling Mao\u003csup\u003e1†\u003c/sup\u003e\n(\u003csup\u003e†\u003c/sup\u003e Corresponding Author, \u003csup\u003e*\u003c/sup\u003e Equal Contribution)\n\n\u003csup\u003e1\u003c/sup\u003e Beijing Institute of Technology, \u003csup\u003e2\u003c/sup\u003e Shanghai AI Laboratory\n\n[![arXiv](https://img.shields.io/badge/arXiv-2307.04725-b31b1b.svg)](https://arxiv.org/abs/2402.13764)\n[![license](https://img.shields.io/github/license/InternLM/opencompass.svg)](./LICENSE)\n\n[[Dataset on HF](https://huggingface.co/datasets/opencompass/CriticBench)]\n[[Project Page](https://open-compass.github.io/CriticEval/)]\n[[Subjective LeaderBoard](https://open-compass.github.io/CriticEval/leaderboard_subjective.html)]\n[[Objective LeaderBoard](https://open-compass.github.io/CriticEval/leaderboard_objective.html)]\n\n\u003e Critique ability are crucial in the scalable oversight and self-improvement of Large Language Models (LLMs). While many recent studies explore the critique ability of LLMs to judge and refine flaws in generations, how to comprehensively and reliably measure the critique abilities of LLMs is under-explored. This paper introduces \u003cb\u003eCriticEval\u003c/b\u003e, a novel benchmark designed to comprehensively and reliably evaluate four key critique ability dimensions of LLMs: feedback, comparison, refinement and meta-feedback. \u003cb\u003eCriticEval\u003c/b\u003e encompasses nine diverse tasks, each assessing the LLMs' ability to critique responses at varying levels of quality granularity. Our extensive evaluations of open-source and closed-source LLMs reveal intriguing relationships between the critique ability and tasks, response qualities, and model scales.\n\n\u003cimg src=\"./figs/overview.png\" alt=\"overview\" align=center /\u003e\n\n## What's New\n\n* **[2024.9.25]** CriticEval is accepted by NeurIPS 2024🥳🥳🥳\n* **[2024.2.21]** Release paper, codes, data and other resources of CriticEval v1.3.\n\n## Next\n\n- [ ] Evaluate Qwen-1.5 series models\n- [ ] Improve the reliability of subjective evaulation in CriticEval (v1.4)\n- [ ] Expand to more diverse tasks\n- [ ] Expand to Chinese applications\n- [ ] Prepare and clean the codebase for OpenCompass\n- [ ] Release the train set of CriticEval\n- [ ] Support inference on Opencompass.\n\n## Quick Start\n\n### 1. Prepare\n\n#### 1.1 Prepare Dataset\n\nDownload the dataset from [huggingface dataset](https://huggingface.co/datasets/opencompass/CriticBench) by running this command:\n```bash\nmkdir data\ncd data\ngit clone https://huggingface.co/datasets/opencompass/CriticBench\n```\nwhich get into the `data` folder and clone the CriticEval dataset.\nNote that the human-annotated Likert scores, preference labels, and critiques in `test` set are excluded.\nYou can submit your inference results on the `test` set (via run codes under `inference` folder) to this [email](lantiangmftby@gmail.com). We will run your predictions and update the results in our leaderboard. Please also provide the scale of your tested model. \nThe structure of your submission should be similar to that in `example_data`.\n\n#### 1.2 Prepare Code and Env\n\n```bash\ngit clone https://github.com/open-compass/CriticEval.git\n# prepare the env for evaluation toolkit\ncd critic_bench\npip install -r requirements.txt\n# prepare the env for LLM inference\ncd ../inference\npip install -r requirements.txt\n```\n\n### 2. Inference LLMs on CriticEval\n\nYou need to inference LLMs to be evaluated on our proposed CriticEval, and generation results on CriticEval can be found in `inference/outputs` folder. \nIf you are interested with our prompts for LLM, they are shown in [inference/utils/prompts.py](inference/utils/prompts.py).\nSpecifically, the inference code should be like:\n```python\n# this line loads all the evaluation dataset in CriticEval from `inference/utils`\ndatasets = load_all_datasets(args['data_dir'])\n\n# these lines init the tokenizer and models from huggingface\ntokenizer = AutoTokenizer.from_pretrained(\n    args['model_name'],\n    trust_remote_code=True\n)\nmodel = AutoModelForCausalLM.from_pretrained(\n    args['model_name'], \n    device_map=\"auto\", \n    trust_remote_code=True\n).cuda().eval()\n\n...\n\n# inference the LLM and save the results in json file format\nfor abbr, dataset in tqdm(datasets.items()):\n    path = os.path.join(folder_path, abbr + \".json\")\n    results = {}\n    for item in tqdm(dataset['dev']):        \n        \n        # If you want to inference other LLMs, please revise this line\n        response, history = model.chat(tokenizer, item['question'], history=[])\n            \n        results[str(len(results))] = {\n            'origin_prompt': item['question'],\n            'prediction': response\n        }\n    # save the results into json file, with the abbr as the file name\n    with open(path, 'w') as f:\n        json.dump(results, f, ensure_ascii=False, indent=4)\n```\n\nWe only provide the inference codebase for our [InternLM2-7B-Chat](https://huggingface.co/internlm/internlm2-chat-7b), but it is easy to revise the inference code for evaluating your own LLMs (more details are in [inference/internlm2.py](./inference/internlm2.py)).\n\n#### Example Inference Data of Representative LLMs\n\nWe have already released the generation results of some representative LLMs on CriticEval, and you could found them in [example_data/prediction_v1.3.tgz](example_data/prediction_v1.3.tgz).\n\n```bash\ntar -xzvf example_data/prediction_v1.3.tgz\n```\n\nAfter unzipping, you could found the details of the predictions of LLMs on CriticEval. Typically, the format of the evaluation files are: `{split}_{domain}_{dimension}_{format}.json`, where `split`, `dimension`, and `format` are described above. The `domain` represents 9 task scenarios in our proposed CriticEval, consisting of `translate`, `chat`, `qa`, `harmlessness`, `summary`, `math_cot`, `math_pot`, `code_exec`, `code_not_exec`. Refer to more details in our paper.\nHere are some notes: \n* the `comp_feedback` critique dimension always company with an `reverse` file which is used to address the well-known positional bias problem for LLM-as-a-judge procedure. Refer to more details in Section 4 of our paper.\n* For `feedback` critique dimension, each `domain` has additional `*_correction_part.json` files, saving the evaluation results of critiques for the correct or the very high-quality responses. Refer to more details about these response in our paper.\n\nThe format of the evaluation result file is:\n```python\n{\n    '0': {\n        'origin_prompt': 'The original prompt for LLMs to be evaluated',\n        'prediction': 'The generated critiques to be evaluated'\n    }\n}\n```\n\n### 3. Compute the Evaluation Results on CriticEval\n\nAfter getting the generation results under `inference/outputs`, your next step is to compute the objective and subjective scores in our proposed CriticEval using our toolkit.\nSee more details about the objective and subjective scores in Section 4 of our paper.\n\nWe provide two ways for computing the `objective` and `subjective` scores in `critic_bench` folder.\n* Objective scores could be computed automatically without any cost\n* Subjective scores rely on the advanced GPT-4-turbo model for automatic evaluation\n\n#### Compute Scores\n\nIt is easy to compute the scores by running following commands.\n\nBefore running this code, please make sure that your own OpenAI API key in [critic_bench/run.sh](critic_bench/run.sh) is set.\n\n```bash\nexport OPENAI_API_KEY=...\n```\n\nThen, running the following codes for evaluation:\n\n```bash\n./run.sh \u003cdimension\u003e \u003cformat\u003e \u003csplit\u003e \u003csave_dir\u003e\n```\n\n* `dimension` denotes critique dimensions defined in our proposed CriticEval, which are `feedback`, `correction`, `comp_feedback`, and `meta_feedback`. Refer to more details about these critique dimensions in Section 2 of our paper.\n* `format` denotes the critique format `objective` and `subjective`. Objective scores are spearman correlation, pass rate, preference accuracy that can be computed automatically without any cost, while subjective scores are computed by prompting GPT-4-turbo to compare generated critiques and our human-annotated high-quality critiques in CriticEval.\n* `split` denotes the `test` or `dev` set to be evaluated.\n* `save_dir` is any text path saving the evaluation results.\n\nIn [run.sh](critic_bench/run.sh) file, you could find the corresponding commands for objective and subjective evaluation process. \nFor example, for the feedback critique dimension, the objective evaluation is like:\n```bash\npython run_feedback.py --root_dir \"../data/CriticEval\" --prediction_dir \"../example_data/prediction_v1.3\" --split $3 --obj True\n```\n* `root_dir` contains the path of the `test` and `dev` set in CriticEval.\n* `prediction_dir` contains the inference results of LLMs to be evaluated. We also provide the inference results of some representation LLMs  in `example_data`. If you want to evaluate your own LLMs, please refer to `inference/README.md` for more details, and the `prediction_dir` could be set as `../inference/outputs`.\n* `split` denotes whether the `test` or the `dev` set is used.\n* `obj` denotes that the objective evaluation is activated\n\nFor the subjective evaluation of the feedback critique dimension, the evaluation command is like:\n```bash\npython run_feedback.py --root_dir \"../data/CriticEval\" --prediction_dir \"../example_data/prediction_v1.3\" --evaluation_dir \"../example_data/evaluation_v1.3/\" --batch_size 1 --split $3 --obj False\n```\n* `evaluation_dir` saves the subjective evaluation scores of GPT-4, which can be re-loaded if the subjective evaluation process borke off. The order of the samples in each file in `evaluation_dir` follows the order in the original data in CriticEval (`data/CriticEval`).\n* `batch_size` controls the number of the process for access GPT-4 API under multiprocessing setting.\n\nThe evaluation results of GPT-4 under `save_dir` is `jsonl`, and each line contains the evaluation results. The chain-of-thought evaluation results prompted by GPT-4 is in the `evaluation` key-value in each line, which is a `dict` consisting of the chain-of-thought rationale about GPT-4 (key-value `cot`) and a Likert score (key-value `score`) for each critiques, ranging from 1 to 10.\n* 1 denotes the worst performance\n* 10 denotes the best performance\n* 8 denotes the comparable performance with our human-annotated high-quality critiques, and scores higher than 8 denotes the better performance of evaluated critiques.\n\n\nTo compute the overall score, please run this command:\n```bash\n# the results of GPT-4 on the test set\npython compute_overall.py --mode obj --feedback_overall 63.54 --correction_overall 69.67 --comp_feedback_overall 57.33 --meta_feedback_overall 62.90\n```\n* replacing the `mode` with `sub` could compute the overall score for subjective evaluation\n* ensure that at least one dimension's overall score is provided among the four dimensions\n\n## Benchmark Results\n\nThe subjective evaluation results of some representation LLMs are shown:\n\n\u003cimg src=\"./figs/subjective_score.png\" alt=\"sujective\" align=center /\u003e\n\nThe objective evaluation results of some representation LLMs are shown:\n\n\u003cimg src=\"./figs/objective_score.png\" alt=\"objective\" align=center /\u003e\n\nRefer to our [Project Page](https://open-compass.github.io/CriticEval/) for the complete evaluation results on \u003cb\u003eCriticEval\u003c/b\u003e.\n\n\n## Acknowledgements\n\n\u003cb\u003eCriticEval\u003c/b\u003e is built with [OpenCompass](https://github.com/open-compass/opencompass). Thanks for their awesome work! \n\nThe quota for API-based LLMs are supported by Beijing Institute of Technology and Shanghai AI Laboratory. Thank you so much!\n\n## Contact Us\n\n* **Tian Lan**: lantiangmftby@gmail.com\n* **Wenwei Zhang**: zhangwenwei@pjlab.org.cn\n\n## BibTeX\n\n```\n@misc{lan2024criticevalevaluatinglargelanguage,\n      title={CriticEval: Evaluating Large Language Model as Critic}, \n      author={Tian Lan and Wenwei Zhang and Chen Xu and Heyan Huang and Dahua Lin and Kai Chen and Xian-ling Mao},\n      year={2024},\n      eprint={2402.13764},\n      archivePrefix={arXiv},\n      primaryClass={cs.CL},\n      url={https://arxiv.org/abs/2402.13764}, \n}\n```\n\n## License\n\nThis project is released under the Apache 2.0 [license](./LICENSE).\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/open-compass.github.io%2FCriticEval%2F","html_url":"https://awesome.ecosyste.ms/projects/open-compass.github.io%2FCriticEval%2F","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/open-compass.github.io%2FCriticEval%2F/lists"}