https://github.com/TheDuckAI/arb

Advanced Reasoning Benchmark Dataset for LLMs
https://github.com/TheDuckAI/arb

benchmark dataset llm

Last synced: about 1 month ago
JSON representation

Advanced Reasoning Benchmark Dataset for LLMs

• Host: GitHub
• URL: https://github.com/TheDuckAI/arb
• Owner: TheDuckAI
• License: mit
• Created: 2023-07-19T22:27:12.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2023-11-19T20:40:33.000Z (about 1 year ago)
• Last Synced: 2024-08-03T09:06:48.767Z (5 months ago)
• Topics: benchmark, dataset, llm
• Language: TypeScript
• Homepage: https://advanced-reasoning-benchmark.netlify.app
• Size: 1.49 MB
• Stars: 44
• Watchers: 3
• Forks: 2
• Open Issues: 6
• Metadata Files:
  • Readme: README.md
  • License: LICENSE.txt

Awesome Lists containing this project

• StarryDivineSky - TheDuckAI/arb

README

        


  

  


  Advanced Reasoning Benchmark

  





    

       

    



A DuckAI project in collaboration with the Georgia Institute of Technology, ETH Zürich, Nomos AI, Stanford University Center for Legal Informatics, and the Mila - Quebec AI Institute


### Abstract

ARB is a novel benchmark dataset composed of advanced reasoning problems designed to evaluate LLMs on text comprehension and expert domain reasoning, presenting a more challenging test than prior benchmarks, featuring questions that test deeper knowledge of mathematics, physics, biology, chemistry, and law.

### API Usage

Endpoint url: https://advanced-reasoning-benchmark.netlify.app/api/




The documentation for the complete REST API of the ARB dataset is [here](https://advanced-reasoning-benchmark.netlify.app/api/).

Copyright © 2023 [DuckAI](https://github.com/TheDuckAI)