https://github.com/huggingface/hf_benchmarks
A starter kit for evaluating benchmarks on the 🤗 Hub
https://github.com/huggingface/hf_benchmarks
evaluation huggingface machine-learning
Last synced: 9 months ago
JSON representation
A starter kit for evaluating benchmarks on the 🤗 Hub
- Host: GitHub
- URL: https://github.com/huggingface/hf_benchmarks
- Owner: huggingface
- License: apache-2.0
- Created: 2021-06-30T14:47:59.000Z (about 5 years ago)
- Default Branch: main
- Last Pushed: 2023-12-29T07:09:05.000Z (over 2 years ago)
- Last Synced: 2025-09-30T18:02:28.518Z (9 months ago)
- Topics: evaluation, huggingface, machine-learning
- Language: Python
- Homepage:
- Size: 122 KB
- Stars: 15
- Watchers: 5
- Forks: 3
- Open Issues: 8
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Hugging Face Benchmarks
> A toolkit for evaluating benchmarks on the [Hugging Face Hub](https://huggingface.co)
## Hosted benchmarks
The list of hosted benchmarks is shown in the table below:
| Benchmark | Description | Submission | Leaderboard |
| :---: | :---: | :---: | :---: |
| RAFT | A benchmark to test few-shot learning in NLP | [`ought/raft-submission`](https://huggingface.co/datasets/ought/raft-submission) | [`ought/raft-leaderboard`](https://huggingface.co/spaces/ought/raft-leaderboard) |
| GEM | A large-scale benchmark for natural language generation | [`GEM/submission-form`](https://huggingface.co/spaces/GEM/submission-form) | [`GEM/results`](https://huggingface.co/spaces/GEM/results) |
## Developer installation
Clone the repository and install the requirements:
```
git clone git@github.com:huggingface/hf_benchmarks.git
cd hf_benchmarks
pip install '.[dev]'
```