https://github.com/embeddings-benchmark/leaderboard
Code for the MTEB leaderboard
https://github.com/embeddings-benchmark/leaderboard
Last synced: over 1 year ago
JSON representation
Code for the MTEB leaderboard
- Host: GitHub
- URL: https://github.com/embeddings-benchmark/leaderboard
- Owner: embeddings-benchmark
- Archived: true
- Created: 2024-07-04T17:36:28.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2025-02-04T10:24:12.000Z (over 1 year ago)
- Last Synced: 2025-04-13T21:08:06.245Z (over 1 year ago)
- Language: Python
- Homepage: https://hf.co/spaces/mteb/leaderboard
- Size: 25.6 MB
- Stars: 23
- Watchers: 4
- Forks: 14
- Open Issues: 17
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
---
title: MTEB Leaderboard
emoji: 🥇
colorFrom: blue
colorTo: indigo
sdk: gradio
sdk_version: 4.20.0
app_file: app.py
pinned: false
tags:
- leaderboard
startup_duration_timeout: 1h
fullWidth: true
---
> As of February 4, 2025, this repository will no longer be under active maintenance, and will be replaced by the new version of the leaderboard, which is integrated into the [mteb package](https://github.com/embeddings-benchmark/mteb). All requested modifications to the leaderboard or model submissions should start in the MTEB repository.
## The Legacy MTEB Leaderboard repository
This repository contains legacy code for pushing and updating the MTEB leaderboard daily.
| Relevant Links | Decription |
|------------------------------------------|------------------------------|
| [mteb](https://github.com/embeddings-benchmark/mteb) | The implementation of the benchmark. Here you e.g. find the code to run your model on the benchmark. |
| [leaderboard](https://huggingface.co/spaces/mteb/leaderboard) | The leaderboard itself, here you can view results of model run on MTEB. |
| [results](https://github.com/embeddings-benchmark/results) | The results of MTEB is stored here. To learn how to add results to the Leaderboard, refer to the documentation: [Adding a Model to the Leaderboard](https://github.com/embeddings-benchmark/mteb/blob/main/docs/adding_a_model.md).
|
## Developer setup
To setup the repository:
```bash
git clone https://github.com/embeddings-benchmark/leaderboard.git
cd leaderboard
# install requirements
pip install -r requirements.txt
# fetch new results
# python refresh.py
# if you'd like to add results to previously cached models, you may have to remove these models in `EXTERNAL_MODEL_RESULTS.json`
# you can also directly delete `EXTERNAL_MODEL_RESULTS.json` and it will recreate it (but be much slower)
# run the leaderboard
python app.py
```