https://github.com/thunlp-mt/cospace
Official Repository for “CoSpace: Benchmarking Continuous Space Perception Ability for Vision-Language Models" [CVPR2025]
https://github.com/thunlp-mt/cospace
Last synced: 3 months ago
JSON representation
Official Repository for “CoSpace: Benchmarking Continuous Space Perception Ability for Vision-Language Models" [CVPR2025]
- Host: GitHub
- URL: https://github.com/thunlp-mt/cospace
- Owner: THUNLP-MT
- Created: 2024-12-02T10:31:50.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-03-21T08:39:28.000Z (about 1 year ago)
- Last Synced: 2025-03-21T09:32:12.625Z (about 1 year ago)
- Homepage: https://thunlp-mt.github.io/CoSpace/
- Size: 13.7 MB
- Stars: 0
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# CoSpace: Benchmarking Continuous Space Perception Ability for Vision-Language Models
## Quick Start
### Installation
All the images for **CoSpace** can be downloaded from [Hugging Face](https://huggingface.co/datasets/StephenZhu/CoSpace/). After successfully downloading the images, please rename `panorama_images` into `images` and keep the directory as follows:
```bash
src/
├── data/
│ ├── anno/
│ │ ├── direction.jsonl
│ │ ├── object.jsonl
│ │ └── ...
│ │
│ ├── images/
│ ├── habitat_scenes/
│ ├── frames_30/
│ └── frames_30/
│
│
├── pipeline/
│ ├── scripts/
│ │ ├── pred_gpt.py
│ │ ├── pred_claude.py
│ │ └── ...
│ │
│ ├── pred.sh
│ ├── pred_and_eval.sh
│ ├── load_data.py
│ ├── prompt.py
│ ├── eval.py
│ └── utils.py
│
└── pipeline-single/
└── ...
```
### Evaluation
To evaluate on our **CoSpace**, you can directly run the provided scripts:
```bash
# Prediction
bash pred.sh
# Prediction and evaluation
bash pred_and_eval.sh
# Also, you can directly evaluate the prediction file
python eval.py --data_file path_to_the_prediction_file --save_dir result_save_directory
```
To evaluate a new model, please add your `pred_{your_model}.py`, and modify the `eval.py` to fit the output format.