https://github.com/cyberagentailab/graphic-design-evaluation
This is the official repository for "Can GPTs Evaluate Graphic Design Based on Design Principles?".
https://github.com/cyberagentailab/graphic-design-evaluation
Last synced: 9 months ago
JSON representation
This is the official repository for "Can GPTs Evaluate Graphic Design Based on Design Principles?".
- Host: GitHub
- URL: https://github.com/cyberagentailab/graphic-design-evaluation
- Owner: CyberAgentAILab
- License: apache-2.0
- Created: 2024-10-02T07:31:32.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-02-10T09:46:43.000Z (over 1 year ago)
- Last Synced: 2025-09-10T07:42:49.460Z (9 months ago)
- Language: Jupyter Notebook
- Homepage:
- Size: 93.5 MB
- Stars: 12
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.txt
Awesome Lists containing this project
README
## Paper: Can GPTs Evaluate Graphic Design Based on Design Principles?
Daichi Haraguchi1, Naoto Inoue1, Wataru Shimoda1, Hayato Mitani2, Seiichi Uchida2, Kota Yamaguchi1
1CyberAgent.Inc, 2 Kyushu University
Accepted to SIGGRAPH Asia (Technical Communications Track).
[[Project-page](https://cyberagentailab.github.io/Graphic-design-evaluation/)]
## Introduction
This repository contains the data for "Can GPTs Evaluate Graphic Design Based on Design Principles?".
## Data
./GPT_eval_data
├── gpt_eval
├── gpt_abs_alignment.csv
...
└── gpt_comp_whitespace.csv
├── human_eval_data
├── human_abs_alignment.csv
...
└── human_comp_whitespace.csv
└── images.zip
Each CSV file includes the evaluation results.
File names consist of "[eval method]_[eval type]\_[design principle].csv."
Note:
"eval. method" is "gpt" or "human."
"eval. type" is "abs" (absolute evaluation) or "comp" (comparative (relative) evaluation).
"design principle" is "alignment," "overlap," or "whitespace."
Example:
"gpt_abs_alignment.csv" is the absolute evaluation result by GPT for alignment.
"images.zip" includes images.
The structure in the "images.zip" is below.
images
├── lefttop_large
├── [image_ID].png
...
└── [image_ID].png
...
└── org
├── [image_ID].png
...
└── [image_ID].png
Note:
The directory name consists of "[perturbation element]_[perturbation size]"
"Org" includes original images without perturbation.
Each image name is an image ID that is written in the CSV files.
## The structure of each CSV file
CSV for absolute evaluation
| id | perturbation | 0 | 1 | 2 | 3 | 4 | avg |
| ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |
| image ID | perturbation size | score | score | score | score | score |average score|
CSV for relative evaluation
| id | comparative | better_design_0 | better_design_1 | better_design_2 | better_design_3 | better_design_4 | voting_res |
| ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |
| image ID | perturbation size of comparison image | voting result | voting result| voting result | voting result | voting result | aggregation of voting results|
## Visualization
We provides a notebook for showing results.
- `scatter_plot.ipynb` shows the scatter plot between human evaluation and GPT evaluation.
## Reference
```bibtex
@inproceedings{haraguchi2024can,
title={Can GPTs Evaluate Graphic Design Based on Design Principles?},
author={Haraguchi, Daichi and Inoue, Naoto and Shimoda, Wataru and Mitani, Hayato and Uchida, Seiichi and Yamaguchi, Kota},
booktitle={SIGGRAPH Asia 2024 Technical Communications},
pages={1--4},
year={2024}
}
```
## Contact
This repository is maintained by Daichi Haraguchi.
E-mail: haraguchi_daichi_xa[at]cyberagent.co.jp