Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/facebookresearch/clevr-dataset-gen
A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
https://github.com/facebookresearch/clevr-dataset-gen
Last synced: 4 months ago
JSON representation
A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
- Host: GitHub
- URL: https://github.com/facebookresearch/clevr-dataset-gen
- Owner: facebookresearch
- License: other
- Archived: true
- Created: 2017-07-05T16:54:21.000Z (over 7 years ago)
- Default Branch: main
- Last Pushed: 2021-08-30T20:51:49.000Z (over 3 years ago)
- Last Synced: 2024-08-01T13:29:19.631Z (7 months ago)
- Language: Python
- Size: 5.49 MB
- Stars: 577
- Watchers: 16
- Forks: 204
- Open Issues: 25
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
# CLEVR Dataset Generation
This is the code used to generate the [CLEVR dataset](http://cs.stanford.edu/people/jcjohns/clevr/) as described in the paper:
**[CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](http://cs.stanford.edu/people/jcjohns/clevr/)**
Justin Johnson,
Bharath Hariharan,
Laurens van der Maaten,
Fei-Fei Li,
Larry Zitnick,
Ross Girshick
Presented at [CVPR 2017](http://cvpr2017.thecvf.com/)
Code and pretrained models for the baselines used in the paper [can be found here](https://github.com/facebookresearch/clevr-iep).
You can use this code to render synthetic images and compositional questions for those images, like this:
**Q:** How many small spheres are there?
**A:** 2
**Q:** What number of cubes are small things or red metal objects?
**A:** 2
**Q:** Does the metal sphere have the same color as the metal cylinder?
**A:** Yes
**Q:** Are there more small cylinders than metal things?
**A:** No
**Q:** There is a cylinder that is on the right side of the large yellow object behind the blue ball; is there a shiny cube in front of it?
**A:** Yes
If you find this code useful in your research then please cite
```
@inproceedings{johnson2017clevr,
title={CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning},
author={Johnson, Justin and Hariharan, Bharath and van der Maaten, Laurens
and Fei-Fei, Li and Zitnick, C Lawrence and Girshick, Ross},
booktitle={CVPR},
year={2017}
}
```
All code was developed and tested on OSX and Ubuntu 16.04.
## Step 1: Generating Images
First we render synthetic images using [Blender](https://www.blender.org/), outputting both rendered images as well as a JSON file containing ground-truth scene information for each image.
Blender ships with its own installation of Python which is used to execute scripts that interact with Blender; you'll need to add the `image_generation` directory to Python path of Blender's bundled Python. The easiest way to do this is by adding a `.pth` file to the `site-packages` directory of Blender's Python, like this:
```bash
echo $PWD/image_generation >> $BLENDER/$VERSION/python/lib/python3.5/site-packages/clevr.pth
```
where `$BLENDER` is the directory where Blender is installed and `$VERSION` is your Blender version; for example on OSX you might run:
```bash
echo $PWD/image_generation >> /Applications/blender/blender.app/Contents/Resources/2.78/python/lib/python3.5/site-packages/clevr.pth
```
You can then render some images like this:
```bash
cd image_generation
blender --background --python render_images.py -- --num_images 10
```
On OSX the `blender` binary is located inside the blender.app directory; for convenience you may want to
add the following alias to your `~/.bash_profile` file:
```bash
alias blender='/Applications/blender/blender.app/Contents/MacOS/blender'
```
If you have an NVIDIA GPU with CUDA installed then you can use the GPU to accelerate rendering like this:
```bash
blender --background --python render_images.py -- --num_images 10 --use_gpu 1
```
After this command terminates you should have ten freshly rendered images stored in `output/images` like these:
The file `output/CLEVR_scenes.json` will contain ground-truth scene information for all newly rendered images.
You can find [more details about image rendering here](image_generation/README.md).
## Step 2: Generating Questions
Next we generate questions, functional programs, and answers for the rendered images generated in the previous step.
This step takes as input the single JSON file containing all ground-truth scene information, and outputs a JSON file
containing questions, answers, and functional programs for the questions in a single JSON file.
You can generate questions like this:
```bash
cd question_generation
python generate_questions.py
```
The file `output/CLEVR_questions.json` will then contain questions for the generated images.
You can [find more details about question generation here](question_generation/README.md).