Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/gjy3035/gcc-cl

GCC dataset Collector and Labeler (GCC CL) [CVPR2019]
https://github.com/gjy3035/gcc-cl

computer-vision crowd-analysis crowd-counting crowd-segmentation cvpr2019 gtav

Last synced: 8 days ago
JSON representation

GCC dataset Collector and Labeler (GCC CL) [CVPR2019]

Host: GitHub
URL: https://github.com/gjy3035/gcc-cl
Owner: gjy3035
License: mit
Created: 2018-12-20T01:46:25.000Z (almost 6 years ago)
Default Branch: master
Last Pushed: 2023-07-24T14:35:53.000Z (over 1 year ago)
Last Synced: 2024-07-24T04:34:51.894Z (4 months ago)
Topics: computer-vision, crowd-analysis, crowd-counting, crowd-segmentation, cvpr2019, gtav
Language: C++
Homepage: https://gjy3035.github.io/GCC-CL/
Size: 16 MB
Stars: 134
Watchers: 6
Forks: 31
Open Issues: 10
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

README

# GCC-CL

GCC-CL(**GCC** dataset **C**ollector and **L**abeler) is a tool for generating synethic crowd image datasets. It consists of two parts, collector and labeler. The former is used to generate crowd image information; the latter produce input and output file through those information for crowd count models.

## Demonstration

[![GCC-CL video](https://github.com/gjy3035/GCC-CL/blob/gh-pages/images/video.jpg?raw=true)](https://youtu.be/Hvl7xWkIueo "GCC dataset Collector and Labeler")

## GCC-Collector

GCC-Collector is a tool to generate crowd image and head points in GTA V. It is a custom plugin running along with GTAV, written in **C++**.

**Attention!!!** All mods must be used in the offline version of GTA V.

### Request
- [ScriptHookV](http://www.dev-c.com/gtav/scripthookv/)
- [DirectXTK](https://github.com/Microsoft/DirectXTK)
- [minhook](https://github.com/TsudaKageyu/minhook)
- [eigen](http://eigen.tuxfamily.org/index.php?title=Main_Page)

### Compilation

1. Navigate to GCC-Collector directory, use Visual Studio to open the project `GCC-Collector.sln`.
2. Open the project (GCC-Collector) property pages, `General/Windos SDK Version`, select **the latest win10 SDK** existed in your computer.
3. Install `minhook` and `eigen` using **NuGet**.
4. Make sure Configution is **release**, Platform is **x64**, and then use hot key `ctrl + shift + B` to compile the project. As a result, asi file `GCC-Collector.asi` will be generated in */GCC-CL/GCC-Collector/x64/Release*

### Installation

1. Following [Script Hook V](http://www.dev-c.com/gtav/scripthookv/) installation steps, copy `ScriptHookV.dll`, `dinput8.dll` and compiled `GCC-Collector.asi` to the game's root folder (where GTA5.exe is located);
2. Use [this mod(No Chromatic aberration & Lens distortion)](https://www.gta5-mods.com/misc/no-chromatic-aberration-lens-distortion-1-41) to avoid chromatic aberration and lens distortion;
3. Compile or directly use the compiled plugin-in `unlimitedLife.asi`, `noVehicle.asi`. Just copy these two asi to the games's root folder.
> - `unlimitedLife.asi` makes sure the player undead.
> - `noVehicle.asi` is optional. When you are creating some crowd images in the street, this plugin-in helps you avoid accidents caused by vehicles entering the scene. And it also set the number of pedestrians created by game scenario close to zero.

### How to work

To create a series of crowd images in one scene, You should follow the steps below:
1. download [GCC-COllector-Tool](https://github.com/gjy3035/GCC-CL/releases) and put `SceneDirectorAnim.txt` in the game's root folder.
2. Create a directory named `data` in the game's root folder.
3. Open GTA V in the **offline version**.
4. Control your role in GTAV go to the scene where you want use it as the background of the crowd images which would be generated.
5. Press `F9` to start a scene script;
6. Press `F10` to adjust the camera. use `W, A, S, D` to move the camera forward, left, backward and right, mouse button(left or right button both works) to move up or down, `shift/ctrl` to speed up or slow down, and `+/-` to adjust the field of view. When the camera has the desired angle, press `F10` again to stop adjust camera and record camera location information automatically.
7. Press `F11` to start drawing the crowd generation area. Use the method of connecting multiple points to form a polygon by moving your game character in the scene. Press `I` to set a polygon vertex at current position. Press `F11` again to end the drawing. The program automatically connects the points in a sequence set by the user. In the process of setting the vertices, you can press `Tab` to move the focus to a specific vertex, and then you can reset that point. During setting the vertices, you can press `F10` to enter the camera view to observe the vertices or modify the camera settings. Press `F10` again to return to set vertices. (Note that it will be better to set a convex polygon area, do not let the sides of the polygon cross.) Following the guide to readjust the camera, and other three cameras (from four different angle).
8. At last, you should confirm how many pedestrians you want to generate in the scene. Press `F12` and follow the guide in the game, you will know how to do it.
9. Now one scene has been created and saved, press `F5` to go back to original state, and do above steps(except step 1) again to recorded another scene.

After a series of scene have been created. restart the game, after the opening animation
1. colse the minimap if it is shown in the lower left corner of screen, or generating segement maps do not work well, as [issue#7](https://github.com/gjy3035/GCC-CL/issues/7) demonstrated;
2. press `L` to launch the GCC-Collector to generate crowd with recorded.information
3. press 'F5' if you want to leave the process and go back to normal. The process would end after one-round of collecting.(Not immediately)

## GCC-Labeler

GCC-Labeler is written in Python3. It needs the following Python libraries:
- scipy
- numpy
- matplotlib
- PIL

You can install these libraries using `pip`.

`main.py` is the entrance of the project. Line 22 and 23 define the source path and the target path.
``` python
source_dir = 'source'
target_dir = 'target'
```
Source path is the path where the file you generated using GCC-Collector is located. The target path is used to specify where to store the final produced image files and the annotation files. Change two paths according to your actual situation. Just run `python main.py`, the final crowd count images and labels will be generated in `target_dir`.

3 directories, 24 files
```
After we run the `main.py`, we can get the target directory like this:
```
target/
`-- scene_11_2
|-- jpgs
| |-- 1534540881.jpg
| `-- 1534575913.jpg
|-- jsons
| |-- 1534540881.json
| `-- 1534575913.json
|-- mats
| |-- 1534540881.mat
| `-- 1534575913.mat
|-- pngs
| |-- 1534540881.png
| `-- 1534575913.png
|-- segs
| |-- 1534540881.raw
| `-- 1534575913.raw
`-- vis
|-- 1534540881.jpg
`-- 1534575913.jpg

7 directories, 12 files
```

## Acknowledgments

Some code borrows from [gtav-mod-scene-director](https://github.com/elsewhat/gtav-mod-scene-director) and [GTAVisionExport](https://github.com/umautobots/GTAVisionExport). The former gave us so many examples for how to use these functions in Script Hook V. The latter inspired us to extract crowd mask.

## Citation

If you find this project useful for your research, please cite:
```
@inproceedings{wang2019learning,
title={Learning from Synthetic Data for Crowd Counting in the Wild},
author={Wang, Qi and Gao, Junyu and Lin, Wei and Yuan, Yuan},
booktitle={Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
pages={8198--8207},
year={2019}
}
```