https://github.com/ptrvilya/object-popup
[CVPR'23] Method to infer objects from human interactions
https://github.com/ptrvilya/object-popup
3d-human cvpr cvpr2023 cvpr23 human-object human-object-interaction
Last synced: 3 months ago
JSON representation
[CVPR'23] Method to infer objects from human interactions
- Host: GitHub
- URL: https://github.com/ptrvilya/object-popup
- Owner: ptrvilya
- License: other
- Created: 2023-04-19T15:12:11.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2025-05-30T07:05:50.000Z (about 1 year ago)
- Last Synced: 2025-09-09T23:53:53.204Z (9 months ago)
- Topics: 3d-human, cvpr, cvpr2023, cvpr23, human-object, human-object-interaction
- Language: Python
- Homepage: https://virtualhumans.mpi-inf.mpg.de/object_popup/
- Size: 8.4 MB
- Stars: 46
- Watchers: 5
- Forks: 3
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
Object pop-up: Can we infer 3D objects and their poses from human interactions alone?
Ilya A. Petrov
Riccardo Marin
Julian Chibane
Gerard Pons-Moll
CVPR 2023
## Environment
The code was tested under `Ubuntu 22.04, Python 3.10, CUDA 11.8, PyTorch 2.0.1`.\
Use the following command to create a conda environment with necessary dependencies:
```bash
conda env create -f environment.yml
```
## Data downloading and processing
The steps are described in [docs/data.md](./docs/data.md).
## Pre-trained models and evaluation
Pre-trained models can be obtained from the [link](https://nc.mlcloud.uni-tuebingen.de/index.php/s/8H4EHmx9MA8sBEs). With the commands:
```bash
wget https://nc.mlcloud.uni-tuebingen.de/index.php/s/PG8wZ5HRKytEY8S/download/object_pop_up_noclass.tar -P ./assets
wget https://nc.mlcloud.uni-tuebingen.de/index.php/s/Dfx9rfQ2tW4ZsEY/download/object_pop_up_class.tar -P ./assets
```
Use the following commands to run evaluation:
```bash
# model without class prediction (assumes 24GB GPU memory)
python evaluate.py scenarios/gb_PNv2_noclass.toml -b 64 -w 20 -d grab behave -g -rc ./assets/object_pop_up_noclass.tar -c configs/smplh.toml
# model with class prediction (assumes 24GB GPU memory)
python evaluate.py scenarios/gb_PNv2_class.toml -b 64 -w 20 -d grab behave -g -rc ./assets/object_pop_up_class.tar -c configs/smplh.toml
```
## Training
Use the following command to run the training:
```bash
python train.py scenarios/gb_PNv2_noclass.toml -b 32 -w 10 -nowb -ep 0001_smplh -c configs/smplh.toml
```
## Citation
```bibtex
@inproceedings{petrov2023popup,
title={Object pop-up: Can we infer 3D objects and their poses from human interactions alone?},
author={Petrov, Ilya A and Marin, Riccardo and Chibane, Julian and Pons-Moll, Gerard},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
year={2023}
}
```
## Acknowledgements
This project benefited from the following resources:
* [BEHAVE](https://virtualhumans.mpi-inf.mpg.de/behave/license.html) and [GRAB](https://grab.is.tue.mpg.de/) datasets;
* [grab](https://github.com/otaheri/GRAB) preprocessing code;
* [smplx](https://github.com/vchoutas/smplx) repository: SMPL-X to SMPL+H conversion;
* [PointNet v2 implementation](https://github.com/yanx27/Pointnet_Pointnet2_pytorch/): encoder model;
* [blendify](https://github.com/ptrvilya/blendify/): all visualizations;
* [blogpost](http://danshiebler.com/2016-09-14-parallel-progress-bar/): parallel map implementation.