{"id":17260339,"url":"https://github.com/mkocabas/spec","last_synced_at":"2025-10-18T11:38:36.300Z","repository":{"id":41078271,"uuid":"416389088","full_name":"mkocabas/SPEC","owner":"mkocabas","description":"Code for ICCV2021 paper SPEC: Seeing People in the Wild with an Estimated Camera ","archived":false,"fork":false,"pushed_at":"2024-02-26T17:12:59.000Z","size":7408,"stargazers_count":228,"open_issues_count":8,"forks_count":31,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-04-02T17:01:41.543Z","etag":null,"topics":["3d-human-mesh","3d-human-pose","3d-human-reconstruction","3d-human-shape-and-pose-estimation","camera-calibration","computer-graphics","computer-vision","human-pose-estimation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mkocabas.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-10-12T15:18:05.000Z","updated_at":"2024-12-18T05:51:24.000Z","dependencies_parsed_at":"2024-10-31T08:03:38.052Z","dependency_job_id":"c801c43e-9225-41d6-9c8d-bbcad9ceab1d","html_url":"https://github.com/mkocabas/SPEC","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkocabas%2FSPEC","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkocabas%2FSPEC/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkocabas%2FSPEC/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkocabas%2FSPEC/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mkocabas","download_url":"https://codeload.github.com/mkocabas/SPEC/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248085324,"owners_count":21045139,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-human-mesh","3d-human-pose","3d-human-reconstruction","3d-human-shape-and-pose-estimation","camera-calibration","computer-graphics","computer-vision","human-pose-estimation"],"created_at":"2024-10-15T07:47:55.623Z","updated_at":"2025-10-18T11:38:36.192Z","avatar_url":"https://github.com/mkocabas.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SPEC: Seeing People in the Wild with an Estimated Camera [ICCV 2021]\n\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)]()\n[![report](https://img.shields.io/badge/Project-Page-blue)](https://spec.is.tue.mpg.de/)\n[![report](https://img.shields.io/badge/ArXiv-Paper-red)](https://arxiv.org/abs/2110.00620)\n\n\u003e [**SPEC: Seeing People in the Wild with an Estimated Camera**](https://arxiv.org/abs/2110.00620),            \n\u003e [Muhammed Kocabas](https://ps.is.tuebingen.mpg.de/person/mkocabas), \n\u003e [Chun-Hao Paul Huang](https://ps.is.tuebingen.mpg.de/person/chuang2), \n\u003e [Joachim Tesch](https://ps.is.tuebingen.mpg.de/person/jtesch), \n\u003e [Lea Müller](https://ps.is.tuebingen.mpg.de/person/lmueller2), \n\u003e [Otmar Hilliges](https://ait.ethz.ch/people/hilliges/),\n[Michael J. Black](https://ps.is.tuebingen.mpg.de/person/black),        \n\u003e *International Conference on Computer Vision (ICCV), 2021* \n\n\u003cp float=\"center\"\u003e\n  \u003cimg src=\"docs/assets/spec_gif.gif\" width=\"100%\" /\u003e\n\u003c/p\u003e\n\n## Features\n\nSPEC is a camera-aware human body pose and shape estimation method. \nIt both predicts the camera parameters and SMPL body model for a given image.\n[CamCalib](camcalib) predicts the camera parameters. \n[SPEC](spec) uses these parameters to predict SMPL body model parameters.\n\nThis implementation:\n- has the demo code for SPEC and CamCalib implemented in PyTorch.\n- achieves SOTA results in SPEC-SYN and SPEC-MTP datasets.\n- shows how to perform evaluation on SPEC-SYN and SPEC-MTP datasets.\n\n## Updates\n\n- 13/10/2021: Demo and evaluation code is released.\n\n## Getting Started\n\nSPEC has been implemented and tested on Ubuntu 18.04 with python \u003e= 3.7.\nIf you don't have a suitable device, try running our Colab demo.\n\nClone the repo:\n\n```shell\ngit clone https://github.com/mkocabas/SPEC.git\n```\n\nInstall the requirements using virtualenv or conda:\n\n```shell\n# pip\nsource scripts/install_pip.sh\n\n# conda\nsource scripts/install_conda.sh\n```\n\n## Running the Demo\n\n### SPEC\n\nFirst, you need to download the required data \n(i.e our trained model and SMPL model parameters). It is approximately 1GB. \nTo do this you can just run:\n\n```shell\nsource scripts/prepare_data.sh\n```\n\nThen, running the demo is as simple as:\n\n```shell\npython scripts/spec_demo.py \\\n  --image_folder data/sample_images \\\n  --output_folder logs/spec/sample_images\n```\nSample demo output:\n\n\u003cp float=\"left\"\u003e\n  \u003cimg src=\"docs/assets/COCO_val2014_000000386912_000000.jpg\" width=\"70%\" /\u003e\n\u003c/p\u003e\n\nHere the green line is the horizon obtained using estimated camera parameters.\nOn the right, the ground plane is visualized to show how accurate the global translation is.\n\n### CamCalib\n\nIf you are only interested in estimating the camera parameters of an image, \nrun the CamCalib demo:\n\n```shell\npython scripts/camcalib_demo.py \\\n  --img_folder \u003cinput image folder\u003e \\\n  --out_folder \u003coutput folder\u003e \\\n  --show # visualize the raw network predictions\n```\n\nThis script outputs a pickle file which contains the predicted camera parameters for each input image along with an output\nimage which visualizes the camera parameters as a horizon line. Pickle file contains:\n```\n'vfov' : vertical field of view in radians\n'f_pix': focal length in pixels\n'pitch': pitch in radians\n'roll' : roll in radians\n```\n\n## Google Colab\n\n## Training\n\nTraining instructions will follow soon.\n\n## Datasets\n\nPano360, SPEC-MTP, and SPEC-SYN are new datasets introduced in our paper.\nYou can download them from the [Downloads](https://spec.is.tue.mpg.de/download.php) \nsection of our [project page](https://spec.is.tue.mpg.de).\n\nFor Pano360 dataset, we have released the Flickr image ids which can be used to download images\nusing FlickrAPI. \nWe have provided a [download script](scripts/download_flickr.py) in this repo.\nSome of the images will be missing due to users deleting their photos. \nIn this case, you can also use `scrape_and_download` function provided \nin the script to find and download more photos.\n\nAfter downloading the SPEC-SYN, SPEC-MTP, Pano360, and [3DPW](https://virtualhumans.mpi-inf.mpg.de/3DPW/) \ndatasets, the `data` folder should look like:\n\n```shell\ndata/\n├── body_models\n│   └── smpl\n├── camcalib\n│   └── checkpoints\n├── dataset_extras\n├── dataset_folders\n│   ├── 3dpw\n│   ├── pano360\n│   ├── spec-mtp\n│   └── spec-syn\n├── sample_images\n└── spec\n    └── checkpoints\n```\n\n## Evaluation\nYou can evaluate SPEC on SPEC-SYN, SPEC-MTP, and 3DPW datasets by running:\n```shell\npython scripts/spec_eval.py \\\n  --cfg data/spec/checkpoints/spec_config.yaml \\\n  --opts DATASET.VAL_DS spec-syn_spec-mtp_3dpw-test-cam\n```\n\nRunning this script should give results reported in this table:\n\n| | W-MPJPE | PA-MPJPE | W-PVE|\n|--- | --- | --- | ---|\n|SPEC-MTP | 124.3 | 71.8 | 147.1|\n|SPEC-SYN | 74.9 | 54.5 | 90.5|\n|3DPW | 106.7 | 53.3 | 124.7|\n\n## Citation\n\n```bibtex\n@inproceedings{SPEC:ICCV:2021,\n  title = {{SPEC}: Seeing People in the Wild with an Estimated Camera},\n  author = {Kocabas, Muhammed and Huang, Chun-Hao P. and Tesch, Joachim and M\\\"uller, Lea and Hilliges, Otmar and Black, Michael J.},\n  booktitle = {Proc. International Conference on Computer Vision (ICCV)},\n  pages = {11035--11045},\n  month = oct,\n  year = {2021},\n  doi = {},\n  month_numeric = {10}\n}\n```\n## License\n\nThis code is available for **non-commercial scientific research purposes** as defined in the [LICENSE file](LICENSE). By downloading and using this code you agree to the terms in the [LICENSE](LICENSE). Third-party datasets and software are subject to their respective licenses.\n\n## References\n\nWe indicate if a function or script is borrowed externally inside each file.\nHere are some great resources we benefit:\n\n- Pano360 dataset preprocessing is borrowed from [Single View Metrology in the Wild](https://github.com/Jerrypiglet/ScaleNet).\n- Most of the utility functions depends on our another paper [PARE](github.com/mkocabas/PARE).\n- Some functions are borrowed from [SPIN](https://github.com/nkolot/SPIN).\n\nConsider citing these works if you use them in your project.\n\n\n\n## Contact\n\nFor questions, please contact spec@tue.mpg.de\n\nFor commercial licensing (and all related questions for business applications), please contact ps-licensing@tue.mpg.de.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmkocabas%2Fspec","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmkocabas%2Fspec","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmkocabas%2Fspec/lists"}