{"id":19196282,"url":"https://github.com/kupynorest/head_detector","last_synced_at":"2025-04-05T00:08:19.558Z","repository":{"id":250297504,"uuid":"757456161","full_name":"KupynOrest/head_detector","owner":"KupynOrest","description":"Official repo for VGGHeads: 3D Multi Head Alignment with a Large-Scale Synthetic Dataset..","archived":false,"fork":false,"pushed_at":"2024-12-20T17:28:42.000Z","size":51036,"stargazers_count":172,"open_issues_count":4,"forks_count":9,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-03-28T23:06:52.841Z","etag":null,"topics":["3dmm","computer-vision","diffusion-models","face-detection","face-mesh-detection","generative-ai","head-detection","mesh-network"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/KupynOrest.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-14T14:33:14.000Z","updated_at":"2025-03-21T03:25:53.000Z","dependencies_parsed_at":"2024-12-15T07:04:48.347Z","dependency_job_id":"7e32f004-7b06-415b-b0ea-6ba4efcb4721","html_url":"https://github.com/KupynOrest/head_detector","commit_stats":null,"previous_names":["kupynorest/head_detector"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KupynOrest%2Fhead_detector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KupynOrest%2Fhead_detector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KupynOrest%2Fhead_detector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KupynOrest%2Fhead_detector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/KupynOrest","download_url":"https://codeload.github.com/KupynOrest/head_detector/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247266564,"owners_count":20910836,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3dmm","computer-vision","diffusion-models","face-detection","face-mesh-detection","generative-ai","head-detection","mesh-network"],"created_at":"2024-11-09T12:13:16.642Z","updated_at":"2025-04-05T00:08:19.539Z","avatar_url":"https://github.com/KupynOrest.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n# VGGHeads: 3D Multi Head Alignment with a Large-Scale Synthetic Dataset\n\n[**Orest Kupyn**](https://github.com/KupynOrest)\u003csup\u003e13\u003c/sup\u003e · [**Eugene Khvedchenia**](https://github.com/BloodAxe)\u003csup\u003e2\u003c/sup\u003e · [**Christian Rupprecht**](https://chrirupp.github.io/)\u003csup\u003e1\u003c/sup\u003e ·\n\n\u003csup\u003e1\u003c/sup\u003eUniversity of Oxford · \u003csup\u003e2\u003c/sup\u003eUkrainian Catholic University · \u003csup\u003e3\u003c/sup\u003ePiñataFarms AI\n\n\u003ca href='https://www.robots.ox.ac.uk/~vgg/research/vgg-heads/'\u003e\u003cimg src='https://img.shields.io/badge/Project-Page-green'\u003e\u003c/a\u003e\n\u003ca href='https://arxiv.org/abs/2407.18245'\u003e\u003cimg src='https://img.shields.io/badge/arXiv Paper-red'\u003e\u003c/a\u003e\n\u003ca href='https://huggingface.co/spaces/okupyn/vgg_heads'\u003e\u003cimg src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue'\u003e\u003c/a\u003e\n\u003ca href='https://huggingface.co/okupyn/head-mesh-controlnet-xl'\u003e\u003cimg src='https://img.shields.io/badge/%F0%9F%A4%97%20ControlNet%20XL-blue'\u003e\u003c/a\u003e\n[![Model](https://img.shields.io/badge/Model-Weights-blue)](https://huggingface.co/okupyn/vgg_heads)\n\n\u003c/div\u003e\n\nVGGHeads is a large-scale fully synthetic dataset for human head detection and 3D mesh estimation with over 1 million images generated with diffusion models. A model trained only on synthetic data generalizes well to real-world and is capable of simultaneous heads detection and head meshes reconstruction from a single image in a single step.\n\n![banner](./images/banner.jpg)\n\n## News\n- [2024/08/30] 🔥 Release Version 0.1.0. Added examples of Head Alignment and Saving Meshes as .obj\n- [2024/08/29] 🔥🔥 We release the dataset, training instructions and ONNX weights!!\n- [2024/08/09] 🔥 We release VGGHeads_L Checkpoint and [Mesh ControlNet](https://huggingface.co/okupyn/head-mesh-controlnet-xl)\n- [2024/07/26] 🔥 We release the initial version of the codebase, the paper, project webpage and an image demo!!\n\n## VGGHeads Dataset Download Instructions\n\n### 1. Download the Dataset\n\nTo download the VGGHeads dataset, you have two options:\n\n1. Torrent download (preferred method):\n \u003ca href='https://academictorrents.com/docs/downloading.html'\u003eHow To Download\u003c/a\u003e\n```bash\npip install academictorrents\nat-get 1ac36f16386061685ed303dea6f0d6179d2e2121\n```\nor use  \u003ca href='https://aria2.github.io/'\u003earia2c\u003c/a\u003e\n\n```bash\naria2c --seed-time=0 --max-overall-download-limit=10M --file-allocation=none https://academictorrents.com/download/1ac36f16386061685ed303dea6f0d6179d2e2121.torrent\n```\n\n\n\n\u003ca href='https://academictorrents.com/download/1ac36f16386061685ed303dea6f0d6179d2e2121.torrent'\u003eFull Torrent Link\u003c/a\u003e\n\nWe recommend using the torrent method as it's typically faster and helps reduce the load on our servers.\n\n2. Direct download:\n   \n```bash\nwget https://thor.robots.ox.ac.uk/vgg-heads/VGGHeads.tar\n```\n\nThis will download a file named `VGGHeads.tar` to your current directory.\n\n### 2. Download the MD5 Checksums\n\nTo verify the integrity of the downloaded file, we'll need the MD5 checksums. Download them using:\n\n```bash\nwget https://thor.robots.ox.ac.uk/vgg-heads/MD5SUMS\n```\n\n### 3. Verify the Download\n\nAfter both files are downloaded, verify the integrity of the `VGGHeads.tar` file:\n\n```bash\nmd5sum -c MD5SUMS\n```\n\nIf the download was successful and the file is intact, you should see an \"OK\" message.\n\n### 4. Extract the Dataset\n\nIf the verification was successful, extract the contents of the tar file:\n\n```bash\ntar -xvf VGGHeads.tar\n```\n\nThis will extract the contents of the archive into your current directory.\n\nNotes:\n\n- The size of the dataset is approximately 187 GB. Ensure you have sufficient disk space before downloading and extracting.\n- The download and extraction process may take some time depending on your internet connection and computer speed.\n- If you encounter any issues during the download or extraction process, try the download again or check your system's tar utility.\n\n## Installation\n\n#### Create a Conda virtual environment\n\n```bash\nconda create --name vgg_heads python=3.10\nconda activate vgg_heads\n```\n\n#### Clone the project and install the package\n\n```bash\ngit clone https://github.com/KupynOrest/head_detector.git\ncd head_detector\n\npip install -e ./\n```\n\nOr simply install\n\n```bash\npip install git+https://github.com/KupynOrest/head_detector.git\n```\n\n## Usage\n\nTo test VGGHeads model on your own images simply use this code:\n\n```python\nfrom head_detector import HeadDetector\nimport cv2\ndetector = HeadDetector()\nimage_path = \"your_image.jpg\"\npredictions = detector(image_path)\n# predictions.heads contain a list of heads with .bbox, .vertices_3d, .head_pose params\nresult_image = predictions.draw() # draw heads on the image\ncv2.imwrite(\"result.png\",result_image) # save result image to preview it.\n```\n\n### Exporting Head Meshes\n\nYou can export head meshes as OBJ files using the `save_meshes` method:\n\n```python\n# After getting predictions\nsave_folder = \"path/to/save/folder\"\npredictions.save_meshes(save_folder)\n```\n\nThis will save individual OBJ files for each detected head in the specified folder.\n\n### Getting Aligned Head Crops\n\nTo obtain aligned head crops, use the `get_aligned_heads` method:\n\n```python\n# After getting predictions\naligned_heads = predictions.get_aligned_heads()\n\n# Process or save aligned head crops\nfor i, head in enumerate(aligned_heads):\n    cv2.imwrite(f\"aligned_head_{i}.png\", head)\n```\n\nThis returns a list of aligned head crops that you can further process or save.\n\n### Extended Example\n\nHere's a complete example incorporating all features:\n\n```python\nfrom head_detector import HeadDetector\nimport cv2\nimport os\n\n# Initialize the detector\ndetector = HeadDetector()\n\n# Specify the path to your image\nimage_path = \"your_image.jpg\"\n\n# Get predictions\npredictions = detector(image_path)\n\n# Draw heads on the image\nresult_image = predictions.draw()\ncv2.imwrite(\"result.png\", result_image)\n\n# Save head meshes\nsave_folder = \"head_meshes\"\nos.makedirs(save_folder, exist_ok=True)\npredictions.save_meshes(save_folder)\n\n# Get and save aligned head crops\naligned_heads = predictions.get_aligned_heads()\nfor i, head in enumerate(aligned_heads):\n    cv2.imwrite(f\"aligned_head_{i}.png\", head)\n\nprint(f\"Detected {len(predictions.heads)} heads.\")\nprint(f\"Result image saved as 'result.png'\")\nprint(f\"Head meshes saved in '{save_folder}' folder\")\nprint(f\"Aligned head crops saved as 'aligned_head_*.png'\")\n```\n\nThis extended example demonstrates how to use all the features of the VGGHeads model, including basic head detection, drawing results, exporting head meshes, and obtaining aligned head crops.\n\nAdditionally, the ONNX weights are available at \u003ca href='https://huggingface.co/okupyn/vgg_heads/tree/main'\u003eHuggingFace\u003c/a\u003e. The example of the inference can be found at: \u003ca href='https://colab.research.google.com/drive/1EJn9dPdlX2qIWrZok9LF185ZJwAGOr9Y'\u003eColab\u003c/a\u003e\n\n## Gradio Demo\n\nWe also provide a Gradio \u003ca href='https://github.com/gradio-app/gradio'\u003e\u003cimg src='https://img.shields.io/github/stars/gradio-app/gradio'\u003e\u003c/a\u003e demo, which you can run locally:\n\n```bash\ncd gradio\npip install -r requirements.txt\npython app.py\n```\nYou can specify the `--server_port`, `--share`, `--server_name` arguments to satisfy your needs!\n\n## Training\n\nCheck `yolo_head_training/Makefile` for examples of train scripts.\n\nTo run the training on all data with Distributed Data Parallel (DDP), use the following command:\n\n```bash\ntorchrun --standalone --nnodes=1 --nproc_per_node=NUM_GPUS train.py --config-name=yolo_heads_l \\\n    dataset_params.train_dataset_params.data_dir=DATA_FOLDER/large \\\n    dataset_params.val_dataset_params.data_dir=DATA_FOLDER/large \\\n    num_gpus=NUM_GPUS multi_gpu=DDP\n```\n\nReplace the following placeholders:\n- `NUM_GPUS`: The number of GPUs you want to use for training.\n- `DATA_FOLDER`: The path to the directory containing your extracted dataset.\n\n### Additional Training Options\n\n1. Single GPU Training:\n   If you're using a single GPU, you can simplify the command:\n   ```bash\n   python train.py --config-name=yolo_heads_l \\\n       dataset_params.train_dataset_params.data_dir=DATA_FOLDER/large \\\n       dataset_params.val_dataset_params.data_dir=DATA_FOLDER/large\n   ```\n\n2. Custom Configuration:\n   You can modify the `--config-name` parameter to use different model configurations. Check the configuration files in the project directory for available options.\n\n3. Adjusting Hyperparameters:\n   You can adjust various hyperparameters by adding them to the command line. For example:\n   ```bash\n   python train.py --config-name=yolo_heads_l \\\n       dataset_params.train_dataset_params.data_dir=DATA_FOLDER/large \\\n       dataset_params.val_dataset_params.data_dir=DATA_FOLDER/large \\\n       training_hyperparams.initial_lr=0.001 \\\n       training_hyperparams.max_epochs=100\n   ```\n\n4. Resuming Training:\n   If you need to resume training from a checkpoint, you can use the `training_hyperparams.resume` flag:\n   ```bash\n   python train.py --config-name=yolo_heads_l \\\n       dataset_params.train_dataset_params.data_dir=DATA_FOLDER/large \\\n       dataset_params.val_dataset_params.data_dir=DATA_FOLDER/large \\\n       training_hyperparams.resume=True\n   ```\n\n### Monitoring Training\n\nYou can monitor the training progress through the console output. Consider using tools like TensorBoard for more detailed monitoring and visualization of training metrics.\n\n## Cite\n\nIf you find VGGHeads useful for your research and applications, please cite us using this BibTeX:\n\n```bibtex\n@article{vggheads,\n      title={VGGHeads: 3D Multi Head Alignment with a Large-Scale Synthetic Dataset},\n      author={Orest Kupyn and Eugene Khvedchenia and Christian Rupprecht},\n      year={2024},\n      eprint={2407.18245},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV},\n      url={https://arxiv.org/abs/2407.18245},\n}\n```\n\n [![CC BY-NC 4.0][cc-by-nc-shield]][cc-by-nc]\n\nThis work is licensed under a\n[Creative Commons Attribution-NonCommercial 4.0 International License][cc-by-nc].\n\n[![CC BY-NC 4.0][cc-by-nc-image]][cc-by-nc]\n\n[cc-by-nc]: https://creativecommons.org/licenses/by-nc/4.0/\n[cc-by-nc-image]: https://licensebuttons.net/l/by-nc/4.0/88x31.png\n[cc-by-nc-shield]: https://img.shields.io/badge/License-CC%20BY--NC%204.0-lightgrey.svg\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkupynorest%2Fhead_detector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkupynorest%2Fhead_detector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkupynorest%2Fhead_detector/lists"}