{"id":21192631,"url":"https://github.com/cr00z/virtual-tryon","last_synced_at":"2025-10-19T20:45:35.302Z","repository":{"id":192066210,"uuid":"456887909","full_name":"cr00z/virtual-tryon","owner":"cr00z","description":"Виртуальная примерка 3D модели одежды на видео (что-то в духе Clometrica)","archived":false,"fork":false,"pushed_at":"2023-11-16T04:07:33.000Z","size":11593,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2023-11-16T05:21:58.486Z","etag":null,"topics":["frankmocap","multigarmentnetwork","python","pytorch3d","smpl"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cr00z.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-02-08T10:37:28.000Z","updated_at":"2023-10-03T01:39:51.000Z","dependencies_parsed_at":"2023-09-02T11:27:44.180Z","dependency_job_id":null,"html_url":"https://github.com/cr00z/virtual-tryon","commit_stats":null,"previous_names":["cr00z/virtual-tryon"],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cr00z%2Fvirtual-tryon","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cr00z%2Fvirtual-tryon/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cr00z%2Fvirtual-tryon/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cr00z%2Fvirtual-tryon/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cr00z","download_url":"https://codeload.github.com/cr00z/virtual-tryon/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225615205,"owners_count":17496946,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["frankmocap","multigarmentnetwork","python","pytorch3d","smpl"],"created_at":"2024-11-20T19:10:31.553Z","updated_at":"2025-10-19T20:45:30.273Z","avatar_url":"https://github.com/cr00z.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Virtual try-on\n\nВиртуальная примерка 3D модели одежды на видео\n(что-то в духе [Clometrica](https://www.clometrica.com/))\n\n## Output\n\nПример вывода\n\n* [Штаны](https://github.com/cr00z/virtual-tryon/blob/master/output/out2.mp4)\n* [Майка](https://github.com/cr00z/virtual-tryon/blob/master/output/out1.mp4)\n\n## Описание\n\n\u003cimg align=\"right\" alt=\"demo2\" src=\"https://raw.githubusercontent.com/cr00z/virtual-tryon/master/output/demo2.jpg\" width=\"144\" height=\"256\"  border=\"1\" hspace=\"5\"/\u003e\n\u003cimg align=\"right\" alt=\"demo1\" src=\"https://raw.githubusercontent.com/cr00z/virtual-tryon/master/output/demo1.jpg\" width=\"144\" height=\"256\" border=\"1\" hspace=\"5\"/\u003e\n\n## Установка (Colab)\n\nДля установки и запуска демо можно использовать файл demo.ipynb. Загрузите его на Colab и выполните.\n\n1. Clone repository\n```\n!git clone https://github.com/cr00z/virtual-tryon\n!cp -r virtual-tryon/* .\n```\n2. Install detectron2 (for bbox detection)\n```\n!pip install 'git+https://github.com/facebookresearch/detectron2.git'\n```\n3. Install opendr\n```\n!sudo apt-get --yes install libglu1-mesa-dev freeglut3-dev mesa-common-dev\n!sudo apt-get --yes install libosmesa6-dev\n!pip install opendr\n```\n4. Install pytorch3d (for rendering)\n\n*I ran on version 0.6.1, the old version 0.3.0 is installed on the colab by default, so we install from github*\n\nAchtung! May take up to 10-20 minutes, please wait! \n```\n%%time\npip -v install \"git+https://github.com/facebookresearch/pytorch3d.git\"\n```\n5. Install [Mesh](https://github.com/MPI-IS/mesh)\n```\n!sudo apt-get --yes install libboost-dev\n!pip install 'git+https://github.com/MPI-IS/mesh.git'\n```\n6. Download extra data\n\nDownload the neutral SMPL model from http://smplify.is.tue.mpg.de/ and place it in the assets folder.\n```\n!echo \"Download the neutral SMPL model\"\n!wget https://github.com/classner/up/raw/master/models/3D/basicModel_neutral_lbs_10_207_0_v1.0.0.pkl -P assets\n\n!echo \"Downloading extra data from SPIN\"\n!wget http://visiondata.cis.upenn.edu/spin/data.tar.gz \u0026\u0026 tar -xvf data.tar.gz \u0026\u0026 rm data.tar.gz \u0026\u0026 mv data/smpl_mean_params.npz assets \u0026\u0026 rm -rf data\n\n!echo \"Downloading pretrained model\"\n!wget https://dl.fbaipublicfiles.com/eft/2020_05_31-00_50_43-best-51.749683916568756.pt -P assets\n\n!echo \"Download garment fts from MultiGarmentNetwork\"\n!wget https://github.com/bharat-b7/MultiGarmentNetwork/raw/master/assets/garment_fts.pkl -P assets\n```\n7. Download Multi-Garment dataset\n```\n!wget https://datasets.d2.mpi-inf.mpg.de/MultiGarmentNetwork/Multi-Garmentdataset.zip\n!unzip Multi-Garmentdataset.zip\n```\n\n## Запуск\n\nВ файле main.py отредактировать путь к входному видеофайлу:\n```\ninput_path = './sample_data/single_totalbody.mp4'\n```\nRun it:\n```\n!python main.py\n```\nКак забрать выходной файл output/out.mp4 через блокнот:\n```\nfrom IPython.display import FileLink\nFileLink(r'output/out.mp4')\n```\n\n### Основная часть (пайплайн):\n\n- берется готовый меш одежды\n- на вход приходит видео человека в полный рост (перед зеркалом/снятого от 3-го лица)\n- предсказывается 3D-поза человека по этому видео (покадрово)\n- далее на каждом кадре видео:\n  - поза конвертируется в нужный формат для перепозирования 3D модели одежды \n  - перепозированный меш одежды рендерится (отрисовывается) поверх картинки\n\n### Задача:\n\n1) ~~Изучить материалы.~~\n2) ~~Скачать и настроить датасет с 3D-одеждой (лучше оба), которая позволяет менять свою позу.~~\n3) ~~Придумать решение проблемы адаптации формы одежды под конкретное тело человека.~~\n4) ~~Запустить и добиться корректности работы хотя бы 1-го метода предсказания 3D позы человека по видео.~~\n5) ~~Реализовать конвертацию формата 3D позы человека в 3D позу меша одежды.~~\n6) ~~Реализовать рендеринг (отрисовку) 3D модели одежды поверх видео.~~\n\n## Полезные ссылки:\n\n1. Статьи на русском про 3DML:\n   * https://m.habr.com/ru/company/itmai/blog/503358/\n2. Рассказ про моделирование 3D одежды:\n   * https://youtu.be/ySx-iE-mb0s \n3. Модель камеры:\n   * http://www.cse.psu.edu/~rtc12/CSE486/lecture12.pdf\n   * http://www.cse.psu.edu/~rtc12/CSE486/lecture13.pdf \n4. Библиотеки:\n   * pytorch3d: https://pytorch3d.org\n   * https://pytorch3d.org/tutorials/render_textured_meshes\n   * open3d: http://www.open3d.org\n   * pyrender: https://github.com/mmatl/pyrender\n5. SMPL: \n   * https://www.youtube.com/watch?v=kuBlUyHeV5U\n   * https://khanhha.github.io/posts/SMPL-model-introduction/ \n6. Предсказание 3D-позы: \n   * https://github.com/facebookresearch/frankmocap\n   * https://github.com/vchoutas/expose\n   * https://google.github.io/mediapipe/solutions/pose\n7. Датасеты мешей одежды: \n   * https://github.com/bharat-b7/MultiGarmentNetwork\n   * https://github.com/jby1993/BCNet\n   * https://github.com/aymenmir1/pix2surf\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcr00z%2Fvirtual-tryon","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcr00z%2Fvirtual-tryon","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcr00z%2Fvirtual-tryon/lists"}