{"id":26189511,"url":"https://github.com/rose-stl-lab/bige","last_synced_at":"2025-10-04T21:16:10.258Z","repository":{"id":270758053,"uuid":"906398520","full_name":"Rose-STL-Lab/BIGE","owner":"Rose-STL-Lab","description":"BIGE is a framework for generative models to adhere to clinician-defined constraints. To generate realistic motion, our method uses a biomechanically informed surrogate model to guide the generation process..","archived":false,"fork":false,"pushed_at":"2025-06-27T05:47:57.000Z","size":37808,"stargazers_count":2,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-27T06:30:27.009Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Rose-STL-Lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-12-20T20:19:00.000Z","updated_at":"2025-06-27T05:48:04.000Z","dependencies_parsed_at":"2025-06-27T06:35:19.029Z","dependency_job_id":null,"html_url":"https://github.com/Rose-STL-Lab/BIGE","commit_stats":null,"previous_names":["rose-stl-lab/bige"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Rose-STL-Lab/BIGE","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rose-STL-Lab%2FBIGE","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rose-STL-Lab%2FBIGE/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rose-STL-Lab%2FBIGE/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rose-STL-Lab%2FBIGE/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Rose-STL-Lab","download_url":"https://codeload.github.com/Rose-STL-Lab/BIGE/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rose-STL-Lab%2FBIGE/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278373516,"owners_count":25976151,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-04T02:00:05.491Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-12T00:38:45.044Z","updated_at":"2025-10-04T21:16:10.252Z","avatar_url":"https://github.com/Rose-STL-Lab.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# BIGE: Biomechanics-informed GenAI for Exercise Science \n \n[![](https://img.shields.io/badge/Project-Page-blue?style=flat\u0026logo=Google%20chrome\u0026logoColor=blue)](https://rose-stl-lab.github.io/UCSD-OpenCap-Fitness-Dataset/)\n[![](https://img.shields.io/badge/Paper-PDF-green?style=flat\u0026logo=arXiv\u0026logoColor=green)](https://rose-stl-lab.github.io/UCSD-OpenCap-Fitness-Dataset/static/pdfs/L4DC_2025_paper_177.pdf)\n[![](https://img.shields.io/badge/Code-Github-red?style=flat\u0026logo=github)](https://gitlab.nrp-nautilus.io/shmaheshwari/digital-coach-anwesh.git)\n[![](https://img.shields.io/badge/tests-passing-brightgreen)](https://github.com/your-repo/actions)\n![](https://img.shields.io/badge/Ubuntu20.04-E95420?style=flat\u0026logo=ubuntu\u0026logoColor=white)\n\u003c!-- ![](https://img.shields.io/badge/Windows-0078D6?style=flat\u0026logo=windows\u0026logoColor=white) --\u003e\n\n\n![Teaser figure](img/BIGE-Diff-Subs.gif)\n\n\u003e BIGE can generate motions with varying muscle activations for subjects with different anthropometrics. \n\n\n## Summary\n\n\n\n\u003e **BIGE** is a framework for generative models to adhere to clinician-defined constraints.  To generate realistic motion, our method uses a biomechanically informed surrogate model to guide the generation process.\n\n![Teaser figure](https://rose-stl-lab.github.io/UCSD-OpenCap-Fitness-Dataset/static/images/Sports_Analytic_Mockup_1_V4.png)\n\n\n\n\nProper movements enhance mobility, coordination, and muscle activation, which are crucial for performance, injury prevention, and overall fitness. However, traditional simulation tools rely on strong modeling assumptions, are difficult to set up, and are computationally expensive. On the other hand, generative AI approaches provide efficient alternatives to motion generation. However they often lack physiological relevance and do not incorporate biomechanical constraints, limiting their practical applications in sports and exercise science. To address these limitations:\n\n- We propose a novel framework, BIGE, that combines bio-mechanically meaningful scoring metrics with generative modeling.\n- BIGE integrates a differentiable surrogate model for muscle activation to reverse optimize the latent space of the generative model.\n- Enables the retrieval of physiologically valid motions through targeted search.\n- Through extensive experiments on squat exercise data, our framework demonstrates superior performance in generating diverse, physically plausible motions while maintaining high fidelity to clinician-defined objectives compared to existing approaches.\n\n\n## Table of Content\n* [1. Visual Results](#1-visual-results)\n* [2. Installation](#2-installation)\n* [3. DEMO](#3-demo)\n* [4. VQ-VAE](#4-vq-vae)\n* [5. Surrogate](#5-surrogate)\n* [6. Guidance](#6-guidance)\n* [7. Metrics](#7-metrics)\n* [8. Rendering](#8-rendering)\n* [9. Acknowledgement](#9-acknowledgement)\n\n\n\n\n\n\n\n## 1. Visual Results (More results can be found on our [project page](https://rose-stl-lab.github.io/UCSD-OpenCap-Fitness-Dataset))\n\u003cimg src=\"https://rose-stl-lab.github.io/UCSD-OpenCap-Fitness-Dataset/static/images/BIGE_Qualitative_Diagram_V4.jpeg\" alt=\"alt text\" style=\"max-height: 960px;\"\u003e\n\n\u003e Our guidance strategy leads to a more physiologically accurate squat motion as evidenced by the increased squat depth. The generated motion samples are ordered by the peak muscle activation. The red and green lines at 50% squat cycle represent the depth of the squat. \n\n\n\u003cvideo controls style=\"max-height: 960px;\"\u003e\n  \u003csource src=\"https://rose-stl-lab.github.io/UCSD-OpenCap-Fitness-Dataset/static/videos/comparision.mp4\" type=\"video/mp4\"\u003e\n  Your browser does not support the video tag.\n\u003c/video\u003e\n \n\u003e Comparison of generated samples from baselines and BIGE. The yellow curve represents the movement of the hip joint over the entire squat cycle. BIGE generates a more realistic squat motion compared to baselines.\n\n## 2. Installation\n\n### 2.1. Environment\n\n\nOur model can be learned in a **single GPU V100-32G**\n\n```bash\nconda env create -f environment.yml\nconda activate T2M-GPT\n\nulimit -n 1000000 # Need to run pytorch open multiple files https://stackoverflow.com/questions/71642653/how-to-resolve-the-error-runtimeerror-received-0-items-of-ancdata\n```\n\nThe code was tested on Python 3.8 and PyTorch 1.8.1.\n\n### 2.3. Datasets\n\nMore details about the dataset used can be found [[here]](https://rose-stl-lab.github.io/UCSD-OpenCap-Fitness-Dataset/ucsd-opencap-dataset.html)\n\n### 2.5. Pre-trained models \n\nThe pretrained model files will be stored in the 'pretrained' folder:\n\n## 3. DEMO (Coming Soon)\nWebsite: https://bige25f-chat.nrp-nautilus.io/\n\nDocker Image\n\n## 4. VQ-VAE\n\nTrain the VQVAE model with specified parameters.\n```bash\npython3 train_vq.py --batch-size 256 --lr 2e-4 --total-iter 300000 --lr-scheduler 200000 --nb-code 512 --down-t 2 --depth 3 --dilation-growth-rate 3 --out-dir output --dataname mcs --vq-act relu --quantizer ema_reset --loss-vel 0.5 --recons-loss l1_smooth --exp-name VQVAE9\n```\n\u003cdetails\u003e\n\u003csummary\u003eVQVAE Training without DeepSpeed\u003c/summary\u003e\nTrain the VQVAE model using DeepSpeed for optimized performance.\n\n```bash\npython3.8 /home/ubuntu/.local/bin/deepspeed train_vq.py --batch-size 32 --lr 2e-4 --total-iter 300000 --lr-scheduler 200000 --nb-code 512 --down-t 2 --depth 3 --dilation-growth-rate 3 --out-dir output --dataname mcs --vq-act relu --quantizer ema_reset --loss-vel 0.5 --recons-loss l1_smooth --exp-name VQVAE9_Bs32 --loss_mode separate\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e \u003csummary\u003eVQVAE Reconstruction\u003c/summary\u003e\n\nGenerate samples using the trained VQVAE model.\n\n```bash\npython MOT_eval.py --dataname mcs --out-dir output --exp-name VQVAE5_v2 --resume-pth output/VQVAE5_v2/300000.pth\n```\n\n\u003c/details\u003e\n\n\n## 5. Surrogate\nRun surrogate training for the model.\n```bash\npython3.8 surrogate_training.py\n```\n\n## 6. Guidance\n\n```bash\npython LIMO_Surrogate.py --exp-name TestBIGE --vq-name /home/ubuntu/data/T2M-GPT/output/VQVAE14/120000.pth  --dataname mcs --seq-len 49 --total-iter 3000 --lr 0.5 --num-runs 3000 --min-samples 20  --subject /data/panini/MCS_DATA/Data/d66330dc-7884-4915-9dbb-0520932294c4 --low 0.35 --high 0.45\n```\n\n\u003cdetails\u003e\u003csummary\u003eRun guidance for all subjects\u003c/summary\u003e\n\n```python\nimport os\nmcs_sessions = [\"349e4383-da38-4138-8371-9a5fed63a56a\",\"015b7571-9f0b-4db4-a854-68e57640640d\",\"c613945f-1570-4011-93a4-8c8c6408e2cf\",\"dfda5c67-a512-4ca2-a4b3-6a7e22599732\",\"7562e3c0-dea8-46f8-bc8b-ed9d0f002a77\",\"275561c0-5d50-4675-9df1-733390cd572f\",\"0e10a4e3-a93f-4b4d-9519-d9287d1d74eb\",\"a5e5d4cd-524c-4905-af85-99678e1239c8\",\"dd215900-9827-4ae6-a07d-543b8648b1da\",\"3d1207bf-192b-486a-b509-d11ca90851d7\",\"c28e768f-6e2b-4726-8919-c05b0af61e4a\",\"fb6e8f87-a1cc-48b4-8217-4e8b160602bf\",\"e6b10bbf-4e00-4ac0-aade-68bc1447de3e\",\"d66330dc-7884-4915-9dbb-0520932294c4\",\"0d9e84e9-57a4-4534-aee2-0d0e8d1e7c45\",\"2345d831-6038-412e-84a9-971bc04da597\",\"0a959024-3371-478a-96da-bf17b1da15a9\",\"ef656fe8-27e7-428a-84a9-deb868da053d\",\"c08f1d89-c843-4878-8406-b6f9798a558e\",\"d2020b0e-6d41-4759-87f0-5c158f6ab86a\",\"8dc21218-8338-4fd4-8164-f6f122dc33d9\"]\nexp_name = \"FinalFinalHigh\"\nfor session in mcs_sessions:\n  os.system(f\"python LIMO_Surrogate.py --exp-name {exp_name} --vq-name /data/panini/T2M-GPT/output/VQVAE14/120000.pth  --dataname mcs --seq-len 49 --total-iter 3000 --lr 0.5 --num-runs 3000 --min-samples 20  --subject /data/panini/MCS_DATA/Data/{session} --low 0.35 --high 0.45\")\n```\n\n\u003c/details\u003e\n\n\n\n## 7. Metrics \n### 7.1 Calculate Wasserstein Distance \u0026 Entropy\n\n\n  - For BIGE\n    ```\n    python calculate_wasserstein.py --file_type mot --folder_path /home/ubuntu/data/MCS_DATA/LIMO/FinalFinalHigh/mot_visualization/\n    ``` \n\n - For motion capture\n    ```bash\n    python wasserstein_mocap.py --file_type mot --folder_path /home/ubuntu/data/MCS_DATA/Data/\n    ```\n  - For baselines \n    ```bash\n    python calculate_wasserstein.py --file_type mot --folder_path /home/ubuntu/data/MCS_DATA/baselines/mdm_baseline/\n    python calculate_wasserstein.py --file_type mot --folder_path /home/ubuntu/data/MCS_DATA/baselines/t2m_baseline/\n    python calculate_wasserstein.py --file_type mot --folder_path /home/ubuntu/data/MCS_DATA/LIMO/VQVAE-Generations/mot_visualization/\n    ```\n\n\n### 7.2 Calculate Guidance metrics\n\n  - For references \n    ```bash\n    python calculate_guidance.py --file_type mocap --folder_path /home/ubuntu/data/MCS_DATA/Data/ \n    ```\n\n  - For baselines \n    ```\n      python calculate_guidance.py  --file_type mot --folder_path /home/ubuntu/data/MCS_DATA/baselines/mdm_baseline/\n      python calculate_guidance.py --file_type mot --folder_path /home/ubuntu/data/MCS_DATA/baselines/t2m_baseline/\n      python calculate_guidance.py --file_type mot --folder_path /home/ubuntu/data/MCS_DATA/LIMO/VQVAE-Generations/mot_visualization/\n    ``` \n\n  - For BIGE \n    ```   \n    python calculate_guidance.py --file_type mot --folder_path /home/ubuntu/data/MCS_DATA/LIMO/FinalFinalHigh/mot_visualization/\n    ```\n\n## 8. Rendering\n\nTo render videos and images, install the [UCSD-OpenCap-Fitness-Dataset](https://rose-stl-lab.github.io/UCSD-OpenCap-Fitness-Dataset/ucsd-opencap-dataset.html)  \n\n### Installation \n\n```\ngit clone https://github.com/Rose-STL-Lab/UCSD-OpenCap-Fitness-Dataset\n```\n\n\nGenerate MP4 videos from MOT files.\n```bash\ncd UCSD-OpenCap-Fitness_Dataset/\nexport DISPLAY=:99.0\n```\n\n\n### Compare MoCap and generated sample\n\n```\npython src/opencap_reconstruction_render.py \u003cabsolute subject-path\u003e \u003cabsolute mot-path\u003e \u003cabsolute save-path\u003e # Compare \n```\n\n\n### Compare 3 .mot files \n\n```\npython src/plot_3.py \u003cmot-path-1\u003e \u003cmot-path-2\u003e \u003cmot-path-3\u003e \u003coptonal-video-path\u003e\n```\n\n\u003cdetails\u003e\u003csummary\u003eExamples: \u003c/summary\u003e   \n- BIGE\n\n```\npython src/plot_3.py  MCS_DATA/LIMO/FinalFinalHigh/mot_visualization/latents_subject_run_d66330dc-7884-4915-9dbb-0520932294c4/entry_{0,2,19}_FinalFinalHigh.mot render/bige_3.mp4\n```\n\n- MDM  \n```\npython src/plot_3.py  MCS_DATA/mdm_baseline/015b7571-9f0b-4db4-a854-68e57640640d/results_*_radians.mot\n```\n\n\n- T2M\n```\npython src/plot_3.py  MCS_DATA/mdm_baseline/015b7571-9f0b-4db4-a854-68e57640640d/results_*_radians.mot\n```\n\n\n- Simulation \n```\n python src/plot_3.py  MCS_DATA/Data/c613945f-1570-4011-93a4-8c8c6408e2cf/OpenSimData/Dynamics/SQT01_segment_?/kinematics_activations_SQT01_segment_?_muscle_driven.mot\n```\n\n\n- For MoCap data collected using opencap\n```\n python src/plot_3.py MCS_DATA/Data/fb6e8f87-a1cc-48b4-8217-4e8b160602bf/MarkerData/SQT01.trc\n```\n\n\n- For Simulation data\n```\npython src/plot_3.py  MCS_DATA/Data/\u003csubject-path\u003e/OpenSimData/Dynamics/SQT01_segment_?/kinematics_activations_SQT01_segment_?_muscle_driven.mot \n\n```\n\n- OpenCap Dataset \n\n\n\n\u003c/details\u003e\n\n\n\n\n### 9. Acknowledgment\n\nWe extend our gratitude to The Wu Tsai Human Performance Alliance for their invaluable support and resources. Their dedication to advancing human performance through interdisciplinary research has been instrumental to our project.\n\nThis work utilized resources provided by the National Research Platform (NRP) at the University of California, San Diego. The NRP is developed and supported in part by funding from the National Science Foundation under awards 1730158, 1540112, 1541349, 1826967, 2112167, 2100237, and 2120019, along with additional support from community partners.\n\nWe also acknowledge the contributions of public code repositories such as [text-to-motion](https://github.com/EricGuo5513/text-to-motion), [T2M-GPT](https://github.com/Mael-zys/T2M-GPT), [MDM](https://github.com/GuyTevet/motion-diffusion-model), and [MotionDiffuse](https://github.com/mingyuan-zhang/MotionDiffuse).\n\nAdditionally, we appreciate the open-source motion capture systems like [OpenCap](https://github.com/stanfordnmbl/opencap-processing).\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frose-stl-lab%2Fbige","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frose-stl-lab%2Fbige","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frose-stl-lab%2Fbige/lists"}