{"id":16147857,"url":"https://github.com/eigenvivek/diffpose","last_synced_at":"2025-05-08T05:55:18.146Z","repository":{"id":211344075,"uuid":"721755894","full_name":"eigenvivek/DiffPose","owner":"eigenvivek","description":"[CVPR 2024] Intraoperative 2D/3D registration via differentiable X-ray rendering","archived":false,"fork":false,"pushed_at":"2025-04-21T19:39:28.000Z","size":127922,"stargazers_count":138,"open_issues_count":3,"forks_count":18,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-05-08T05:55:04.048Z","etag":null,"topics":["2d-3d-registration","camera-pose-estimation","differentiable-rendering","medical-imaging"],"latest_commit_sha":null,"homepage":"http://vivekg.dev/DiffPose/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eigenvivek.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-11-21T17:54:43.000Z","updated_at":"2025-05-02T04:18:01.000Z","dependencies_parsed_at":"2025-04-23T04:01:50.484Z","dependency_job_id":null,"html_url":"https://github.com/eigenvivek/DiffPose","commit_stats":null,"previous_names":["eigenvivek/diffpose"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eigenvivek%2FDiffPose","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eigenvivek%2FDiffPose/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eigenvivek%2FDiffPose/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eigenvivek%2FDiffPose/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eigenvivek","download_url":"https://codeload.github.com/eigenvivek/DiffPose/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253009851,"owners_count":21839713,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["2d-3d-registration","camera-pose-estimation","differentiable-rendering","medical-imaging"],"created_at":"2024-10-10T00:28:04.042Z","updated_at":"2025-05-08T05:55:18.133Z","avatar_url":"https://github.com/eigenvivek.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DiffPose\n\n\u003c!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! --\u003e\n\n\u003e Intraoperative 2D/3D registration via differentiable X-ray rendering\n\n[![CI](https://github.com/eigenvivek/DiffPose/actions/workflows/test.yaml/badge.svg)](https://github.com/eigenvivek/DiffPose/actions/workflows/test.yaml)\n[![Paper\nshield](https://img.shields.io/badge/arXiv-2312.06358-red.svg)](https://arxiv.org/abs/2312.06358)\n[![License:\nMIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)\n[![Docs](https://github.com/eigenvivek/DiffPose/actions/workflows/deploy.yaml/badge.svg)](https://vivekg.dev/DiffPose)\n[![Code style:\nblack](https://img.shields.io/badge/Code%20style-black-black.svg)](https://github.com/psf/black)\n\n![](experiments/test_time_optimization.gif)\n\n\u003e [!NOTE]\n\u003e\n\u003e If you're considering using `DiffPose` for your own data, instead use [`xvr`](https://github.com/eigenvivek/xvr), the\n\u003e successor project from the same authors.\n\u003e\n\u003e `xvr` is actively maintained and offers many features not present in `DiffPose`, such as a Python API and CLI for \n\u003e training your own patient-specific pose regression networks (in about 5 minutes!) and performing test-time optimization.\n\n## Install\n\nTo install `DiffPose` and the requirements in\n[`environment.yml`](https://github.com/eigenvivek/DiffPose/blob/main/environment.yml),\nrun:\n\n``` zsh\npip install diffpose\n```\n\nThe differentiable X-ray renderer that powers the backend of `DiffPose`\nis available at [`DiffDRR`](https://github.com/eigenvivek/DiffDRR).\n\n## Datasets\n\nWe evaluate `DiffPose` networks on the following open-source datasets:\n\n| **Dataset**                                                                | **Anatomy**        | **\\# of Subjects** | **\\# of 2D Images** | **CTs** | **X-rays** | Fiducials |\n|----------------------------------------------------------------------------|--------------------|:------------------:|:-------------------:|:-------:|:----------:|:---------:|\n| [`DeepFluoro`](https://github.com/rg2/DeepFluoroLabeling-IPCAI2020)        | Pelvis             |         6          |         366         |   ✅    |     ✅     |    ❌     |\n| [`Ljubljana`](https://lit.fe.uni-lj.si/en/research/resources/3D-2D-GS-CA/) | Cerebrovasculature |         10         |         20          |   ✅    |     ✅     |    ✅     |\n\n\u003c!-- | [`2D-3D-GS`](https://lit.fe.uni-lj.si/en/research/resources/2D-3D-GS/)     | Lumbar Spine       |          1         |          18         |    ✅    |      ✅     |     ✅     |\n| [`VerSe`](https://github.com/anjany/verse)                                 | Spine              |         355        |         N/A         |    ✅    |      ❌     |     ❌     | --\u003e\n\n- `DeepFluoro` ([**Grupp et al.,\n  2020**](https://link.springer.com/article/10.1007/s11548-020-02162-7))\n  provides paired X-ray fluoroscopy images and CT volume of the pelvis.\n  The data were collected from six cadaveric subjects at John Hopkins\n  University. Ground truth camera poses were estimated with an offline\n  registration process. A visualization of one X-ray / CT pair in the\n  `DeepFluoro` dataset is available\n  [here](https://vivekg.dev/DiffPose/experiments/render.html).\n\n``` zsh\nmkdir -p data/\nwget --no-check-certificate -O data/ipcai_2020_full_res_data.zip \"http://archive.data.jhu.edu/api/access/datafile/:persistentId/?persistentId=doi:10.7281/T1/IFSXNV/EAN9GH\"\nunzip -o data/ipcai_2020_full_res_data.zip -d data\nrm data/ipcai_2020_full_res_data.zip\n```\n\n- `Ljubljana` ([**Mitrovic et al.,\n  2013**](https://ieeexplore.ieee.org/abstract/document/6507588))\n  provides paired 2D/3D digital subtraction angiography (DSA) images.\n  The data were collected from 10 patients undergoing endovascular\n  image-guided interventions at the University of Ljubljana. Ground\n  truth camera poses were estimated by registering surface fiducial\n  markers.\n\n``` zsh\nmkdir -p data/\nwget --no-check-certificate -O data/ljubljana.zip \"https://drive.google.com/uc?export=download\u0026confirm=yes\u0026id=1x585pGLI8QGk21qZ2oGwwQ9LMJ09Tqrx\"\nunzip -o data/ljubljana.zip -d data\nrm data/ljubljana.zip\n```\n\n\u003c!-- - `2D-3D-GS` ([**Tomaževič et al., 2004**](https://pubmed.ncbi.nlm.nih.gov/16192053/)) ...\n\u0026#10;- `VerSe` ([**Sekuboyina et al., 2020**](https://pubs.rsna.org/doi/10.1148/ryai.2020190074)) ... --\u003e\n\n## Experiments\n\nTo run the experiments in `DiffPose`, run the following scripts (ensure\nyou’ve downloaded the data first):\n\n``` zsh\n# DeepFluoro dataset\ncd experiments/deepfluoro\nsrun python train.py     # Pretrain pose regression CNN on synthetic X-rays\nsrun python register.py  # Run test-time optimization with the best network per subject\n```\n\n``` zsh\n# Ljubljana dataset\ncd experiments/ljubljana\nsrun python train.py\nsrun python register.py\n```\n\nThe training and test-time optimization scripts use SLURM to run on all\nsubjects in parallel:\n\n- `experiments/deepfluoro/train.py` is configured to run across six\n  A6000 GPUs\n- `experiments/deepfluoro/register.py` is configured to run across six\n  2080 Ti GPUs\n- `experiments/ljubljana/train.py` is configured to run across twenty\n  2080 Ti GPUs\n- `experiments/ljubljana/register.py` is configured to run on twenty\n  2080 Ti GPUs\n\nThe GPU configurations can be changed at the end of each script using\n[`submitit`](https://github.com/facebookincubator/submitit).\n\n## Development\n\n`DiffPose` package, docs, and CI are all built using\n[`nbdev`](https://nbdev.fast.ai/). To get set up with`nbdev`, install\nthe following\n\n``` zsh\nconda install jupyterlab nbdev -c fastai -c conda-forge \nnbdev_install_quarto      # To build docs\nnbdev_install_hooks       # Make notebooks git-friendly\npip install -e  \".[dev]\"  # Install the development verison of DiffPose\n```\n\nRunning `nbdev_help` will give you the full list of options. The most\nimportant ones are\n\n``` zsh\nnbdev_preview  # Render docs locally and inspect in browser\nnbdev_clean    # NECESSARY BEFORE PUSHING\nnbdev_test     # tests notebooks\nnbdev_export   # builds package and builds docs\nnbdev_readme   # Render the readme\n```\n\nFor more details, follow this [in-depth\ntutorial](https://nbdev.fast.ai/tutorials/tutorial.html).\n\n## Citing `DiffPose`\n\nIf you find `DiffPose` or\n[`DiffDRR`](https://github.com/eigenvivek/DiffDRR) useful in your work,\nplease cite the appropriate papers:\n\n```\n@article{gopalakrishnan2023intraoperative,\n  title={Intraoperative {2D/3D} Image Registration via Differentiable X-ray Rendering},\n  author={Gopalakrishnan, Vivek and Dey, Neel and Golland, Polina},\n  journal={arXiv preprint arXiv:2312.06358},\n  year={2023}\n}\n\n@inproceedings{gopalakrishnan2022fast,\n  title={Fast Auto-Differentiable Digitally Reconstructed Radiographs for Solving Inverse Problems in Intraoperative Imaging},\n  author={Gopalakrishnan, Vivek and Golland, Polina},\n  booktitle={Workshop on Clinical Image-Based Procedures},\n  pages={1--11},\n  year={2022},\n  organization={Springer}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feigenvivek%2Fdiffpose","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feigenvivek%2Fdiffpose","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feigenvivek%2Fdiffpose/lists"}