{"id":50615557,"url":"https://github.com/lucidrains/d4rt","last_synced_at":"2026-06-06T08:01:46.006Z","repository":{"id":357207236,"uuid":"1234717086","full_name":"lucidrains/d4rt","owner":"lucidrains","description":"Implementation of D4RT, Efficiently Reconstructing Dynamic Scenes, Deepmind","archived":false,"fork":false,"pushed_at":"2026-05-11T19:07:37.000Z","size":665,"stargazers_count":39,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2026-05-11T21:15:02.143Z","etag":null,"topics":["4d","artificial-intelligence","attention","deep-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lucidrains.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-10T14:52:54.000Z","updated_at":"2026-05-11T19:07:54.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/lucidrains/d4rt","commit_stats":null,"previous_names":["lucidrains/d4rt"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/lucidrains/d4rt","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Fd4rt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Fd4rt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Fd4rt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Fd4rt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lucidrains","download_url":"https://codeload.github.com/lucidrains/d4rt/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Fd4rt/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33973868,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-06T02:00:07.033Z","response_time":107,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["4d","artificial-intelligence","attention","deep-learning"],"created_at":"2026-06-06T08:01:45.314Z","updated_at":"2026-06-06T08:01:45.996Z","avatar_url":"https://github.com/lucidrains.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"./d4rt.png\" width=\"400px\"\u003e\u003c/img\u003e\n\n## d4rt\n\nImplementation of [D4RT](https://d4rt-paper.github.io/), Efficiently Reconstructing Dynamic Scenes, by Chuhan Zhang et al. from Deepmind\n\n## install\n\n```shell\n$ pip install d4rt\n```\n\n## usage\n\n```python\nfrom torch import randn, randint\nfrom d4rt import D4RT\n\nmodel = D4RT(\n    dim = 512,\n    video_image_size = 128,\n    video_patch_size = 32,\n    video_max_time_len = 10,\n    enc_depth = 6,\n    dec_depth = 6\n)\n\nvideos = randn(2, 10, 3, 128, 128)\n\nvideo_lens = randint(2, 10, (2,)) # handle variable lengthed video, can be None for max length always\n\n# inputs\n\ncoors = randint(0, 128, (2, 5, 2))\ntime_src = randint(0, 10, (2, 5))\ntime_tgt = randint(0, 10, (2, 5))\ntime_camera = randint(0, 10, (2, 5))\n\nquery_lens = randint(1, 5, (2,)) # handle variable lengthed queries\n\n# output\n\npoints = randn(2, 5, 3)\n\nloss = model(\n    videos,\n    video_lens = video_lens,\n    coors = coors,\n    time_src = time_src,\n    time_tgt = time_tgt,\n    time_camera = time_camera,\n    query_lens = query_lens,\n    points = points,\n)\n\nloss.backward()\n\n# without giving the output, it returns the prediction\n\npred = model(\n    videos,\n    coors = coors,\n    time_src = time_src,\n    time_tgt = time_tgt,\n    time_camera = time_camera\n)\n\nassert pred.shape == (2, 5, 3)\n```\n\n## contribute\n\nJust add your code and your tests in the `tests/` folder and run `pytest` in the project root\n\nVibing with attention models are welcomed\n\n## citations\n\n```bibtex\n@article{zhang2025d4rt,\n    title   = {Efficiently Reconstructing Dynamic Scenes One D4RT at a Time},\n    author  = {Zhang, Chuhan and Le Moing, Guillaume and Koppula, Skanda and Rocco, Ignacio and Momeni, Liliane and Xie, Junyu and Sun, Shuyang and Sukthankar, Rahul and Barral, Jo{\\\"e}lle K. and Hadsell, Raia and Ghahramani, Zoubin and Zisserman, Andrew and Zhang, Junlin and Sajjadi, Mehdi S. M.},\n    journal = {arXiv preprint},\n    year    = {2025}\n}\n```\n\n```bibtex\n@inproceedings{liu2026geometryaware,\n    title   = {Geometry-aware 4D Video Generation for Robot Manipulation},\n    author  = {Zeyi Liu and Shuang Li and Eric Cousineau and Siyuan Feng and Benjamin Burchfiel and Shuran Song},\n    booktitle = {The Fourteenth International Conference on Learning Representations},\n    year    = {2026},\n    url     = {https://openreview.net/forum?id=18gC6pZVVc}\n}\n```\n\n```bibtex\n@misc{joseph2026interpretingphysicsvideoworld,\n    title   = {Interpreting Physics in Video World Models},\n    author  = {Sonia Joseph and Quentin Garrido and Randall Balestriero and Matthew Kowal and Thomas Fel and Shahab Bakhtiari and Blake Richards and Mike Rabbat},\n    year    = {2026},\n    eprint  = {2602.07050},\n    archivePrefix = {arXiv},\n    primaryClass = {cs.CV},\n    url={https://arxiv.org/abs/2602.07050},\n}\n```\n\n```bibtex\n@misc{li2025basicsletdenoisinggenerative,\n    title   = {Back to Basics: Let Denoising Generative Models Denoise},\n    author  = {Tianhong Li and Kaiming He},\n    year    = {2025},\n    eprint  = {2511.13720},\n    archivePrefix = {arXiv},\n    primaryClass = {cs.CV},\n    url     = {https://arxiv.org/abs/2511.13720},\n}\n```\n\n```bibtex\n@misc{li2025basicsletdenoisinggenerative,\n    title   = {Back to Basics: Let Denoising Generative Models Denoise},\n    author  = {Tianhong Li and Kaiming He},\n    year    = {2025},\n    eprint  = {2511.13720},\n    archivePrefix = {arXiv},\n    primaryClass = {cs.CV},\n    url     = {https://arxiv.org/abs/2511.13720},\n}\n```\n\n```bibtex\n@misc{charpentier2024gptbertboth,\n    title   = {GPT or BERT: why not both?},\n    author  = {Lucas Georges Gabriel Charpentier and David Samuel},\n    year    = {2024},\n    eprint  = {2410.24159},\n    archivePrefix = {arXiv},\n    primaryClass = {cs.CL},\n    url     = {https://arxiv.org/abs/2410.24159},\n}\n```\n\n```bibtex\n@misc{balestriero2025lejepa,\n    title   = {LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics},\n    author  = {Randall Balestriero and Yann LeCun},\n    year    = {2025},\n    eprint  = {2511.08544},\n    archivePrefix = {arXiv},\n    primaryClass = {cs.LG},\n    url     = {https://arxiv.org/abs/2511.08544},\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucidrains%2Fd4rt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flucidrains%2Fd4rt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucidrains%2Fd4rt/lists"}