{"id":23859922,"url":"https://github.com/pointrix-project/pointrix","last_synced_at":"2025-09-08T07:31:02.942Z","repository":{"id":251738030,"uuid":"825238733","full_name":"pointrix-project/pointrix","owner":"pointrix-project","description":"A differentiable point-based rendering framework.","archived":false,"fork":false,"pushed_at":"2025-03-28T06:04:10.000Z","size":17934,"stargazers_count":184,"open_issues_count":3,"forks_count":3,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-03-28T07:20:25.426Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pointrix-project.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-07T08:04:40.000Z","updated_at":"2025-03-28T06:04:14.000Z","dependencies_parsed_at":null,"dependency_job_id":"9a77b1bc-f446-4436-b99d-8c6398b7ea6e","html_url":"https://github.com/pointrix-project/pointrix","commit_stats":null,"previous_names":["pointrix-project/pointrix"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/pointrix-project/pointrix","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pointrix-project%2Fpointrix","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pointrix-project%2Fpointrix/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pointrix-project%2Fpointrix/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pointrix-project%2Fpointrix/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pointrix-project","download_url":"https://codeload.github.com/pointrix-project/pointrix/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pointrix-project%2Fpointrix/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274152132,"owners_count":25231285,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-08T02:00:09.813Z","response_time":121,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-03T04:01:27.663Z","updated_at":"2025-09-08T07:31:02.926Z","avatar_url":"https://github.com/pointrix-project.png","language":"Python","funding_links":[],"categories":["Open Source Implementations","Tools, Pipeline \u0026 Utilities"],"sub_categories":["Framework","Gaussian Splatting \u0026 NeRF"],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003cp align=\"center\"\u003e\n      \u003cpicture\u003e\n      \u003csource srcset=\"https://github.com/user-attachments/assets/14d54372-01e6-4e16-aa20-91ec9fc5c257\" media=\"(prefers-color-scheme: dark)\"\u003e\n      \u003csource srcset=\"https://github.com/user-attachments/assets/a83ee3b1-5452-4614-84f0-662d8d0d9a7f\" media=\"(prefers-color-scheme: light)\"\u003e\n      \u003cimg alt=\"Pointrix\" src=\"https://github.com/user-attachments/assets/a83ee3b1-5452-4614-84f0-662d8d0d9a7f\" width=\"80%\"\u003e\n      \u003c/picture\u003e\n\n  \u003c/p\u003e\n  \u003cp align=\"center\"\u003e\n    A differentiable point-based rendering framework\n    \u003cbr /\u003e\n    \u003ca href=\"https://pointrix-project.github.io/pointrix/\"\u003e\n    \u003cstrong\u003eDocument🏠\u003c/strong\u003e\u003c/a\u003e  | \n    \u003ca href=\"https://pointrix-project.github.io/pointrix/index_cn.html\"\u003e\n    \u003cstrong\u003e中文文档🏠\u003c/strong\u003e\u003c/a\u003e | \n    \u003ca href=\"https://pointrix-project.github.io/pointrix/\"\u003e\n    \u003cstrong\u003ePaper(Comming soon)📄\u003c/strong\u003e\u003c/a\u003e | \n    \u003ca href=\"https://github.com/pointrix-project/msplat\"\u003e\n    \u003cstrong\u003eMsplat Backend🌐\u003c/strong\u003e\u003c/a\u003e |\n    \u003ca href=\"https://www.bilibili.com/video/BV1GaeJepEij/?vd_source=8cf77152a94231ac96b3a3732b42cf30#reply112960710116213\"\u003e\n    \u003cstrong\u003e教程视频🔗\u003c/strong\u003e\u003c/a\u003e\n    \u003cbr /\u003e\n    \u003cbr /\u003e\n    \u003c!-- \u003ca href=\"https://github.com/othneildrew/Best-README-Template\"\u003eView Demo\u003c/a\u003e\n    ·\n    \u003ca href=\"https://github.com/othneildrew/Best-README-Template/issues\"\u003eReport Bug\u003c/a\u003e\n    ·\n    \u003ca href=\"https://github.com/othneildrew/Best-README-Template/issues\"\u003eRequest Feature\u003c/a\u003e --\u003e\n  \u003c/p\u003e\n\u003c/div\u003e\n\n## News\n- 2024-10-23: We add basic viser support for pointrix. We will update documentation soon.\n- 2024-09-11: We have included instructions for hyperparameter search in the documentation.\n- 2024-09-02: We support all hyperparameter turning based on wandb, including \"**random**\", \"**grid**\" and \"**bayes**\" sweep configuration.\n- 2024-08-18: We have released Pointrix: v1.0\n- 2024-08-18: We add exporter which supports metric/video/mesh export.\n\n## Features\nPointrix is a differentiable point-based rendering framework which has following properties:\n\n- **Highly Extensible**:\n  - Python API\n  - Modular design for both researchers and beginners\n  - Implementing your own method without touching CUDA\n- **Powerful Backend**:\n  - CUDA Backend\n  - Forward Anything: rendering image, depth, normal, optical flow, etc.\n  - Backward Anything: optimizing even intrinsics and extrinsics.\n- **Rich Features**:\n  - Support camera parameters optimization.\n  - Support Dynmamic scene reconstruction task and Generation task (WIP).\n  - Support mesh extraction and different type of initialization.\n\n\u003c!-- ## Comparation with original 3D gaussian code\n\n### nerf_synthetic dataset (PSNR)\n\n| Method                  | lego        | chair        | ficus        | drums        | hotdog        | ship        | materials        | mic        | average        |\n| -----------             | ----------- | ----------- | ----------- | ----------- | ----------- | ----------- | ----------- | ----------- | ----------- |\n| Pointrix | 35.84       | 36.12       | 35.02       | 26.18       | 37.81       | 30.98       | 29.95       | 35.34       |  33.40       |\n| [original](https://github.com/graphdeco-inria/gaussian-splatting)        | 35.88        | 35.92        | 35.00        | 26.21        | 37.81        | 30.95        | 30.02        | 35.35        |   33.39       |\n\nwe obtain the result of 3D gaussian code by running following command in their repository.\n```bash\n python train.py -s nerf_synthetic_root --eval -w\n``` --\u003e\n\n## Quickstart\n\n### Installation\n\n\nClone pointrix:\n\n```bash\ngit clone https://github.com/pointrix-project/pointrix.git  --recursive\ncd pointrix\n```\n\nCreate a new conda environment with pytorch:\n\n```bash\nconda create -n pointrix python=3.9\nconda activate pointrix\nconda install pytorch==2.1.1 torchvision==0.16.1 pytorch-cuda=12.1 -c pytorch -c nvidia\n```\n\nInstall Pointrix and MSplat:\n\n```bash\ncd msplat\npip install .\n\ncd ..\npip install -r requirements.txt\npip install -e .\n```\n\n(Optional) You can also install gsplat or diff-gaussian-rasterization:\n\n```bash\npip install gsplat\n\ngit clone https://github.com/graphdeco-inria/diff-gaussian-rasterization.git\ncd diff-gaussian-rasterization\npython setup.py install\npip install .\n```\n\n\n###  Train Your First 3D Gaussian\n\n#### Tanks and Temples Dataset Demo (Colmap format dataset)\nDownload the demo truck scene [data](https://pan.baidu.com/s/1NlhxylY7q3SmVf9j29we3Q?pwd=f95m) and run:\n```bash\ncd examples/gaussian_splatting\n# For Tanks and Temples data which have high-res images and need to downsample.\npython launch.py --config ./configs/colmap.yaml trainer.datapipeline.dataset.data_path=your_data_path trainer.datapipeline.dataset.scale=0.5 trainer.output_path=your_log_path\n\n# you can also use GaussianSplatting renderer or GSplat renderer\npython launch.py --config ./configs/colmap.yaml trainer.datapipeline.dataset.data_path=your_data_path trainer.datapipeline.dataset.scale=0.5 trainer.output_path=your_log_path trainer.model.renderer.name=GaussianSplattingRender\n\npython launch.py --config ./configs/colmap.yaml trainer.datapipeline.dataset.data_path=your_data_path trainer.datapipeline.dataset.scale=0.5 trainer.output_path=your_log_path trainer.controller.normalize_grad=True trainer.model.renderer.name=GsplatRender\n```\nYou can visualize the training process of rendering by enabling webgui (viser):\n```bash\ntrainer.enable_gui=True\n```\n\n\n![2024-10-29 17-07-13屏幕截图](https://github.com/user-attachments/assets/1ec5270e-cf83-4fe9-80a9-8dc10c08ef67)\n\nThe scale should be set as 0.25 for mipnerf 360 datasets.\n\nFor other colmap dataset which do not need to downsample:\n\n```bash\npython launch.py --config ./configs/colmap.yaml trainer.datapipeline.dataset.data_path=your_data_path trainer.datapipeline.dataset.scale=1.0 trainer.output_path=your_log_path\n```\nif you want test your model:\n\n```bash\ncd examples/gaussian_splatting\n# For Tanks and Temples data which have high-res images and need to downsample.\npython launch.py --config ./configs/colmap.yaml trainer.datapipeline.dataset.data_path=your_data_path trainer.datapipeline.dataset.scale=0.25 trainer.output_path=your_log_path trainer.training=False trainer.test_model_path=your_model_path\n```\n\n#### NeRF-Lego (NeRF-Synthetic format dataset)\nDownload the lego data:\n\n```bash\nwget http://cseweb.ucsd.edu/~viscomp/projects/LF/papers/ECCV20/nerf/nerf_example_data.zip\n```\n\nRun the following (with adjusted data path):\n\n```bash\ncd examples/gaussian_splatting\npython launch.py --config ./configs/nerf.yaml trainer.datapipeline.dataset.data_path=your_data_path trainer.output_path=your_log_path\n```\n\nif you want to test the model:\n\n```bash\npython launch.py --config ./configs/nerf.yaml trainer.training=False trainer.datapipeline.dataset.data_path=your_data_path trainer.test_model_path=your_model_path\n```\n\n## Advanced Approaches\n\n#### Turning your hyperparameters\n\nPointrix support turning of hyperparameters based on sweep configuration in wandb, try this feature by running following command:\n\n```bash\ncd examples/gaussian_splatting_sweep\npython launch_sweep.py --config configs/colmap.yaml --config_sweep configs/colmap_sweep.yaml trainer.datapipeline.dataset.data_path=your_data_path  trainer.output_path=your_log_path\n```\n\n![2024-09-02 18-33-13屏幕截图](https://github.com/user-attachments/assets/e25ea893-b3ba-4f1d-ae2d-78834588d42c)\n\n\n\n#### Camera optimization\n\nTo enable camera optimization, you should set trainer.model.camera_model.enable_training=True and trainer.optimizer.optimizer_1.camera_params.lr=1e-3:\nThe renderer must be setted as MsplatRender.\n\n```bash\npython launch.py --config ./configs/colmap.yaml trainer.datapipeline.dataset.data_path=your_data_path trainer.datapipeline.dataset.scale=1.0 trainer.output_path=your_log_path trainer.model.renderer.name=MsplatRender trainer.model.camera_model.enable_training=True trainer.optimizer.optimizer_1.camera_params.lr=1e-3\n```\n\n![pose](https://github.com/user-attachments/assets/42f20422-45be-463a-8b4b-744ede05de84)\n\n#### Post-Processing Results Extraction (Metric, Mesh, Video)\n\nPointrix uses exporters to obtain desired post-processing results, such as mesh and video. The relevant configuration is as follows:\n\n```yaml\ntrainer:\n    exporter:\n        exporter_a:\n            type: MetricExporter\n        exporter_b:\n            type: TSDFFusion\n            extra_cfg:\n                voxel_size: 0.02\n                sdf_trunc: 0.08\n                total_points: 8_000_000 \n        exporter_c:\n            type: VideoExporter\n```\nUsers can specify multiple exporters to obtain various post-processing results. For example, with the above configuration, users can get Metric and Mesh extraction results as well as Video post-processing results. \nMesh is obtained using the TSDF fusion method by default.\nThe renderer must be set as MsplatRender or GsplatRender. You need to set trainer.model.renderer.render_depth as True to enable TSDFFusion.\n\n```bash\ncd pointrix/projects/gaussian_splatting\npython launch.py --config ./configs/nerf.yaml trainer.training=False trainer.datapipeline.dataset.data_path=your_data_path trainer.test_model_path=your_model_path trainer.model.renderer.render_depth=True\n```\n\n#### Dust3r initialization (Beta)\n1. Switch to the Beta branch.\n\n2. Download Dust3r to examples/dust3r_init and follow the installation instructions.\n\n3. Move convert_dust3r.py to the examples/dust3r_init/dust3r folder.\n\n4. Navigate to examples/dust3r_init/dust3r, and then use Dust3r to extract point cloud priors and camera priors:\n\n```bash\npython convert_dust3r.py --model_path your_dust3r_weights --filelist your_image_path\n```\n5. Run the program\n\n```bash\npython launch.py --config config.yaml trainer.datapipeline.dataset.data_path=your_data_path trainer.output_path=your_log_path\n```\n\n\nWelcome to discuss with us and submit PR on new ideas and methods.\n\n## Acknowledgment\nThanks to the developers and contributors of the following open-source repositories, whose invaluable work has greatly inspire our project:\n\n- [3D Gaussian Splatting](https://github.com/graphdeco-inria/gaussian-splatting): 3D Gaussian Splatting for Real-Time Radiance Field Rendering.\n- [Threestudio](https://github.com/threestudio-project): A unified framework for 3D content creation\n- [OmegaConf](https://github.com/omry/omegaconf): Flexible Python configuration system.\n- [SSIM](https://github.com/Po-Hsun-Su/pytorch-ssim): pytorch SSIM loss implemetation.\n- [GSplat](https://github.com/nerfstudio-project/gsplat): An open-source library for CUDA accelerated rasterization of gaussians with python bindings. \n- [detectron2](https://github.com/facebookresearch/detectron2): Detectron2 is Facebook AI Research's next generation library that provides state-of-the-art detection and segmentation algorithms. \n- [DN-Splatter](https://github.com/maturk/dn-splatter): Depth and Normal Priors for Gaussian Splatting and Meshing\n- [GOF](https://github.com/autonomousvision/gaussian-opacity-fields): Efficient and Compact Surface Reconstruction in Unbounded Scenes\n- [Viser](https://viser.studio/latest/): A library for interactive 3D visualization in Python.\n- [2D-GS-Viser-Viewer](https://github.com/hwanhuh/2D-GS-Viser-Viewer): Simple Viser Viewer for 2D Gaussian Splatting for Geometrically Accurate Radiance Fields.\n\nThis is project is licensed under Apache License. However, if you use MSplat or the original 3DGS kernel in your work, please follow their license.\n\n## Contributors\n\u003ca href=\"https://github.com/pointrix-project/pointrix/graphs/contributors\"\u003e\n  \u003cimg src=\"https://contrib.rocks/image?repo=pointrix-project/pointrix\" /\u003e\n\u003c/a\u003e\n\nMade with [contrib.rocks](https://contrib.rocks).\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpointrix-project%2Fpointrix","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpointrix-project%2Fpointrix","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpointrix-project%2Fpointrix/lists"}