{"id":20296838,"url":"https://github.com/thended/torch-ash","last_synced_at":"2025-10-04T13:27:36.710Z","repository":{"id":186445282,"uuid":"612330505","full_name":"theNded/torch-ash","owner":"theNded","description":"[PAMI 2022, CVPR 2023] ASH: Parallel Spatial Hashing for Fast Scene Reconstruction ","archived":false,"fork":false,"pushed_at":"2023-09-14T04:10:56.000Z","size":1658,"stargazers_count":154,"open_issues_count":10,"forks_count":7,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-04-11T12:34:06.157Z","etag":null,"topics":["hashing","sdf"],"latest_commit_sha":null,"homepage":"https://dongwei.info/publication/ash-mono/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/theNded.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-10T17:46:40.000Z","updated_at":"2025-01-22T12:47:20.000Z","dependencies_parsed_at":null,"dependency_job_id":"83f9adce-d2cb-4550-9dbc-7f52042b396a","html_url":"https://github.com/theNded/torch-ash","commit_stats":null,"previous_names":["thended/torch-ash"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/theNded/torch-ash","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theNded%2Ftorch-ash","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theNded%2Ftorch-ash/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theNded%2Ftorch-ash/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theNded%2Ftorch-ash/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/theNded","download_url":"https://codeload.github.com/theNded/torch-ash/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theNded%2Ftorch-ash/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274140084,"owners_count":25229138,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-08T02:00:09.813Z","response_time":121,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hashing","sdf"],"created_at":"2024-11-14T15:41:21.794Z","updated_at":"2025-10-04T13:27:31.668Z","avatar_url":"https://github.com/theNded.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# torch-ash\ntorch-ash is the missing piece of collision-free extendable parallel spatial hashing for torch modules. It includes two paper's core implementations:\n\n[[PAMI 2022]](https://arxiv.org/abs/2110.00511) | [[CVPR 2023]](https://arxiv.org/abs/2305.13220)\n```\n@article{dong2022ash,\n  title={ASH: A modern framework for parallel spatial hashing in 3D perception},\n  author={Dong, Wei and Lao, Yixing and Kaess, Michael and Koltun, Vladlen},\n  journal={PAMI},\n  year={2022},\n}\n\n@inproceedings{dong2023ash-mono,\n  title={Fast Monocular Scene Reconstruction with Global-Sparse Local-Dense Grids},\n  author={Dong, Wei and Choy, Chris and Loop, Charles and Zhu, Yuke and Litany, Or and Anandkumar, Anima},\n  booktitle={CVPR},\n  year={2023},\n}\n```\n\u003ctable\u003e\n  \u003ctr\u003e\n  \u003ctd\u003e\u003cimg src=\"https://github.com/theNded/theNded.github.io/blob/master/assets/images/ash-mono/0050.gif\" width=\"480\"/\u003e\u003c/td\u003e\n  \u003ctd\u003e\u003cimg src=\"https://github.com/theNded/theNded.github.io/blob/master/assets/images/ash-mono/lounge.gif\" width=\"480\"/\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\nNote for a more user-friendly interface and further extensions, I have fully rewritten everything from scratch in this repo. Discrepancies from the reported results in the aforementioned papers are expected. Updates and more examples will come. \n\n\n## Install\nFirst, install [PyTorch](https://pytorch.org/get-started/locally/).\nOptionally install [nerfacc](https://pypi.org/project/nerfacc/0.2.1/) for volume rendering.\n\ncmake is required in the conda environment for compiling the source code.\n```\ngit clone --recursive git@github.com:theNded/torch-ash.git\npip install . --verbose\n```\n\n## Engine (ASH)\n- The core is `ASHEngine`, a PyTorch module implementing a parallel, collision-free, dynamic hash map from coordinates (`torch.IntTensor`) to indices (`torch.LongTensor`). It depends on [stdgpu](https://github.com/stotko/stdgpu).\n- Above `ASHEngine`, there are `HashSet` and `HashMap` which are wrappers around ASHEngine. A `HashSet` maps a coordinate to a boolean value, usually used for the `unique` operation. A `HashMap` maps a coordinate to a (dictionary) of values, and allows fast insertion and accessing coordinate-value pairs.\n- Similar to `HashMap`, `HashEmbedding` maps coordinates to embeddings and is akin to `torch.nn.Embedding`.\n\n### Usage\n```python\nhashmap = HashMap(key_dim=3, value_dims={\"color\": 3, \"depth\": 1}, capacity=100, device=torch.device(\"cuda:0\"))\n\n# To insert\nkeys = (torch.rand(10, 3) * 100).int().cuda()\nvalues = {\"colors\": torch.rand(10, 3).float().cuda(), \"depth\": torch.rand(10, 1).float().cuda()}\nhashmap.insert(keys, values)\n\n# To query\nquery_keys = (torch.rand(10, 3) * 100).int().cuda()\nindices, masks = hashmap.find(query_keys)\n\n# To enumerate\nall_indices, all_values = hashmap.items(return_indices=True, return_values=True)\n```\n\n## SparseDenseGrids for Surface Reconstruction\n`SparseDenseGrid` is the engine for direct/neural scene representation. It consists of sparse arrays of grids and dense arrays of cells. The idea is similar to [Instant-NGP](https://github.com/NVlabs/instant-ngp) and [Plenoxels](https://github.com/sxyu/svox2), but precise sparsity is achieved through spatial initialization and collision-free hashing. Essentially it is a modern version of [VoxelHashing](https://github.com/niessner/VoxelHashing).\n\nIt has two wrappers for coordinate transform, `UnboundedSparseDenseGrid` for potentially dynamically increasing metric scenes, and `BoundedSparseDenseGrid` for scenes bounded in unit cubes. Trilinear interpolation and double backward are implemented to support differentiable gradient computation. All these modules can be converted to and from state dicts by serializing the underlying hash map.\n\nThe `SparseDenseGrid` does a good job without an MLP in fast reconstruction tasks (e.g. RGB-D fusion, differentiable volume rendering with a decent initialization), but with an MLP, there seem no advantages in comparison to Instant-NGP as of now. Potential extensions in this line are still in progress.\n\n### Demo: RGB-D fusion [PAMI 22]\nRGB-D fusion takes in posed RGB-D images and creates colorized mesh, raw and filtered. Here, depth can either be sensor depth, or generated from a monocular depth prediction model (e.g. [omnidata](https://github.com/theNded/mini-omnidata)) with calibrated scales via [COLMAP](https://colmap.github.io/). Example datasets can be downloaded at [Google Drive](https://drive.google.com/drive/folders/12E4cTIIxmShV_ENkcvzKOQunsa0TqDVQ?usp=drive_link). Instructions for custom datasets will be available soon.\n\nThese datasets are organized by\n```\n- image/ # for RGB images [jpg|png]\n- depth/ # for sensor depth [optional, png]\n- omni_depth/ # for learned depth generated from RGB [npy]\n- depth_scales.txt # calculated between learned depth and SfM\n- omni_normal/ # for learned normals generated from RGB [optional, npy]\n- poses.txt\n- intrinsic.txt\n```\n\nTo run the demo,\n```sh\n# Unbounded scenes, sensor depth\npython demo/rgbd_fusion.py --path /path/to/dataset/samples --voxel_size 0.015 --depth_type sensor\n\n# Bounded scenes, learned depth\npython demo/rgbd_fusion.py --path /path/to/dataset/samples --resolution 512 --depth_type learned\n```\n\n### Demo: surface refinement [CVPR 23]\nWith learned depth, the fusion result is usually noisy. We can apply volume rendering to further optimize the shape:\n```\npython demo/train_scene_recon.py --path /path/to/dataset/samples --voxel_size 0.015 --depth_type learned\n```\nWe start with a local 7x7x7 Gaussian filter to smooth the initialization.\n\u003ctable\u003e\n  \u003ctr\u003e\n  \u003ctd\u003e\u003cimg src=\"https://github.com/theNded/theNded.github.io/blob/master/assets/images/ash-nofilter.png\" width=\"480\"/\u003e\u003c/td\u003e\n  \u003ctd\u003e\u003cimg src=\"https://github.com/theNded/theNded.github.io/blob/master/assets/images/ash-filtered.png\" width=\"480\"/\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\nVolume rendering follows the initialization. The results will be written in `logs/datetime`. At every 500 iterations, mesh will be extracted and stored. The optimization will start with ripples on the surfaces, but finally converge to smooth reconstructions as shown above.\n\n\n## API Usage\nHere is a brief summary of basic usage, doc will be online soon.\n### Allocation\nWe first initialize a 3D sparse-dense grid with 10000 sparse grid blocks. Each sparse grid contains a dense 8^3=512 array of cells, whose size is 0.01m.\n```python\ngrid = UboundedSparseDenseGrid(in_dim=3,\n                               num_embeddings=10000,\n                               grid_dim=16,\n                               embedding_dims=8,\n                               cell_size=0.01)\n```\n\n### Initialization\nWe then spatially initialize the grid at input points (e.g. obtained point cloud, RGB-D scans). This results in coordinates and indices that support index-based access.\n```python\nwith torch.no_grad():\n    grid_coords, cell_coords, grid_indices, cell_indices = grid.spatial_init_(points)\n\n    # [Optional] direct assignment\n    grid.embeddings[grid_indices, cell_indices] = attributes\n```\n\n### Optimization\nAs a PyTorch extension, first and second-order autodiff are enabled by a differentiable query.\n```python\noptim = torch.optim.SGD(grid.parameters(), lr=1e-3)\nfor x, gt in batch:\n    optim.zero_grad()\n    x.requires_grad_(True)\n    embedding, mask = grid(x, interpolation=\"linear\")\n\n    output = forward_fn(embedding, mask)\n\n    doutput_dx = torch.autograd.grad(\n        outputs=output,\n        inputs=x,\n        grad_outputs=torch.ones_like(output, requires_grad=False),\n        create_graph=True,\n        retain_graph=True)[0]\n\n    (loss_fn(output) + grad_loss_fn(doutput_dx)).backward()\n    optim.step()\n```\n\n## Milestones\n- [x] Initial release\n- [x] Demo: RGB-(pseudo)D SDF fusion\n- [x] Demo: SDF refinement from volume rendering\n- [ ] Better instructions and documentation\n- [ ] Demo: LiDAR SDF fusion\n- [ ] Demo: MLP integration\n- [ ] CPU counterpart\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthended%2Ftorch-ash","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthended%2Ftorch-ash","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthended%2Ftorch-ash/lists"}