{"id":13936171,"url":"https://github.com/uber-research/sbnet","last_synced_at":"2025-04-06T02:11:26.279Z","repository":{"id":91801776,"uuid":"114788303","full_name":"uber-research/sbnet","owner":"uber-research","description":"Sparse Blocks Networks","archived":false,"fork":false,"pushed_at":"2018-11-06T04:22:50.000Z","size":114,"stargazers_count":433,"open_issues_count":4,"forks_count":89,"subscribers_count":20,"default_branch":"master","last_synced_at":"2025-03-30T01:11:16.772Z","etag":null,"topics":["neuralnetwork","neuralnetworks","python","sbnet","tensorflow","uber"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/uber-research.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-12-19T16:37:57.000Z","updated_at":"2025-03-03T12:40:11.000Z","dependencies_parsed_at":null,"dependency_job_id":"8e3a75c0-c666-4e38-be4e-804450d520bc","html_url":"https://github.com/uber-research/sbnet","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uber-research%2Fsbnet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uber-research%2Fsbnet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uber-research%2Fsbnet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uber-research%2Fsbnet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/uber-research","download_url":"https://codeload.github.com/uber-research/sbnet/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247423515,"owners_count":20936626,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["neuralnetwork","neuralnetworks","python","sbnet","tensorflow","uber"],"created_at":"2024-08-07T23:02:25.930Z","updated_at":"2025-04-06T02:11:26.259Z","avatar_url":"https://github.com/uber-research.png","language":"Python","readme":"# Sparse Blocks Network (SBNet)\n\nThis repository releases code for our paper [*SBNet: Sparse Blocks Network for Fast Inference*](https://arxiv.org/abs/1801.02108). Please refer to our [blog post](https://eng.uber.com/sbnet) for more context.\nNote that benchmarking in the paper was performed with an older version of this repo using TensorFlow 1.2, cuDNN 6.1 and commit cf8ea06.\n\nThis repository contains \n1. a TensorFlow custom operations library that implements SBNet,\n2. a Python implementation of sparse ResNet blocks, and\n3. a benchmark for performance comparison with [Submanifold Sparse Convolutional Networks](https://arxiv.org/abs/1706.01307).\n\n## Prerequisites\n\nInstallation was tested under Ubuntu 14.04 and 16.04 with TensorFlow 1.8, CUDA 9.0 and cuDNN 7.1.\n\n## Hardware requirements\n\nCode was tested on and compiled for NVIDIA CUDA 6.1, 6.0, 5.2 and 7.0 architectures (Titan XP, GTX 1080Ti, GTX 1080, P100, V100, TitanV, and most Maxwell cards).\nTo compile for an older architecture please modify the Makefile and add the corresponding line, such as `-gencode arch=compute_50,code=sm_50` for older cards such as laptop Maxwell.\nPlease refer to [CUDA Wikipedia](https://en.wikipedia.org/wiki/CUDA) page to lookup the architecture code for your graphics card.\n\n\n## Setup\n\nTo build a release version of the library, run\n\n`cd sbnet_tensorflow/sbnet_ops \u0026\u0026 make`\n\nTo run tests:\n\n`cd sbnet_tensorflow/sbnet_ops \u0026\u0026 make test`\n\nThe library will be built in sbnet_tensorflow/sbnet_ops/build/libsbnet.so and symlinked to sbnet_tensorflow/sbnet_ops/libsbnet.so.\nTo import the library into your TensorFlow Python code use the following command:\n\n```\nsbnet_module = tf.load_op_library('path_to_library/libsbnet.so')\n```\n\nThe following Tensorflow ops are implemented in the op library:\n\n```sbnet_module.reduce_mask```\n\n```sbnet_module.sparse_gather```\n\n```sbnet_module.sparse_scatter```\n\n\n`reduce_mask` op converts a dense mask to a list of active block indices.\n\nIn the following snippet the mask is expected to be a tensor of dimensions `[N,H,W,1]`:\n\n```\n    indices = sbnet_module.reduce_mask(\n        mask, tf.constant([BCH, BCW], dtype=tf.int32),\n        bsize=[BSZH, BSZW],\n        boffset=[BOFFSH, BOFFSW],\n        bstride=[BSTRH, BSTRW],\n        tol=0.5, # pooling threshold to consider a block as active\n        avgpool=True) # max pooling by default\n```\n\n[BCH, BCW] are block counts in height and width dimensions.\n[BSZH, BSZW], [BOFFSH, BOFSFW] and [BSTRH, BSTRW] are block sizes, offsets and strides in H and W dimensions.\n`reduce_mask` performs a combined max pooling (or average pooling) operation localized to each block followed by generating\na list of triples of indices `[(ni, hi, wi)]` for blocks where either max or average pooling value exceeds specified tolerance `tol`.\nIn numpy terms each block is defined as a slice from the input mask of dimensions `[N,H,W,1]`, with following dimensions:\n`[ni, BOFFSH+BSTRH*hi : BOFFSH+BSTRH*hi+BSZH, BOFFSW+BSTRW*wi : BOFFSW+BSTRW*wi+BSZW, :]`.\n\nThe resulting list of indices can then be passed to two other operations: `sbnet_module.sparse_scatter` and `sbnet_module.sparse_gather`.\n\nThe following snippets illustrate the use of these operations:\n```\n    blockStack = sbnet_module.sparse_gather(\n        x,\n        indices.bin_counts,\n        indices.active_block_indices,\n        bsize=[BSZH, BSZW], # block size\n        boffset=[BOFFSH, BOFFSW], # block offset\n        bstride=[BSTRH, BSTRW], # block stride\n        transpose=do_transpose)\n```\n\nThis operation will use the indices generated by reduce_mask and slice out tensors of channel depth C out of input tensor `x` of dimensions `[N,H,W,C]` as illustrated in the following pseudo-code snippet:\n\n```\n    for (ni, hi, wi) in indices.active_block_indices:\n        channel_slice = x[ni, BOFFSH+BSTRH*hi : BOFFSH+BSTRH*hi+BSZH, BOFFSW+BSTRW*wi : BOFFSW+BSTRW*wi+BSZW, :]\n        blockStack[ni, :, :, :] = channel_slice\n```\n\nIf `do_transpose` is true, a fused transpose operation will also be performed and the resulting tensor will have dimensions `[nBlocks, C, BSZH, BSZW]`.\nAny out-of-range values will be padded with zeroes.\n\nThe inverse operation is `sbnet_module.sparse_scatter`. The following snippet illustrates it's use:\n\n```\n    y = sbnet_module.sparse_scatter(\n        blockStack,\n        indices.bin_counts,\n        indices.active_block_indices,\n        x, # base tensor to copy to output and overwrite on top of\n        bsize=[BSZH, BSZW],\n        boffset=[BOFFSH, BOFFSW],\n        bstride=[BSTRH, BSTRW],\n        add=do_add,\n        atomic=False, # use atomic or regular adds\n        transpose=do_transpose)\n```\n\nNote that due to a limitation of TensorFlow API an intermediate tensor cannot be modified in place unless it's specified to be a tf.Variable.\nThis necessitates creating an intermediate tensor inside the op and performing a copy which has negative implications for performance.\nSo we created a second version of the op `sbnet_module.sparse_scatter_var` that expects x to be a `tf.Variable` and modifies it in place.\nUsing `sparse_scatter_var` is strongly recommended for maximum performance.\n\nThe effect of this operation is opposite to `sparse_gather` - the input blocks will be written on top of base tensor x, or added to it's contents if `do_add` is True.\nThe following pseudo-code snippet illustrates the semantics of `sparse_scatter`:\n\n```\n    for (ni, hi, wi) in indices.active_block_indices:\n        if do_add:\n            x[ni, BOFFSH+BSTRH*hi : BOFFSH+BSTRH*hi+BSZH, BOFFSW+BSTRW*wi : BOFFSW+BSTRW*wi+BSZW, :]\\\n                += blockStack[ni, :, :, :]\n        else:\n            x[ni, BOFFSH+BSTRH*hi : BOFFSH+BSTRH*hi+BSZH, BOFFSW+BSTRW*wi : BOFFSW+BSTRW*wi+BSZW, :]\\\n                = blockStack[ni, :, :, :]\n```\n\nSo the blocks are 'put back in place', however the sizes and strides can be different from those passed to sparse_gather. This enables implementation of sparse ResNet blocks where output resolution is reduced\nafter a 'VALID' convolution. Similar to `sparse_gather`, if `do_transpose` is true, a fused transpose operation will also be performed by sparse_scatter, permuting the input `[N,C,H,W]` dimensions to `[N,H,W,C]` in the output.\nTypically the block size for a 'VALID' convolution is reduced by 2 in each spatial dimension for each 3x3 convolution, thus creating non-overlapping outputs.\nNote that even though currently we support atomic adds in scatter with add=True, the gradient is not implemented at this time if overlapping scatters are used the forward pass. \n\n## Benchmarks and tests\n\nBenchmarks for SBNet are located in sbnet_tensorflow/benchmarks/ subdirectory.\n\nTo run benchmarks execute:\n\n```\ncd sbnet_tensorflow/benchmarks \u0026\u0026 ./run_all_behchmarks.bash\n```\n\nNote that we average over a number of runs and test many permutations of parameters so this may take about 20 minutes (on a Titan XP) and will produce a number of .csv files in your /home/user/ directory.\nWe benchmark individual sparse convolutions and entire sparse ResNet blocks on a synthetic mask with variable sparsity.\n\nTo run unit tests execute:\n```\ncd sbnet_tensorflow/sbnet_ops \u0026\u0026 make tests\n```\n\n\n## Submanifold Sparse Convolutional Networks Benchmark\n\nFor comparison we implemented benchmarking code for [Submanifold Sparse Convolutional Networks](https://github.com/facebookresearch/SparseConvNet).\nRunning this benchmark requires Submanifold Sparse Convolutions python package to be installed:\n``` \ngit clone https://github.com/facebookresearch/SparseConvNet.git \n```\nFollow the setup instructions in SparseConvNet repo.\n\nCode integration with Submanifold Sparse Convolutions was tested with git sha 609224df3c0e42b8a1dd4073aaa56fab805096c6. To reset the repo to this sha use the following sequence of commands:\n```\ncd SparseConvNet\ngit checkout 609224df3c0e42b8a1dd4073aaa56fab805096c6\n```\n\nThe benchmark code is located in sbnet_tensorflow/benchmark_submanifold directory.\n\n\n## Other notes\n\nCurrent code is not tuned for performance with non-square block sizes and has specialized implementations for a specific list of block sizes. This includes square blocks of sizes 1 to 34 and a few others. To achieve maximum performance for these sizes you would need to add your custom template instantiations by modifying SIZE_TEMPLATES macro in `sparse_gather.cu`.\n\n\n## Contributing to this repository\n\nFor now, we do not accept pull request to this repo, as we are currently setting up automated CI.\nIf you would like to contribute to this repository, feel free create a GitHub issue.\n\n## Citation\n\nIf you use our code, please consider cite the following:\nM. Ren, A. Pokrovsky, B. Yang, and R. Urtasun. SBNet: Sparse Blocks Network for Fast Inference. \n*CoRR*, abs/1801.02108, 2018.\n\n```\n@article{ren18sbnet,\n  author    = {Mengye Ren and \n               Andrei Pokrovsky and\n               Bin Yang and\n               Raquel Urtasun},\n  title     = {SBNet: Sparse Blocks Network for Fast Inference},\n  journal   = {CoRR},\n  volume    = {abs/1801.02108},\n  year      = {2018},\n}\n```\n","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuber-research%2Fsbnet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fuber-research%2Fsbnet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuber-research%2Fsbnet/lists"}