{"id":13476893,"url":"https://github.com/pytorch/nestedtensor","last_synced_at":"2025-09-30T05:31:29.926Z","repository":{"id":41322298,"uuid":"217161669","full_name":"pytorch/nestedtensor","owner":"pytorch","description":"[Prototype] Tools for the concurrent manipulation of variably sized Tensors.","archived":true,"fork":false,"pushed_at":"2022-11-14T12:30:49.000Z","size":13546,"stargazers_count":252,"open_issues_count":49,"forks_count":28,"subscribers_count":16,"default_branch":"master","last_synced_at":"2024-09-27T19:23:51.440Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pytorch.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-10-23T22:04:33.000Z","updated_at":"2024-06-06T09:07:27.000Z","dependencies_parsed_at":"2023-01-21T09:17:02.277Z","dependency_job_id":null,"html_url":"https://github.com/pytorch/nestedtensor","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pytorch%2Fnestedtensor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pytorch%2Fnestedtensor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pytorch%2Fnestedtensor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pytorch%2Fnestedtensor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pytorch","download_url":"https://codeload.github.com/pytorch/nestedtensor/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":234707373,"owners_count":18874671,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T16:01:35.827Z","updated_at":"2025-09-30T05:31:24.464Z","avatar_url":"https://github.com/pytorch.png","language":"Jupyter Notebook","funding_links":[],"categories":["Jupyter Notebook"],"sub_categories":[],"readme":"# BIG UPDATE: NestedTensor [in core](https://pytorch.org/docs/master/nested.html)!\n\n## March 15 2022\nAs of recently we landed a minimal version of NestedTensor [in core PyTorch](https://pytorch.org/docs/master/nested.html)!\nOperator coverage and migration of features is possible, but must be backed by issues (feature requests). If you have demand for specific NestedTensor operators, please open a feature request on [pytorch/pytorch](https://github.com/pytorch/pytorch/issues/new?assignees=\u0026labels=\u0026template=feature-request.yml). For a more impactful submission please include your motivation, use case and list of operators.\n\u003cbr /\u003e\n\u003cbr /\u003e\n\u003cbr /\u003e\n\u003cbr /\u003e\n\u003cbr /\u003e\n\u003cbr /\u003e\n\n# The nestedtensor package [prototype](https://pytorch.org/blog/pytorch-feature-classification-changes/#prototype)\n\nIf you are here because you ran into a runtime error due to a missing feature or some kind of bug, please [open an issue and fill in the appropiate template](https://github.com/pytorch/nestedtensor/issues/new/choose). If you have general feedback about this prototype [you can use our suggested template](https://github.com/pytorch/nestedtensor/issues/new?assignees=\u0026labels=\u0026template=prototype-feedback.md\u0026title=) or just open a free-form issue if you like. Thank you for contributing to this project!\n\n## Tutorials\n\nIf you are new to this project, we recommend you take a look at our [whirlwind introduction](https://colab.research.google.com/github/pytorch/nestedtensor/blob/master/tutorials/notebooks/basic.ipynb) to get started.\n\n## Autograd support\n\nDue to missing extensibility features of PyTorch nestedtensor currently lacks autograd support. We're actively working on this and recognize that it severely limits the applicability of the project. Please run nestedtensor operations within the [inference mode](https://github.com/ailzhang/rfcs/blob/rfc0011/RFC-0011-InferenceMode.md) context to prevent any adverse interactions with the autograd system.\n\nFor example\n```\nsentences = [torch.randn(10, 5), torch.randn(5, 5), torch.randn(9, 5)]\nwith torch.inference_mode():    \n    nt = nestedtensor.nested_tensor(sentences)\n    nt.sum(1)\n```\n\n## Binaries\n\nDue to the development velocity of PyTorch the nestedtensor project is built on top of and dependent on a fixed, recent PyTorch nightly.\n\n| Version | Python | CUDA | Wheels |\n| --- | ---- | ------ | ---- |\n| 0.1.1 | 3.6 | CPU-only | [nestedtensor](https://download.pytorch.org/nestedtensor/whl/nightly/cpu/py3.6/nestedtensor-0.1.1_cpu-cp36-cp36m-linux_x86_64.whl) |\n| 0.1.1 | 3.7 | CPU-only | [nestedtensor](https://download.pytorch.org/nestedtensor/whl/nightly/cpu/py3.7/nestedtensor-0.1.1_cpu-cp37-cp37m-linux_x86_64.whl) |\n| 0.1.1 | 3.8 | CPU-only | [nestedtensor](https://download.pytorch.org/nestedtensor/whl/nightly/cpu/py3.8/nestedtensor-0.1.1_cpu-cp38-cp38m-linux_x86_64.whl) |\n| 0.1.1 | 3.6 | CUDA 10.2 | [nestedtensor](https://download.pytorch.org/nestedtensor/whl/nightly/cpu/py3.6/nestedtensor-0.1.1_cu102-cp36-cp36m-linux_x86_64.whl) |\n| 0.1.1 | 3.7 | CUDA 10.2 | [nestedtensor](https://download.pytorch.org/nestedtensor/whl/nightly/cpu/py3.7/nestedtensor-0.1.1_cu102-cp37-cp37m-linux_x86_64.whl) |\n| 0.1.1 | 3.8 | CUDA 10.2 | [nestedtensor](https://download.pytorch.org/nestedtensor/whl/nightly/cpu/py3.8/nestedtensor-0.1.1_cu102-cp38-cp38m-linux_x86_64.whl) |\n\nWhen installing a binary please specify the corresponding torch nightly link archive to automatically pull in the correct PyTorch nightly.\n\nCPU\n```\npip install https://download.pytorch.org/nestedtensor/whl/nightly/cpu/py3.7/nestedtensor-0.1.1_cpu-cp37-cp37m-linux_x86_64.whl -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html\n```\n\nCUDA 10.2\n```\npip install https://download.pytorch.org/nestedtensor/whl/nightly/cu102/py3.7/nestedtensor-0.1.1_cu102-cp37-cp37m-linux_x86_64.whl -f https://download.pytorch.org/whl/nightly/cu102/torch_nightly.html\n```\n\n## Why consider using this? / Dealing with dynamic shapes\n\nIn general we batch data for efficiency, but usually batched kernels need, or greatly benefit from, regular, statically-shaped data.\n\nOne way of dealing with dynamic shapes then, is via padding and masking.\n[Various](https://github.com/pytorch/fairseq/blob/54b934417d95baa1b0076089c61bde32728e34cf/fairseq/data/audio/raw_audio_dataset.py#L92)\n[projects](https://github.com/facebookresearch/ParlAI/blob/8200396cdd08cfd26b01fe52b4a3bd0654081182/parlai/agents/drqa/utils.py#L143)\n[construct](https://github.com/facebookresearch/detr/blob/4e1a9281bc5621dcd65f3438631de25e255c4269/util/misc.py#L306)\n[masks](https://github.com/pytorch/vision/blob/24f16a338391d6f45aa6291c48eb6d5513771631/references/detection/utils.py#L102)\n[that](https://github.com/pytorch/audio/blob/3250d3df168c956389bd16956aa458ce111570d0/examples/pipeline_wav2letter/datasets.py#L90), together with a data Tensor, are used as a representation for lists of dynamically shaped Tensors.\n\nObviously this is inefficient from a memory and compute perspective if the Tensors within this list are sufficiently diverse.\n\nYou can also trace through the codebase where these masks are used and observe the kind of code this approach often leads to. See for example [universal_sentence_embedding](https://github.com/facebookresearch/ParlAI/blob/8200396cdd08cfd26b01fe52b4a3bd0654081182/parlai/agents/drqa/utils.py#L143).\n\nOtherwise we also have \n[one-off](https://pytorch.org/docs/master/generated/torch.nn.utils.rnn.pack_padded_sequence.html?highlight=pack_padded_sequence)\n[operator](https://pytorch.org/docs/master/generated/torch.nn.CrossEntropyLoss.html#torch.nn.CrossEntropyLoss)\n[support](https://pytorch.org/docs/master/generated/torch.nn.MultiheadAttention.html#torch.nn.MultiheadAttention)\n[in](https://pytorch.org/docs/master/generated/torch.nn.EmbeddingBag.html#torch.nn.EmbeddingBag) \nPyTorch that aims to support dynamic shapes via extra arguments such as a\n[padding index](https://pytorch.org/docs/master/generated/torch.nn.CrossEntropyLoss.html#torch.nn.CrossEntropyLoss).\nOf course, while these functions are fast and sometimes memory efficient, they don't provide a consistent interface.\n\nOther users simply gave up and started writing [for-loops](https://github.com/pytorch/vision/blob/1aef87d01eec2c0989458387fa04baebcc86ea7b/torchvision/models/detection/transform.py#L97), or discovered that batching didn't help.\n\nWe want to have a single abstraction that is consistent, fast, memory efficient and readable and the nestedtensor project aims to provide that.\n\n## How does nestedtensor help here?\n\nNestedTensors are a generalization of torch Tensors which eases working with data of different shapes and lengths. \nIn a nutshell, Tensors have scalar entries (e.g. floats) and NestedTensors have Tensor entries. However, note that\na NestedTensor is still a Tensor. That means it needs to have a single dimension, single dtype, single device and single layout.\n\n Tensor entry constraints:\n - Each Tensor constituent is of the dtype, layout and device of the containing NestedTensor.\n - The dimension of a constituent Tensor must be less than the dimension of the NestedTensor. \n - An empty NestedTensor is of dimension zero.\n\n## Prototype classification\n\nThe nestedtensor package is a prototype intended for early stage feedback and testing. It is on the road to a beta classification, but there is no definitive timeline yet. See [PyTorch feature classification](https://pytorch.org/docs/stable/index.html) for what prototype, beta and stale means.\n\n## Dependencies\n\n- pytorch (installed from nestedtensor/third_party/pytorch submodule)\n- torchvision (needed for examples and tests)\n- ipython (needed for examples)\n- notebook (needed for examples)\n\n## Contribution\nThe project is under active development. If you have a suggestions or found a bug, please file an issue!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpytorch%2Fnestedtensor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpytorch%2Fnestedtensor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpytorch%2Fnestedtensor/lists"}