{"id":13471790,"url":"https://github.com/parrt/tensor-sensor","last_synced_at":"2025-10-24T04:20:34.095Z","repository":{"id":45488050,"uuid":"291160909","full_name":"parrt/tensor-sensor","owner":"parrt","description":"The goal of this library is to generate more helpful exception messages for matrix algebra expressions for numpy, pytorch, jax, tensorflow, keras, fastai. ","archived":false,"fork":false,"pushed_at":"2022-04-07T20:52:55.000Z","size":50857,"stargazers_count":805,"open_issues_count":8,"forks_count":40,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-04-08T17:18:07.758Z","etag":null,"topics":["debugging","deep-learning","jax","matrix","numpy","python","pytorch","tensorflow","tracing","vector"],"latest_commit_sha":null,"homepage":"https://github.com/parrt/tensor-sensor","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/parrt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"contributing.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-08-28T22:54:04.000Z","updated_at":"2025-03-31T09:10:53.000Z","dependencies_parsed_at":"2022-07-19T00:02:01.035Z","dependency_job_id":null,"html_url":"https://github.com/parrt/tensor-sensor","commit_stats":null,"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/parrt%2Ftensor-sensor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/parrt%2Ftensor-sensor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/parrt%2Ftensor-sensor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/parrt%2Ftensor-sensor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/parrt","download_url":"https://codeload.github.com/parrt/tensor-sensor/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254478190,"owners_count":22077676,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["debugging","deep-learning","jax","matrix","numpy","python","pytorch","tensorflow","tracing","vector"],"created_at":"2024-07-31T16:00:49.279Z","updated_at":"2025-10-24T04:20:34.012Z","avatar_url":"https://github.com/parrt.png","language":"Jupyter Notebook","readme":"# Tensor Sensor\n\nSee article [Clarifying exceptions and visualizing tensor operations in deep learning code](https://explained.ai/tensor-sensor/index.html) and [TensorSensor implementation slides](https://github.com/parrt/tensor-sensor/raw/master/talks/tensor-sensor.pdf) (PDF).\n\n(*As of September 2021, M1 macs experience illegal instructions in many of the tensor libraries installed via Anaconda, so you should expect TensorSensor to work only on Intel-based Macs at the moment. PyTorch appears to work.*)\n\n\u003cimg src=\"images/dtypes.png\" width=\"50%\" align=\"right\"\u003eOne of the biggest challenges when writing code to implement deep learning networks, particularly for us newbies, is getting all of the tensor (matrix and vector) dimensions to line up properly. It's really easy to lose track of tensor dimensionality in complicated expressions involving multiple tensors and tensor operations.  Even when just feeding data into predefined [Tensorflow](https://www.tensorflow.org/) network layers, we still need to get the dimensions right. When you ask for improper computations, you're going to run into some less than helpful exception messages.  \n\nTo help myself and other programmers debug tensor code, I built this library.  TensorSensor clarifies exceptions by augmenting messages and visualizing Python code to indicate the shape of tensor variables (see figure to the right for a teaser). It works with [Tensorflow](https://www.tensorflow.org/), [PyTorch](https://pytorch.org/), [JAX](https://github.com/google/jax), and [Numpy](https://numpy.org/), as well as higher-level libraries like [Keras](https://keras.io/) and [fastai](https://www.fast.ai/).\n\n*TensorSensor is currently at 1.0 (December 2021)*.\n\n## Visualizations\n\nFor more, see [examples.ipynb at colab](https://colab.research.google.com/github/parrt/tensor-sensor/blob/master/testing/examples.ipynb). (The github rendering does not show images for some reason: [examples.ipynb](testing/examples.ipynb).)\n\n```python\nimport numpy as np\n\nn = 200         # number of instances\nd = 764         # number of instance features\nn_neurons = 100 # how many neurons in this layer?\n\nW = np.random.rand(d,n_neurons)\nb = np.random.rand(n_neurons,1)\nX = np.random.rand(n,d)\nwith tsensor.clarify() as c:\n    Y = W @ X.T + b\n```\n\nDisplays this in a jupyter notebook or separate window:\n\n\u003cimg src=\"images/mm.svg\"\u003e\n\nInstead of the following default exception message:\n\n```\nValueError: matmul: Input operand 1 has a mismatch in its core dimension 0, with gufunc signature (n?,k),(k,m?)-\u003e(n?,m?) (size 764 is different from 100)\n```\n\nTensorSensor augments the message with more information about which operator caused the problem and includes the shape of the operands:\n\n```\nCause: @ on tensor operand W w/shape (764, 100) and operand X.T w/shape (764, 200)\n```\n\nYou can also get the full computation graph for an expression that includes all of the sub-expression shapes.\n \n```python\nW = torch.rand(size=(2000,2000), dtype=torch.float64)\nb = torch.rand(size=(2000,1), dtype=torch.float64)\nh = torch.zeros(size=(1_000_000,), dtype=int)\nx = torch.rand(size=(2000,1))\nz = torch.rand(size=(2000,1), dtype=torch.complex64)\n\ntsensor.astviz(\"b = W@b + (h+3).dot(h) + z\", sys._getframe())\n```\n\nyields the following abstract syntax tree with shapes:\n\n\u003cimg src=\"images/ast.svg\" width=\"250\"\u003e\n\n## Install\n\n```\npip install tensor-sensor             # This will only install the library for you\npip install tensor-sensor[torch]      # install pytorch related dependency\npip install tensor-sensor[tensorflow] # install tensorflow related dependency\npip install tensor-sensor[jax]        # install jax, jaxlib\npip install tensor-sensor[all]        # install tensorflow, pytorch, jax\n```\n\nwhich gives you module `tsensor`. I developed and tested with the following versions\n\n```\n$ pip list | grep -i flow\ntensorflow                         2.5.0\ntensorflow-estimator               2.5.0\n$ pip list | grep -i numpy\nnumpy                              1.19.5\nnumpydoc                           1.1.0\n$ pip list | grep -i torch\ntorch                              1.10.0\ntorchvision                        0.10.0\n$ pip list | grep -i jax\njax                                0.2.20\njaxlib                             0.1.71\n```\n\n### Graphviz for tsensor.astviz()\n\nFor displaying abstract syntax trees (ASTs) with `tsensor.astviz(...)`, you need the `dot` executable from graphviz, not just the python library. \n\nOn **Mac**, do this before or after tensor-sensor install:\n\n```\nbrew install graphviz\n```\n\nOn **Windows**, apparently you need\n\n```\nconda install python-graphviz  # Do this first; get's dot executable and py lib\npip install tensor-sensor      # Or one of the other installs\n```\n\n\n## Limitations\n\nI rely on parsing lines that are assignments or expressions only so the clarify and explain routines do not handle methods expressed like:\n\n```\ndef bar(): b + x * 3\n```\n\nInstead, use\n\n```\ndef bar():\n\tb + x * 3\n```\n\nwatch out for side effects!  I don't do assignments, but any functions you call with side effects will be done while I reevaluate statements.\n\nCan't handle `\\` continuations.\n\nWith Python `threading` package, don't use multiple threads calling clarify(). `multiprocessing` package should be fine.\n\nAlso note: I've built my own parser to handle just the assignments / expressions tsensor can handle.\n\n## Deploy (parrt's use)\n\n```bash\n$ python setup.py sdist upload \n```\n\nOr download and install locally\n\n```bash\n$ cd ~/github/tensor-sensor\n$ pip install .\n```\n\n### TODO\n\n* can i call pyviz in debugger?\n","funding_links":[],"categories":["Jupyter Notebook","Pytorch实用程序"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparrt%2Ftensor-sensor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fparrt%2Ftensor-sensor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparrt%2Ftensor-sensor/lists"}