{"id":25523297,"url":"https://github.com/adam-mazur/tinytensor","last_synced_at":"2026-04-29T22:31:54.182Z","repository":{"id":278423928,"uuid":"935572902","full_name":"Adam-Mazur/TinyTensor","owner":"Adam-Mazur","description":"A from-scratch PyTorch's Tensor reimplementation in C++","archived":false,"fork":false,"pushed_at":"2025-02-19T17:11:11.000Z","size":75,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-19T18:24:35.744Z","etag":null,"topics":["cpp","cpp17","from-scratch","pytorch","pytorch-implementation","tensor"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Adam-Mazur.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-19T17:01:03.000Z","updated_at":"2025-02-19T17:11:15.000Z","dependencies_parsed_at":"2025-02-19T18:24:39.923Z","dependency_job_id":null,"html_url":"https://github.com/Adam-Mazur/TinyTensor","commit_stats":null,"previous_names":["adam-mazur/tinytensor"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Adam-Mazur%2FTinyTensor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Adam-Mazur%2FTinyTensor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Adam-Mazur%2FTinyTensor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Adam-Mazur%2FTinyTensor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Adam-Mazur","download_url":"https://codeload.github.com/Adam-Mazur/TinyTensor/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239718423,"owners_count":19685725,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp","cpp17","from-scratch","pytorch","pytorch-implementation","tensor"],"created_at":"2025-02-19T19:18:40.305Z","updated_at":"2025-12-29T09:30:14.959Z","avatar_url":"https://github.com/Adam-Mazur.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TinyTensor\n![License](https://img.shields.io/badge/license-MIT-blue)\n![C++](https://img.shields.io/badge/C++-17-blue)\n![Contributions welcome](https://img.shields.io/badge/contributions-welcome-brightgreen)\n![Dependencies](https://img.shields.io/badge/dependencies-none-green)\n![Made with Love](https://img.shields.io/badge/made%20with-%E2%9D%A4-red)\n\nThis project is a from-scratch reimplementation of PyTorch’s Tensor object in C++. It supports fundamental tensor operations, including indexing, broadcasting, automatic differentiation, and various linear algebra functions. The goal is to provide a lightweight yet fully capable Tensor implementation that serves as a learning tool for exploring PyTorch internals.\n\nI started this project as a university assignment for Object Programming classes, but I decided to continue developing it as a personal project. While it is still far from fully replicating PyTorch’s features, it is already capable of training simple neural networks, such as a CNN for MNIST, with reasonable performance.\n\n## Features\nHere is a list of features that the project supports:\n- Strided array data structure \n- Memory management and reference counting\n- Basic arithmetic operations (addition, subtraction, multiplication, division, etc.)\n- Indexing and slicing (designed to closely match PyTorch behavior)\n- Broadcasting (for element-wise operations and matrix multiplication)\n- Linear algebra operations (e.g., matmul, mm)\n- Math functions (e.g., exp, log, sum, mean, var, max, min)\n- Machine learning operations (e.g., ReLU, softmax, cross-entropy)\n- Automatic differentiation (backpropagation)\n- Other PyTorch-inspired functions (e.g., stack, unfold, view, transpose) \n\n## Dependencies\nThis project uses the following dependencies:\n- CMake\n- Catch2 (installed automatically with CMake)\n- Valgrind\n\n## Installation\n1. Clone the repository:\n```bash\ngit clone https://github.com/Adam-Mazur/TinyTensor.git\ncd tiny_tensor\n```\n2. Create a build directory:\n```bash\nmkdir build\ncd build\n```\n3. Run CMake:\n```bash\ncmake ..\n```\n4. Build the project:\n```bash\nmake\n```\n5. Run the tests:\n```bash\nctest\n```\n## Usage\nGenerally, the project is designed to match PyTorch’s API as closely as possible. However, some differences exist mainly because some simplifications were made to keep the project manageable. Here is an example of how to use the project to train a simple linear regressor:\n\n```cpp\n// ...\nTensor\u003cfloat\u003e x = Tensor\u003cfloat\u003e::randn({NUM_SAMPLES, 1}) * 10.0;\nTensor\u003cfloat\u003e y = x * TRUE_W + TRUE_B + Tensor\u003cfloat\u003e::randn({NUM_SAMPLES, 1}) * 0.1;\n\nTensor\u003cfloat\u003e w = Tensor\u003cfloat\u003e::randn({1}, true);\nTensor\u003cfloat\u003e b = Tensor\u003cfloat\u003e::randn({1}, true);\n\nfor (int i = 0; i \u003c EPOCHS; i++)\n{\n    w.zero_grad();\n    b.zero_grad();\n\n    Tensor\u003cfloat\u003e y_pred = x * w + b;\n    Tensor\u003cfloat\u003e loss = (y_pred - y).pow(2).mean();\n\n    loss.backward();\n\n    w += (*w.grad) * (-LEARNING_RATE);\n    b += (*b.grad) * (-LEARNING_RATE);\n}\n// ...\n```\n\nFor the full example, see `tests/integration_test.cpp` file. \n\n---\nAnother example is training a simple CNN for the Binary MNIST dataset. The code below shows the forward pass of the network:\n\n```cpp\n// ...\nTensor\u003cfloat\u003e out1 = conv2d(x, w1, 16, 3, 2, 1);\nTensor\u003cfloat\u003e out2 = conv2d(Tensor\u003cfloat\u003e::relu(out1), w2, 32, 3, 2, 1);\nTensor\u003cfloat\u003e out3 = conv2d(Tensor\u003cfloat\u003e::relu(out2), w3, 64, 3, 2, 1);\nTensor\u003cfloat\u003e out4 = Tensor\u003cfloat\u003e::relu(out3).view({x.size()[0], -1}); // Flatten\nTensor\u003cfloat\u003e out5 = Tensor\u003cfloat\u003e::matmul(out4, w4) + b1;\nTensor\u003cfloat\u003e out6 = Tensor\u003cfloat\u003e::relu(out5);\nreturn Tensor\u003cfloat\u003e::matmul(out6, w5) + b2;\n// ...\n```\nFor the full example, see the `src/demo.cpp` file. To run this code, you need to download the data with the following command:\n```bash\npython3 download_data.py\n```  \nAnd then run the following command (inside the build folder):\n```bash\n./demo\n```\n\n## Contributing\nContributions are welcome! If you'd like to improve this project, here’s how you can help:\n\n### How to Contribute\n1. Fork the repository and clone it to your local machine.\n2. Create a new branch for your feature or bug fix.\n3. Make your changes and ensure the code is clean and well-documented.\n4. Commit and push your changes.\n5. Open a pull request on GitHub.\n\n### Reporting Issues\nIf you find a bug or have a feature request, please open an issue with a detailed description.\n\n\n## License\nThis project is licensed under the **MIT License**. ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadam-mazur%2Ftinytensor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadam-mazur%2Ftinytensor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadam-mazur%2Ftinytensor/lists"}