{"id":21270661,"url":"https://github.com/timstr/scorch","last_synced_at":"2025-03-15T12:12:17.277Z","repository":{"id":130203206,"uuid":"388328306","full_name":"timstr/scorch","owner":"timstr","description":"Like torch, but rather than seeing the light, you get burnt.","archived":false,"fork":false,"pushed_at":"2021-07-27T04:41:24.000Z","size":67,"stargazers_count":3,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-22T02:31:23.947Z","etag":null,"topics":["automatic-differentiation","deep-learning","gradient-descent","linear-algebra","math","matrix","numeric","tensor"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/timstr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-07-22T04:41:18.000Z","updated_at":"2022-06-05T15:06:34.000Z","dependencies_parsed_at":"2023-07-12T13:16:25.184Z","dependency_job_id":null,"html_url":"https://github.com/timstr/scorch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timstr%2Fscorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timstr%2Fscorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timstr%2Fscorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timstr%2Fscorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/timstr","download_url":"https://codeload.github.com/timstr/scorch/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243725636,"owners_count":20337670,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automatic-differentiation","deep-learning","gradient-descent","linear-algebra","math","matrix","numeric","tensor"],"created_at":"2024-11-21T08:18:13.790Z","updated_at":"2025-03-15T12:12:17.271Z","avatar_url":"https://github.com/timstr.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Scorch\n\nA lightweight, cross-platform, header-only library written in standard C++ for tensor arithmetic with automatic differentiation, designed to closely mimick the famous PyTorch library in usage and appearance but with improved compile-time safety checks.\n\nHere's an example that trains a two-layer neural network to learn the identity function. This code can be run in `demo.cpp`. Most of this should look and feel immediately familiar to a PyTorch user.\n```C++\n// layer sizes\nconstexpr std::size_t InputDim = 4;\nconstexpr std::size_t HiddenDim = 16;\nconstexpr std::size_t OutputDim = 4;\n\n// learnable network parameters\nauto W0 = scorch::rand\u003cfloat, HiddenDim, InputDim\u003e();\nauto b0 = scorch::rand\u003cfloat, HiddenDim\u003e();\nauto W1 = scorch::rand\u003cfloat, HiddenDim, HiddenDim\u003e();\nauto b1 = scorch::rand\u003cfloat, HiddenDim\u003e();\nauto W2 = scorch::rand\u003cfloat, OutputDim, HiddenDim\u003e();\nauto b2 = scorch::rand\u003cfloat, OutputDim\u003e();\n\n// optimizer\n// learning rate, momentum ratio, parameters...\nauto opt = scorch::optim::SGD(0.1f, 0.8f, W0, b0, W1, b1);\n\n// batch size\nconstexpr std::size_t BatchDim = 16;\n\nfor (auto i = 0; i \u003c 100; ++i) {\n    // random input\n    auto x = scorch::rand\u003cfloat, BatchDim, InputDim\u003e();\n\n    // identity function: output is equal to input\n    auto y = copy(x);\n\n    // compute the network output\n    // Yes, it's actually this simple\n    auto y_hat = sigmoid(sigmoid(x % W0 + b0) % W1 + b1) % W2 + b2;\n\n    // compute the loss\n    auto l = mean((y_hat - y) ^ 2.0f);\n\n    // don't forget to zero the gradients before back-propagation\n    opt.zero_grad();\n\n    // compute the gradients of all parameters w.r.t. the loss\n    l.backward();\n\n    // take a training step\n    opt.step();\n}\n```\n\nNotable features:\n - Support for vector, matrix, and tensor variables with arbitrarily many dimensions\n - Support for scalar variables\n - Element-wise functions, broadcasting semantics*, matrix-vector mulitplication, and more.\n - The usual overloaded operators, plus `%` for matrix-vector multiplication and `^` for exponentiation.\n - Extremely ergonomic syntax for writing expressions (see the example)\n - Compile-time checking of tensor shape compatibility (!!!)\n - Automatic differentiation using reverse-mode gradient computation\n - Dynamic computational graphs\n - Optimizers (only SGD for now)\n - Tested with MSVC, GCC, and Clang\n\n\\* Broadcasting semantics are only supported for pairs of tensors whose shapes are identical except that one may have additional higher dimensions. For example, a size 3x5x7 tensor is broadcastable with a size 5x7 tensor and a size 7 tensor, but a size 3x5x7 tensor is **not** broadcastable with a size 1x1x7 tensor, or a size 1x1x1 tensor.\n\nFeatures that are not supported but are probably coming soon:\n - Tensor views, clever indexing, and differentiation through tensor scalar element access\n - Convolutions\n - Matrix-matrix multiplication\n - Some remaining basic mathematical functions (e.g. `cbrt`, `atan`, etc...)\n - Smarter optimizers (e.g. Adam, RMSProp, if I can understand them)\n - Higher-order derivatives (maybe)\n\nFeatures that not supported and probably never will be:\n - GPU acceleration\n - Dynamically-sized tensors\n\nThis code was written by Tim Straubinger and is made available for free use under the MIT license.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimstr%2Fscorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftimstr%2Fscorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimstr%2Fscorch/lists"}