{"id":13593946,"url":"https://github.com/eduardoleao052/js-pytorch","last_synced_at":"2025-05-14T21:11:13.292Z","repository":{"id":229770650,"uuid":"764272743","full_name":"eduardoleao052/js-pytorch","owner":"eduardoleao052","description":"A JavaScript library like PyTorch, with GPU acceleration.","archived":false,"fork":false,"pushed_at":"2024-11-15T18:32:17.000Z","size":28692,"stargazers_count":1161,"open_issues_count":7,"forks_count":51,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-05-06T12:48:57.458Z","etag":null,"topics":["automatic-differentiation","deep-learning","javascript-library","neural-networks","pytorch"],"latest_commit_sha":null,"homepage":"https://eduardoleao052.github.io/js-pytorch/site/index.html","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eduardoleao052.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-27T19:33:50.000Z","updated_at":"2025-05-06T11:04:56.000Z","dependencies_parsed_at":"2024-04-16T02:32:15.957Z","dependency_job_id":"f50a45ff-8303-45ab-ab11-71a72f4b794c","html_url":"https://github.com/eduardoleao052/js-pytorch","commit_stats":null,"previous_names":["eduardoleao052/js-torch","eduardoleao052/js-pytorch"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eduardoleao052%2Fjs-pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eduardoleao052%2Fjs-pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eduardoleao052%2Fjs-pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eduardoleao052%2Fjs-pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eduardoleao052","download_url":"https://codeload.github.com/eduardoleao052/js-pytorch/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254227631,"owners_count":22035671,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automatic-differentiation","deep-learning","javascript-library","neural-networks","pytorch"],"created_at":"2024-08-01T16:01:26.728Z","updated_at":"2025-05-14T21:11:08.247Z","avatar_url":"https://github.com/eduardoleao052.png","language":"JavaScript","funding_links":[],"categories":["deep-learning","Recently Updated","pytorch","JavaScript","Machine Learning"],"sub_categories":["[Sep 03, 2024](/content/2024/09/03/README.md)","Runner"],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./assets/docs_logo.png\" alt=\"js-torch\" height=\"135\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://github.com/eduardoleao052/js-torch/actions/workflows/test.yml/badge.svg\" alt=\"Unit Tests\"\u003e\n        \u003cimg src=\"https://github.com/eduardoleao052/js-torch/actions/workflows/test.yml/badge.svg\" /\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/eduardoleao052/js-torch/pulse\" alt=\"Activity\"\u003e\n        \u003cimg src=\"https://img.shields.io/github/commit-activity/m/eduardoleao052/js-torch\" /\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/eduardoleao052/js-torch/graphs/contributors\" alt=\"Contributors\"\u003e\n        \u003cimg src=\"https://img.shields.io/github/contributors/eduardoleao052/js-torch\" /\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/eduardoleao052/js-torch\"\u003e\n        \u003cimg src=\"https://img.shields.io/badge/language-JavaScript-yellow\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"mailto:eduardoleao052@usp.br\"\u003e\n        \u003cimg src=\"https://img.shields.io/badge/-Email-red?style=flat-square\u0026logo=gmail\u0026logoColor=white\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://www.linkedin.com/in/eduardoleao052/\"\u003e\n        \u003cimg src=\"https://img.shields.io/badge/-Linkedin-blue?style=flat-square\u0026logo=linkedin\"\u003e\n    \u003c/a\u003e\n\u003c/p\u003e\n\n# PyTorch in JavaScript\n\n- JS-PyTorch is a Deep Learning **JavaScript library** built from scratch, to closely follow PyTorch's syntax.\n- This library has **GPU support**, using GPU.js.\n- If you want to run it yourself, check out the \u003ca href=\"https://eduardoleao052.github.io/js-pytorch/site/index.html\" target=\"blank\"\u003eDocumentation\u003c/a\u003e.\n- Try out the \u003ca href=\"https://eduardoleao052.github.io/js-pytorch/site/demo/\" target=\"blank\"\u003eWeb Demo\u003c/a\u003e!\n\n\u003e **Note:** You can install the package locally with: `npm install js-pytorch`\n\n\u003cbr\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e Implemented Tensor \u003cb\u003eOperations\u003c/b\u003e: \u003c/summary\u003e\n\n\u003cbr/\u003e\n\n- [Add](https://github.com/eduardoleao052/js-torch/blob/07c1286867b952f32c0e904033214253e8812090/src/tensor.js#L346-L401)\n- [Subtract](https://github.com/eduardoleao052/js-torch/blob/07c1286867b952f32c0e904033214253e8812090/src/tensor.js#L404-L438)\n- [Multiply](https://github.com/eduardoleao052/js-torch/blob/07c1286867b952f32c0e904033214253e8812090/src/tensor.js#L441-L496)\n- [Divide](https://github.com/eduardoleao052/js-torch/blob/07c1286867b952f32c0e904033214253e8812090/src/tensor.js#L498-L557)\n- [Matrix Multiply](https://github.com/eduardoleao052/js-torch/blob/07c1286867b952f32c0e904033214253e8812090/src/tensor.js#L560-L621)\n- [Power](https://github.com/eduardoleao052/js-torch/blob/07c1286867b952f32c0e904033214253e8812090/src/tensor.js#L625-L663)\n- [Square Root](https://github.com/eduardoleao052/js-torch/blob/07c1286867b952f32c0e904033214253e8812090/src/tensor.js#L666-L704)\n- [Exponentiate](https://github.com/eduardoleao052/js-torch/blob/07c1286867b952f32c0e904033214253e8812090/src/tensor.js#706-L744)\n- [Log](https://github.com/eduardoleao052/js-torch/blob/07c1286867b952f32c0e904033214253e8812090/src/tensor.js#L746-L785)\n- [Sum](https://github.com/eduardoleao052/js-torch/blob/07c1286867b952f32c0e904033214253e8812090/src/tensor.js#L790-L842)\n- [Mean](https://github.com/eduardoleao052/js-torch/blob/07c1286867b952f32c0e904033214253e8812090/src/tensor.js#L844-L894)\n- [Variance](https://github.com/eduardoleao052/js-torch/blob/07c1286867b952f32c0e904033214253e8812090/src/tensor.js#L896-L949)\n- [Transpose](https://github.com/eduardoleao052/js-torch/blob/07c1286867b952f32c0e904033214253e8812090/src/tensor.js#L953-L1008)\n- [At](https://github.com/eduardoleao052/js-torch/blob/07c1286867b952f32c0e904033214253e8812090/src/tensor.js#L1010-L1060)\n- [MaskedFill](https://github.com/eduardoleao052/js-torch/blob/07c1286867b952f32c0e904033214253e8812090/src/tensor.js#L1062-L1095)\n- [Reshape](https://github.com/eduardoleao052/js-torch/blob/07c1286867b952f32c0e904033214253e8812090/src/tensor.js#L1097-L1129)\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e Implemented Deep Learning \u003cb\u003eLayers\u003c/b\u003e: \u003c/summary\u003e\n\n\u003cbr/\u003e\n\n- [nn.Linear](https://github.com/eduardoleao052/js-torch/blob/a158c91db9775a88fae6ed2d0f76d6d8ee6f9d23/src/layers.js#L60-L88)\n- [nn.MultiHeadSelfAttention](https://github.com/eduardoleao052/js-torch/blob/a158c91db9775a88fae6ed2d0f76d6d8ee6f9d23/src/layers.js#L90-L163)\n- [nn.FullyConnected](https://github.com/eduardoleao052/js-torch/blob/a158c91db9775a88fae6ed2d0f76d6d8ee6f9d23/src/layers.js#L165-L194)\n- [nn.Block](https://github.com/eduardoleao052/js-torch/blob/a158c91db9775a88fae6ed2d0f76d6d8ee6f9d23/src/layers.js#L196-L226)\n- [nn.Embedding](https://github.com/eduardoleao052/js-torch/blob/a158c91db9775a88fae6ed2d0f76d6d8ee6f9d23/src/layers.js#L231-L260)\n- [nn.PositionalEmbedding](https://github.com/eduardoleao052/js-torch/blob/a158c91db9775a88fae6ed2d0f76d6d8ee6f9d23/src/layers.js#L262-L291)\n- [nn.ReLU](https://github.com/eduardoleao052/js-torch/blob/a158c91db9775a88fae6ed2d0f76d6d8ee6f9d23/src/layers.js#L296-L325)\n- [nn.Softmax](https://github.com/eduardoleao052/js-torch/blob/a158c91db9775a88fae6ed2d0f76d6d8ee6f9d23/src/layers.js#L327-L346)\n- [nn.Dropout](https://github.com/eduardoleao052/js-torch/blob/a158c91db9775a88fae6ed2d0f76d6d8ee6f9d23/src/layers.js#L351-L376)\n- [nn.LayerNorm](https://github.com/eduardoleao052/js-torch/blob/a158c91db9775a88fae6ed2d0f76d6d8ee6f9d23/src/layers.js#L378-L397)\n- [nn.CrossEntropyLoss](https://github.com/eduardoleao052/js-torch/blob/a158c91db9775a88fae6ed2d0f76d6d8ee6f9d23/src/layers.js#L400-L441)\n\n\u003c/details\u003e\n\u003cbr/\u003e\n\n## 1.Table of Contents\n\n* [Installation](#2-installation)\n* [Running it Yourself](#3-Running-it-Yourself)\n  * [Simple Autograd Example](#simple-autograd-example)\n  * [Complex Autograd Example (Transformer)](#complex-autograd-example-transformer)\n  * [Saving and Loading models](#saving-and-loading-models)\n* [Distribution \u0026 Devtools](#4-distribution--devtools)\n* [Future Work](#5-future-work)\n\n## 2. Installation\n\n- On **MacOS**, **Windows**, and **Ubuntu**, you can install the library with `npm install js-pytorch`.\n- On **Windows**, if you run into an error, you might need to install the latest version of [Visual Studio](https://visualstudio.microsoft.com/downloads/?cid=learn-navbar-download-cta), including the \"Desktop development with C++\" workload.\n- To run in the **Browser**, paste the following tag in the `\u003chead\u003e` of your HTML file:\n  \n```html\n\u003cscript src=\"https://cdnjs.cloudflare.com/ajax/libs/js-pytorch/0.7.2/js-pytorch-browser.js\"\n        integrity=\"sha512-l22t7GnqXvHBMCBvPUBdFO2TEYxnb1ziCGcDQcpTB2un16IPA4FE5SIZ8bUR+RwoDZGikQkWisO+fhnakXt9rg==\"\n        crossorigin=\"anonymous\"\n        referrerpolicy=\"no-referrer\"\u003e\u003c/script\u003e\n```\n\n- After that, you can use JS-PyTorch freely in any `\u003cscript\u003e` in your HTML file:\n\n```html\n\u003chead\u003e\n    \u003ctitle\u003eMy Project\u003c/title\u003e\n    \u003c!-- New script goes here --\u003e\n    \u003cscript src=\"https://cdnjs.cloudflare.com/ajax/libs/js-pytorch/0.7.2/js-pytorch-browser.js\" \n            integrity=\"sha512-l22t7GnqXvHBMCBvPUBdFO2TEYxnb1ziCGcDQcpTB2un16IPA4FE5SIZ8bUR+RwoDZGikQkWisO+fhnakXt9rg==\"\n            crossorigin=\"anonymous\" \n            referrerpolicy=\"no-referrer\"\u003e\n    \u003c/script\u003e\n    \u003c!----\u003e\n\u003c/head\u003e\n\u003cbody\u003e\n    \u003cscript\u003e\n        let x = torch.randn([10,5])\n        let linear = new torch.nn.Linear(5,1,'gpu',true)\n        let z = linear.forward(x)\n        console.log(z.data)\n    \u003c/script\u003e\n\u003c/body\u003e\n```\n\n## 3. Running it Yourself\n\n### Simple Autograd Example:\n\n```typescript\n// Require the Library if running in node (not necessary in the browser):\nconst { torch } = require(\"js-pytorch\");\n\n// Pass device as an argument to a Tensor or nn.Module (same as PyTorch):\nconst device = 'gpu';\n\n// Instantiate Tensors:\nlet x = torch.randn([8, 4, 5]);\nlet w = torch.randn([8, 5, 4], true, device);\nlet b = torch.tensor([0.2, 0.5, 0.1, 0.0], true);\n\n// Make calculations:\nlet out = torch.matmul(x, w);\nout = torch.add(out, b);\n\n// Compute gradients on whole graph:\nout.backward();\n\n// Get gradients from specific Tensors:\nconsole.log(w.grad);\nconsole.log(b.grad);\n```\n\n### Complex Autograd Example (Transformer):\n\n```typescript\n// Require the Library if running in node (not necessary in the browser):\nconst { torch } = require(\"js-pytorch\");\nconst nn = torch.nn;\nconst optim = torch.optim;\n\nconst device = 'gpu';\n\n// Define training hyperparameters:\nconst vocab_size = 52;\nconst hidden_size = 32;\nconst n_timesteps = 16;\nconst n_heads = 4;\nconst dropout_p = 0;\nconst batch_size = 8;\n\n// Create Transformer decoder Module:\nclass Transformer extends nn.Module {\n  constructor(vocab_size, hidden_size, n_timesteps, n_heads, dropout_p, device) {\n    super();\n    // Instantiate Transformer's Layers:\n    this.embed = new nn.Embedding(vocab_size, hidden_size);\n    this.pos_embed = new nn.PositionalEmbedding(n_timesteps, hidden_size);\n    this.b1 = new nn.Block(hidden_size, hidden_size, n_heads, n_timesteps, dropout_p, device);\n    this.b2 = new nn.Block(hidden_size, hidden_size, n_heads, n_timesteps, dropout_p, device);\n    this.ln = new nn.LayerNorm(hidden_size);\n    this.linear = new nn.Linear(hidden_size, vocab_size, device);\n  }\n\n  forward(x) {\n    let z;\n    z = torch.add(this.embed.forward(x), this.pos_embed.forward(x));\n    z = this.b1.forward(z);\n    z = this.b2.forward(z);\n    z = this.ln.forward(z);\n    z = this.linear.forward(z);\n    return z;\n  }\n}\n\n// Instantiate your custom nn.Module:\nconst model = new Transformer(vocab_size, hidden_size, n_timesteps, n_heads, dropout_p, device);\n\n// Define loss function and optimizer:\nconst loss_func = new nn.CrossEntropyLoss();\nconst optimizer = new optim.Adam(model.parameters(), (lr = 5e-3), (reg = 0));\n\n// Instantiate sample input and output:\nlet x = torch.randint(0, vocab_size, [batch_size, n_timesteps, 1]);\nlet y = torch.randint(0, vocab_size, [batch_size, n_timesteps]);\nlet loss;\n\n// Training Loop:\nfor (let i = 0; i \u003c 40; i++) {\n  // Forward pass through the Transformer:\n  let z = model.forward(x);\n\n  // Get loss:\n  loss = loss_func.forward(z, y);\n\n  // Backpropagate the loss using torch.tensor's backward() method:\n  loss.backward();\n\n  // Update the weights:\n  optimizer.step();\n\n  // Reset the gradients to zero after each training step:\n  optimizer.zero_grad();\n\n  // Print loss at every iteration:\n  console.log(`Iter ${i} - Loss ${loss.data[0].toFixed(4)}`)\n}\n```\n\n### Saving and Loading models:\n\n```typescript\n// Instantiate your model:\nconst model = new Transformer(vocab_size, hidden_size, n_timesteps, n_heads, dropout_p);\n\n// Train the model:\ntrainModel(model);\n\n// Save model to JSON file:\ntorch.save(model, 'model.json')\n\n// To load, instantiate placeHolder using the original model's architecture:\nconst placeHolder = new Transformer(vocab_size, hidden_size, n_timesteps, n_heads, dropout_p);\n\n// Load weights into placeHolder:\nconst newModel = torch.load(placeHolder, 'model.json')\n```\n\n\n\u003cbr/\u003e\n\n## 4. Distribution \u0026 Devtools\n\n- **Build for Distribution** by running `npm run build`. CJS and ESM modules and `index.d.ts` will be output in the `dist/` folder.\n- **Check the Code** with ESLint at any time, running `npm run lint`.\n- **Run tests** run `npm test`.\n- **Improve Code Formatting** with prettier, running `npm run prettier`.\n- **Performance Benchmarks** are also included in the `tests/benchmarks/` directory. Run all benchmarks with `npm run bench` and save new benchmarks with `npm run bench:update`.\n\n\n## 5. Future Work\n\n- This package is not as optimized as PyTorch yet, but I tried making it more interpretable. Efficiency improvements are incoming!\n- Feel free to **contribute**! Create a merge request to the `develop` branch, and also feel free to reach out. I'll try to answer as soon as possible.\n- Hope you enjoy!\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feduardoleao052%2Fjs-pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feduardoleao052%2Fjs-pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feduardoleao052%2Fjs-pytorch/lists"}