{"id":13717477,"url":"https://github.com/mctorch/mctorch","last_synced_at":"2025-05-07T07:31:34.219Z","repository":{"id":48035544,"uuid":"149773189","full_name":"mctorch/mctorch","owner":"mctorch","description":"A manifold optimization library for deep learning","archived":false,"fork":false,"pushed_at":"2021-08-10T17:05:32.000Z","size":192596,"stargazers_count":244,"open_issues_count":10,"forks_count":13,"subscribers_count":15,"default_branch":"master","last_synced_at":"2025-04-19T13:43:49.233Z","etag":null,"topics":["deep-learning","manifold-optimization"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mctorch.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-09-21T14:16:20.000Z","updated_at":"2025-03-31T15:57:09.000Z","dependencies_parsed_at":"2022-08-12T17:11:09.584Z","dependency_job_id":null,"html_url":"https://github.com/mctorch/mctorch","commit_stats":null,"previous_names":[],"tags_count":20,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mctorch%2Fmctorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mctorch%2Fmctorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mctorch%2Fmctorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mctorch%2Fmctorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mctorch","download_url":"https://codeload.github.com/mctorch/mctorch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252833539,"owners_count":21811205,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","manifold-optimization"],"created_at":"2024-08-03T00:01:22.855Z","updated_at":"2025-05-07T07:31:32.356Z","avatar_url":"https://github.com/mctorch.png","language":"Python","readme":"# McTorch Lib, a manifold optimization library for deep learning \n\nMcTorch is a Python library that adds manifold optimization functionality to [PyTorch](https://github.com/pytorch/pytorch).  \n\nMcTorch:\n - Leverages tensor computation and GPU acceleration from PyTorch.\n - Enables optimization on manifold constrained tensors to address nonlinear optimization problems.\n - Facilitates constrained weight tensors in deep learning layers.\n\nSections:\n- [More about McTorch](#more-about-mctorch)\n  - [Using McTorch for Optimization](#using-mctorch-for-optimization)\n  - [Using McTorch for Deep Learning](#using-mctorch-for-deep-learning)\n- [Functionality Supported](#functionality-supported)\n- [Installation](#installation)\n- [Release and Contribution](#release-and-contribution)\n- [Team](#team)\n- [Reference](#reference)\n\n## More about McTorch\nMcTorch builds on top of PyTorch and supports all PyTorch functions in addition to Manifold optimization. This is done to ensure researchers and developers using PyTorch can easily experiment with McTorch functions. McTorch's manifold implementations and optimization methods are derived from the Matlab toolbox [Manopt](http://manopt.org/) and the Python toolbox [Pymanopt](https://pymanopt.github.io/).\n\n### Using McTorch for Optimization\n\n1. **Initialize Parameter** - McTorch manifold parameters are same as PyTorch parameters (`mctorch.nn.Parameter`) and requires just addition of one property to parameter initialization to constrain the parameter values. \n2. **Define Cost** - Cost function can be any PyTorch function using the above parameter mixed with non constrained parameters.\n3. **Optimize** - Any optimizer from `mctorch.optim` can be used to optimize the cost function using same functionality as any PyTorch code.\n\n**PCA Example**\n```python\nimport torch\nimport mctorch.nn as mnn\nimport mctorch.optim as moptim\n\n# Random data with high variance in first two dimension\nX = torch.diag(torch.FloatTensor([3,2,1])).matmul(torch.randn(3,200))\n\n# 1. Initialize Parameter\nmanifold_param = mnn.Parameter(manifold=mnn.Stiefel(3,2))\n\n# 2. Define Cost - squared reconstruction error\ndef cost(X, w):\n    wTX = torch.matmul(w.transpose(1,0), X)\n    wwTX = torch.matmul(w, wTX)\n    return torch.sum((X - wwTX)**2)\n\n# 3. Optimize\noptimizer = moptim.rAdagrad(params = [manifold_param], lr=1e-2)\n\nfor epoch in range(30):\n    cost_step = cost(X, manifold_param)\n    print(cost_step)\n    cost_step.backward()\n    optimizer.step()\n    optimizer.zero_grad()\n```\n\n### Using McTorch for Deep Learning\n**Multi Layer Perceptron Example**\n```python\nimport torch\nimport mctorch.nn as mnn\nimport torch.nn.functional as F\n\n# a torch module using constrained linear layers\nclass ManifoldMLP(nn.Module):\n    def __init__(self):\n        super(ManifoldMLP, self).__init__()\n        self.layer1 = mnn.rLinear(in_features=28*28, out_features=100, weight_manifold=mnn.Stiefel)\n        self.layer2 = mnn.rLinear(in_features=100, out_features=100, weight_manifold=mnn.PositiveDefinite)\n        self.output = mnn.rLinear(in_features=100, out_features=10, weight_manifold=mnn.Stiefel)\n\n    def forward(self, x):\n        x = F.relu(self.layer1(x))\n        x = F.relu(self.layer2(x))\n        x = F.log_softmax(self.output(x), dim=0)\n        return x\n\n# create module object and compute cost by applying module on inputs\nmlp_module = ManifoldMLP()\ncost = mlp_module(inputs)\n\n```\n\nMore examples added - [here](examples)\n\n## Functionality Supported\nThis would be an ever increasing list of features. McTorch currently supports:\n\n### Manifolds\n- Stiefel\n- Positive Definite\n- Hyperbolic\n- Doubly Stochastic\n\nAll manifolds support k multiplier as well.\n\n### Optimizers\n- rSGD\n- rAdagrad\n- rASA\n- rConjugateGradient\n\n### Layers\n- Linear\n- Conv1d, Conv2d, Conv3d\n\n\n## Installation\nAfter installing PyTorch can be installed with `python setup.py install`\n\n### Linux\n```bash\nsource activate myenv\nconda install numpy setuptools\n# Add LAPACK support for the GPU if needed\nconda install -c pytorch magma-cuda90 # or [magma-cuda80 | magma-cuda92 | magma-cuda100 ] depending on your cuda version\nconda install pytorch torchvision cudatoolkit=9.0 -c pytorch # or cudatoolkit=10.0 | cudatoolkit=10.1 | .. depending on your cuda version\npip install mctorch-lib\n```\n\n## Release and Contribution\nMcTorch is currently under development and any contributions, suggestions and feature requests are welcome. We'd closely follow PyTorch stable versions to keep the base updated and will have our own versions for other additions.\n\nMcTorch is released under the open source [3-clause BSD License](LICENSE).\n\n## Team \n- [Mayank Meghwanshi](https://github.com/mayank127/)\n- [Satyadev Ntv](https://github.com/satyadevntv/)\n- [Bamdev Mishra](https://github.com/bamdevm)\n- [Pratik Jawanpuria](https://pratikjawanpuria.com)\n- [Hiroyuki Kasai](https://github.com/hiroyuki-kasai)\n- [Anoop Kunchukuttan](https://github.com/anoopkunchukuttan)\n\n## Reference\nPlease cite [[1](https://arxiv.org/abs/1810.01811)] if you found this code useful.\n#### McTorch, a manifold optimization library for deep learning\n[1] M. Meghawanshi, P. Jawanpuria, A. Kunchukuttan, H. Kasai, and B. Mishra, [McTorch, a manifold optimization library for deep learning](https://arxiv.org/abs/1810.01811)\n\n```\n@techreport{meghwanshi2018mctorch,\n  title={McTorch, a manifold optimization library for deep learning},\n  author={Meghwanshi, Mayank and Jawanpuria, Pratik and Kunchukuttan, Anoop and Kasai, Hiroyuki and Mishra, Bamdev},\n  institution={arXiv preprint arXiv:1810.01811},\n  year={2018}\n}\n```\n","funding_links":[],"categories":["Pytorch \u0026 related libraries｜Pytorch \u0026 相关库","Pytorch \u0026 related libraries"],"sub_categories":["Other libraries｜其他库:","Other libraries:"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmctorch%2Fmctorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmctorch%2Fmctorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmctorch%2Fmctorch/lists"}