{"id":30882347,"url":"https://github.com/dbsxdbsx/only_torch","last_synced_at":"2025-09-08T08:46:33.638Z","repository":{"id":62924422,"uuid":"563666929","full_name":"dbsxdbsx/only_torch","owner":"dbsxdbsx","description":"Rust写的简易ML库（尚很不成熟）","archived":false,"fork":false,"pushed_at":"2025-08-01T06:49:09.000Z","size":15157,"stargazers_count":4,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-09-02T08:47:11.232Z","etag":null,"topics":["ai","neat-python","pytorch","rust"],"latest_commit_sha":null,"homepage":"https://crates.io/crates/only_torch","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dbsxdbsx.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-11-09T04:21:25.000Z","updated_at":"2025-08-02T05:22:05.000Z","dependencies_parsed_at":"2024-02-24T04:23:12.008Z","dependency_job_id":"8b2c52dd-9e70-4958-9a20-738afcf661b7","html_url":"https://github.com/dbsxdbsx/only_torch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/dbsxdbsx/only_torch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dbsxdbsx%2Fonly_torch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dbsxdbsx%2Fonly_torch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dbsxdbsx%2Fonly_torch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dbsxdbsx%2Fonly_torch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dbsxdbsx","download_url":"https://codeload.github.com/dbsxdbsx/only_torch/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dbsxdbsx%2Fonly_torch/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274159643,"owners_count":25232636,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-08T02:00:09.813Z","response_time":121,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","neat-python","pytorch","rust"],"created_at":"2025-09-08T08:46:29.684Z","updated_at":"2025-09-08T08:46:33.603Z","avatar_url":"https://github.com/dbsxdbsx.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"## 这是啥？\n\n一个用纯Rust（不用C++）打造的仿Pytorch的玩具型AI框架（目前尚不成熟，请勿使用）。该项目不打算支持GPU--因后期可能要支持安卓等平台，不想受制于某（几）种非CPU设备。但可能会加入NEAT等网络进化的算法。\n\n### 名字由来\n\n一部分原因是受到pytorch的影响，希望能写个和pytorch一样甚至更易用的AI框架；另一部分是希望本框架只触及（touch）一些关键的东西：\n\n- only torch Rust --- 只用Rust（不用C++是因为其在复杂逻辑项目中容易写出内存不安全代码）；也不打算支持Python接口）；亦不用第三方lib（所以排除[tch-rs](https://github.com/LaurentMazare/tch-rs)），这样对跨平台支持会比较友好。\n- only torch CPU --- 不用GPU，因要照顾多平台也不想被某个GPU厂商制约，且基于NEAT进化的网络结构也不太好被GPU优化（也省得考虑数据从CPU的堆栈迁移到其他设备内存的开销问题了）。\n- only torch node --- 没有全连接、卷积、resnet这类先入为主的算子概念，具体模型结构均基于NEAT进化。\n- only torch tensor --- 所有的数据类型都是内置类型tensor（实现可能会参考[peroxide](https://crates.io/crates/peroxide)），不需要第三方处理库，如[numpy](https://github.com/PyO3/Rust-numpy)，[array](https://doc.Rust-lang.org/std/primitive.array.html)或[openBLAS](https://github.com/xianyi/OpenBLAS/wiki/User-Manual)（[关于blas的一些说明](https://blog.csdn.net/u013677156/article/details/77865405)）。\n- only torch f32 --- 网络的参数（包括模型的输入、输出）不需要除了f32外的数据类型。\n\n## 文档\n\n目前无人性化的文档。可直接看Rust自动生成的[Api Doc](https://docs.rs/only_torch)即可。\n\n### 使用示例\n\n- **[Adaline自适应线性神经元](tests/test_ada_line.rs)** - 经典二分类算法实现，本例使用了最原始的写来构建计算图、自动微分和参数更新，测试显示1000样本10轮训练可达95%+准确率（运行：`cargo test test_adaline -- --show-output`）\n\n## TODO\n\n- still need teset `test_duplicate_computation_avoidance`?\n- (back/forward)pass_id相关的graph测试？\n- （最后用AI优化下backward的逻辑）\n- 1次， 一次？用哪个？\n- `assert_eq!( graph.backward_nodes(\u0026[input], input), Err(GraphError::InvalidOperation(format!( \"输入节点[id=1, name=input, type=Input]不应该有雅可比矩阵\" ))) ); `添加一个 `assert_err`的宏，可才参考 `assert_panic`宏\n- graph反向传播中有些节点没有值需要过滤怎么添加（如多个output的网络结构）？\n- 针对 `loss1.backward(retain_graph=True)`和 `detach()`还有多output输出，还有rnn节点的反向传播，还有多次backward的问题；\n- 对于Input节点的 `set_jacobi`和 `jacobi`是否可用更好的panic或error取代，毕竟Input节点不该有梯度相关的概念；\n- 是否需要添加一个sign节点来取代step直接forward输出[-1,1]？\n- 直接实现CrossEntropyLoss（训练）和SoftMax（推理），不用LogisticLoss（Sigmoid）了；\n- unit test for Graph, and parent/children\n- Graph测试中该包含各种pub method的正确及错误测试\n- Graph测试中最好添加某个节点后，测试该节点还有其父节点的parents/children属性（又比如：同2个节点用于不同图的add节点，测试其parents/children属性是否正确）(Variable 节点无父节点)、“节点var1在图default_graph中重复”\n- add a `graph` for unit test to test the 多层的jacobi计算，就像ada_line那样?\n- 各种assign类的op（如：add_assign）是否需要重载而不是复用基本算子？\n- 在python中仿造ada_Line构造一个复合多节点，然后基于此在rust中测试这种复合节点，已验证在复合多层节点中的反向传播正确性\n- jacobi到底该测试对parent还是children？\n- how to expose only `in crate::nn` to the nn::Graph`?\n- should completely hide the NodeHandle?\n- Graph/NodeHandle rearrange blocks due to visibility and funciontality\n- NodeHandle重命名为Node? 各种 `parent/children/node_id`重命名为 `parents/children/id`?\n- should directly use `parents` but not `parents_ids`?\n- check other unused methods\n- draw_graph(graphvis画图)\n- save/load网络模型（已有test_save_load_tensor）\n- 也许后期可给Graph添加一个 `forward_batch`方法，用于批量forward(参考adaline_batch.py)？\n- 后期当引入NEAT机制后，可以给已存在节点添加父子节点后，需要把现有节点检测再完善下；\n- 当后期（NEAT阶段）需要在一个已经forwarded的图中添加节点（如将已经被使用过的var1、var2结合一个新的未使用的var3构建一个add节点），可能需要添加一个 `reset_forward_cnt`方法来保证图forward的一致性。\n- NEAT之后，针对图backward的 `loss1.backward(retain_graph=True)`和 `detach()`机制的实现（可在GAN和强化学习算法实例中针对性实现测试），可能须和 `forward_cnt`机制结合, 还要考虑一次forward后多次backward()后的结果。\n- Tensor 真的需要uninit吗？\n- 各种命名规范“2维”，“二维”，“二阶”，“2阶”，“一个”，“两个”，“三个”，“需要”，“需”，“须要”，“须”，“值/value”,\"变量/variable\",\"node/handle\"，“注/注意：”，”dim/dimension/rank“,\"维/阶\",\",\"改为\", \",\"仍然\"改为”仍“\n- 添加一个表格，说明下本crate中，阶、维、标量、向量、矩阵、张量的概念异同；\n- 根据matrixSlow+我笔记重写全部实现！保证可以后期以NEAT进化,能ok拓展至linear等常用层，还有detach，，容易添加edge(如已存在的add节点的父节点)，。\n- 等ada_line例子跑通后：`Variable`节点做常见的运算重载（如此便不需要用那些丑陋的节点算子了）\n- 图错误“InvalidOperation” vs “ComputationError”\n- Tensor类的index将 `[[`优化成 `[`?\n- Tensor类的 `slice(\u0026[0..m, j..j+1])`是否需要？\n- use approx::assert_abs_diff_eq; need or not?\n- [get_node_grad函数优化分析](.doc/get_node_grad_optimization_analysis.md)\n\n**目前需要先解决有没有的问题，而不是好不好**\n\n- [] 实现类似tch-rs中 `tch::no_grad(|| {});`的无梯度功能；\n- [] 常用激活函数，tanh，Softplus，[sech](https://discuss.pytorch.org/t/implementing-sech/66862)\n- [] 基于本框架解决XOR监督学习问题\n- [] 基于本框架解决Mnist（数字识别）的监督学习问题\n- [] 基于本框架解决CartPole（需要openAI Gym或相关crate支持）的深度强化学习问题\n- [] 尝试实现下[CFC](https://github.com/raminmh/CfC)\n\n## 笔记\n\n- [计算图执行机制重构方案](.doc/graph_execution_refactor.md) - 阐述了从基于 `pass_id`的指令式执行到基于“版本号”和“拓扑排序”的反应式更新模型的重构计划。\n- [MatrixSlow项目识别文档](.doc/python_MatrixSlow_pid.md) - 基于MatrixSlow的Python深度学习框架分析，包含计算图、自动求导、静态图执行等核心概念的详细说明\n- [本项目的梯度设计机制说明](.doc/gradient_clear_and_accumulation_design.md) - 详细说明了梯度/雅可比矩阵相关的设计决策，包括手动清除梯度的原理、累计机制等的使用模式和最佳实践\n\n## 参考资料\n\n### 训练用数据集（包括强化学习gym）\n\n- [Mnist](http://yann.lecun.com/exdb/mnist/)\n- [FashionMnist](https://www.kaggle.com/datasets/zalando-research/fashionmnist?resource=download)\n- [ChineseMnist](https://www.kaggle.com/datasets/gpreda/chinese-mnist)\n- [训练用的各种数据集（包括强化学习）](https://huggingface.co/FUXI)\n- [bevy_rl](https://crates.io/crates/bevy_rl)\n- [pure_rust_gym](https://github.com/MathisWellmann/gym-rs/tree/master)\n- [老式游戏rom](https://www.myabandonware.com/)\n\n### 数学/IT原理\n\n- [早期pytorch关于Tensor、Variable等的探讨](https://pytorch.org/blog/pytorch-0_4_0-migration-guide/#merging-tensor-and-variable-and-classes)\n- [矩阵和向量的各种乘法](https://www.jianshu.com/p/9165e3264ced)\n- [神经网络与记忆](https://www.bilibili.com/video/BV1fV4y1i7hZ/?spm_id_from=333.1007.0.0\u0026vd_source=3facc3cb195be0a27a0ea9a4eb3bb6fe)\n- [陈天奇的机器学习编译课](https://www.bilibili.com/video/BV15v4y1g7EU/?is_story_h5=false\u0026p=1\u0026share_from=ugc\u0026share_medium=android\u0026share_plat=android\u0026share_session_id=5a312434-ccf7-4cb9-862a-17a601cc4d35\u0026share_source=COPY\u0026share_tag=s_i\u0026timestamp=1661386914\u0026unique_k=zCWMKGC\u0026vd_source=3facc3cb195be0a27a0ea9a4eb3bb6fe)\n- [基于梯度的机器学习IT原理](https://zhuanlan.zhihu.com/p/518198564)\n\n### 开源示例\n\n- [KAN 2.0](https://blog.csdn.net/qq_44681809/article/details/141355718)\n- [radiate--衍生NEAT的纯Rust库](https://github.com/pkalivas/radiate)\n- [neat-rs](https://github.com/dbsxdbsx/neat-rs)\n- [纯Rust的NEAT+GRU](https://github.com/sakex/neat-gru-Rust)\n- [Rusty_sr-纯Rust的基于dl的图像超清](https://github.com/millardjn/Rusty_sr)\n- [ndarray_glm(可参考下 `array!`，分布，以及原生的BLAS)](https://docs.rs/ndarray-glm/latest/ndarray_glm/)\n- [PyToy--基于MatrixSlow的Python机器学习框架](https://github.com/ysj1173886760/PyToy)\n- [MatrixSlow--纯python写的神经网络库](https://github.com/zc911/MatrixSlow)\n- [python：遗传算法（GE）玩FlappyBird](https://github.com/ShuhuaGao/gpFlappyBird)\n- [python包：遗传规划gplearn](https://gplearn.readthedocs.io/en/stable/examples.html)\n- [python包：遗传规划deap](https://deap.readthedocs.io/en/master/examples/gp_symbreg.html)\n- [python包：特征自动提取](https://github.com/IIIS-Li-Group/OpenFE)\n- [NTK网络](https://zhuanlan.zhihu.com/p/682231092)\n\n（较为成熟的3方库）\n\n- [Burn—纯rust深度学习库](https://github.com/Tracel-AI/burn)\n- [Candle:纯rust较成熟的机器学习库](https://github.com/huggingface/candle)\n- [用纯numpy写各类机器学习算法](https://github.com/ddbourgin/numpy-ml)\n  （自动微分参考）\n- [手工微分：Rust-CNN](https://github.com/goldstraw/RustCNN)\n- [neuronika--纯Rust深度学习库（更新停滞了，参考下自动微分部分）](https://github.com/neuronika/neuronika)\n- [基于TinyGrad的python深度学习库的RL示例](https://github.com/DHDev0/TinyRL/tree/main)\n- [重点：Rust- ---支持cuda的Rust深度学习库(参考下自动微分部分)](https://docs.rs/dfdx/latest/dfdx/)\n- [重点：基于ndarray的反向autoDiff库](https://github.com/raskr/rust-autograd)\n- [前向autoDiff(貌似不成熟)](https://github.com/elrnv/autodiff)\n- []\n- [深度学习框架InsNet简介](https://zhuanlan.zhihu.com/p/378684569)\n- [C++机器学习库MLPACK](https://www.mlpack.org/)\n- [经典机器学习算法Rust库](https://github.com/Rust-ml/linfa)\n- [peroxide--纯Rust的线代及周边库](https://crates.io/crates/peroxide)\n- [C++实现的NEAT+LSTM/GRU/CNN](https://github.com/travisdesell/exact)\n- [pytorch+NEAT](https://github.com/ddehueck/pytorch-neat)\n- [avalog--基于avatar的Rust逻辑推理库](https://crates.io/crates/avalog)\n\n### NEAT、神经架构进化\n\n- [用梯度指导神经架构进化：Splitting Steepest Descent](https://www.cs.utexas.edu/~qlearning/project.html?p=splitting)\n- [Deep Mad，将卷积网络设计为一个数学建模问题](https://www.bilibili.com/video/BV1HP411R74T/?spm_id_from=333.999.0.0\u0026vd_source=3facc3cb195be0a27a0ea9a4eb3bb6fe)\n- [动态蛇形卷积DSCNet](https://www.bilibili.com/video/BV1J84y1d7yG/?spm_id_from=333.1007.0.0\u0026vd_source=3facc3cb195be0a27a0ea9a4eb3bb6fe)\n- [autoML介绍](https://www.zhihu.com/question/554255720/answer/2750670583)\n\n### 符号派：逻辑/因果推断\n\n- [scryer-prolog--Rust逻辑推理库](https://github.com/mthom/scryer-prolog)\n- [vampire:自动证明器](https://github.com/vprover/vampire?tab=readme-ov-file)\n- [那迷人的被遗忘的语言：Prolog](https://zhuanlan.zhihu.com/p/41908829)\n- [结合prolog和RL](https://arxiv.org/abs/2004.06997)\n- [prolog与4证人难题](https://prolog.longluntan.com/t9-topic)\n- [logic+mL提问](https://ai.stackexchange.com/questions/16224/has-machine-learning-been-combined-with-logical-reasoning-for-example-prolog)\n- [prolog解决数独问题](https://prolog.longluntan.com/t107-topic)\n- [贝叶斯与逻辑推理](https://stats.stackexchange.com/questions/243746/what-is-probabilistic-inference)\n- [用一阶逻辑辅佐人工神经网络](https://www.cs.cmu.edu/~hovy/papers/16ACL-NNs-and-logic.pdf)\n- [二阶逻辑杂谈](https://blog.csdn.net/VucNdnrzk8iwX/article/details/128928166)\n- [关于二阶逻辑的概念问题](https://www.zhihu.com/question/321025032/answer/702580771?utm_id=0)\n- [PWL:基于贝叶斯的自然语言处理](https://github.com/asaparov/PWL)\n- [Symbolic Learning Enables Self-Evolving Agents](https://arxiv.org/abs/2406.18532)\n- ASTRID系统（Mind|Construct, 2017）\n- 归纳逻辑编程（Inductive Logic Programming, ILP）\n- 书：《The Book of Why》\n- 书：《Causality:Models,Reasoning,and Inference》\n- [知乎：因果推断杂谈](https://www.zhihu.com/question/266812683/answer/895210894)\n- [信息不完备下基于贝叶斯推断的可靠度优化方法](https://www.docin.com/p-2308549828.html)\n- [贝叶斯网络中的因果推断](https://www.docin.com/p-1073204271.html?docfrom=rrela)\n\n### 神经网络的可解释性\n\n- [可解释性核心——神经网络的知识表达瓶颈](https://zhuanlan.zhihu.com/p/422420088/)\n- [神经网络可解释性：论统一14种输入重要性归因算法](https://zhuanlan.zhihu.com/p/610774894/)\n- [神经网络的可解释性](https://zhuanlan.zhihu.com/p/341153242)\n- [可解释的哈萨尼网络](https://zhuanlan.zhihu.com/p/643213054)\n\n### 超参数优化\n\n- [mle-hyperopt](https://github.com/mle-infrastructure/mle-hyperopt)\n\n### CPU加速\n\n- [SLIDE](https://arxiv.org/abs/2103.10891)\n- [Rust+AVX](https://medium.com/@Razican/learning-simd-with-Rust-by-finding-planets-b85ccfb724c3)\n- [矩阵加速-GEMM](https://www.jianshu.com/p/6d3f013d8aba)\n\n### 强化学习\n\n- [Sac用以复合Action](https://arxiv.org/pdf/1912.11077v1.pdf)\n- [EfficientZero](https://arxiv.org/abs/2111.00210)\n- [EfficientZero Remastered](https://www.gigglebit.net/blog/efficientzero)\n- [EfficientZero V2: Mastering Discrete and Continuous Control with Limited Data](https://arxiv.org/abs/2403.00564v2)\n- [SpeedyZero](https://openreview.net/forum?id=Mg5CLXZgvLJ)\n- [LightZero系列](https://github.com/opendilab/LightZero?tab=readme-ov-file)\n- [随机MuZero代码](https://github.com/DHDev0/Stochastic-muzero)\n- [Redeeming Intrinsic Rewards via Constrained Optimization](https://williamd4112.github.io/pubs/neurips22_eipo.pdf)\n- [Learning Reward Machines for Partially Observable Reinforcement Learning](https://arxiv.org/abs/2112.09477)\n- [combo代码](https://github.com/Shylock-H/COMBO_Offline_RL)\n- [2023最新model-based offline算法：MOREC](https://arxiv.org/abs/2310.05422)\n- [众多model-base/free的offline算法](https://github.com/yihaosun1124/OfflineRL-Kit)\n- [model-free offline算法：MCQ解析](https://zhuanlan.zhihu.com/p/588444380)\n- [RL论文列表（curiosity、offline、uncertainty，safe）](https://github.com/yingchengyang/Reinforcement-Learning-Papers)\n- [代替Gym的综合库](https://gymnasium.farama.org/)\n\n### rust+大语言模型（LLM）\n\n- [BionicGpt](https://github.com/bionic-gpt/bionic-gpt)\n- [适用对话的Rust终端UI？](https://dustinblackman.com/posts/oatmeal/)\n- [chatGpt相关论文](https://arxiv.org/abs/2203.02155)\n\n### （自动、交互式）定理证明\n\n- [关于lean的一篇文章](https://zhuanlan.zhihu.com/p/183902909#%E6%A6%82%E8%A7%88)\n- [Lean+LLM](https://github.com/lean-dojo/LeanDojo)\n- [陶哲轩使用Lean4](https://mp.weixin.qq.com/s/TYB6LgbhjvHYvkbWrEoDOg)\n\n```\nFormal Verification\n├── Theorem Proving（定理证明）\n│   ├── Interactive Theorem Proving（交互式）\n│   │   ├── Coq\n│   │   ├── Lean\n│   │   └── Isabelle/HOL\n│   └── Automated Theorem Proving（自动式）\n└── Model Checking（模型检测）\n```\n\n### 博弈论（game）\n\n- [Sprague-Grundy介绍1](https://zhuanlan.zhihu.com/p/157731188)\n- [Sprague-Grundy介绍2](https://zhuanlan.zhihu.com/p/20611132)\n- [Sprague-Grundy介绍3](https://zhuanlan.zhihu.com/p/357893255)\n\n### 其他\n\n- [动手学深度学习-李沐著](https://zh-v2.d2l.ai/chapter_preliminaries/linear-algebra.html#subsec-lin-algebra-norms)\n- [openMMLab-Yolo](https://github.com/open-mmlab/mmyolo)\n- [GRU解释](https://www.pluralsight.com/guides/lstm-versus-gru-units-in-rnn)\n- [基于人类语音指挥的AI](https://arxiv.org/abs/1703.09831)\n- [webGPT会上网的gpt](https://arxiv.org/abs/2112.09332)\n- [LeCun的自监督世界模型](https://zhuanlan.zhihu.com/p/636997984)\n- [awesome Rust](https://github.com/Rust-unofficial/awesome-Rust#genetic-algorithms)\n- [去雾算法](https://blog.csdn.net/IT_job/article/details/78864236)\n- [rust人工智能相关的项目](https://github.com/rust-unofficial/awesome-rust#artificial-intelligence)\n- [《千脑智能》及相关github项目](https://www.numenta.com/thousand-brains-project/)\n\n## 遵循协议\n\n本项目遵循MIT协议（简言之：不约束，不负责）。\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdbsxdbsx%2Fonly_torch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdbsxdbsx%2Fonly_torch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdbsxdbsx%2Fonly_torch/lists"}