{"id":20947385,"url":"https://github.com/computemachines/nnfs-rust","last_synced_at":"2026-04-21T22:03:45.946Z","repository":{"id":155571661,"uuid":"630570758","full_name":"computemachines/nnfs-rust","owner":"computemachines","description":"Following along with Neural Networks from Scratch by Kinsley \u0026 Kekiela","archived":false,"fork":false,"pushed_at":"2023-06-30T00:54:59.000Z","size":100613,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-01-02T01:42:18.859Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/computemachines.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-20T17:01:24.000Z","updated_at":"2023-04-21T02:19:16.000Z","dependencies_parsed_at":null,"dependency_job_id":"ef4bf562-0001-498f-8625-22a05f7e2859","html_url":"https://github.com/computemachines/nnfs-rust","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/computemachines/nnfs-rust","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/computemachines%2Fnnfs-rust","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/computemachines%2Fnnfs-rust/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/computemachines%2Fnnfs-rust/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/computemachines%2Fnnfs-rust/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/computemachines","download_url":"https://codeload.github.com/computemachines/nnfs-rust/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/computemachines%2Fnnfs-rust/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32112030,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-21T11:25:29.218Z","status":"ssl_error","status_checked_at":"2026-04-21T11:25:28.499Z","response_time":128,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-19T00:11:03.940Z","updated_at":"2026-04-21T22:03:45.918Z","avatar_url":"https://github.com/computemachines.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Chapter 19\n## Fashion MNIST data\nYou will need to extract the zip file manually after downloading it. The zip crate can't handle the compression algorithm used.\nI took some pictures of my own clothes and some random objects I not included in the training set. \n\nUsing GIMP, I cropped the images to 28x28 and converted them to grayscale, inverted the colors, and did a auto white balance. The backgrounds are not perfectly black like the training images.\n\nRaw image:\n![T-shirt](/my_images/2.jpg)\n\nPreprocessed image:\n![T-shirt](/my_images/2.png)\n\n\n```shell\n$ cargo run -- ch19 train\n...\nepoch: 10, acc: 0.915, loss: 10.0595011732 (data_loss: 10.060, reg_loss: 0.000)\n\n$ cargo run -- ch19 inference --filename my_images/2.png \n    Finished dev [unoptimized + debuginfo] target(s) in 0.08s\n     Running `target/debug/nnfs-rust ch19 inference --filename my_images/2.png`\nLoading model...done\nLoading image...\ndone\nRunning inference...done\n\n\n---- Report ----\nBag: 27.11%\nShirt: 21.69%\nCoat: 19.01%\nPullover: 15.71%\nT-shirt/top: 10.75%\nTrouser: 3.24%\nDress: 2.43%\nAnkle boot: 0.04%\nSandal: 0.02%\nSneaker: 0.00%\n```\nThe results could definity be inproved by manipulating the training images to make it look superficially more like my images. I get over 90% accuracy on the testing set.\n# Chapter 18\n## The model api\nExample usage:\n```rust\nlet binary_inputs = array![[0., 0.], [0., 1.], [1., 0.], [1., 1.]];\nlet and_outputs =   array![[0.],     [0.],     [0.],     [1.]];\nlet xor_outputs =   array![[0.],     [1.],     [1.],     [0.]];\n\n// Instantiate the (uninitialized) model\nlet mut model = Model::new();\n\n// Add perceptron layers\nmodel.add(LayerDense::new(2, 1));\nmodel.add_final_activation(Sigmoid::new());\n    \n// Set the loss, optimizer, and accuracy type\nmodel.set(\n    BinaryCrossentropy::new(),\n    OptimizerSDG::from(OptimizerSDGConfig {\n        learning_rate: 1.0,\n        ..Default::default()\n    }),\n    AccuracyBinary,\n);\n\n// Finalize the model. This changes the type of the model to permit training.\nlet mut model = model.finalize();\n\n// Train the model\nmodel.train(\n    \u0026binary_inputs,\n    \u0026and_outputs,\n    ModelTrainConfig {\n        epochs: 30,\n        print_every: 1,\n    },\n);\n\n\n// show output from trained model\nmodel.forward(\u0026binary_inputs);\nlet inference = model.output();\nprintln!(\"Inference: \\n{}\", inference);\n```\nRust really shines here, the compiler will catch incompatible uses of Loss, input dimension, and Accuracy types. Python however is much much easier to do exploratory data analysis and visualization with matplotlib. This was the only chapter where I thought that rust was as pleasant to do ML in as python.\n```rust\n## Example output\n```shell\n$ cargo run -- ch18\n    Finished dev [unoptimized + debuginfo] target(s) in 2.00s\n     Running `target/debug/nnfs-rust ch18`\nepoch: 5, acc: 0.750, loss: 0.357 (data_loss: 0.357, reg_loss: 0.000)\nepoch: 10, acc: 1.000, loss: 0.233 (data_loss: 0.233, reg_loss: 0.000)\nepoch: 15, acc: 1.000, loss: 0.177 (data_loss: 0.177, reg_loss: 0.000)\nepoch: 20, acc: 1.000, loss: 0.150 (data_loss: 0.150, reg_loss: 0.000)\nepoch: 25, acc: 1.000, loss: 0.138 (data_loss: 0.138, reg_loss: 0.000)\nepoch: 30, acc: 1.000, loss: 0.132 (data_loss: 0.132, reg_loss: 0.000)\nInference: \n[[0.000015782928969143955],\n [0.0047410583767099794],\n [0.004878318834536076],\n [0.5967060593546276]]\n```\n## Unfinished work\nI did not implement the combined Softmax + Categorical Cross Entropy backward pass with the model but left it open to be done using generics. All I would need to do is write a specialized implementation:\n```rust\nimpl Model\u003cArray1\u003cf64\u003e, Softmax, CategoricalCrossEntropy\u003e {\n    ...\n    fn backward(\u0026mut self, y_pred: \u0026Array1\u003cf64\u003e, y_true: \u0026Array1\u003cf64\u003e) {\n        ...\n    }\n}\n```\nCode that used both Softmax and CategoricalCrossEntropy would then get a free speedup without needing to be updated. \n\n# Chapter 17\nRegression\n- Requires at least 2 nonlinear activation layers\n- Very sensitive to initial weights\n- Different activation functions call for different weight initialization strategies\n- I struggled to get good training results. \n```shell\n$ cargo run -- ch17 adam -l 0.05 -d 5e-3\n```\n![Prediction vs Training Sine Data](/plots/ch17-sine-prediction-vs-training.png)\n\n# Chapter 16\nBinary Logistic Regression\n```shell\n$ cargo run -- ch16 loss-only adam -l 0.01 -d 5e-7\n...\nEpoch: 9800,\nData Loss: 0.920, Regularization Loss: 1.082, Accuracy: 0.870,\nTest Data Loss: 1.319, Test Accuracy: 0.800\n\nEpoch: 9900,\nData Loss: 0.937, Regularization Loss: 1.132, Accuracy: 0.870,\nTest Data Loss: 1.358, Test Accuracy: 0.795\n```\n\n# Chapter 15\nDropout\n![Training Forward Run Visualization](/plots/ch15-adam-2x64-l0.05-d0.0000005-e0.0000001-b1_0.9-b2_0.999-animation.gif)\n```shell\n$ cargo run -- ch15 -n 10000 --l2reg 1e-5 --dropout 0.1 loss-only adam\n```\n# Chapter 14\nRegularization\n```shell\n$ cargo run -- ch14 -n 10000 --l2reg=5e-4 loss-only adam -l 0.05 -d 5e-7 --beta-1 0.9 --beta-2 0.999\n    Finished dev [unoptimized + debuginfo] target(s) in 0.05s\n     Running `target/debug/nnfs-rust ch14 -n 850 --l2reg 5e-4 animate adam -l 0.05 -d 5e-7 --beta-1 0.9 --beta-2 0.999`\n\nEpoch: 0,\nData Loss: 1.609, Regularization Loss: 0.000, Accuracy: 0.226,\nTest Data Loss: 1.609, Test Regularization Loss: 0.000, Test Accuracy: 0.234\n\nEpoch: 1,\nData Loss: 1.601, Regularization Loss: 0.001, Accuracy: 0.276,\nTest Data Loss: 1.602, Test Regularization Loss: 0.001, Test Accuracy: 0.268\n...\n\nEpoch: 9998,\nData Loss: 2.499, Regularization Loss: 7.817, Accuracy: 0.538,\nTest Data Loss: 2.510, Test Regularization Loss: 7.817, Test Accuracy: 0.534\n\nEpoch: 9999,\nData Loss: 2.500, Regularization Loss: 7.820, Accuracy: 0.540,\nTest Data Loss: 2.510, Test Regularization Loss: 7.820, Test Accuracy: 0.534\n```\n\n\n# Chapter 13\ndatasets should be about ~ +-1 or preprocessed\nsmall datasets can be augmented with geometric manipulations of the original data\n\n# Chapter 12\ncross-validation\n\n# Chapter 11\nOverfitting\n\n# Chapter 10\n\n`cargo run -- ch10 animate adam -l 0.05 -d 5e-7 --beta-1 0.9 --beta-2 0.999`\n![Training Forward Run Visualization](/plots/ch10-adam-l0.05-d0.0000005-e0.0000001-b1_0.9-b2_0.999-animation.gif)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcomputemachines%2Fnnfs-rust","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcomputemachines%2Fnnfs-rust","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcomputemachines%2Fnnfs-rust/lists"}