{"id":16092506,"url":"https://github.com/gabrielmfern/intricate","last_synced_at":"2025-03-17T17:31:05.367Z","repository":{"id":48324984,"uuid":"516544176","full_name":"gabrielmfern/intricate","owner":"gabrielmfern","description":"A gpu accelerated neural network Rust crate.","archived":false,"fork":false,"pushed_at":"2023-04-17T10:13:50.000Z","size":12745,"stargazers_count":13,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-12T22:16:30.837Z","etag":null,"topics":["deep-learning","gpu-accelerated-library","gpu-acceleration","machine-learning","opencl","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gabrielmfern.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-07-21T22:59:07.000Z","updated_at":"2024-08-28T18:50:00.000Z","dependencies_parsed_at":"2024-10-27T15:47:56.916Z","dependency_job_id":"3cb071f9-2520-4347-a247-3af08b9a9d64","html_url":"https://github.com/gabrielmfern/intricate","commit_stats":{"total_commits":279,"total_committers":2,"mean_commits":139.5,"dds":"0.11469534050179209","last_synced_commit":"be14500943e033bd378168bf3ccfd354807b7ce2"},"previous_names":[],"tags_count":20,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gabrielmfern%2Fintricate","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gabrielmfern%2Fintricate/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gabrielmfern%2Fintricate/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gabrielmfern%2Fintricate/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gabrielmfern","download_url":"https://codeload.github.com/gabrielmfern/intricate/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243872376,"owners_count":20361464,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","gpu-accelerated-library","gpu-acceleration","machine-learning","opencl","rust"],"created_at":"2024-10-09T16:08:51.442Z","updated_at":"2025-03-17T17:31:04.930Z","avatar_url":"https://github.com/gabrielmfern.png","language":"Rust","funding_links":[],"categories":["Machine Learning"],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\r\n  \u003cimg \r\n    src=\"https://github.com/gabrielmfern/intricate/blob/main/icon.png\"\r\n    height=\"200px\"\r\n    stlye=\"margin-top: 20px; margin-bottom: 20px\"\r\n  \u003e\r\n\u003c/div\u003e\r\n\r\n[![Crates.io](https://img.shields.io/crates/v/intricate.svg?label=intricate)](https://crates.io/crates/intricate)\r\n[![Crates.io](https://img.shields.io/crates/dv/intricate)](https://crates.io/crates/intricate)\r\n![github.com](https://img.shields.io/github/license/gabrielmfern/intricate)\r\n![github.com](https://img.shields.io/github/commit-activity/m/gabrielmfern/intricate)\r\n\r\nA GPU accelerated library that creates/trains/runs neural networks in safe Rust code.\r\n\r\n---\r\n\r\n### Table of contents\r\n\r\n* [Architecture overview](#architecture-overview)\r\n    * [Models](#models)\r\n    * [Layers](#layers)\r\n    * [Optimizers](#optimizers)\r\n    * [Loss Functions](#loss-functions)\r\n* [XoR using Intricate](#xor-using-intricate)\r\n    * [Setting up the training data](#setting-up-the-training-data)\r\n    * [Setting up the layers](#setting-up-the-layers)\r\n    * [Setting up OpenCL](#setting-up-opencls-state)\r\n    * [Fitting our Model](#fitting-our-model)\r\n* [How to save and load models](#how-to-save-and-load-models)\r\n    * [Saving the Model](#saving-the-model)\r\n    * [Loading the Model](#loading-the-model)\r\n* [Things to be done still](#things-to-be-done-still)\r\n\r\n---\r\n\r\n## Architecture overview\r\n\r\nIntricate has a layout very similar to popular libraries out there such as Keras.\r\n\r\nIt consists at the surface of a [Model](#models), which consists then \r\nof [Layers](#layers) which can be adjusted using a [Loss Function](#loss-functions)\r\nthat is also helped by an [Optimizer](#optimizers).\r\n\r\n### Models\r\n\r\nAs said before, similar to Keras, Intricate defines Models as basically\r\na list of [Layers](#layers).\r\n\r\nA model does not have much logic in it, mostly it delegates most of the work to the layers,\r\nall that it does is orchestrate how the layers should work together and how the data goes from\r\na layer to another.\r\n\r\n### Layers\r\n\r\nEvery layer receives **inputs** and returns **outputs** following some rule that they must define. \r\n\r\nThey must also implement four methods that together constitute back-propagation:\r\n\r\n- `optimize_parameters`\r\n- `compute_gradients`\r\n- `apply_gradients`\r\n- `compute_loss_to_input_derivatives`\r\n\r\nMostly the optimize_parameters will rely on an [Optimizer](#optimizers) that will try to improve\r\nthe parameters that the Layer allows it to optimize.\r\n\r\nThese methods together will be called sequentially to do back-propagation in the Model and\r\nusing the results from the `compute_loss_to_input_derivatives` we will then do the same for\r\nthe last layer and so on.\r\n\r\nThese layers can be really any type of transformation on the inputs and outputs.\r\nAn example of this is the activation functions in Intricate which are actual \r\nlayers instead of being one with other layers\r\nwhich does simplify calculations tremendously and works like a charm.\r\n\r\n### Optimizers\r\n\r\nOptimizers the do just what you might think, they optimize.\r\n\r\nSpecifically they optimize both the parameters a Layer allows them to optimize, and \r\nthe [Layer](#layers)'s gradients so that the Layer can use them to apply the optimized gradients on itself.\r\n\r\nThis is useful because one using Intricate can develop, and perhaps debug, an Optimizer to see how well it does\r\nfor certain use cases which is very good for where I want Intricate to go. All you have to do is create some struct\r\nthat implements the `Optimizer` trait.\r\n\r\n### Loss Functions\r\n\r\nLoss Functions are just basically some implementations of a certain trait that are used\r\nto determine how bad a Model is. \r\n\r\nLoss Functions are **NOT** used in a layer, they are used\r\nfor the Model itself. Even though a Layer will use derivatives with respect \r\nto the loss they don't really communicate with the Loss Function directly.\r\n\r\n---\r\n\r\n## XoR using Intricate\r\n\r\nIf you look at the `examples/` in the repository \r\nyou will find XoR implemented using Intricate. \r\nThe following is basically just that example with some separate explanation.\r\n\r\n### Setting up the training data\r\n\r\n```rust\r\nlet training_inputs = vec![\r\n    vec![0.0, 0.0],\r\n    vec![0.0, 1.0],\r\n    vec![1.0, 0.0],\r\n    vec![1.0, 1.0],\r\n];\r\n\r\nlet expected_outputs = vec![\r\n    vec![0.0],\r\n    vec![1.0],\r\n    vec![1.0],\r\n    vec![0.0],\r\n];\r\n```\r\n\r\n### Setting up the layers\r\n\r\n```rust\r\nuse intricate::layers::{\r\n    activations::TanH,\r\n    Dense\r\n};\r\nlet mut layers: Vec\u003cModelLayer\u003e = vec![\r\n    Dense::new(2, 3), // inputs amount, outputs amount\r\n    TanH::new (3),\r\n    Dense::new(3, 1),\r\n    TanH::new (1),\r\n];\r\n```\r\n\r\n### Creating the model with the layers\r\n\r\n```rust\r\nuse intricate::Model;\r\n// Instantiate our model using the layers\r\nlet mut xor_model = Model::new(layers);\r\n```\r\n\r\nWe make the model `mut` because we will call `fit` for training our model\r\nwhich will tune each of the layers when necessary.\r\n\r\n### Setting up OpenCL's state\r\n\r\nSince Intricate does use OpenCL under the hood for doing calculations,\r\nwe do need to initialize a `OpenCLState` which is just a struct\r\ncontaining some necessary OpenCL stuff:\r\n\r\n```rust\r\nuse intricate::utils::{\r\n    setup_opencl,\r\n    DeviceType\r\n}\r\n//              you can change this device type to GPU if you want\r\nlet opencl_state = setup_opencl(DeviceType::CPU).unwrap();\r\n```\r\n\r\nFor our Model to be able to actually do computations, we need to pass the OpenCL state \r\ninto the `init` method inside the Model as follows:\r\n\r\n```rust\r\nxor_model.init(\u0026opencl_state).unwrap();\r\n```\r\n\r\n### Fitting our model\r\n\r\nFor training our Model we just need to call the `fit`\r\nmethod and pass in some parameters as follows:\r\n\r\n```rust\r\nuse intricate::{\r\n    loss_functions::MeanSquared,\r\n    optimizers,\r\n    types::{TrainingOptions, TrainingVerbosity},\r\n};\r\n\r\nlet mut loss = MeanSquared::new(); // the type of loss function that should be used for Intricate\r\n                                   // to determine how bad the Model is\r\nlet mut optimizer = optimizers::Basic::new(0.1); // the optimizer tries to use the gradients to optimize the training\r\n                                                 // process\r\n\r\n// Fit the model however many times we want\r\nxor_model\r\n    .fit(\r\n        \u0026training_inputs,\r\n        \u0026expected_outputs,\r\n        \u0026mut TrainingOptions::new(\u0026mut loss, \u0026mut optimizer)\r\n            .set_epochs(10000)\r\n            .set_batch_size(4) // the size of the mini-batch being used in Intricate's Mini-batch\r\n                               // Gradient Descent\r\n            .should_compute_accuracy(true).unwrap() // if Intricate should compute the accuracy after each\r\n                                                    // training step\r\n            .should_print_accuracy(true).unwrap() // should print the accuracy after each epoch\r\n            //                 a condition for stopping the training if a min accuracy is reached\r\n            .set_halting_condition(HaltingCondition::MinAccuracyReached(0.95)).unwrap()\r\n            .should_show_halting_condition_warning(true).unwrap(),\r\n    )\r\n    .unwrap();\r\n```\r\n\r\nAs you can see it is extremely easy creating these models, and blazingly fast as well.\r\n\r\n---\r\n\r\n## How to save and load models\r\n\r\nFor saving and loading models Intricate uses the [savefile](https://github.com/avl/savefile) \r\ncrate which makes it very simple and fast to save models.\r\n\r\n### Saving the model\r\n\r\nAs an example let's try saving and loading our XoR model.\r\n\r\nFor doing that we will first need to sync all the relevant layer information\r\nof the Model with OpenCL's `host`, (or just with the CPU), and then we will need\r\nto call the `save_file` method as follows:\r\n\r\n```rust\r\nxor_model.sync_data_from_buffers_to_host().unwrap(); // sends the weights and biases from \r\n                                                     // OpenCL buffers to Rust Vec's\r\nsave_file(\"xor-model.bin\", 0, \u0026xor_model).unwrap();\r\n```\r\n\r\n### Loading the model\r\n\r\nAs for loading our XoR model, we just need to call the \r\ncounterpart of the save_file method: `load_file`.\r\n\r\n```rust\r\nlet mut loaded_xor_model: Model = load_file(\"xor-model.bin\", 0).unwrap();\r\n```\r\n\r\nNow of curse, the savefile crate cannot load in the data to the GPU, so if you want\r\nto use the Model after loading it, you **must** call the `init` method in the `loaded_xor_model`\r\n(done in examples/xor.rs).\r\n\r\n## Things to be done still\r\n\r\n- implement Max Pooling or other Pooling type of layers;\r\n- add a way to send into the training process a callback closure that would be called every time an epoch finished or even a step too with some cool info;\r\n- make an example after doing the thing above ^, that uses that same function to plot the loss real time using a crate like `textplots`;\r\n- add embedding layers for text such as bag of words with an expected vocabulary size;\r\n- add a way to show inputs and outputs not matching error be more clear and perhaps even appear at compile time;\r\n- add a way to choose what type of accuracy should be calculated to avoid weird and unuseful accuracies being calculated\r\n- make Intricate GPU parallel (gonna take a long time to implement and can't do it rn since I don't have multiple GPUs available to me)\r\n- add a way to epxose Intricate's used crates such as SaveFile and OpenCL3 to be used outside by a user if needed without having to install them","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgabrielmfern%2Fintricate","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgabrielmfern%2Fintricate","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgabrielmfern%2Fintricate/lists"}