{"id":13483559,"url":"https://github.com/NeuraLegion/shainet","last_synced_at":"2025-03-27T14:31:25.483Z","repository":{"id":46108664,"uuid":"113332848","full_name":"NeuraLegion/shainet","owner":"NeuraLegion","description":"SHAInet - a pure Crystal machine learning library","archived":false,"fork":false,"pushed_at":"2023-01-11T12:13:47.000Z","size":27157,"stargazers_count":185,"open_issues_count":0,"forks_count":18,"subscribers_count":30,"default_branch":"master","last_synced_at":"2025-03-14T09:04:52.582Z","etag":null,"topics":["convolutional-neural-networks","crystal","deep-learning","deep-neural-networks","machine-learning","neural-network"],"latest_commit_sha":null,"homepage":"","language":"Crystal","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NeuraLegion.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-12-06T15:25:15.000Z","updated_at":"2025-02-15T23:31:17.000Z","dependencies_parsed_at":"2023-02-09T03:00:33.653Z","dependency_job_id":null,"html_url":"https://github.com/NeuraLegion/shainet","commit_stats":null,"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NeuraLegion%2Fshainet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NeuraLegion%2Fshainet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NeuraLegion%2Fshainet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NeuraLegion%2Fshainet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NeuraLegion","download_url":"https://codeload.github.com/NeuraLegion/shainet/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245863092,"owners_count":20684787,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["convolutional-neural-networks","crystal","deep-learning","deep-neural-networks","machine-learning","neural-network"],"created_at":"2024-07-31T17:01:12.747Z","updated_at":"2025-03-27T14:31:24.479Z","avatar_url":"https://github.com/NeuraLegion.png","language":"Crystal","funding_links":[],"categories":["Machine Learning"],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\u003cimg src=\"logo/logotype_vertical.png\" alt=\"shainet\" height=\"200px\"\u003e\u003c/p\u003e\n\n[![Crystal CI](https://github.com/NeuraLegion/shainet/actions/workflows/crystal.yml/badge.svg)](https://github.com/NeuraLegion/shainet/actions/workflows/crystal.yml)\n\n\nSHAInet - stands for Super Human Artificial Intelligence network\na neural network in pure [Crystal](https://crystal-lang.org/)\n\nThis is a free-time project, happily hosted by NeuraLegion that was created as part of some internal research. We started it with research in mind, rather than production, and just kept going, also thanks to members of the community.\n\nWe wanted to try and implement some inspiration from the biological world into this project. In addition to that, we wanted to try an approach for NNs using object-oriented modeling instead of matrices. The main reason behind that was, to try new types of neurons aiming for more robust learning (if possible) or at least have more fine-tuned control over the manipulation of each neuron (which is difficult using a matrix-driven approach).\n\nAt the [Roadmap](https://github.com/NeuraLegion/shainet#development) you can see what we plan to add to the network as the project will progress.  \n\n\n## Installation\n\nAdd this to your application's `shard.yml`:\n\n```yaml\ndependencies:\n  shainet:\n    github: NeuraLegion/shainet\n```\n\n## Usage\n\nMore usage examples can be found in the specs\n\n### Standard training on XOR example  \n```crystal\nrequire \"shainet\"\n\ntraining_data = [\n  [[0, 0], [0]],\n  [[1, 0], [1]],\n  [[0, 1], [1]],\n  [[1, 1], [0]],\n]\n# Initialize a new network\nxor = SHAInet::Network.new\n# Add a new layer of the input type with 2 neurons and classic neuron type (memory)\nxor.add_layer(:input, 2, :memory, SHAInet.sigmoid)\n# Add a new layer of the hidden type with 2 neurons and classic neuron type (memory)\nxor.add_layer(:hidden, 2, :memory, SHAInet.sigmoid)\n# Add a new layer of the output type with 1 neurons and classic neuron type (memory)\nxor.add_layer(:output, 1, :memory, SHAInet.sigmoid)\n# Fully connect the network layers\nxor.fully_connect\n\n# Adjust network parameters\nxor.learning_rate = 0.7\nxor.momentum = 0.3\n\n# data, training_type, cost_function, activation_function, epochs, error_threshold (sum of errors), learning_rate, momentum)\nxor.train(\n      data: training_data,\n      training_type: :sgdm,\n      cost_function: :mse,\n      epochs: 5000,\n      error_threshold: 0.000001,\n      log_each: 1000)\n\n# Run the trained network\nxor.run([0, 0])\n```\n\n\n### Batch training on the iris dataset using adam\n```crystal\n# Create a new Data object based on a CSV\ndata = SHAInet::Data.new_with_csv_input_target(\"iris.csv\", 0..3, 4)\n\n# Split the data in a training set and a test set\ntraining_set, test_set = data.split(0.67)\n\n# Initiate a new network\niris = SHAInet::Network.new\n\n# Add layers\niris.add_layer(:input, 4, :memory, SHAInet.sigmoid)\niris.add_layer(:hidden, 5, :memory, SHAInet.sigmoid)\niris.add_layer(:output, 3, :memory, SHAInet.sigmoid)\niris.fully_connect\n\n# Adjust network parameters\nxor.learning_rate = 0.7\nxor.momentum = 0.3\n\n# Train the network\niris.train_batch(\n      data: normalized.data.shuffle,\n      training_type: :adam,\n      cost_function: :mse,\n      epochs: 20000,\n      error_threshold: 0.000001,\n      log_each: 1000)\n\n# Test the network's performance\niris.test(test_set)\n```\n\n### Using convolutional network\n```crystal\n\n# Load training data (partial dataset)\nraw_data = Array(Array(Float64)).new\ncsv = CSV.new(File.read(__DIR__ + \"/test_data/mnist_train.csv\"))\n10000.times do\n  # CSV.each_row(File.read(__DIR__ + \"/test_data/mnist_train.csv\")) do |row|\n  csv.next\n  new_row = Array(Float64).new\n  csv.row.to_a.each { |value| new_row \u003c\u003c value.to_f64 }\n  raw_data \u003c\u003c new_row\nend\nraw_input_data = Array(Array(Float64)).new\nraw_output_data = Array(Array(Float64)).new\n\nraw_data.each do |row|\n  raw_input_data \u003c\u003c row[1..-1]\n  raw_output_data \u003c\u003c [row[0]]\nend\n\ntraining_data = SHAInet::CNNData.new(raw_input_data, raw_output_data)\ntraining_data.for_mnist_conv\ntraining_data.data_pairs.shuffle!\n\n# Load test data (partial dataset)\nraw_data = Array(Array(Float64)).new\ncsv = CSV.new(File.read(__DIR__ + \"/test_data/mnist_test.csv\"))\n1000.times do\n  csv.next\n  new_row = Array(Float64).new\n  csv.row.to_a.each { |value| new_row \u003c\u003c value.to_f64 }\n  raw_data \u003c\u003c new_row\nend\n\nraw_input_data = Array(Array(Float64)).new\nraw_output_data = Array(Array(Float64)).new\n\nraw_data.each do |row|\n  raw_input_data \u003c\u003c row[1..-1]\n  raw_output_data \u003c\u003c [row[0]]\nend\n\n# Load data to a CNNData helper class\ntest_data = SHAInet::CNNData.new(raw_input_data, raw_output_data)\ntest_data.for_mnist_conv # Normalize and make labels into 'one-hot' vectors\n\n# Initialize Covnolutional network\ncnn = SHAInet::CNN.new\n\n# Add layers to the model\ncnn.add_input([height = 28, width = 28, channels = 1]) # Output shape = 28x28x1\ncnn.add_conv(\n  filters_num: 20,\n  window_size: 5,\n  stride: 1,\n  padding: 2,\n  activation_function: SHAInet.none)  # Output shape = 28x28x20\ncnn.add_relu(0.01)                    # Output shape = 28x28x20\ncnn.add_maxpool(pool: = 2, stride = 2) # Output shape = 14x14x20\ncnn.add_conv(\n  filters_num: 20,\n  window_size: 5,\n  stride: 1,\n  padding: 2,\n  activation_function: SHAInet.none)  # Output shape = 14x14x40\ncnn.add_maxpool(pool:2, stride: 2)    # Output shape = 7x7x40\ncnn.add_fconnect(l_size: 10, activation_function: SHAInet.sigmoid)\ncnn.add_fconnect(l_size: 10, activation_function: SHAInet.sigmoid)\ncnn.add_softmax\n\ncnn.learning_rate = 0.005\ncnn.momentum = 0.02\n\n# Train the model on the training-set\ncnn.train_batch(\n  data: training_data.data_pairs,\n  training_type: :sgdm,\n  cost_function: :mse,\n  epochs: 3,\n  error_threshold: 0.0001,\n  log_each: 1,\n  mini_batch_size: 50)\n\n# Evaluate accuracy on the test-set\ncorrect_answers = 0\ntest_data.data_pairs.each do |data_point|\n  result = cnn.run(data_point[:input], stealth: true)\n  if (result.index(result.max) == data_point[:output].index(data_point[:output].max))\n    correct_answers += 1\n  end\nend\n\n# Print the layer activations\ncnn.inspect(\"activations\")\nputs \"We managed #{correct_answers} out of #{test_data.data_pairs.size} total\"\nputs \"Cnn output: #{cnn.output}\"\n```\n\n### Evolutionary optimizer example:\n```crystal\nlabel = {\n      \"setosa\"     =\u003e [0.to_f64, 0.to_f64, 1.to_f64],\n      \"versicolor\" =\u003e [0.to_f64, 1.to_f64, 0.to_f64],\n      \"virginica\"  =\u003e [1.to_f64, 0.to_f64, 0.to_f64],\n    }\n\n    iris = SHAInet::Network.new\n    iris.add_layer(:input, 4, :memory, SHAInet.sigmoid)\n    iris.add_layer(:hidden, 4, :memory, SHAInet.sigmoid)\n    iris.add_layer(:output, 3, :memory, SHAInet.sigmoid)\n    iris.fully_connect\n\n    # Get data from a local file\n    outputs = Array(Array(Float64)).new\n    inputs = Array(Array(Float64)).new\n    CSV.each_row(File.read(__DIR__ + \"/test_data/iris.csv\")) do |row|\n      row_arr = Array(Float64).new\n      row[0..-2].each do |num|\n        row_arr \u003c\u003c num.to_f64\n      end\n      inputs \u003c\u003c row_arr\n      outputs \u003c\u003c label[row[-1]]\n    end\n    data = SHAInet::TrainingData.new(inputs, outputs)\n    data.normalize_min_max\n\n    training_data, test_data = data.split(0.9)\n\n    iris.train_es(\n      data: training_data,\n      pool_size: 50,\n      learning_rate: 0.5,\n      sigma: 0.1,\n      cost_function: :c_ent,\n      epochs: 500,\n      mini_batch_size: 15,\n      error_threshold: 0.00000001,\n      log_each: 100,\n      show_slice: true)\n\n    # Test the trained model\n    correct = 0\n    test_data.data.each do |data_point|\n      result = iris.run(data_point[0], stealth: true)\n      expected = data_point[1]\n      # puts \"result: \\t#{result.map { |x| x.round(5) }}\"\n      # puts \"expected: \\t#{expected}\"\n      error_sum = 0.0\n      result.size.times do |i|\n        error_sum += (result[i] - expected[i]).abs\n      end\n      correct += 1 if error_sum \u003c 0.3\n    end\n    puts \"Correct answers: (#{correct} / #{test_data.size})\"\n    (correct \u003e 10).should eq(true)\n```\n\n\n## Development\n\n### Basic Features  \n  - [x] Train network\n  - [x] Save/load\n  - [x] Activation functions:\n    - [x] Sigmoid\n    - [x] Bipolar sigmoid\n    - [x] log-sigmoid\n    - [x] Tanh\n    - [x] ReLU\n    - [x] Leaky ReLU\n    - [x] Softmax\n  - [x] Cost functions:\n    - [x] Quadratic\n    - [x] Cross-entropy\n  - [x] Gradient optimizers\n    - [x] SGD + momentum\n    - [x] iRprop+  \n    - [x] ADAM\n    - [x] ES (evolutionary strategy, non-backprop)\n  - [x] Autosave during training\n\n### Advanced Features\n  - [x] Support activation functions as Proc\n  - [x] Support cost functions as Proc\n  - [x] Convolutional Neural Net.  \n  - [ ] Add support for multiple neuron types.  \n  - [ ] Bind and use CUDA (GPU acceleration)  \n  - [ ] graphic printout of network architecture.  \n  \n### Possible Future Features\n  - [ ] RNN (recurant neural network)\n  - [ ] LSTM (long-short term memory)\n  - [ ] GNG (growing neural gas).  \n  - [ ] SOM (self organizing maps).  \n  - [ ] DBM (deep belief network).  \n\n\n\n\n\n## Contributing\n\n1. Fork it ( https://github.com/NeuraLegion/shainet/fork )\n2. Create your feature branch (git checkout -b my-new-feature)\n3. Commit your changes (git commit -am 'Add some feature')\n4. Push to the branch (git push origin my-new-feature)\n5. Create a new Pull Request\n\n## Contributors\n\n- [ArtLinkov](https://github.com/ArtLinkov) - creator, maintainer\n- [bararchy](https://github.com/bararchy) - creator, maintainer\n- [drujensen](https://github.com/drujensen) - contributor\n- [hugoabonizio](https://github.com/hugoabonizio) - contributor\n- [Rémy Marronnier](https://github.com/rmarronnier) - contributor\n- [psikoz](https://github.com/psikoz) - logo desgin\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FNeuraLegion%2Fshainet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FNeuraLegion%2Fshainet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FNeuraLegion%2Fshainet/lists"}