{"id":18292889,"url":"https://github.com/pashashiz/scanet3","last_synced_at":"2025-07-24T11:42:59.615Z","repository":{"id":44697899,"uuid":"255042070","full_name":"pashashiz/scanet3","owner":"pashashiz","description":"Type-safe, high performance, distributed Neural networks in Scala","archived":false,"fork":false,"pushed_at":"2023-11-20T07:21:01.000Z","size":97769,"stargazers_count":29,"open_issues_count":1,"forks_count":2,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-05T10:33:42.925Z","etag":null,"topics":["functional-programming","linalg","math","neural-networks","scala","tensorflow"],"latest_commit_sha":null,"homepage":null,"language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pashashiz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-04-12T08:44:04.000Z","updated_at":"2024-10-05T21:40:58.000Z","dependencies_parsed_at":"2023-02-18T22:30:20.713Z","dependency_job_id":"72a01fb9-720a-450f-a138-ce3cde4aba4a","html_url":"https://github.com/pashashiz/scanet3","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/pashashiz/scanet3","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pashashiz%2Fscanet3","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pashashiz%2Fscanet3/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pashashiz%2Fscanet3/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pashashiz%2Fscanet3/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pashashiz","download_url":"https://codeload.github.com/pashashiz/scanet3/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pashashiz%2Fscanet3/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266836310,"owners_count":23992589,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-24T02:00:09.469Z","response_time":99,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["functional-programming","linalg","math","neural-networks","scala","tensorflow"],"created_at":"2024-11-05T14:19:48.493Z","updated_at":"2025-07-24T11:42:59.592Z","avatar_url":"https://github.com/pashashiz.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Scanet\n\n[![Build Status](https://travis-ci.org/pashashiz/scanet3.svg?branch=master)](https://travis-ci.org/pashashiz/scanet3)\n\nType-safe, high performance, distributed Neural networks in Scala (not Python, finally...).\n\n## Intro\n\nLow level (linear algebra) operations powered by low level TensorFlow API (C, C++ bindings via JNI).\n\nScala used to build computation graphs and compile them into native tensor graphs.\nCompiled graphs are fully calculated in native code (on CPU, GPU or TPU)\nand only result is returned back via `DirectBuffer`which points into native memory.\n\n`DirectBuffer` is wrapped with `Tensor` read-only object which allows\nto slice and read data in a convenient way (just like `Breeze` or `Numpy` does).\n\nThe optimizer is built on top of `Spark` and can optimize the model in a distributed/parallel way.\nThe chosen algorithm - `Data parallelism with synchronous model averaging`. The dataset is split between\nthe workers and each epoch is run independently on each data split, at the end of each epoch\nparameters are averaged and broadcasted back to each worker.\n\nThe input data is expected to be `Dataset[Array[TensorType]` and it contains a shape of the tensors in metadata.\nUsually, `TensorType` is choosen to be `Float` since it performs best on GPU, also `Double` can be used.\n\n## Examples\n\n### ANN\n\nExample of a simple MNIST dataset classifier with Fully Connected Neural Network:\n\n``` scala\nval (trainingDs, testDs) = MNIST.load(sc, trainingSize = 30000)\nval model = Dense(50, Sigmoid) \u003e\u003e Dense(10, Softmax)\nval trained = trainingDs.train(model)\n  .loss(CategoricalCrossentropy)\n  .using(Adam(0.01f))\n  .batch(1000)\n  .each(1.epochs, RecordLoss(tensorboard = true))\n  .each(10.epochs, RecordAccuracy(testDs, tensorboard = true))\n  .stopAfter(200.epochs)\n  .run()\naccuracy(trained, testDs) should be \u003e= 0.95f\n```\n\nHere, `loss` and `accuracy` will be logged and added to `TensorBoard` as live trends. To run tensorboard execute:\n\n```sh\npip install tensorboard\ntensorboard --logdir board\n```\n\n### CNN\n\nSame but with CNN (Convolutional Neural Network)\n\n```scala\nval (trainingDs, testDs) = MNIST()\nval model =\n  Conv2D(32, activation = ReLU()) \u003e\u003e Pool2D() \u003e\u003e\n    Conv2D(64, activation = ReLU()) \u003e\u003e Pool2D() \u003e\u003e\n    Flatten \u003e\u003e Dense(10, Softmax)\nval trained = trainingDs\n  .train(model)\n  .loss(CategoricalCrossentropy)\n  .using(Adam(0.001f))\n  .batch(100)\n  .each(1.epochs, RecordLoss(tensorboard = true))\n  .each(1.epochs, RecordAccuracy(testDs, tensorboard = true))\n  .stopAfter(3.epochs)\n  .run()\naccuracy(trained, testDs) should be \u003e= 0.98f\n```\n\n### RNN\n\nLSTM Layer to forecast sunspots\n\n```scala\nval Array(train, test) = monthlySunspots(12).randomSplit(Array(0.8, 0.2), 1)\nval model = LSTM(2) \u003e\u003e Dense(1, Tanh)\nval trained = train\n  .train(model)\n  .loss(MeanSquaredError)\n  .using(Adam())\n  .batch(10)\n  .each(1.epochs, RecordLoss(tensorboard = true))\n  .stopAfter(100.epochs)\n  .run()\nRMSE(trained, test) should be \u003c 0.2f\nR2Score(trained, test) should be \u003e 0.8f\n```\n\n## Road Map\n\n### Tensor Flow Low Level API\n\n- [x] Tensor\n- [x] DSL for computation DAG\n- [x] TF Session\n- [x] Core ops\n- [x] Math ops\n- [x] Logical ops\n- [x] String ops\n- [x] TF Functions, Placeholders, Session caching\n- [x] Tensor Board basic support\n\n### Optimizer engine\n\n- [x] Spark\n- [ ] Hyper parameter tuning\n- [ ] Model Import/Export\n\n### Optimizer algorithms\n\n- [x] SGD\n- [x] AdaGrad\n- [x] AdaDelta\n- [x] RMSProp\n- [x] Adam\n- [x] Nadam\n- [x] Adamax\n- [x] AMSGrad\n\n### Statistics\n\n- [x] Variance/STD\n- [ ] Covariance/Correlation Matrix\n- [ ] Lots of other useful algs to analyze the data set\n\n### Models\n\n- [x] Linear Regression\n- [x] Binary Logistic Regression\n- [x] ANN (Multilayer Perceptron NN)\n- [x] Kernel regularization\n- [x] Convolutional NN\n- [x] Recurrent NN (Simple, LSTM))\n- [x] Recurrent NN Enhancements\n    - Add GRU Cell\n    - Add LSTM GPU implementation LSTM: add GPU implementation, see `tensorflow/python/keras/layers/recurrent_v2.py`\n      line 1655\n    - Add RNN unroll option, see [tf.while_loop](https://www.tensorflow.org/api_docs/python/tf/while_loop)\n    - Add state between batches\n    - Try LSTM weights fusing (x4 less weights)\n- [ ] Layers Dropout (provide random generator to layers)(Wanted!)\n- [x] Batch Normalization\n- [ ] others\n\n# Localization \u0026 Object Detection \u0026 Instance Segmentation\n\n- [ ] Object Localization\n- [ ] Region Proposals (Selective Search, EdgeBoxes, etc..)\n- [ ] R-CNN\n- [ ] Fast R-CNN\n- [ ] Faster R-CNN\n- [ ] YOLO (You only look once)\n- [ ] SSD (Single-Shot MultiBox Detector)\n\n### Activation functions\n\n- [x] Sigmoid\n- [x] Tanh\n- [x] RELU\n- [x] Softmax\n- [ ] Exp\n- [ ] SELU\n- [ ] ELU\n- [ ] Sofplus\n\n### Loss functions\n\n- [x] RMSE (Mean Squared Error)\n- [x] Binary Crossentropy\n- [x] Categorical Crossentropy\n- [ ] Sparse Categorical Crossentropy\n\n### Benchmark Datasets\n\n- [ ] Boston Housing price regression dataset\n- [x] MNIST\n- [ ] Fashion MNIST\n- [ ] CIFAR-10\n- [ ] CIFAR-100\n- [ ] ILSVRC (ImageNet-1000)\n- [ ] Pascal VOC\n\n### Preprocessing\n\n- [ ] SVD/PCA/Whitening\n- [ ] Feature scalers\n- [ ] Feature embedding\n- [ ] Hashed features\n- [ ] Crossed features\n\n### Estimators\n\n- [x] r2 score\n- [x] accuracy estimator,\n- [ ] confusion matrix, precision, recall, f1 score\n- [ ] runtime estimating and new stop condition based on that\n\n### Benchmarks\n\n- [x] LeNet\n- [ ] AlexNet\n- [ ] ZF Net\n- [ ] ZF Net\n- [ ] VGGNet\n- [ ] GoogLeNet\n- [ ] ResNet\n- [ ] ...\n\n### CPU vs GPU vs TPU\n\n- [ ] Create computation intensive operation, like `matmul` multiple times large tensors\n  and compare with Scala `breeze`, python `tensorflow`, python `numpy`\n- [ ] Compare with existing implementations using local CPU\n- [ ] Compare with existing implementations using one GPU\n- [ ] Compare with existing implementations using distributed mode on GCP DataProc\n\n### Other useful things\n\n- [ ] While training analyze the weights histograms to make sure the deep NN do not saturate\n- [ ] Grid/Random hyper parameters search\n- [x] Different weight initializers (Xavier)\n- [ ] Decay learning rate over time (step, exponential, 1/t decay)\n- [ ] Try using in interactive notebook\n- [ ] Add graph library so we could plot some charts and publish them in `tensorboard` or `notebook` (maybe fork and\n  upgrade `vegas` to scala `2.12` ot try `evil-plot`)\n\n### Refactoring\n\n- Redefine the way we train model on a dataset and make a prediction.\n  We should cover 2 cases: BigData with `spark` which can train and predict on large datasets and single (batch)\n  prediction without `spark` dependency (to be able to expose model via API or use in realtime).\n  For that we need to:\n    + separate project into `core` + `spark` modules.\n    + implement model weights export/import\n    + implement feature preprocessing, for training use case try using `MLib`,\n      yet we need to figure out how to transform features via regular function without `spark` involved\n    + integrating with `MLib` might require redefining the `Dataset[Record[A]]` we have right now\n      probably better to use any abstract dataset which contains 2 required columns `features` + `labels`\n      for training and `features` for prediction.\n- Add DSL to build tensor requirements like `tensor require rank(4)`, `tensor require shape squratedMatrix`\n- We might need to define high level untyped trait `Node` which `Expr[A]` trait will extend\n  Such `Node` will have a defined compiler, to make it `Expr` we would need choose an output and assign a type\n\nIf you want to become a contributor, you are welcome!!! You can pick anything from a Road Map or propose your idea.\n\nPlease, contact:\n\n- `ppoh@softserveinc.com`\n- `yatam@softserveinc.com`","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpashashiz%2Fscanet3","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpashashiz%2Fscanet3","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpashashiz%2Fscanet3/lists"}