{"id":23626369,"url":"https://github.com/yetanalytics/dl4clj","last_synced_at":"2025-12-12T01:23:06.981Z","repository":{"id":62432447,"uuid":"51748595","full_name":"yetanalytics/dl4clj","owner":"yetanalytics","description":null,"archived":false,"fork":false,"pushed_at":"2018-07-17T17:04:01.000Z","size":9895,"stargazers_count":99,"open_issues_count":1,"forks_count":18,"subscribers_count":30,"default_branch":"master","last_synced_at":"2024-05-22T13:09:18.441Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yetanalytics.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-02-15T10:36:45.000Z","updated_at":"2022-11-18T04:52:26.000Z","dependencies_parsed_at":"2022-11-01T21:00:55.489Z","dependency_job_id":null,"html_url":"https://github.com/yetanalytics/dl4clj","commit_stats":null,"previous_names":["engagor/dl4clj"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yetanalytics%2Fdl4clj","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yetanalytics%2Fdl4clj/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yetanalytics%2Fdl4clj/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yetanalytics%2Fdl4clj/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yetanalytics","download_url":"https://codeload.github.com/yetanalytics/dl4clj/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":231548413,"owners_count":18393559,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-27T22:53:08.907Z","updated_at":"2025-12-12T01:23:06.934Z","avatar_url":"https://github.com/yetanalytics.png","language":"Clojure","readme":"# dl4clj\n\nPort of [deeplearning4j](https://github.com/deeplearning4j/) to clojure\n\n# Contact info\nIf you have any questions,\n- my email is will@yetanalytics.com\n- I'm will_hoyt in the clojurians slack\n- twitter is @FeLungz (don't check very often)\n\n## TODO\n- update examples dir\n- finish README\n  - add in examples using Transfer Learning\n- finish tests\n  - eval is missing regression tests, roc tests\n  - nn-test is missing regression tests\n  - spark tests need to be redone\n  - need dl4clj.core tests\n- revist spark for updates\n- write specs for user facing functions\n  - this is very important, match isnt strict for maps\n  - provides 100% certianty of the input -\u003e output flow\n  - check the args as they come in, dispatch once I know its safe, test the pure output\n- collapse overlapping api namespaces\n- add to core use case flows\n\n## Features\n\n### Stable Features with tests\n\n- Neural Networks DSL\n- Early Stopping Training\n- Transfer Learning\n- Evaluation\n- Data import\n\n### Features being worked on for 0.1.0\n\n- Clustering (testing in progress)\n- Spark (currently being refactored)\n- Front End (maybe current release, maybe future release. Not sure yet)\n- Version of dl4j is 0.0.8 in this project.  Current dl4j version is 0.0.9\n- Parallelism\n- Kafka support\n- Other items mentioned in TODO\n\n### Features being worked on for future releases\n\n- NLP\n- Computational Graphs\n- Reinforement Learning\n- Arbiter\n\n## Artifacts\n\nNOT YET RELEASED TO CLOJARS\n- fork or clone to try it out\n\nIf using Maven add the following repository definition to your pom.xml:\n\n```\n\u003crepository\u003e\n  \u003cid\u003eclojars.org\u003c/id\u003e\n  \u003curl\u003ehttp://clojars.org/repo\u003c/url\u003e\n\u003c/repository\u003e\n```\n\n## Latest release\n\nWith Leiningen:\n\n```\nn/a\n\n```\n\nWith Maven:\n\nn/a\n\n```\n\u003cdependency\u003e\n  \u003cgroupId\u003e_\u003c/groupId\u003e\n  \u003cartifactId\u003e_\u003c/artifactId\u003e\n  \u003cversion\u003e_\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n## Usage\n\n### Things you need to know\n- All functions for creating dl4j objects return code by default\n  - All of these functions have an option to return the dl4j object\n    - :as-code? = false\n  - This because all builders require the code representation of dl4j objects\n    - this requirement is not going to change\n  - INDarray creation fns default to objects, this is for convenience\n    - :as-code? is still respected\n\n- API functions return code when all args are provided as code\n- API functions return the value of calling the wrapped method when\n  args are provided as a mixture of objects and code or just objects\n\n- The tests are there to help clarify behavior, if you are unsure of how\n  to use a fn, search the tests\n  - for questions about spark, refer to the spark section bellow\n\n### Example of obj/code duality\n\n``` clojure\n(ns my.ns\n  (:require [dl4clj.nn.conf.builders.layers :as l]))\n\n;; as code (the default)\n\n(l/dense-layer-builder\n :activation-fn :relu\n :learning-rate 0.006\n :weight-init :xavier\n :layer-name \"example layer\"\n :n-in 10\n :n-out 1)\n\n;; =\u003e\n\n(doto\n (org.deeplearning4j.nn.conf.layers.DenseLayer$Builder.)\n (.nOut 1)\n (.activation (dl4clj.constants/value-of {:activation-fn :relu}))\n (.weightInit (dl4clj.constants/value-of {:weight-init :xavier}))\n (.nIn 10)\n (.name \"example layer\")\n (.learningRate 0.006))\n\n;; as an object\n\n(l/dense-layer-builder\n :activation-fn :relu\n :learning-rate 0.006\n :weight-init :xavier\n :layer-name \"example layer\"\n :n-in 10\n :n-out 1\n :as-code? false)\n\n;; =\u003e\n\n#object[org.deeplearning4j.nn.conf.layers.DenseLayer 0x69d7d160 \"DenseLayer(super=FeedForwardLayer(super=Layer(layerName=example layer, activationFn=relu, weightInit=XAVIER, biasInit=NaN, dist=null, learningRate=0.006, biasLearningRate=NaN, learningRateSchedule=null, momentum=NaN, momentumSchedule=null, l1=NaN, l2=NaN, l1Bias=NaN, l2Bias=NaN, dropOut=NaN, updater=null, rho=NaN, epsilon=NaN, rmsDecay=NaN, adamMeanDecay=NaN, adamVarDecay=NaN, gradientNormalization=null, gradientNormalizationThreshold=NaN), nIn=10, nOut=1))\"]\n\n```\n\n### General usage examples\n\n#### Importing data\n\nLoading data from a file (here its a csv)\n\n``` clojure\n\n(ns my.ns\n (:require [dl4clj.datasets.input-splits :as s]\n           [dl4clj.datasets.record-readers :as rr]\n           [dl4clj.datasets.api.record-readers :refer :all]\n           [dl4clj.datasets.iterators :as ds-iter]\n           [dl4clj.datasets.api.iterators :refer :all]\n           [dl4clj.helpers :refer [data-from-iter]]))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; file splits (convert the data to records)\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n(def poker-path \"resources/poker-hand-training.csv\")\n;; this is not a complete dataset, it is just here to sever as an example\n\n(def file-split (s/new-filesplit :path poker-path))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; record readers, (read the records created by the file split)\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n(def csv-rr (initialize-rr! :rr (rr/new-csv-record-reader :skip-n-lines 0 :delimiter \",\")\n                                 :input-split file-split))\n\n;; lets look at some data\n(println (next-record! :rr csv-rr :as-code? false))\n;; =\u003e #object[java.util.ArrayList 0x2473e02d [1, 10, 1, 11, 1, 13, 1, 12, 1, 1, 9]]\n;; this is our first line from the csv\n\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; record readers dataset iterators (turn our writables into a dataset)\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n(def rr-ds-iter (ds-iter/new-record-reader-dataset-iterator\n                 :record-reader csv-rr\n                 :batch-size 1\n                 :label-idx 10\n                 :n-possible-labels 10))\n\n;; we use our record reader created above\n;; we want to see one example per dataset obj returned (:batch-size = 1)\n;; we know our label is at the last index, so :label-idx = 10\n;; there are 10 possible types of poker hands so :n-possible-labels = 10\n;; you can also set :label-idx to -1 to use the last index no matter the size of the seq\n\n(def other-rr-ds-iter (ds-iter/new-record-reader-dataset-iterator\n                       :record-reader csv-rr\n                       :batch-size 1\n                       :label-idx -1\n                       :n-possible-labels 10))\n\n(str (next-example! :iter rr-ds-iter :as-code? false))\n;; =\u003e\n;;===========INPUT===================\n;;[1.00, 10.00, 1.00, 11.00, 1.00, 13.00, 1.00, 12.00, 1.00, 1.00]\n;;=================OUTPUT==================\n;;[0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 1.00]\n\n\n;; and to show that :label-idx = -1 gives us the same output\n\n(= (next-example! :iter rr-ds-iter :as-code? false)\n   (next-example! :iter other-rr-ds-iter :as-code? false)) ;; =\u003e true\n\n```\n\n#### INDArrays and Datasets from clojure data structures\n\n``` clojure\n\n(ns my.ns\n  (:require [nd4clj.linalg.factory.nd4j :refer [vec-\u003eindarray matrix-\u003eindarray\n                                                indarray-of-zeros indarray-of-ones\n                                                indarray-of-rand vec-or-matrix-\u003eindarray]]\n            [dl4clj.datasets.new-datasets :refer [new-ds]]\n            [dl4clj.datasets.api.datasets :refer [as-list]]\n            [dl4clj.datasets.iterators :refer [new-existing-dataset-iterator]]\n            [dl4clj.datasets.api.iterators :refer :all]\n            [dl4clj.datasets.pre-processors :as ds-pp]\n            [dl4clj.datasets.api.pre-processors :refer :all]\n            [dl4clj.core :as c]))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; INDArray creation\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n;;TODO: consider defaulting to code\n\n;; can create from a vector\n\n(vec-\u003eindarray [1 2 3 4])\n;; =\u003e #object[org.nd4j.linalg.cpu.nativecpu.NDArray 0x269df212 [1.00, 2.00, 3.00, 4.00]]\n\n;; or from a matrix\n\n(matrix-\u003eindarray [[1 2 3 4] [2 4 6 8]])\n;; =\u003e #object[org.nd4j.linalg.cpu.nativecpu.NDArray 0x20aa7fe1\n;; [[1.00, 2.00, 3.00, 4.00], [2.00, 4.00, 6.00, 8.00]]]\n\n\n;; will fill in spareness with zeros\n\n(matrix-\u003eindarray [[1 2 3 4] [2 4 6 8] [10 12]])\n;; =\u003e #object[org.nd4j.linalg.cpu.nativecpu.NDArray 0x8b7796c\n;;[[1.00, 2.00, 3.00, 4.00],\n;; [2.00, 4.00, 6.00, 8.00],\n;; [10.00, 12.00, 0.00, 0.00]]]\n\n;; can create an indarray of all zeros with specified shape\n;; defaults to :rows = 1 :columns = 1\n\n(indarray-of-zeros :rows 3 :columns 2)\n;; =\u003e #object[org.nd4j.linalg.cpu.nativecpu.NDArray 0x6f586a7e\n;;[[0.00, 0.00],\n;; [0.00, 0.00],\n;; [0.00, 0.00]]]\n\n(indarray-of-zeros) ;; =\u003e #object[org.nd4j.linalg.cpu.nativecpu.NDArray 0xe59ffec 0.00]\n\n;; and if only one is supplied, will get a vector of specified length\n\n(indarray-of-zeros :rows 2)\n;; =\u003e #object[org.nd4j.linalg.cpu.nativecpu.NDArray 0x2899d974 [0.00, 0.00]]\n\n(indarray-of-zeros :columns 2)\n;; =\u003e #object[org.nd4j.linalg.cpu.nativecpu.NDArray 0xa5b9782 [0.00, 0.00]]\n\n;; same considerations/defaults for indarray-of-ones and indarray-of-rand\n\n(indarray-of-ones :rows 2 :columns 3)\n;; =\u003e #object[org.nd4j.linalg.cpu.nativecpu.NDArray 0x54f08662 [[1.00, 1.00, 1.00], [1.00, 1.00, 1.00]]]\n\n(indarray-of-rand :rows 2 :columns 3)\n;; all values are greater than 0 but less than 1\n;; =\u003e #object[org.nd4j.linalg.cpu.nativecpu.NDArray 0x2f20293b [[0.85, 0.86, 0.13], [0.94, 0.04, 0.36]]]\n\n\n\n;; vec-or-matrix-\u003eindarray is built into all functions which require INDArrays\n;; so that you can use clojure data structures\n;; but you still have the option of passing existing INDArrays\n\n(def example-array (vec-or-matrix-\u003eindarray [1 2 3 4]))\n;; =\u003e #object[org.nd4j.linalg.cpu.nativecpu.NDArray 0x5c44c71f [1.00, 2.00, 3.00, 4.00]]\n\n(vec-or-matrix-\u003eindarray example-array)\n;; =\u003e #object[org.nd4j.linalg.cpu.nativecpu.NDArray 0x607b03b0 [1.00, 2.00, 3.00, 4.00]]\n\n(vec-or-matrix-\u003eindarray (indarray-of-rand :rows 2))\n;; =\u003e #object[org.nd4j.linalg.cpu.nativecpu.NDArray 0x49143b08 [0.76, 0.92]]\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; data-set creation\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n(def ds-with-single-example (new-ds :input [1 2 3 4]\n                                    :output [0.0 1.0 0.0]))\n\n(as-list :ds ds-with-single-example :as-code? false)\n;; =\u003e\n;; #object[java.util.ArrayList 0x5d703d12\n;;[===========INPUT===================\n;;[1.00, 2.00, 3.00, 4.00]\n;;=================OUTPUT==================\n;;[0.00, 1.00, 0.00]]]\n\n(def ds-with-multiple-examples (new-ds\n                                :input [[1 2 3 4] [2 4 6 8]]\n                                :output [[0.0 1.0 0.0] [0.0 0.0 1.0]]))\n\n(as-list :ds ds-with-multiple-examples :as-code? false)\n;; =\u003e\n;;#object[java.util.ArrayList 0x29c7a9e2\n;;[===========INPUT===================\n;;[1.00, 2.00, 3.00, 4.00]\n;;=================OUTPUT==================\n;;[0.00, 1.00, 0.00],\n;;===========INPUT===================\n;;[2.00, 4.00, 6.00, 8.00]\n;;=================OUTPUT==================\n;;[0.00, 0.00, 1.00]]]\n\n;; we can create a dataset iterator from the code which creates datasets\n;; and set the labels for our outputs (optional)\n\n(def ds-with-multiple-examples\n  (new-ds\n   :input [[1 2 3 4] [2 4 6 8]]\n   :output [[0.0 1.0 0.0] [0.0 0.0 1.0]]))\n\n;; iterator\n(def training-rr-ds-iter\n  (new-existing-dataset-iterator\n   :dataset ds-with-multiple-examples\n   :labels [\"foo\" \"baz\" \"foobaz\"]))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; data-set normalization\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n;; this gathers statistics on the dataset and normalizes the data\n;; and applies the transformation to all dataset objects in the iterator\n(def train-iter-normalized\n  (c/normalize-iter! :iter training-rr-ds-iter\n                     :normalizer (ds-pp/new-standardize-normalization-ds-preprocessor)\n                     :as-code? false))\n\n;; above returns the normalized iterator\n;; to get fit normalizer\n\n(def the-normalizer\n  (get-pre-processor train-iter-normalized))\n\n```\n\n#### Model configuration\n\nCreating a neural network configuration with singe and multiple layers\n\n``` clojure\n(ns my.ns\n  (:require [dl4clj.nn.conf.builders.layers :as l]\n            [dl4clj.nn.conf.builders.nn :as nn]\n            [dl4clj.nn.conf.distributions :as dist]\n            [dl4clj.nn.conf.input-pre-processor :as pp]\n            [dl4clj.nn.conf.step-fns :as s-fn]))\n\n;; nn/builder has 3 types of args\n;; 1) args which set network configuration params\n;; 2) args which set default values for layers\n;; 3) args which set multi layer network configuration params\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; single layer nn configuration\n;; here we are setting network configuration\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n(nn/builder :optimization-algo :stochastic-gradient-descent\n            :seed 123\n            :iterations 1\n            :minimize? true\n            :use-drop-connect? false\n            :lr-score-based-decay-rate 0.002\n            :regularization? false\n            :step-fn :default-step-fn\n            :layers {:dense-layer {:activation-fn :relu\n                                   :updater :adam\n                                   :adam-mean-decay 0.2\n                                   :adam-var-decay 0.1\n                                   :learning-rate 0.006\n                                   :weight-init :xavier\n                                   :layer-name \"single layer model example\"\n                                   :n-in 10\n                                   :n-out 20}})\n\n;; there are several options within a nn-conf map which can be configuration maps\n;; or calls to fns\n;; It doesn't matter which option you choose and you don't have to stay consistent\n;; the list of params which can be passed as config maps or fn calls will\n;; be enumerated at a later date\n\n(nn/builder :optimization-algo :stochastic-gradient-descent\n            :seed 123\n            :iterations 1\n            :minimize? true\n            :use-drop-connect? false\n            :lr-score-based-decay-rate 0.002\n            :regularization? false\n            :step-fn (s-fn/new-default-step-fn)\n            :build? true\n            ;; dont need to specify layer order, theres only one\n            :layers (l/dense-layer-builder\n                    :activation-fn :relu\n                    :updater :adam\n                    :adam-mean-decay 0.2\n                    :adam-var-decay 0.1\n                    :dist (dist/new-normal-distribution :mean 0 :std 1)\n                    :learning-rate 0.006\n                    :weight-init :xavier\n                    :layer-name \"single layer model example\"\n                    :n-in 10\n                    :n-out 20))\n\n;; these configurations are the same\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; multi-layer configuration\n;; here we are also setting layer defaults\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n;; defaults will apply to layers which do not specify those value in their config\n\n(nn/builder\n :optimization-algo :stochastic-gradient-descent\n :seed 123\n :iterations 1\n :minimize? true\n :use-drop-connect? false\n :lr-score-based-decay-rate 0.002\n :regularization? false\n :default-activation-fn :sigmoid\n :default-weight-init :uniform\n\n ;; we need to specify the layer order\n :layers {0 (l/activation-layer-builder\n             :activation-fn :relu\n             :updater :adam\n             :adam-mean-decay 0.2\n             :adam-var-decay 0.1\n             :learning-rate 0.006\n             :weight-init :xavier\n             :layer-name \"example first layer\"\n             :n-in 10\n             :n-out 20)\n          1 {:output-layer {:n-in 20\n                            :n-out 2\n                            :loss-fn :mse\n                            :layer-name \"example output layer\"}}})\n\n;; specifying multi-layer config params\n\n(nn/builder\n ;; network args\n :optimization-algo :stochastic-gradient-descent\n :seed 123\n :iterations 1\n :minimize? true\n :use-drop-connect? false\n :lr-score-based-decay-rate 0.002\n :regularization? false\n\n ;; layer defaults\n :default-activation-fn :sigmoid\n :default-weight-init :uniform\n\n ;; the layers\n :layers {0 (l/activation-layer-builder\n             :activation-fn :relu\n             :updater :adam\n             :adam-mean-decay 0.2\n             :adam-var-decay 0.1\n             :learning-rate 0.006\n             :weight-init :xavier\n             :layer-name \"example first layer\"\n             :n-in 10\n             :n-out 20)\n          1 {:output-layer {:n-in 20\n                            :n-out 2\n                            :loss-fn :mse\n                            :layer-name \"example output layer\"}}}\n ;; multi layer network args\n :backprop? true\n :backprop-type :standard\n :pretrain? false\n :input-pre-processors {0 (pp/new-zero-mean-pre-pre-processor)\n                        1 {:unit-variance-processor {}}})\n\n```\n\n#### Configuration to Trained models\n\nMulti Layer models\n- an implementation of the dl4j [mnist classification example](https://github.com/deeplearning4j/dl4j-examples/blob/master/dl4j-examples/src/main/java/org/deeplearning4j/examples/feedforward/mnist/MLPMnistTwoLayerExample.java)\n\n\n``` clojure\n(ns my.ns\n  (:require [dl4clj.datasets.iterators :as iter]\n            [dl4clj.datasets.input-splits :as split]\n            [dl4clj.datasets.record-readers :as rr]\n            [dl4clj.optimize.listeners :as listener]\n            [dl4clj.nn.conf.builders.nn :as nn]\n            [dl4clj.nn.multilayer.multi-layer-network :as mln]\n            [dl4clj.nn.api.model :refer [init! set-listeners!]]\n            [dl4clj.nn.api.multi-layer-network :refer [evaluate-classification]]\n            [dl4clj.datasets.api.record-readers :refer [initialize-rr!]]\n            [dl4clj.eval.api.eval :refer [get-stats get-accuracy]]\n            [dl4clj.core :as c]))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; nn-conf -\u003e multi-layer-network\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n(def nn-conf\n  (nn/builder\n   ;; network args\n   :optimization-algo :stochastic-gradient-descent\n   :seed 123 :iterations 1 :regularization? true\n\n   ;; setting layer defaults\n   :default-activation-fn :relu :default-l2 7.5e-6\n   :default-weight-init :xavier :default-learning-rate 0.0015\n   :default-updater :nesterovs :default-momentum 0.98\n\n   ;; setting layer configuration\n   :layers {0 {:dense-layer\n               {:layer-name \"example first layer\"\n                :n-in 784 :n-out 500}}\n            1 {:dense-layer\n               {:layer-name \"example second layer\"\n                :n-in 500 :n-out 100}}\n            2 {:output-layer\n               {:n-in 100 :n-out 10\n                ;; layer specific params\n                :loss-fn :negativeloglikelihood\n                :activation-fn :softmax\n                :layer-name \"example output layer\"}}}\n\n   ;; multi layer args\n   :backprop? true\n   :pretrain? false))\n\n(def multi-layer-network (c/model-from-conf nn-conf))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; local cpu training with dl4j pre-built iterators\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n;; lets use the pre-built Mnist data set iterator\n\n(def train-mnist-iter\n  (iter/new-mnist-data-set-iterator\n   :batch-size 64\n   :train? true\n   :seed 123))\n\n(def test-mnist-iter\n  (iter/new-mnist-data-set-iterator\n   :batch-size 64\n   :train? false\n   :seed 123))\n\n;; and lets set a listener so we can know how training is going\n\n(def score-listener (listener/new-score-iteration-listener :print-every-n 5))\n\n;; and attach it to our model\n\n;; TODO: listeners are broken, look into log4j warnning\n(def mln-with-listener (set-listeners! :model multi-layer-network\n                                       :listeners [score-listener]))\n\n(def trained-mln (mln/train-mln-with-ds-iter! :mln mln-with-listener\n                                              :iter train-mnist-iter\n                                              :n-epochs 15\n                                              :as-code? false))\n\n;; training happens because :as-code? = false\n;; if it was true, we would still just have a data structure\n;; we now have a trained model that has seen the training dataset 15 times\n;; time to evaluate our model\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;;Create an evaluation object\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n(def eval-obj (evaluate-classification :mln trained-mln\n                                       :iter test-mnist-iter))\n\n;; always remember that these objects are stateful, dont use the same eval-obj\n;; to eval two different networks\n;; we trained the model on a training dataset.  We evaluate on a test set\n\n(println (get-stats :evaler eval-obj))\n;; this will print the stats to standard out for each feature/label pair\n\n;;Examples labeled as 0 classified by model as 0: 968 times\n;;Examples labeled as 0 classified by model as 1: 1 times\n;;Examples labeled as 0 classified by model as 2: 1 times\n;;Examples labeled as 0 classified by model as 3: 1 times\n;;Examples labeled as 0 classified by model as 5: 1 times\n;;Examples labeled as 0 classified by model as 6: 3 times\n;;Examples labeled as 0 classified by model as 7: 1 times\n;;Examples labeled as 0 classified by model as 8: 2 times\n;;Examples labeled as 0 classified by model as 9: 2 times\n;;Examples labeled as 1 classified by model as 1: 1126 times\n;;Examples labeled as 1 classified by model as 2: 2 times\n;;Examples labeled as 1 classified by model as 3: 1 times\n;;Examples labeled as 1 classified by model as 5: 1 times\n;;Examples labeled as 1 classified by model as 6: 2 times\n;;Examples labeled as 1 classified by model as 7: 1 times\n;;Examples labeled as 1 classified by model as 8: 2 times\n;;Examples labeled as 2 classified by model as 0: 3 times\n;;Examples labeled as 2 classified by model as 1: 2 times\n;;Examples labeled as 2 classified by model as 2: 1006 times\n;;Examples labeled as 2 classified by model as 3: 2 times\n;;Examples labeled as 2 classified by model as 4: 3 times\n;;Examples labeled as 2 classified by model as 6: 3 times\n;;Examples labeled as 2 classified by model as 7: 7 times\n;;Examples labeled as 2 classified by model as 8: 6 times\n;;Examples labeled as 3 classified by model as 2: 4 times\n;;Examples labeled as 3 classified by model as 3: 990 times\n;;Examples labeled as 3 classified by model as 5: 3 times\n;;Examples labeled as 3 classified by model as 7: 3 times\n;;Examples labeled as 3 classified by model as 8: 3 times\n;;Examples labeled as 3 classified by model as 9: 7 times\n;;Examples labeled as 4 classified by model as 2: 2 times\n;;Examples labeled as 4 classified by model as 3: 1 times\n;;Examples labeled as 4 classified by model as 4: 967 times\n;;Examples labeled as 4 classified by model as 6: 4 times\n;;Examples labeled as 4 classified by model as 7: 1 times\n;;Examples labeled as 4 classified by model as 9: 7 times\n;;Examples labeled as 5 classified by model as 0: 2 times\n;;Examples labeled as 5 classified by model as 3: 6 times\n;;Examples labeled as 5 classified by model as 4: 1 times\n;;Examples labeled as 5 classified by model as 5: 874 times\n;;Examples labeled as 5 classified by model as 6: 3 times\n;;Examples labeled as 5 classified by model as 7: 1 times\n;;Examples labeled as 5 classified by model as 8: 3 times\n;;Examples labeled as 5 classified by model as 9: 2 times\n;;Examples labeled as 6 classified by model as 0: 4 times\n;;Examples labeled as 6 classified by model as 1: 3 times\n;;Examples labeled as 6 classified by model as 3: 2 times\n;;Examples labeled as 6 classified by model as 4: 4 times\n;;Examples labeled as 6 classified by model as 5: 4 times\n;;Examples labeled as 6 classified by model as 6: 939 times\n;;Examples labeled as 6 classified by model as 7: 1 times\n;;Examples labeled as 6 classified by model as 8: 1 times\n;;Examples labeled as 7 classified by model as 1: 7 times\n;;Examples labeled as 7 classified by model as 2: 4 times\n;;Examples labeled as 7 classified by model as 3: 3 times\n;;Examples labeled as 7 classified by model as 7: 1005 times\n;;Examples labeled as 7 classified by model as 8: 2 times\n;;Examples labeled as 7 classified by model as 9: 7 times\n;;Examples labeled as 8 classified by model as 0: 3 times\n;;Examples labeled as 8 classified by model as 2: 3 times\n;;Examples labeled as 8 classified by model as 3: 2 times\n;;Examples labeled as 8 classified by model as 4: 4 times\n;;Examples labeled as 8 classified by model as 5: 3 times\n;;Examples labeled as 8 classified by model as 6: 2 times\n;;Examples labeled as 8 classified by model as 7: 4 times\n;;Examples labeled as 8 classified by model as 8: 947 times\n;;Examples labeled as 8 classified by model as 9: 6 times\n;;Examples labeled as 9 classified by model as 0: 2 times\n;;Examples labeled as 9 classified by model as 1: 2 times\n;;Examples labeled as 9 classified by model as 3: 4 times\n;;Examples labeled as 9 classified by model as 4: 8 times\n;;Examples labeled as 9 classified by model as 6: 1 times\n;;Examples labeled as 9 classified by model as 7: 4 times\n;;Examples labeled as 9 classified by model as 8: 2 times\n;;Examples labeled as 9 classified by model as 9: 986 times\n\n;;==========================Scores========================================\n;; Accuracy:        0.9808\n;; Precision:       0.9808\n;; Recall:          0.9807\n;; F1 Score:        0.9807\n;;========================================================================\n\n;; can get the stats that are printed via fns in the evaluation namespace\n;; after running eval-model-whole-ds\n\n(get-accuracy :evaler evaler-with-stats) ;; =\u003e 0.9808\n\n\n```\n\n#### Model Tuning\n\nEarly Stopping (controlling training)\n- it is recommened you start here when designing models\n\n- using dl4clj.core\n\n``` clojure\n\n(ns my.ns\n  (:require [dl4clj.earlystopping.termination-conditions :refer :all]\n            [dl4clj.earlystopping.model-saver :refer [new-in-memory-saver]]\n            [dl4clj.nn.api.multi-layer-network :refer [evaluate-classification]]\n            [dl4clj.eval.api.eval :refer [get-stats]]\n            [dl4clj.nn.conf.builders.nn :as nn]\n            [dl4clj.datasets.iterators :as iter]\n            [dl4clj.core :as c]))\n\n(def nn-conf\n  (nn/builder\n   ;; network args\n   :optimization-algo :stochastic-gradient-descent\n   :seed 123\n   :iterations 1\n   :regularization? true\n\n   ;; setting layer defaults\n   :default-activation-fn :relu\n   :default-l2 7.5e-6\n   :default-weight-init :xavier\n   :default-learning-rate 0.0015\n   :default-updater :nesterovs\n   :default-momentum 0.98\n\n   ;; setting layer configuration\n   :layers {0 {:dense-layer\n               {:layer-name \"example first layer\"\n                :n-in 784 :n-out 500}}\n            1 {:dense-layer\n               {:layer-name \"example second layer\"\n                :n-in 500 :n-out 100}}\n            2 {:output-layer\n               {:n-in 100 :n-out 10\n                ;; layer specific params\n                :loss-fn :negativeloglikelihood\n                :activation-fn :softmax\n                :layer-name \"example output layer\"}}}\n\n   ;; multi layer args\n   :backprop? true\n   :pretrain? false))\n\n(def train-iter\n  (iter/new-mnist-data-set-iterator\n   :batch-size 64\n   :train? true\n   :seed 123))\n\n(def test-iter\n  (iter/new-mnist-data-set-iterator\n   :batch-size 64\n   :train? false\n   :seed 123))\n\n(def invalid-score-condition (new-invalid-score-iteration-termination-condition))\n\n(def max-score-condition (new-max-score-iteration-termination-condition\n                          :max-score 20.0))\n\n(def max-time-condition (new-max-time-iteration-termination-condition\n                         :max-time-val 10\n                         :max-time-unit :minutes))\n\n(def score-doesnt-improve-condition (new-score-improvement-epoch-termination-condition\n                                     :max-n-epoch-no-improve 5))\n\n(def target-score-condition (new-best-score-epoch-termination-condition\n                             :best-expected-score 0.009))\n\n(def max-number-epochs-condition (new-max-epochs-termination-condition :max-n 20))\n\n(def in-mem-saver (new-in-memory-saver))\n\n(def trained-mln\n;; defaults to returning the model\n  (c/train-with-early-stopping\n   :nn-conf nn-conf\n   :training-iter train-mnist-iter\n   :testing-iter test-mnist-iter\n   :eval-every-n-epochs 1\n   :iteration-termination-conditions [invalid-score-condition\n                                      max-score-condition\n                                      max-time-condition]\n   :epoch-termination-conditions [score-doesnt-improve-condition\n                                  target-score-condition\n                                  max-number-epochs-condition]\n   :save-last-model? true\n   :model-saver in-mem-saver\n   :as-code? false))\n\n(def model-evaler\n  (evaluate-classification :mln trained-mln :iter test-mnist-iter))\n\n(println (get-stats :evaler model-evaler))\n\n```\n\n- explicit, step by step way of doing this\n\n``` clojure\n(ns my.ns\n  (:require [dl4clj.earlystopping.early-stopping-config :refer [new-early-stopping-config]]\n            [dl4clj.earlystopping.termination-conditions :refer :all]\n            [dl4clj.earlystopping.model-saver :refer [new-in-memory-saver new-local-file-model-saver]]\n            [dl4clj.earlystopping.score-calc :refer [new-ds-loss-calculator]]\n            [dl4clj.earlystopping.early-stopping-trainer :refer [new-early-stopping-trainer]]\n            [dl4clj.earlystopping.api.early-stopping-trainer :refer [fit-trainer!]]\n            [dl4clj.nn.conf.builders.nn :as nn]\n            [dl4clj.nn.multilayer.multi-layer-network :as mln]\n            [dl4clj.utils :refer [load-model!]]\n            [dl4clj.datasets.iterators :as iter]\n            [dl4clj.core :as c]))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; start with our network config\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n(def nn-conf\n  (nn/builder\n   ;; network args\n   :optimization-algo :stochastic-gradient-descent\n   :seed 123 :iterations 1 :regularization? true\n   ;; setting layer defaults\n   :default-activation-fn :relu :default-l2 7.5e-6\n   :default-weight-init :xavier :default-learning-rate 0.0015\n   :default-updater :nesterovs :default-momentum 0.98\n   ;; setting layer configuration\n   :layers {0 {:dense-layer\n               {:layer-name \"example first layer\"\n                :n-in 784 :n-out 500}}\n            1 {:dense-layer\n               {:layer-name \"example second layer\"\n                :n-in 500 :n-out 100}}\n            2 {:output-layer\n               {:n-in 100 :n-out 10\n                ;; layer specific params\n                :loss-fn :negativeloglikelihood\n                :activation-fn :softmax\n                :layer-name \"example output layer\"}}}\n   ;; multi layer args\n   :backprop? true\n   :pretrain? false))\n\n(def mln (c/model-from-conf nn-conf))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; the training/testing data\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n(def train-iter\n  (iter/new-mnist-data-set-iterator\n   :batch-size 64\n   :train? true\n   :seed 123))\n\n(def test-iter\n  (iter/new-mnist-data-set-iterator\n   :batch-size 64\n   :train? false\n   :seed 123))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; we are going to need termination conditions\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n;; these allow us to control when we exit training\n\n;; this can be based off of iterations or epochs\n\n;; iteration termination conditions\n\n(def invalid-score-condition (new-invalid-score-iteration-termination-condition))\n\n(def max-score-condition (new-max-score-iteration-termination-condition\n                          :max-score 20.0))\n\n(def max-time-condition (new-max-time-iteration-termination-condition\n                         :max-time-val 10\n                         :max-time-unit :minutes))\n\n;; epoch termination conditions\n\n(def score-doesnt-improve-condition (new-score-improvement-epoch-termination-condition\n                                     :max-n-epoch-no-improve 5))\n\n(def target-score-condition (new-best-score-epoch-termination-condition :best-expected-score 0.009))\n\n(def max-number-epochs-condition (new-max-epochs-termination-condition :max-n 20))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; we also need a way to save our model\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n;; can be in memory or to a local directory\n\n(def in-mem-saver (new-in-memory-saver))\n\n(def local-file-saver (new-local-file-model-saver :directory \"resources/tmp/readme/\"))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; set up your score calculator\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n(def score-calcer (new-ds-loss-calculator :iter test-iter\n                                          :average? true))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; create an early stopping configuration\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n;; termination conditions\n;; a way to save our model\n;; a way to calculate the score of our model on the dataset\n\n(def early-stopping-conf\n  (new-early-stopping-config\n   :epoch-termination-conditions [score-doesnt-improve-condition\n                                  target-score-condition\n                                  max-number-epochs-condition]\n   :iteration-termination-conditions [invalid-score-condition\n                                      max-score-condition\n                                      max-time-condition]\n   :eval-every-n-epochs 5\n   :model-saver local-file-saver\n   :save-last-model? true\n   :score-calculator score-calcer))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; create an early stopping trainer from our data, model and early stopping conf\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n(def es-trainer (new-early-stopping-trainer :early-stopping-conf early-stopping-conf\n                                            :mln mln\n                                            :iter train-iter))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; fit and use our early stopping trainer\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n(def es-trainer-fitted (fit-trainer! es-trainer :as-code? false))\n\n;; when the trainer terminates, you will see something like this\n;;[nREPL-worker-24] BaseEarlyStoppingTrainer INFO  Completed training epoch 14\n;;[nREPL-worker-24] BaseEarlyStoppingTrainer INFO  New best model: score = 0.005225599372851298,\n;;                                                   epoch = 14 (previous: score = 0.018243224899038346, epoch = 7)\n;;[nREPL-worker-24] BaseEarlyStoppingTrainer INFO Hit epoch termination condition at epoch 14.\n;;                                           Details: BestScoreEpochTerminationCondition(0.009)\n\n;; and if we look at the es-trainer-fitted object we see\n\n;;#object[org.deeplearning4j.earlystopping.EarlyStoppingResult 0x5ab74f27 EarlyStoppingResult\n;;(terminationReason=EpochTerminationCondition,details=BestScoreEpochTerminationCondition(0.009),\n;; bestModelEpoch=14,bestModelScore=0.005225599372851298,totalEpochs=15)]\n\n;; and our model has been saved to /resources/tmp/readme/bestModel.bin\n;; there we have our model config, model params and our updater state\n\n;; we can then load this model to use it or continue refining it\n\n(def loaded-model (load-model! :path \"resources/tmp/readme/bestModel.bin\"\n                               :load-updater? true))\n\n```\n\nTransfer Learning (freezing layers)\n\n``` clojure\n\n;; TODO: need to write up examples\n\n```\n\n\n### Spark Training\ndl4j [Spark](https://deeplearning4j.org/spark) usage\n\nHow it is done in dl4clj\n\n- Uses dl4clj.core\n  - This example uses a fn which takes care of most steps for you\n    - allows you to pass args as code bc the fn accounts for\n      the multiple spark contexts issue encountered when everything\n      is just a data structure\n\n``` clojure\n\n(ns my.ns\n  (:require [dl4clj.nn.conf.builders.layers :as l]\n            [dl4clj.nn.conf.builders.nn :as nn]\n            [dl4clj.datasets.iterators :refer [new-iris-data-set-iterator]]\n            [dl4clj.eval.api.eval :refer [get-stats]]\n            [dl4clj.spark.masters.param-avg :as master]\n            [dl4clj.spark.data.java-rdd :refer [new-java-spark-context\n                                                java-rdd-from-iter]]\n            [dl4clj.spark.api.dl4j-multi-layer :refer [eval-classification-spark-mln\n                                                       get-spark-context]]\n            [dl4clj.core :as c]))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; Step 1, create your model config\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n(def mln-conf\n  (nn/builder\n   :optimization-algo :stochastic-gradient-descent\n   :default-learning-rate 0.006\n   :layers {0 (l/dense-layer-builder :n-in 4 :n-out 2 :activation-fn :relu)\n            1 {:output-layer\n               {:loss-fn :negativeloglikelihood\n                :n-in 2 :n-out 3\n                :activation-fn :soft-max\n                :weight-init :xavier}}}\n   :backprop? true\n   :backprop-type :standard))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; Step 2, training master\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n(def training-master\n  (master/new-parameter-averaging-training-master\n   :build? true\n   :rdd-n-examples 10\n   :n-workers 4\n   :averaging-freq 10\n   :batch-size-per-worker 2\n   :export-dir \"resources/spark/master/\"\n   :rdd-training-approach :direct\n   :repartition-data :always\n   :repartition-strategy :balanced\n   :seed 1234\n   :save-updater? true\n   :storage-level :none))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; Step 3, spark context\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n(def your-spark-context\n  (new-java-spark-context :app-name \"example app\"))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; Step 4, training data\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n(def iris-iter\n  (new-iris-data-set-iterator\n   :batch-size 1\n   :n-examples 5))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; Step 5, spark mln\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n(def fitted-spark-mln\n  (c/train-with-spark :spark-context your-spark-context\n                      :mln-conf mln-conf\n                      :training-master training-master\n                      :iter iris-iter\n                      :n-epochs 1\n                      :as-code? false))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; Step 5, use spark context from spark-mln to create rdd\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; TODO: eliminate this step\n\n(def our-rdd\n  (let [sc (get-spark-context fitted-spark-mln :as-code? false)]\n    (java-rdd-from-iter :spark-context sc\n                        :iter iris-iter)))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; Step 6, evaluation model and print stats (poor performance of model expected)\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n(def eval-obj\n  (eval-classification-spark-mln\n   :spark-mln fitted-spark-mln\n   :rdd our-rdd))\n\n(println (get-stats :evaler eval-obj))\n\n\n```\n\n- this example demonstrates the dl4j [workflow](https://deeplearning4j.org/spark#Overview)\n  - NOTE: unlike the previous example, this one requires dl4j objects to be used\n    - this is becaues spark only wants you to have one spark context at a time\n\n``` clojure\n(ns my.ns\n  (:require [dl4clj.nn.conf.builders.layers :as l]\n            [dl4clj.nn.conf.builders.nn :as nn]\n            [dl4clj.datasets.iterators :refer [new-iris-data-set-iterator]]\n            [dl4clj.eval.api.eval :refer [get-stats]]\n            [dl4clj.spark.masters.param-avg :as master]\n            [dl4clj.spark.data.java-rdd :refer [new-java-spark-context java-rdd-from-iter]]\n            [dl4clj.spark.dl4j-multi-layer :as spark-mln]\n            [dl4clj.spark.api.dl4j-multi-layer :refer [fit-spark-mln!\n                                                       eval-classification-spark-mln]]))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; Step 1, create your model\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n(def mln-conf\n  (nn/builder\n   :optimization-algo :stochastic-gradient-descent\n   :default-learning-rate 0.006\n   :layers {0 (l/dense-layer-builder :n-in 4 :n-out 2 :activation-fn :relu)\n            1 {:output-layer\n               {:loss-fn :negativeloglikelihood\n                :n-in 2 :n-out 3\n                :activation-fn :soft-max\n                :weight-init :xavier}}}\n   :backprop? true\n   :as-code? false\n   :backprop-type :standard))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; Step 2, create a training master\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n;; not all options specified, but most are\n\n(def training-master\n  (master/new-parameter-averaging-training-master\n   :build? true\n   :rdd-n-examples 10\n   :n-workers 4\n   :averaging-freq 10\n   :batch-size-per-worker 2\n   :export-dir \"resources/spark/master/\"\n   :rdd-training-approach :direct\n   :repartition-data :always\n   :repartition-strategy :balanced\n   :seed 1234\n   :as-code? false\n   :save-updater? true\n   :storage-level :none))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; Step 3, create a Spark Multi Layer Network\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n(def your-spark-context\n  (new-java-spark-context :app-name \"example app\" :as-code? false))\n\n;; new-java-spark-context will turn an existing spark-configuration into a java spark context\n;; or create a new java spark context with master set to \"local[*]\" and the app name\n;; set to :app-name\n\n\n(def spark-mln\n  (spark-mln/new-spark-multi-layer-network\n   :spark-context your-spark-context\n   :mln mln-conf\n   :training-master training-master\n   :as-code? false))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; Step 4, load your data\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n;; one way is via a dataset-iterator\n;; can make one directly from a dataset (iterator data-set)\n;; see: nd4clj.linalg.dataset.api.data-set and nd4clj.linalg.dataset.data-set\n;; we are going to use a pre-built one\n\n(def iris-iter\n  (new-iris-data-set-iterator\n   :batch-size 1\n   :n-examples 5\n   :as-code? false))\n\n;; now lets convert the data into a javaRDD\n\n(def our-rdd\n  (java-rdd-from-iter :spark-context your-spark-context\n                      :iter iris-iter))\n\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n;; Step 5, fit and evaluate the model\n;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n(def fitted-spark-mln\n  (fit-spark-mln!\n   :spark-mln spark-mln\n   :rdd our-rdd\n   :n-epochs 1))\n;; this fn also has the option to supply :path-to-data instead of :rdd\n;; that path should point to a directory containing a number of dataset objects\n\n(def eval-obj\n  (eval-classification-spark-mln\n   :spark-mln fitted-spark-mln\n   :rdd our-rdd))\n;; we would want to have different testing and training rdd's but here we are using\n;; the data we trained on\n\n;; lets get the stats for how our model performed\n\n(println (get-stats :evaler eval-obj))\n\n```\n## Terminology\n\nComing soon\n\n## Packages to come back to:\n\nImplement ComputationGraphs and the classes which use them\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/nn/graph/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/nn/conf/ComputationGraphConfiguration.GraphBuilder.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/nn/conf/ComputationGraphConfiguration.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/nn/conf/graph/package-frame.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/nn/conf/graph/rnn/package-frame.html\u003e\n\nNLP\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/bagofwords/vectorizer/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/datasets/vectorizer/Vectorizer.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/iterator/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/iterator/provider/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/embeddings/WeightLookupTable.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/embeddings/inmemory/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/embeddings/learning/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/embeddings/learning/impl/elements/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/embeddings/learning/impl/sequence/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/embeddings/loader/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/embeddings/reader/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/embeddings/reader/impl/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/embeddings/wordvectors/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/glove/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/glove/count/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/node2vec/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/paragraphvectors/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/sequencevectors/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/sequencevectors/enums/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/sequencevectors/interfaces/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/sequencevectors/iterators/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/sequencevectors/listeners/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/sequencevectors/sequence/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/sequencevectors/serialization/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/sequencevectors/transformers/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/sequencevectors/transformers/impl/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/sequencevectors/transformers/impl/iterables/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/word2vec/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/word2vec/iterator/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/word2vec/wordstore/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/models/word2vec/wordstore/inmemory/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/text/annotator/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/text/corpora/sentiwordnet/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/text/corpora/treeparser/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/text/corpora/treeparser/transformer/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/text/documentiterator/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/text/documentiterator/interoperability/DocumentIteratorConverter.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/text/inputsanitation/InputHomogenization.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/text/invertedindex/InvertedIndex.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/text/labels/LabelsProvider.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/text/movingwindow/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/text/sentenceiterator/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/text/sentenceiterator/interoperability/SentenceIteratorConverter.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/text/sentenceiterator/labelaware/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/text/stopwords/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/text/tokenization/tokenizer/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/text/tokenization/tokenizer/preprocessor/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/text/tokenization/tokenizer/tokenprepreprocessor/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/text/tokenization/tokenizerfactory/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/text/uima/UimaResource.html\u003e\n\nParallelism\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/parallelism/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/parallelism/factory/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/parallelism/main/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/parallelism/parameterserver/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/parallelism/trainer/package-summary.html\u003e\n\nTSNE\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/plot/package-summary.html\u003e\n\nUI\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/activation/PathUpdate.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/api/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/components/chart/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/components/chart/style/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/components/component/ComponentDiv.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/components/component/style/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/components/decorator/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/components/decorator/style/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/components/table/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/components/table/style/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/components/text/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/components/text/style/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/flow/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/flow/beans/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/flow/data/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/i18n/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/module/convolutional/ConvolutionalListenerModule.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/module/defaultModule/DefaultModule.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/module/flow/FlowListenerModule.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/module/histogram/HistogramModule.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/module/remote/RemoteReceiverModule.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/module/train/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/module/tsne/TsneModule.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/nearestneighbors/word2vec/NearestNeighborsQuery.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/play/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/play/misc/FunctionUtil.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/play/staticroutes/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/providers/ObjectMapperProvider.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/renders/PathUpdate.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/standalone/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/stats/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/stats/api/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/stats/impl/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/stats/impl/java/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/stats/sbe/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/storage/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/storage/impl/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/storage/mapdb/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/storage/sqlite/J7FileStatsStorage.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/weights/package-summary.html\u003e\n- \u003chttps://deeplearning4j.org/doc/org/deeplearning4j/ui/weights/beans/CompactModelAndGradient.html\u003e\n\n## License\n\nCopyright © 2016 Engagor\n\nDistributed under the BSD Clause-2 License as distributed in the file LICENSE at the root of this repository.\n\n![](https://thumbs.gfycat.com/IncompleteAncientJanenschia-size_restricted.gif)\n\n*Drops the mic*\n","funding_links":[],"categories":["Clojure","Machine Learning"],"sub_categories":["Tools","[Tools](#tools-1)"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyetanalytics%2Fdl4clj","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyetanalytics%2Fdl4clj","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyetanalytics%2Fdl4clj/lists"}