{"id":19031472,"url":"https://github.com/plandes/clj-ml-model","last_synced_at":"2025-06-11T20:10:56.442Z","repository":{"id":80107483,"uuid":"64147634","full_name":"plandes/clj-ml-model","owner":"plandes","description":"Interface for Machine Learning Modeling","archived":false,"fork":false,"pushed_at":"2018-06-25T02:50:19.000Z","size":232,"stargazers_count":6,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-18T01:48:00.457Z","etag":null,"topics":["clojure","cross-validation","machine-learning","weka"],"latest_commit_sha":null,"homepage":null,"language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/plandes.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-07-25T15:46:56.000Z","updated_at":"2021-09-26T09:49:07.000Z","dependencies_parsed_at":"2023-09-20T18:32:06.296Z","dependency_job_id":null,"html_url":"https://github.com/plandes/clj-ml-model","commit_stats":null,"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plandes%2Fclj-ml-model","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plandes%2Fclj-ml-model/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plandes%2Fclj-ml-model/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plandes%2Fclj-ml-model/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/plandes","download_url":"https://codeload.github.com/plandes/clj-ml-model/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250471858,"owners_count":21436030,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clojure","cross-validation","machine-learning","weka"],"created_at":"2024-11-08T21:23:46.321Z","updated_at":"2025-04-23T16:40:33.149Z","avatar_url":"https://github.com/plandes.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Interface for Machine Learning Modeling, Testing and Training\n\n[![Travis CI Build Status][travis-badge]][travis-link]\n\n  [travis-link]: https://travis-ci.org/plandes/clj-ml-model\n  [travis-badge]: https://travis-ci.org/plandes/clj-ml-model.svg?branch=master\n\nThis repository provides generalized library to train, test and use machine\nlearning models.  Specifically it:\n\n* Wraps [Weka](http://www.cs.waikato.ac.nz/ml/weka/) 3.8.\n* Automation of any combination of classifiers and features.\n* Sort and prints results in many formats and levels of detail.\n* Generate Excel spreadsheet files of multiple run results.\n* Two pass cross validation.\n* Integrates with\n  the [dataset library](https://github.com/plandes/clj-ml-dataset).\n\n\n\u003c!-- markdown-toc start - Don't edit this section. Run M-x markdown-toc-refresh-toc --\u003e\n## Table of Contents\n\n- [Obtaining](#obtaining)\n- [Documentation](#documentation)\n- [Example](#example)\n- [Usage](#usage)\n    - [Create the Corpus](#create-the-corpus)\n    - [Create Features](#create-features)\n    - [Create the Model Configuration](#create-the-model-configuration)\n    - [Create the Model](#create-the-model)\n    - [Evaluating the Model](#evaluating-the-model)\n        - [Creating an ARFF File](#creating-an-arff-file)\n        - [One Pass Train/Test](#one-pass-traintest)\n        - [Cross Validation](#cross-validation)\n        - [Cross Validation Report](#cross-validation-report)\n    - [Persist/Save the Model](#persistsave-the-model)\n    - [Use the Model](#use-the-model)\n    - [Test the Model](#test-the-model)\n    - [Automating testing and overfitting](#automating-testing-and-overfitting)\n- [Building](#building)\n- [Known Bugs](#known-bugs)\n- [Changelog](#changelog)\n- [License](#license)\n\n\u003c!-- markdown-toc end --\u003e\n\n\n## Obtaining\n\nIn your `project.clj` file, add:\n\n[![Clojars Project](https://clojars.org/com.zensols.ml/model/latest-version.svg)](https://clojars.org/com.zensols.ml/model/)\n\n\n## Documentation\n\nAPI [documentation](https://plandes.github.io/clj-ml-model/codox/index.html).\n\n\n## Example\n\nSee the [example repo](https://github.com/plandes/clj-example-nlp-ml) that\nillustrates how to use this library and contains the code from where these\nexamples originate.  It's highly recommended to clone it and follow along as\nyou peruse this README.\n\n\n## Usage\n\nTo create, validate, test and utilize a model you must do the following:\n\n1. [Create the corpus](#create-the-corpus)\n2. [Create features](#create-features)\n3. [Create the model configuration](#create-the-model-configuration)\n4. [Create the model](#create-the-model)\n5. [Evaluating the model](#evaluate-the-model)\n6. [Using the model](#use-the-model)\n7. [Testing the model](#test-the-model)\n8. [Automating testing and overfitting](#automating-testing-and-overfitting)\n\nNote that this example (like `clj-ml-dataset`) uses natural language processing\nbut the library was written to be general purpose and other non-NLP projects\ncan use it.\n\n\n### Create the Corpus\n\nBefore we can do anything, we need a annotated corpus since we'll be using\nsupervised learning methods.  To do that, use the\n[machine learning dataset library](https://github.com/plandes/clj-ml-dataset)\nto pre-parse all utterances in the annotated corpus (follow the readme and\ncreate `zensols.example.anon-db` namesapce).  You'll also need to start a\nDocker instance for the Elasticsearch server as detailed in the docs.\n\n\n### Create Features\n\nFirst we have to generate the features that will be used in our model and train\nour classifier.  We'll generate our features from details that are parsed from\nEnglish utterances for our example so we'll use the\n[NLP library](https://github.com/plandes/clj-nlp-parse) to parse and generate\nthose features from the pre-parsed utterances stored in Elasticsearch using the\n`clj-ml-dataset` library:\n```clojure\n(ns zensols.example.sa-feature\n  (:require [zensols.nlparse.parse :as p]\n            [zensols.nlparse.feature :as fe]\n            [zensols.example.anon-db :as adb]\n            [zensols.model.execute-classifier :refer (with-model-conf)]))\n\n(defn create-features\n  ([panon]\n   (create-features panon nil))\n  ([panon context]\n   (let [tokens (p/tokens panon)]\n     (merge (fe/verb-features (-\u003e\u003e panon :sents first))\n            (fe/token-features panon tokens)\n            (fe/pos-tag-features tokens)\n            (fe/dictionary-features tokens)\n            (fe/tree-features panon)\n            (fe/srl-features tokens)))))\n\n(defn create-feature-sets []\n  (-\u003e\u003e (adb/anons)\n       (map #(merge {:sa (:class-label %)\n                     :utterance (-\u003e\u003e % :annotation :text)}\n                    (create-features (:annotation %))))))\n\n(defn feature-metas []\n  (concat (fe/verb-feature-metas)\n          (fe/token-feature-metas)\n          (fe/pos-tag-feature-metas)\n          (fe/dictionary-feature-metas)\n          (fe/tree-feature-metas)\n          (fe/srl-feature-metas)))\n\n(defn- class-feature-meta []\n  [:sa [\"answer\" \"question\" \"expressive\"]])\n```\n\nIn this example we call `adb/anons` to return the parsed corpus data (see the\nannotation library documentation for how to generate the corpus cache).\n\n\n### Create the Model Configuration\n\nNext we create the model configuration (not the model yet).  The configuration\ngives the framework what needs to create the feature set to generate an\n[weka.core.Instances](http://weka.sourceforge.net/doc.dev/weka/core/Instances.html) used\nby Weka to create, test and utilize the model.\n\n```clojure\n(defn create-model-config []\n  {:name \"speech-act\"\n   :create-feature-sets-fn create-feature-sets\n   :create-features-fn create-features\n   :feature-metas-fn feature-metas\n   :class-feature-meta-fn class-feature-meta\n   :model-return-keys #{:label :distributions :features}})\n```\n\nThe model configuration is a map that refers to functions we already created\nand some other metadata.\n\n\n### Create the Model\n\nNext we define our features and classifiers.\n\nAfter the namesapce declaration we define `feature-sets-set`, which is a\ntwo level hierarchy of features that have the same names as those given in\nthe `feature-metas` [function](#create-features).  The levels are:\n\n1. Feature metadata sets set: list of lists with each list is iterated on while\n   cross-validating to find the feature set to fit the model.\n2. Feature metadata set: the list of features used to create a model for the\n   current feature metadata set iteration.\n\nWe create a `classifiers` binding to store what *genera* of classifiers we want\nto use.  See the [classifiers dynamic binding](https://plandes.github.io/clj-ml-model/codox/zensols.model.weka.html#var-*classifiers*)\nfor more information.\n\n```clojure\n(ns zensols.example.sa-eval\n  (:require [zensols.model.execute-classifier :refer (with-model-conf)]\n            [zensols.model.eval-classifier :as ec])\n  (:require [zensols.example.sa-feature :as sf]))\n\n(defn feature-sets-set []\n  {:set-1 '((token-count))\n   :set-2 '((token-count\n             pos-tag-ratio-verb\n             pos-tag-ratio-adverb\n             pos-tag-ratio-noun\n             pos-tag-ratio-adjective))\n   :set-3 '((token-count stopword-count)\n            (token-count\n             pos-tag-ratio-noun\n             pos-tag-ratio-wh\n             pos-first-tag\n             stopword-count))})\n\n(def classifiers [:zeror :fast])\n```\n\nNext we add an atom to store the `weka.core.Instances` object so we can speed\nup our feature/classifier configuration without having to regenerate feature\nsets for each model testing iteration.\n\n```clojure\n(def cross-fold-instances-inst (atom nil))\n```\n\nFinally we extend the model configuration with the `Instances` atom and our\nfeature metadata sets.\n\n```clojure\n(defn- create-model-config []\n  (merge (sf/create-model-config)\n         {:cross-fold-instances-inst cross-fold-instances-inst\n          :feature-sets-set (feature-sets-set)}))\n```\n\n\n### Evaluating the Model\n\nWhile this step isn't necessary, you'll want to do it to see how well the model\nperforms and optimize it by changing the feature set or swapping and/or\ntweaking the classifier.\n\nTechnically speaking, the actual in memory model is not yet created, but we\nhave now set up everything the framework needs to use it.\n\n\n#### Creating an ARFF File\n\nLet's start by writing out an\n[ARFF](http://www.cs.waikato.ac.nz/ml/weka/arff.html) file:\n```clojure\n(with-model-conf (create-model-config)\n  (ec/write-arff))\n```\n\n\n#### One Pass Train/Test\n\nBy default the system uses [cross validation](#cross-validation).  To train the\nmodel with the training data, then test on the training set you must bind \n[`*default-set-type*`](https://plandes.github.io/clj-ml-model/codox/zensols.model.eval-classifier.html#var-*default-set-type*)\nto `:train-test`.\n\n\n#### Cross Validation\n\nNote that we need to wrap everything in a `with-model-conf`, which is the way\nthe framework receives our model configuration.  In practice you'll wrap more\nthan just one statement and do several things in the lexical context of a\n`with-model-conf`.\n\nNow let's invoke a cross validation and get just an\n[F-measure](https://en.wikipedia.org/wiki/F1_score) score:\n```clojure\n(with-model-conf (create-model-config)\n  (ec/terse-results classifiers :set-3 :only-stats? true))\n```\nThis performs a ten fold cross validation using the using two feature sets:\n\n* `token-count` and `stopword-count`\n* `token-count`, `pos-tag-ratio-noun`, `pos-tag-ratio-wh`, `pos-first-tag`,\n  `stopword-count`\n\nFor both models it tests with the `zeror` and `fast` classifiers.  The first\n`zeror` is a majority rule classifier usually used to generate a baseline to\ngauge relative performance gains.  The `fast` classifier are a group of\nclassifiers that train fast.  See the\n[classifiers dynamic binding](https://plandes.github.io/clj-ml-model/codox/zensols.model.weka.html#var-*classifiers*)\nfor more information on classifier genres.\n\n\n#### Cross Validation Report\n\nYou might have a large dataset and choose to use classifiers that take a long\ntime to train.  This is all multiplied by feature metadata set cardinally,\nwhich drastically compoundes the run time.  In these situations you might want\nto leave it running for a while and generate a spreadsheet report as output.\nThis report contains the feature set, classifiers used, and performance\nmetrics.\n\n```clojure\n(with-model-conf (create-model-config)\n  (ec/eval-and-write classifiers :huge-meta-set))\n```\n\n\n### Persist/Save the Model\n\nOnce you're happy with the performance of your model you can save it and use it\nin the same or different JVM instance.\n\n```clojure\n(with-model-conf (create-model-config)\n  (-\u003e\u003e (ec/create-model classifiers :set-best)\n       ec/train-model\n       ec/write-model))\n```\n\nThis creates a binary model file where you've configured the\n[model output directory](https://plandes.github.io/clj-ml-model/codox/zensols.model.classifier.html#var-model-dir).  More information on how to\nconfigure see the code example in the [example project repository](#example).\n\nThe information encoded in this file includes:\n* The trained classifier\n* The features of the model\n* Performance metrics like F-measure, recall, precision, predictions\n* The context created with the [model configuration's :context-fn](https://plandes.github.io/clj-ml-model/codox/zensols.model.execute-classifier.html#var-with-model-conf) function\n\n\n### Use the Model\n\nFirst let's create a namespace to work with our new model and a function to\ncreate that model:\n\n```clojure\n(ns zensols.example.sa-model\n  (:require [zensols.model.execute-classifier :as exc :refer (with-model-conf)]\n            [zensols.nlparse.parse :as p])\n  (:require [zensols.example.sa-feature :as sf]))\n\n(def model-inst (atom nil))\n\n(defn- model []\n  (swap! model-inst\n         #(or %\n              (with-model-conf (sf/create-model-config)\n                (exc/prime-model (exc/read-model))))))\n```\n\nSince the details of the previous model is encoded in binary you won't be able\nto look at the file to make sense of it.  However, you can output the contents\nof the model (to the REPL and a file respectively) including everything\nmentioned in the [previous section](#persist-model):\n\n```clojure\n(exc/print-model-info (model))\n(exc/dump-model-info (model))\n```\nwhich yields:\n```make\ninstances-total: 382.0\ninstances-correct: 366.0\ninstances-incorrct: 16.0\nname: speech-act\ncreate-time: Mon Jul 25 12:19:24 CDT 2016\naccuracy: 95.81151832460733\nwprecision: 0.9585456987726836\nwrecall: 0.9581151832460733\nwfmeasure: 0.9580599320074707\nfeatures:\n(:token-count :pos-tag-ratio-noun :pos-tag-ratio-wh :pos-first-tag :pos-last-tag :stopword-count)\nclassifier: ...\n```\n\nFinally, we can parse an utterance and use its features to classify our speech\nact:\n```clojure\n(-\u003e\u003e (p/parse \"when are we getting there\")\n     (exc/classify (model))\n     pprint)\n```\nwhich yeilds:\n```clojure\n{:features-set\n {:pos-last-tag \"RB\",\n  ...\n  :pos-tag-ratio-wh 0},\n :label \"question\",\n :distributions\n {\"answer\" 0.033959200182016515,\n   \"question\" 0.9306213420827363,\n   \"expressive\" 0.03541945773524721}}\n```\n\nThis gives us all the results we asked for in the `:model-return-keys` of our\n`create-model-config` function in our\n[feature namespace](#create-the-model-configuration), which is:\n* **:features** the single instance features given to the model for classification,\n  which for us was generate by the `create-features` [function](#create-features).\n* **:label** is the class label for our classification, which in the case is\n  correct for the utterance *when are we getting there*.\n* **:distributions** is the probability distribution over the class label,\n  which in our case is pretty uneven suggesting a high degree of confidence\n\nThe features are availble since there are situations where you might want to do\nsomething with a feature after classification.  More specifically you could\neven generate features as data not used by the model like an ID for an NER tag.\n\nThe probability distribution could be handy for cases where you don't want the\nfirst choice or would like to use the distribution itself as a feature in\nanother model or get an idea of specific classifications' performance.\n\nNow we can create a client friendly (to our new library) function:\n```clojure\n(defn classify-utterance [utterance]\n  (-\u003e\u003e (p/parse utterance)\n       (exc/classify (model) anon)\n       :label))\n```\n\n\n### Test the Model\n\nIn our [example](#use-the-model) we performed with a weighted F-measure\nof 0.96, which seems pretty unbelievable.  Another way to confirm we have a\ngood model is to divide the dataset into a training and test set.  For this\nexample, let's split it right down the middle and retrain:\n```clojure\nuser=\u003e (adb/divide-by-set 0.5)\nuser=\u003e (reset! instance-inst nil) ; invalidate the instances cache\nuser=\u003e (with-model-conf (create-model-config)\n\t\t (-\u003e\u003e (ec/create-model classifiers :set-best)\n\t\t\t  ec/train-model\n\t\t\t  ec/write-model))\nuser=\u003e (reset! model-inst nil) ; invalidate the model\nuser=\u003e (exc/print-model-info)\n```\nyeilds:\n```make\ninstances-total: 239.0\ninstances-correct: 225.0\ninstances-incorrct: 14.0\nname: speech-act\ncreate-time: Mon Jul 25 12:51:18 CDT 2016\naccuracy: 94.14225941422595\nwprecision: 0.9413853837630445\nwrecall: 0.9414225941422594\nwfmeasure: 0.9413703416300443\n```\nwhich is still very good and still hard to believe how well it performs.\nHowever, now we have a better way to prove the model, which is to run it on\ndata we left out, which is the training data.  We'll code to invoke the model\nclassifier on the test data:\n```clojure\n(ns zensols.example.sa-model\n  (:require [clj-excel.core :as excel])\n  (:require [zensols.actioncli.dynamic :refer (dyn-init-var) :as dyn]\n            [zensols.actioncli.log4j2 :as lu]\n            [zensols.actioncli.resource :as res]\n            [zensols.util.spreadsheet :as ss]\n            [zensols.model.execute-classifier :as exc :refer (with-model-conf)]\n            [zensols.example.anon-db :as adb]))\n\n(def preds-inst (atom nil))\n\n(defn- test-annotation [anon-rec]\n  (let [{anon :instance label :class-label} anon-rec\n        sent (:text anon)\n        pred (classify-utterance sent)]\n    (log/debugf \"label: %s, prediction: %s\" label pred)\n    {:label label\n     :sent sent\n     :prediction pred\n     :correct? (= label pred)}))\n\n(defn- predict-test-set []\n  (swap! preds-inst\n         #(or %\n              (let [anons (adb/anons :set-type :test)\n                    results (map test-annotation anons)\n                    preds (map :correct? results)]\n                {:correct (filter true? preds)\n                 :incorrect (filter false? preds)\n                 :predictions preds\n                 :results results}))))\n\n(defn- create-prediction-report []\n  (letfn [(data-sheet [anons]\n            (-\u003e\u003e anons\n                 (map (fn [anon]\n                        [(:class-label anon) (-\u003e\u003e anon :instance :text)]))\n                 (cons [\"Label\" \"Utterance\"])))]\n   (let [out-file (res/resource-path :analysis-report \"sa-predictions.xls\")]\n     (-\u003e (excel/build-workbook\n          (excel/workbook-hssf)\n          {\"Predictions on test data\"\n           (-\u003e\u003e (predict-test-set)\n                :results\n                (map (fn [res]\n                       (let [{:keys [label sent prediction correct?]} res]\n                         [correct? label prediction sent])))\n                (cons [\"Is Correct\" \"Gold Label\" \"Prediction\" \"Utterance\"])\n                (ss/headerize))\n           \"Training\" (data-sheet (adb/anons))\n           \"Test\" (data-sheet (adb/anons :set-type :test))})\n         (ss/autosize-columns)\n         (excel/save out-file)))))\n\n(create-prediction-report)\n```\nInvoking this code creates a report on the desktop with the training data and\nits predictions on the first sheet and the dataset by its set type (training\nand testing) on the second two tabs.  We'll still correctly classify 230 of the\n238 giving a 96% accuracy.\n\n\n### Automating testing and overfitting\n\nThere is an easier way to test and train our model by using the\n`clj-ml-dataset` library, but first we have to make a few changes.  The\n`create-feature-sets` function we wrote earlier needs to take a test/train\nratio parameter so the data set library can re-create the training and testing\nsets:\n```clojure\n(defn create-feature-sets [\u0026 adb-keys]\n  (-\u003e\u003e (apply adb/anons adb-keys)\n       (map #(merge {:sa (:class-label %)\n                     :utterance (-\u003e\u003e % :instance :text)}\n                    (create-features (:instance %))))))\n```\nThe `adb-keys` are the keys that eventually get passed to the\n[instances](https://plandes.github.io/clj-ml-dataset/codox/zensols.dataset.db.html#var-instances) function.\n\nIn our evaluation code we need to create a new atom to cache the results of the\ntesting and training instances:\n```clojure\n(dyn-init-var *ns* 'test-train-instances-inst (atom nil))\n```\n\nThis atom needs to be added to the model configuration.  We also need to tell\nthe framework how to repartition the training and testing data sets and clear\nthe train/test atom that caches the instances:\n```clojure\n(defn- create-model-config []\n  (letfn [(divide-by-set [divide-ratio]\n            (adb/divide-by-set divide-ratio :shuffle? false)\n            (reset! test-train-instances-inst nil))]\n   (merge (sf/create-model-config)\n          {:cross-fold-instances-inst cross-fold-instances-inst\n           :test-train-instances-inst test-train-instances-inst\n           :feature-sets-set (feature-sets-set)\n           :divide-by-set divide-by-set})))\n```\nThe `divide-by-set` function defined above creates a new *division* of testing\nand training data and in our case will incrementall move instances from the\ntraining data to the testing data.  With the `shuffle? false` we do *not*\nshuffle the data set before making the new split so we are effectively\nre-partitioning by moving the train/test data set demarcation line.\n\nNow we're ready to call the framework to train the classifier on the training\ninstances and then test the trained classifier on the test instances:\n```clojure\n(binding [cl/*rand-fn* (fn [] (java.util.Random. 1))]\n  (with-model-conf (create-model-config)\n    (-\u003e\u003e (ec/train-test-series\n          [:j48] :set-best {:start 0.1 :stop 1 :step 0.05})\n         ec/write-csv-train-test-series)))\n```\nIn this example, the `cl/*rand-fn*` tells the framework to use `1` as the seed\nso the ordering of the instances across training/testing data is always the\nsame, which means if running the same tests (including cross validation)\ndoesn't change our outcomes.\n\nThe `ec/write-csv-train-test-series` writes the result outcomes to a CSV file,\nwhich we can then use to find the *elbow* or point where we start to *overfit*\nthe model.  The `R` code to do this and the results are in the\n[example project repository](#example).  This code creates the following graph:\n![Overfitting Example](https://plandes.github.io/clj-example-nlp-ml/results/speech-act-J48-train-test-series.svg)\n\nIn the graph we see the we have just below 0.4 F-measuer for 48 training\ninstances, it then ballons to above 0.9 at 72 instances so the classifier (J48\ndecision tree for this example) learns quickly.  However we see the first drop\nat 120 training instances (the red portion), which is mentioned *elbow* where\nwe typically see the classifier start to overtrain.\n\n\n## Building\n\nTo build from source, do the folling:\n\n- Install [Leiningen](http://leiningen.org) (this is just a script)\n- Install [GNU make](https://www.gnu.org/software/make/)\n- Install [Git](https://git-scm.com)\n- Download the source: `git clone https://github.com/clj-mkproj \u0026\u0026 cd clj-mkproj`\n- Download the make include files:\n```bash\nmkdir ../clj-zenbuild \u0026\u0026 wget -O - https://api.github.com/repos/plandes/clj-zenbuild/tarball | tar zxfv - -C ../clj-zenbuild --strip-components 1\n```\n- Build the distribution binaries: `make dist`\n\nNote that you can also build a single jar file with all the dependencies with: `make uber`\n\n\n## Known Bugs\n\nI am suspicious there is a bug with the *two pass* functionality as I've\nrecently worked with a data set that gave very different performance results\nusing a test/train split.  While I'm not sure, I suspect the bug is somewhere\nin the Clojure -\u003e Weka -\u003e Clojure flow.  Currently, two pass works by\noverloading the `Instances` class and another that uses the overridden class,\nand subsequently doesn't copy that class.  This is invoked from the\n`zensols.model.weka/clone-instances` function that uses the model framework to\ncreate separate train and test folds.\n\nI've been over and over this code and can't find the bug.  There are any Weka\nwizards out there that have some time and can help out, I'd **really**\nappreciate it!\n\n\n## Changelog\n\nAn extensive changelog is available [here](CHANGELOG.md).\n\n\n## License\n\nMIT License\n\nCopyright (c) 2016 - 2018 Paul Landes\n\nPermission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplandes%2Fclj-ml-model","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fplandes%2Fclj-ml-model","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplandes%2Fclj-ml-model/lists"}