{"id":19855446,"url":"https://github.com/gtesei/fast-furious","last_synced_at":"2025-05-02T01:31:01.706Z","repository":{"id":13020163,"uuid":"15699668","full_name":"gtesei/fast-furious","owner":"gtesei","description":"code (R, Matlab/Octave), models and meta-models I needed in my Machine Learning Lab but I didn't found on the shelf","archived":false,"fork":false,"pushed_at":"2019-04-12T01:57:49.000Z","size":168473,"stargazers_count":18,"open_issues_count":0,"forks_count":3,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-04-06T20:47:24.364Z","etag":null,"topics":["ensemble-learning","gradient-boosting-machine","matlab","metamodel","model","neural-network","octave","r","svm"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gtesei.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-01-07T09:00:15.000Z","updated_at":"2024-03-10T17:20:17.000Z","dependencies_parsed_at":"2022-08-28T06:11:11.966Z","dependency_job_id":null,"html_url":"https://github.com/gtesei/fast-furious","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gtesei%2Ffast-furious","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gtesei%2Ffast-furious/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gtesei%2Ffast-furious/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gtesei%2Ffast-furious/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gtesei","download_url":"https://codeload.github.com/gtesei/fast-furious/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251969340,"owners_count":21673188,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ensemble-learning","gradient-boosting-machine","matlab","metamodel","model","neural-network","octave","r","svm"],"created_at":"2024-11-12T14:12:49.241Z","updated_at":"2025-05-02T01:30:56.696Z","avatar_url":"https://github.com/gtesei.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# fast-furious\n\n[![Build Status](https://api.travis-ci.org/gtesei/fast-furious.svg?branch=master)](https://travis-ci.org/gtesei/fast-furious)\n[![Coverage Status](https://coveralls.io/repos/gtesei/fast-furious/badge.svg?branch=master\u0026service=github)](https://coveralls.io/github/gtesei/fast-furious?branch=master)\n[![License](http://img.shields.io/:license-mit-blue.svg?style=flat)](http://badges.mit-license.org)\n[![Badges](http://img.shields.io/:badges-4/4-ff6799.svg)](https://github.com/badges/badgerbadgerbadger)\n\n“*We should forget about small efficiencies, say about 97% of the time:\n__premature optimization is the root of all evil__. Yet we should not pass\nup our opportunities in that critical 3%. A good programmer will\nnot be lulled into complacency by such reasoning, he will be wise to\nlook carefully at the critical code; but only after that code has been\nidentified.*” - Donald Knuth.\n\n## 1. What is it?\n  fast-furiuos gathers code (**R, Matlab/Octave**), models and meta-models I needed in my Machine Learning Lab but I didn't found on the shelf.\n  \n## 2. Requirements, installation and how to use fast-furious in your scripts \nfast-furious has been built in interpretable languages like R, Matlab/Octave, Python (hence, it does not require compilation) and **(Mac) OSX**, **Windows**, **Linux** are **fully supported**. \n\n### 2.1 Requirements\n  * [Octave](http://www.gnu.org/software/octave/download.html) or Matlab is **mandatory** for fast-furious model implementations (*regularized neural networks, regularized linear and polynomial regression, regularized logistic regression*). If you are using only these fast-furious models Octave or Matlab installed on your machine is the only requirement. Currently, I am working on matlab compatibility issues. \n  * [R](http://www.r-project.org/) is **mandatory** for data process, feature engineering, model selection and model ensembling.\n  \n### 2.2 Installation  \n  Installation is pretty easy and quick. You can choose\n  * to download the zip in the directory you like as **fast-furious base dir** and unzip  \n  * or to use ```git``` in the directory you like as **fast-furious base dir** \n  \n  ```\n  git clone https://github.com/gtesei/fast-furious.git\n  ```\n\n### 2.3 Installing only fast-furious R-Package \nR-Package installation is pretty easy and fast from github by using ```devtools::install_github```. Windows user will need to install [RTools](http://cran.r-project.org/bin/windows/Rtools/) first.\n\n```r\ndevtools::install_github('gtesei/fast-furious',subdir='R-package')\n```\n\n### 2.4 How to use fast-furious in your Octave/Matlab scripts  \nAssuming you are launching your Octave/Matlab script in fast-furious base dir, you just need to call at the begin of your script the fast-furious \n```menv``` function to set up the enviroment. Typically, your script should look like this \n\n```matlab\n%% setting enviroment \nmenv;\n\n... here your stuff ...\n```\nFor example, this is the code of fast-furious ```GO_Neural.m``` script located on fast-furious base dir: \n```matlab\n%% setting enviroment \nmenv;\n\n%% load use cases and go  \nREADME_Neural;\ngo();\n```\n\n### 2.5 How to use fast-furious in your R scripts  \nOnce installed, you just need to load the package by using the R ```library``` function. E.g. this is the code sketch for tuning, training, predicting and ensembling an XGBoost model on a binary classification problem. \n```r\nlibrary(fastfurious)\n\n##########################\n## TUNE / TRAIN / PREDICT \n##########################\ncontrolObject = caret::trainControl(method = \"repeatedcv\", repeats = 1, number = 4 , summaryFunction = twoClassSummary , classProbs = TRUE)\nl = ff.trainAndPredict.class (Ytrain=Ytrain ,\n                              Xtrain=Xtrain , \n                              Xtest=Xtest , \n                              model.label=\"xgbTree\" , \n                              controlObject=controlObject, \n                              best.tuning = TRUE, \n                              verbose = TRUE, \n                              removePredictorsMakingIllConditionedSquareMatrix_forLinearModels = F, \n                              xgb.metric.fun = NULL, \n                              xgb.maximize = TRUE, \n                              metric.label = 'auc', \n                              xgb.foldList = NULL,\n                              xgb.eta = 0.02, \n                              xgb.max_depth = 8, \n                              xgb.cv.default = FALSE)\n                              \nAUC.xval = max(l$model$results$ROC)\nbestTune = l$model$bestTune\npred = l$pred\npred.prob = l$pred.prob\nsecs = l$secs \n                                 \n##########################\n## ENSEMB \n##########################\nindex = caret::createMultiFolds(y=Ytrain, controlObject$number, controlObject$repeats)\nindexOut \u003c- lapply(index, function(training, allSamples) allSamples[-unique(training)], allSamples = seq(along = Ytrain))\ncontrolObject = trainControl(method = \"repeatedcv\", \n                               ## The method doesn't really matter\n                               ## since we defined the resamples\n                               index = index, \n                               indexOut = indexOut , \n                               summaryFunction = twoClassSummary , classProbs = TRUE)\n                               \nens = ff.createEnsemble(Xtrain = Xtrain,\n                        Xtest = Xtest,\n                        y = Ytrain,\n                        caretModelName = 'xgbTree', \n                        predTest = pred.prob,\n                        bestTune = expand.grid(\n                          nrounds = bestTune$early.stop ,\n                          max_depth = 8 ,  \n                          eta = 0.02 ),\n                        removePredictorsMakingIllConditionedSquareMatrix_forLinearModels = F, \n                        controlObject = controlObject, \n                        parallelize = TRUE,\n                        verbose = TRUE , \n                        regression = FALSE, \n                             \n                        ### ... \n                        objective = \"binary:logistic\",\n                        eval_metric = \"auc\", \n                        subsample = 0.7 , \n                        colsample_bytree = 0.6 , \n                        scale_pos_weight = 0.8 , \n                        max_delta_step = 2)\n                          \nensemble_pred_train = ens$predTrain\nensemble_pred_test = ens$predTest\n```\n\n## 3. fast-furious model implementations \n### 3.1 Regularized Neural Networks \nPackage ```neural``` **very fast 100% vectorized implementation of backpropagation** in Matlab/Octave.\n\n * for **basic use cases** just run command line (fast-furious base dir) \n \n    ```\u003eoctave GO_Neural.m```\n    \n * for **binary classification problems** use ```nnCostFunction``` cost function wrapped in ```trainNeuralNetwork```. E.g. this is the code for fitting a neural neural network with 400 neurons at input layer, 25 neurons at hidden layer, 1 neuron (= binary classification) at output layer, 0.001 as regularization parameter, where trainset/testset has been already scaled and with the bias term added.\n    ```matlab\n    % y must a 01 vector (e.g. [1 0 1 0 0 0 0 0 1 1 0 1] )\n    % train_data and test_data are the train set and test set \n    \n    %% 400 neurons at input layer\n    %% 25 neurons at hidden layer\n    %% 1 neuron at output layer  \n    NNMeta = buildNNMeta([400 25 1]); \n    \n    %% regularization parameter \n    lambda = 0.001; \n    \n    %% train on train set \n    [Theta] = trainNeuralNetwork(NNMeta, Xtrain, ytrain, lambda , iter = 100, featureScaled = 1); \n    \n    %% predict on train set \n    probs_train = NNPredictMulticlass(NNMeta, Theta , Xtrain , featureScaled = 1);\n    pred_train = (probs_train \u003e 0.5);\n    \n    %% predict on test set \n    probs_test = NNPredictMulticlass(NNMeta, Theta , Xtest , featureScaled = 1);\n    pred_test = (probs_test \u003e 0.5);\n    \n    %% measure accuracy \n    acc_train = mean(double(pred_train == ytrain)) * 100;\n    acc_test = mean(double(pred_test == ytest)) * 100;\n    ```\n * for **tuning parameters on classification problems** (number of neurons per layer, number of hidden layers, regularization parameter) by cross-validation use the ```findOptPAndHAndLambda``` function. E.g. this is the code for finding the best number of neurons per layer (p_opt_acc), the best number of hidden layers (h_opt_acc), the best regularization parameter (lambda_opt_acc), using cross validation on a binary classification problem with accuracy as metric on a train set (80% of data) and cross validation set (20% of data) not scaled.\n    ```matlab\n    \n    % y must a 01 vector (e.g. [1 0 1 0 0 0 0 0 1 1 0 1] )\n    % train_data and test_data are the train set and test set \n    \n    %% scale and add bias term \n    [train_data,mu,sigma] = treatContFeatures(train_data,1);\n    [test_data,mu,sigma] = treatContFeatures(test_data,1,1,mu,sigma);\n    \n    %% split and randomize \n    [Xtrain,ytrain,Xval,yval] = splitTrainValidation(train_data,ytrain,0.80,shuffle=1);\n\n    %% tuning parameters \n    [p_opt_acc,h_opt_acc,lambda_opt_acc,acc_opt,tuning_grid] = findOptPAndHAndLambda(Xtrain, ytrain, Xval, yval, ...\n  \t\t\t\tfeatureScaled = 1 , \n\t\t\t\t\th_vec = [1 2 3 4 5 6 7 8 9 10] , ...\n\t\t\t\t\tlambda_vec = [0 0.001 0.003 0.01 0.03 0.1 0.3 1 3 10] , ...\n\t\t\t\t\tverbose = 1, doPlot=1 , ...\n\t\t\t\t\titer = 200 , ...\n\t\t\t\t\tregression = 0 , num_labels = 1 );\n                      \n    %% train on full train set \n    NNMeta = buildNNMeta([(size(train_data,2)-1) (ones(h_opt_acc,1) .* p_opt_acc)' 1]');\n    [Theta] = trainNeuralNetwork(NNMeta, train_data, ytrain, lambda_opt_acc , iter = 2000, featureScaled = 1);\n  \n    %% predict on train set \n    probs_train = NNPredictMulticlass(NNMeta, Theta , train_data , featureScaled = 1);\n    pred_train = (probs_train \u003e 0.5);\n    acc_train = mean(double(pred_train == ytrain)) * 100;\n\n    %% predict on test set \n    probs_test = NNPredictMulticlass(NNMeta, Theta , test_data , featureScaled = 1); \n    pred_test = (probs_test \u003e 0.5);\n    ```\n* for **multiclass classification problems** use ```nnCostFunction``` cost function wrapped in ```trainNeuralNetwork``` as well. E.g. this is the code for fitting a neural neural network with 400 neurons at input layer, 25 neurons at hidden layer, 5 neurons (= 5 class classification problem) at output layer, 0.001 as regularization parameter, where trainset/testset has been already scaled and with the bias term added.\n    ```matlab\n    % y must be 1-based and, in this case a 12345 vector, (e.g. [1 2 5 4 3 2 3 4 5 2 3 4 1 2 3 4 5] )\n    % train_data and test_data are the train set and test set \n    \n    %% 400 neurons at input layer\n    %% 25 neurons at hidden layer\n    %% 1 neuron at output layer  \n    NNMeta = buildNNMeta([400 25 1]); \n    \n    %% regularization parameter \n    lambda = 0.001; \n    \n    %% train on train set \n    [Theta] = trainNeuralNetwork(NNMeta, Xtrain, ytrain, lambda , iter = 100, featureScaled = 1); \n    \n    %% predict on train set \n    probs_train = NNPredictMulticlass(NNMeta, Theta , Xtrain , featureScaled = 1);\n    pred_train = (probs_train \u003e 0.5);\n    \n    %% predict on test set \n    probs_test = NNPredictMulticlass(NNMeta, Theta , Xtest , featureScaled = 1);\n    \n    %% measure accuracy \n    acc_train = mean(double(pred_train == ytrain)) * 100;\n    acc_test = mean(double(pred_test == ytest)) * 100;\n    ```\n * for **regression problems** use ```nnCostFunctionReg``` cost function wrapped in ```trainNeuralNetworkReg```. E.g. this is the code for fitting a neural neural network with 400 neurons at input layer, 25 neurons at hidden layer, 1 neuron at output layer, 0.001 as regularization parameter, where trainset/testset has been already scaled and with the bias term added.\n    ```matlab\n    %% 400 neurons at input layer\n    %% 25 neurons at hidden layer\n    %% 1 neuron at output layer  \n    NNMeta = buildNNMeta([400 25 1]); \n    \n    %% regularization parameter \n    lambda = 0.001; \n    \n    %% train on train set \n    [Theta] = trainNeuralNetworkReg(NNMeta, Xtrain, ytrain, lambda , iter = 200, featureScaled = 1);\n    \n    %% predict on train set \n    pred_train = NNPredictReg(NNMeta, Theta , Xtrain , featureScaled = 1);\n    \n    %% predict on test set \n    pred_test = NNPredictReg(NNMeta, Theta , Xtest , featureScaled = 1);\n    \n    %% measure RMSE \n    RMSE_train = sqrt(MSE(pred_train, ytrain));\n    RMSE_test = sqrt(MSE(pred_test, ytest));\n    ```\n * for **tuning parameters on regression problems** (number of neurons per layer, number of hidden layers, regularization parameter) by cross-validation use the ```findOptPAndHAndLambda``` function. E.g. this is the code for finding the best number of neurons per layer (p_opt_rmse), the best number of hidden layers (h_opt_rmse), the best regularization parameter (lambda_opt_rmse), using cross validation on a regression problem with RMSE as metric on a train set (80% of data) and cross validation set (20% of data) not scaled.\n    ```matlab\n    %% scale and add bias term \n    [train_data,mu,sigma] = treatContFeatures(train_data,1);\n    [test_data,mu,sigma] = treatContFeatures(test_data,1,1,mu,sigma);\n    \n    %% split and randomize \n    [Xtrain,ytrain,Xval,yval] = splitTrainValidation(train_data,ytrain,0.80,shuffle=1);\n\n    %% tuning parameters \n    [p_opt_rmse,h_opt_rmse,lambda_opt_rmse,rmse_opt,tuning_grid] = findOptPAndHAndLambda(Xtrain, ytrain, Xval, yval, ...\n    \t\t\tfeatureScaled = 1 , \n\t\t\t\t\th_vec = [1 2 3 4 5 6 7 8 9 10] , ...\n\t\t\t\t\tlambda_vec = [0 0.001 0.003 0.01 0.03 0.1 0.3 1 3 10] , ...\n\t\t\t\t\tverbose = 1, doPlot=1 , ...\n\t\t\t\t\titer = 200 , ...\n\t\t\t\t\tregression = 1 );\n                      \n    %% train on full train set \n    NNMeta = buildNNMeta([(size(train_data,2)-1) (ones(h_opt_rmse,1) .* p_opt_rmse)' 1]');\n    [Theta] = trainNeuralNetworkReg(NNMeta, train_data, ytrain, lambda_opt_rmse , iter = 2000, featureScaled = 1);\n  \n    %% predict on train set \n    pred_train = NNPredictReg(NNMeta, Theta , Xtrain , featureScaled = 1);\n    RMSE_train = sqrt(MSE(pred_train, ytrain));\n\n    %% predict on test set \n    pred_test = NNPredictReg(NNMeta, Theta , Xtest , featureScaled = 1);\n    RMSE_test = sqrt(MSE(pred_test, ytest));\n    ```\n * for **large datasets** (e.g. **80GB train set on a machine with 8GB RAM**) use ```nnCostFunction_Buff``` (wrapped in ```trainNeuralNetwork_Buff```) that is a **buffered implementation of batch gradient descent**, i.e. it uses all train observations in each iteration vs. one observation as **stochastic gradient descent** or k (k \u003c number of observations on trainset) observations in each iteration as **mini-batch gradient descent**. E.g. this is the code for for fitting a neural neural network with 400 neurons at input layer, 25 neurons at hidden layer, 1 neuron (= binary classification) at output layer, 0.001 as regularization parameter, from file  ```foXtrain ``` for predictors (columns from  ```ciX ``` to  ```ceX ```), and from file  ```fytrain ``` for labels (columns form  ```ciy ``` to  ```cey ```) and buffer equals to 10000 observations (= you load in memory 10000 observations each time).      \n    ```matlab\n    %% 400 neurons at input layer\n    %% 25 neurons at hidden layer\n    %% 1 neuron at output layer  \n    NNMeta = buildNNMeta([400 25 1]); \n    \n    %% regularization parameter \n    lambda = 0.001; \n  \n    %% train (buffer = 10000 observations) \n    %% from file \u003cfoXtrain\u003e (columns from \u003cciX\u003e to \u003cceX\u003e) as train data\n    %% from file \u003cfytrain\u003e (columns form \u003cciy\u003e to \u003ccey\u003e) as labels \n    [Theta_Buff] = trainNeuralNetwork_Buff(NNMeta,foXtrain,ciX,ceX, ... \n                            fytrain,ciy,cey, ... \n                            sep=',',b=10000, ... \n                            lambda, iter = 50 , ... \n                            featureScaled = 0 , ... \n                            initialTheta = cell(0,0) );\n    \n    %% predict (buffer = 10000 observations) on train set \n    pred_val_bf = NNPredictMulticlass_Buff(NNMeta,foXval,ciX,ceX,Theta_Buff,10000,',',0);\n    \n    %% predict (buffer = 10000 observations) on test set \n    pred_train_bf = NNPredictMulticlass_Buff(NNMeta,foXtrain,ciX,ceX,Theta_Buff,10000,',',0);\n    ```\n * for **Neural Networks with EGS (= Extended Generalized Shuffle) interconnection pattern among layers** in regression problesm use ```nnCostFunctionRegEGS``` cost function wrapped in ```trainNeuralNetworkRegEGS``` function. E.g. this is the code for fitting a neural neural network with 400 neurons at input layer, 25 neurons at hidden layer, 1 neuron (= binary classification) at output layer, 0.001 as regularization parameter, where trainset/testset has been already scaled and with the bias term added. \n    ```matlab\n    %% 400 neurons at input layer\n    %% 25 neurons at hidden layer\n    %% 1 neuron at output layer  \n    NNMeta = buildNNMeta([400 25 1]); \n    \n    %% regularization parameter \n    lambda = 0.001; \n    \n    %% train \n    [Theta] = trainNeuralNetworkRegEGS(NNMeta, Xtrain, ytrain, lambda , iter = 300, featureScaled = 1 );\n    \n    %% predict on train/test set \n    pred_train = NNPredictRegEGS(NNMeta, Theta , Xtrain , featureScaled = 1);\n    pred_test = NNPredictRegEGS(NNMeta, Theta , Xtest , featureScaled = 1);\n    \n    %% measure MSE on train/test predictions \n    MSE_train = MSE(pred_train, ytrain);\n    MSE_test = MSE(pred_test, ytest);\n    ```\n    \n### 3.2 Regularized Linear and Polynomial Regression \nPackage ```linear_reg``` **very fast 100% vectorized implementation** in Matlab/Octave\n * for **basic use cases** just run command line (fast-furious base dir) \n    \n    ```\u003eoctave GO_LinearReg.m```\n * for a **performance comparison** (=RMSE) among **(fast-furiuos) Regularized Polynomial Regression**, **(libsvm) epsilon-SVR**, **(libsvm) nu-SVR**, **(fast-furiuos) Neural Networks** on dataset *solubility* of [AppliedPredictiveModeling](http://appliedpredictivemodeling.com/) run command line \n    \n    ```\u003eoctave linear_reg/____testRegression.m```\n\n * for fitting a **linear regression** model use ```linearRegCostFunction``` wrapped in  ```trainLinearReg``` function. E.g. this is the code for fitting a regularized liner regression model with trainset/testset not scaled and with regularization parameter set to 0.001.   \n    ```matlab\n    %% feature scaling (trainset/testset) \n    [Xtrain,mu,sigma] = treatContFeatures(Xtrain,p = 1);\n    [Xtest,mu,sigma] = treatContFeatures(Xtest,p = 1,1,mu,sigma);\n    \n    %% regularization parameter \n    lambda = 0.001;\n    \n    %% train \n    [theta] = trainLinearReg(Xtrain, ytrain, lambda , 200 );\n    \n    %% predict\n    pred_train =predictLinearReg(Xtrain,theta);\n    pred_test = predictLinearReg(Xtest,theta);\n    \n    %% measure MSE\n    mse_train = MSE(pred_train, ytrain);\n    mse_test = MSE(pred_test, ytest);\n    ```\n * for fitting a **linear regression** model using **the normal equation** instead of **batch gradient descent** use the ```normalEqn_RegLin``` function. **I recommend not to use the normal equation for large datasets**. E.g. this is the code for fitting a regularized liner regression model using **the normal equation** with trainset/testset not scaled and with regularization parameter set to 0.001. \n    ```matlab\n    %% feature scaling (trainset/testset) \n    [Xtrain,mu,sigma] = treatContFeatures(Xtrain,p = 1);\n    [Xtest,mu,sigma] = treatContFeatures(Xtest,p = 1,1,mu,sigma);\n    \n    %% regularization parameter \n    lambda = 0.001;\n    \n    %% train \n    [theta] = normalEqn_RegLin(Xtrain,ytrain,lambda);\n    \n    %% predict \n    pred_train = predictLinearReg(Xtrain,theta);\n    pred_test = predictLinearReg(Xtest,theta);\n    \n    %% measure performance \n    mse_train = MSE(pred_train, ytrain);\n    mse_test = MSE(pred_test, ytest);\n    ```\n * for fitting a **polynomial regression** model use ```linearRegCostFunction``` as well. Just set up the degree of the polynomial trasformation you like in the ```treatContFeatures``` function. E.g. this is the code for fitting a regularized liner regression model with trainset/testset not scaled and with regularization parameter set to 0.001 and **polynomial degree 5**.   \n    ```matlab\n    %% feature scaling (trainset/testset) \n    [Xtrain,mu,sigma] = treatContFeatures(Xtrain,p = 5);\n    [Xtest,mu,sigma] = treatContFeatures(Xtest,p = 5,1,mu,sigma);\n    \n    %% regularization parameter \n    lambda = 0.001;\n    \n    %% train \n    [theta] = trainLinearReg(Xtrain, ytrain, lambda , 200 );\n    \n    %% predict\n    pred_train =predictLinearReg(Xtrain,theta);\n    pred_test = predictLinearReg(Xtest,theta);\n    \n    %% measure MSE\n    mse_train = MSE(pred_train, ytrain);\n    mse_test = MSE(pred_test, ytest);\n    ```\n * for **tuning parameters (on regression problems)** (degree of polynomial trasformation, regularization parameter) by cross-validation use the ```findOptPAndLambdaRegLin``` function. E.g. this is the code for finding the best degree of polynomial trasformation (p_opt_RMSE), the best regularization parameter (lambda_opt_RMSE), using cross validation on a regression problem with RMSE as metric on a train set and test set already scaled.\n \n    ```matlab\n    [p_opt_RMSE,lambda_opt_RMSE,RMSE_opt,grid]  = ... \n          findOptPAndLambdaRegLin(solTrainX, solTrainY, solTestX, solTestY, ...\n            p_vec = [1 2 3 4 5 6 7 8 9 10 12 20]' , ...\n            lambda_vec = [0 0.001 0.003 0.01 0.03 0.1 0.3 1 3 10]' , ...\n            verbose = 1, initGrid = [] , initStart = -1 , iter=1000);\n            \n    printf('\u003e\u003e\u003e\u003e\u003e found min RMSE=%f  with p=%i and lambda=%f \\n', RMSE_opt , p_opt_RMSE , lambda_opt_RMSE );\n    ```\n * for **large datasets** (e.g. **80GB train set on a machine with 8GB RAM**) you can use the ```trainLinearReg_MiniBatch``` function that is a **mini-batch gradient descent** implementation, i.e. it uses k observations (k \u003c number of observations on trainset) in each iteration. E.g. this is the code for for fitting a linear regression model with 0.001 as regularization parameter, from file  ```foXtrain ``` for predictors (columns from  ```ciX ``` to  ```ceX ```), and from file  ```fytrain ``` for labels (columns form  ```ciy ``` to  ```cey ```) and buffer equals to 100 observations (= you load in memory 100 observations each time **and you use only these for complete a gradient descent iteration**).\n \n    ```matlab\n    %% regularization parameter \n    lambda = 0.001; \n  \n    %% train (buffer = 100 observations) \n    %% from file \u003cfoXtrain\u003e (columns from \u003cciX\u003e to \u003cceX\u003e) as train data\n    %% from file \u003cfytrain\u003e (columns form \u003cciy\u003e to \u003ccey\u003e) as labels \n    [theta_mb] = trainLinearReg_MiniBatch(foXtrain,ciX,ceX,fytrain,ciy,cey,lambda, b=100, sep=',' , iter=200);\n    \n    %% predict \n    pred_train = predictLinearReg_Buff(foXtrain,ciX,ceX,theta_mb,b=10000,sep=',');\n    pred_test = predictLinearReg_Buff(foXtest,ciX,ceX,theta_mb,b=10000,sep=',');\n    \n    \n    %% measure performance \n    mse_train = MSE(pred_train, ytrain);\n    mse_test = MSE(pred_test, ytest);\n    ```\n * for **large datasets** (e.g. **80GB train set on a machine with 8GB RAM**) you can use the ```trainLinearReg_Buff``` function that is a **buffered implementation of gradient descent**, i.e. it uses it uses all train observations in each iteration vs. one observation as **stochastic gradient descent** or k (k \u003c number of observations on trainset) observations in each iteration as **mini-batch gradient descent**. E.g. this is the code for for fitting a linear regression model with 0.001 as regularization parameter, from file  ```foXtrain ``` for predictors (columns from  ```ciX ``` to  ```ceX ```), and from file  ```fytrain ``` for labels (columns form  ```ciy ``` to  ```cey ```) and buffer equals to 100 observations (= you load in memory 100 observations each time **but you use all train observations for complete a gradient descent iteration**).\n\n    ```matlab\n    %% regularization parameter \n    lambda = 0.001; \n  \n    %% train (buffer = 100 observations) \n    %% from file \u003cfoXtrain\u003e (columns from \u003cciX\u003e to \u003cceX\u003e) as train data\n    %% from file \u003cfytrain\u003e (columns form \u003cciy\u003e to \u003ccey\u003e) as labels \n    [theta_bf] = trainLinearReg_Buff(foXtrain,ciX,ceX,fytrain,ciy,cey,lambda, b=100, sep=',' , iter=200);\n    \n    %% predict \n    pred_train = predictLinearReg_Buff(foXtrain,ciX,ceX,theta_bf,b=10000,sep=',');\n    pred_test = predictLinearReg_Buff(foXtest,ciX,ceX,theta_bf,b=10000,sep=',');\n    \n    %% measure performance \n    mse_train = MSE(pred_train, ytrain);\n    mse_test = MSE(pred_test, ytest);\n    ```\n### 3.3 Regularized Polynomial Logistic Regression \nPackage ```logistic_reg``` **very fast 100% vectorized implementation** in Matlab/Octave\n\n* for **basic use cases** just run command line (fast-furious base dir) \n    \n    ```\u003eoctave GO_LogisticReg.m```\n* for fitting a **logistic regression** model use ```lrCostFunction``` wrapped in  ```trainLogReg``` function. E.g. this is the code for fitting a regularized logistic regression model with trainset/testset not scaled and with regularization parameter set to 0.001. Note: in this code sketch insteaf of using 0.5 as probability threshold I use the ```selectThreshold``` that select the probability threshold maximizing [F1-score](https://en.wikipedia.org/wiki/F1_score).    \n    ```matlab\n    %% feature scaling (trainset/testset) \n    [Xtrain,mu,sigma] = treatContFeatures(Xtrain,p = 1);\n    [Xtest,mu,sigma] = treatContFeatures(Xtest,p = 1,1,mu,sigma);\n    \n    %% regularization parameter \n    lambda = 0.001;\n    \n    %% train \n    [theta] = trainLogReg(Xtrain, ytrain, lambda , iter = 200 );\n    \n    %% predict probabilities  \n    probs_train = predictLogReg(Xtrain,theta);\n    probs_test = predictLogReg(Xtest,theta);\n\t\n    %% select threshold (instead of 0.5) on train data \n    %% Note: this usually should be done by cross-validation \n    thr = selectThreshold (ytrain,probs_train);\n    \n    %% predict labels   \n   \tpred_train = (probs_train \u003e thr);\n   \tpred_train = (probs_test \u003e thr);\n    ```\n* for fitting a **logistic polynomial regression** model use ```lrCostFunction``` as well. Just set up the degree of the polynomial trasformation you like in the ```treatContFeatures``` function. E.g. this is the code for fitting a regularized logistic regression model with trainset/testset not scaled, with regularization parameter set to 0.001 and **polynomial degree 10**.   \n    ```matlab\n    %% feature scaling (trainset/testset) \n    [Xtrain,mu,sigma] = treatContFeatures(Xtrain,p = 10);\n    [Xtest,mu,sigma] = treatContFeatures(Xtest,p = 10,1,mu,sigma);\n    \n    %% regularization parameter \n    lambda = 0.001;\n    \n    %% train \n    [theta] = trainLogReg(Xtrain, ytrain, lambda , iter = 200 );\n    \n    %% predict probabilities  \n    probs_train = predictLogReg(Xtrain,theta);\n    probs_test = predictLogReg(Xtest,theta);\n  \n    %% select threshold (instead of 0.5) on train data \n    %% Note: this usually should be done by cross-validation \n    thr = selectThreshold (ytrain,probs_train);\n    \n    %% predict labels   \n   \tpred_train = (probs_train \u003e thr);\n   \tpred_train = (probs_test \u003e thr);\n    ```\n* for **tuning parameters (on classification problems)** (degree of polynomial trasformation, regularization parameter) by cross-validation use the ```findOptPAndLambdaRegLog``` function. E.g. this is the code for finding the best degree of polynomial trasformation, the best regularization parameter, using cross validation on a train set and cross-validation set already scaled. **Best parameters are found for metrics** [F1-score](https://en.wikipedia.org/wiki/F1_score), [precision](https://en.wikipedia.org/wiki/Precision_and_recall), [recall](https://en.wikipedia.org/wiki/Precision_and_recall). \n \n    ```matlab\n    [p_opt_recall,lambda_opt_recall,p_opt_accuracy,lambda_opt_accuracy,p_opt_precision,lambda_opt_precision,p_opt_F1,lambda_opt_F1,grid] = ...\n      findOptPAndLambdaRegLog(Xtrain, ytrain, Xval, yval)\n      \n    printf('\u003e\u003e\u003e\u003e\u003e metric: F1        - found optimum with p=%i and lambda=%f \\n', p_opt_F1 , lambda_opt_F1 );\n    printf('\u003e\u003e\u003e\u003e\u003e metric: precision - found optimum with p=%i and lambda=%f \\n', p_opt_precision , lambda_opt_precision );\n    printf('\u003e\u003e\u003e\u003e\u003e metric: recall    - found optimum with p=%i and lambda=%f \\n', p_opt_recall , lambda_opt_recall );\n    ```\n    \n## 4. fast-furious R-Package \nPlease, refer to [fast-furious R-Package PDF manual](https://github.com/gtesei/fast-furious/blob/master/fastfurious-manual.pdf). \n    \n## References \nMost parts of fast-furious are based on the following resources: \n* Stanford professor Andrew NG resources: [1](http://openclassroom.stanford.edu/MainFolder/CoursePage.php?course=MachineLearning), [2](https://www.coursera.org/learn/machine-learning/home/info)\n* J. Friedman, T. Hastie, R. Tibshirani, *The Elements of Statistical Learning*, Springer, 2009\n* Max Kuhn and Kjell Johnson, *Applied Predictive Modeling*, Springer, 2013\n\nOther resources: \n* G. James, D. Witten, T. Hastie, R. Tibshirani, *An Introduction to Statistical Learning*, Springer, 2013\n* Hadley Wickham, *Advanced R*, Chapman \u0026 Hall/CRC The R Series, 2014 \n* Paul S.P. Cowpertwait, Andrew V. Metcalfe, *Introductory Time Series with R*, Springer, 2009","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgtesei%2Ffast-furious","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgtesei%2Ffast-furious","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgtesei%2Ffast-furious/lists"}