{"id":19514614,"url":"https://github.com/jorischau/gslnls","last_synced_at":"2025-04-26T06:30:38.653Z","repository":{"id":56934313,"uuid":"405605628","full_name":"JorisChau/gslnls","owner":"JorisChau","description":"{gslnls}: GSL multi-start nonlinear least-squares fitting in R","archived":false,"fork":false,"pushed_at":"2025-01-17T14:19:09.000Z","size":1702,"stargazers_count":16,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-22T23:41:22.406Z","etag":null,"topics":["gnu-scientific-library","gsl","levenberg-marquardt","multi-start","nonlinear-least-squares","nonlinear-regression","r","r-package","robust-regresssion"],"latest_commit_sha":null,"homepage":"https://CRAN.R-project.org/package=gslnls","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JorisChau.png","metadata":{"files":{"readme":"README.md","changelog":"NEWS.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-09-12T10:00:33.000Z","updated_at":"2025-03-08T19:04:46.000Z","dependencies_parsed_at":"2024-04-30T10:00:12.921Z","dependency_job_id":"2c96fd93-d062-444c-a47d-a7cf9c33f243","html_url":"https://github.com/JorisChau/gslnls","commit_stats":{"total_commits":64,"total_committers":1,"mean_commits":64.0,"dds":0.0,"last_synced_commit":"564de078a6fbf9fbcd51b5238499f2ae94970b49"},"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JorisChau%2Fgslnls","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JorisChau%2Fgslnls/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JorisChau%2Fgslnls/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JorisChau%2Fgslnls/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JorisChau","download_url":"https://codeload.github.com/JorisChau/gslnls/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250943883,"owners_count":21511643,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gnu-scientific-library","gsl","levenberg-marquardt","multi-start","nonlinear-least-squares","nonlinear-regression","r","r-package","robust-regresssion"],"created_at":"2024-11-10T23:37:01.253Z","updated_at":"2025-04-26T06:30:38.644Z","avatar_url":"https://github.com/JorisChau.png","language":"R","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n\u003c!-- README.md is generated from README.Rmd. Please edit that file --\u003e\n\n# {gslnls}: GSL Multi-Start Nonlinear Least-Squares Fitting in R\n\n\u003c!-- badges: start --\u003e\n\n[![CRAN\nversion](https://www.r-pkg.org/badges/version/gslnls)](https://cran.r-project.org/package=gslnls)\n[![R-CMD-check](https://github.com/JorisChau/gslnls/workflows/R-CMD-check/badge.svg)](https://github.com/JorisChau/gslnls/actions)\n[![codecov](https://codecov.io/gh/JorisChau/gslnls/branch/master/graph/badge.svg)](https://app.codecov.io/gh/JorisChau/gslnls)\n[![Total\nDownloads](https://cranlogs.r-pkg.org/badges/grand-total/gslnls)](https://cran.r-project.org/package=gslnls)\n\u003c!-- badges: end --\u003e\n\nThe {gslnls}-package provides R bindings to nonlinear least-squares\noptimization with the [GNU Scientific Library\n(GSL)](https://www.gnu.org/software/gsl/). The function `gsl_nls()`\nsolves small to moderate sized nonlinear least-squares problems with the\n`gsl_multifit_nlinear` interface with built-in support for multi-start\noptimization and nonlinear robust regression. For large problems, where\nfactoring the full Jacobian matrix becomes prohibitively expensive, the\n`gsl_nls_large()` function can be used to solve the system with the\n`gsl_multilarge_nlinear` interface. The `gsl_nls_large()` function is\nalso appropriate for systems with sparse structure in the Jacobian\nmatrix.\n\nThe following trust region methods to solve nonlinear least-squares\nproblems are available in `gsl_nls()` (and `gsl_nls_large()`):\n\n- [Levenberg-Marquardt](https://www.gnu.org/software/gsl/doc/html/nls.html#levenberg-marquardt)\n- [Levenberg-Marquardt with geodesic\n  acceleration](https://www.gnu.org/software/gsl/doc/html/nls.html#levenberg-marquardt-with-geodesic-acceleration)\n- [Dogleg](https://www.gnu.org/software/gsl/doc/html/nls.html#dogleg)\n- [Double\n  dogleg](https://www.gnu.org/software/gsl/doc/html/nls.html#double-dogleg)\n- [Two Dimensional\n  Subspace](https://www.gnu.org/software/gsl/doc/html/nls.html#two-dimensional-subspace)\n- [Steihaug-Toint Conjugate\n  Gradient](https://www.gnu.org/software/gsl/doc/html/nls.html#steihaug-toint-conjugate-gradient)\n  (only available in `gsl_nls_large()`)\n\nThe [Tunable\nparameters](https://www.gnu.org/software/gsl/doc/html/nls.html#tunable-parameters)\navailable for the trust method algorithms can be modified from R in\norder to help accelerating convergence for a specific problem at hand.\n\nSee the [Nonlinear Least-Squares\nFitting](https://www.gnu.org/software/gsl/doc/html/nls.html#nonlinear-least-squares-fitting)\nchapter in the GSL reference manual for a comprehensive overview of the\n`gsl_multifit_nlinear` and `gsl_multilarge_nlinear` interfaces and the\nrelevant mathematical background.\n\n## Installation from source\n\n### System requirements\n\nWhen installing the R-package from source, verify that GSL (\\\u003e= 2.3) is\ninstalled on the system, e.g. on Ubuntu/Debian Linux:\n\n    gsl-config --version\n\nIf GSL (\\\u003e= 2.3) is not available on the system, install GSL from a\npre-compiled binary package (see the examples below) or install GSL from\nsource by downloading the latest stable release\n(\u003chttps://www.gnu.org/software/gsl/\u003e) and following the installation\ninstructions in the included README and INSTALL files.\n\n#### GSL installation examples\n\n##### Ubuntu, Debian\n\n    sudo apt-get install libgsl-dev\n\n##### macOS\n\n    brew install gsl\n\n##### Fedora, RedHat, CentOS\n\n    yum install gsl-devel\n\n##### Windows\n\nUsing Rtools \\\u003e= 42, GSL is available with the Rtools installation. For\nearlier versions of Rtools, a binary version of GSL can be installed\nusing the Rtools package manager (see\ne.g. \u003chttps://github.com/r-windows/docs/blob/master/rtools40.md\u003e):\n\n    pacman -S mingw-w64-{i686,x86_64}-gsl\n\nOn windows, the environment variable `LIB_GSL` must be set to the parent\nof the directory containing `libgsl.a`. Note that forward instead of\nbackward slashes should be used in the directory path\n(e.g. `C:/rtools43/x86_64-w64-mingw32.static.posix`).\n\n### R-package installation\n\nWith GSL available, install the R-package from source with:\n\n``` r\n## Install latest CRAN release:\ninstall.packages(\"gslnls\", type = \"source\")\n```\n\nor install the latest development version from GitHub with:\n\n``` r\n## Install latest GitHub development version:\n# install.packages(\"devtools\")\ndevtools::install_github(\"JorisChau/gslnls\")\n```\n\n## Installation from binary\n\nOn windows and some macOS builds, the R-package can be installed from\nCRAN as a binary package. In this case, GSL does not need to be\navailable on the system.\n\n``` r\n## Install latest CRAN release:\ninstall.packages(\"gslnls\")\n```\n\n## Example usage\n\n### Example 1: Exponential model\n\n#### Data\n\nBelow, we simulate\n![n = 25](https://latex.codecogs.com/png.latex?n%20%3D%2025 \"n = 25\")\nnoisy observations\n![y_1,\\ldots,y_n](https://latex.codecogs.com/png.latex?y_1%2C%5Cldots%2Cy_n \"y_1,\\ldots,y_n\")\nfrom an exponential model with additive (i.i.d.) Gaussian noise\naccording to:\n\n![\\left\\\\\n\\begin{aligned}\nf_i \u0026 = A \\cdot \\exp(-\\lambda \\cdot x_i) + b, \u0026 i = 1,\\ldots, n \\\\\ny_i \u0026 = f_i + \\epsilon_i, \u0026 \\epsilon_i \\overset{\\text{iid}}{\\sim} N(0,\\sigma^2)\n\\end{aligned}\n\\right.](https://latex.codecogs.com/png.latex?%5Cleft%5C%7B%0A%5Cbegin%7Baligned%7D%0Af_i%20%26%20%3D%20A%20%5Ccdot%20%5Cexp%28-%5Clambda%20%5Ccdot%20x_i%29%20%2B%20b%2C%20%26%20i%20%3D%201%2C%5Cldots%2C%20n%20%5C%5C%0Ay_i%20%26%20%3D%20f_i%20%2B%20%5Cepsilon_i%2C%20%26%20%5Cepsilon_i%20%5Coverset%7B%5Ctext%7Biid%7D%7D%7B%5Csim%7D%20N%280%2C%5Csigma%5E2%29%0A%5Cend%7Baligned%7D%0A%5Cright. \"\\left\\{\n\\begin{aligned}\nf_i \u0026 = A \\cdot \\exp(-\\lambda \\cdot x_i) + b, \u0026 i = 1,\\ldots, n \\\\\ny_i \u0026 = f_i + \\epsilon_i, \u0026 \\epsilon_i \\overset{\\text{iid}}{\\sim} N(0,\\sigma^2)\n\\end{aligned}\n\\right.\")\n\nThe exponential model parameters are set to\n![A = 5](https://latex.codecogs.com/png.latex?A%20%3D%205 \"A = 5\"),\n![\\lambda = 1.5](https://latex.codecogs.com/png.latex?%5Clambda%20%3D%201.5 \"\\lambda = 1.5\"),\n![b = 1](https://latex.codecogs.com/png.latex?b%20%3D%201 \"b = 1\"), with\nnoise standard deviation\n![\\sigma = 0.25](https://latex.codecogs.com/png.latex?%5Csigma%20%3D%200.25 \"\\sigma = 0.25\").\n\n``` r\nset.seed(1)\nn \u003c- 25\nx \u003c- (seq_len(n) - 1) * 3 / (n - 1)\nf \u003c- function(A, lam, b, x) A * exp(-lam * x) + b\ny \u003c- f(A = 5, lam = 1.5, b = 1, x) + rnorm(n, sd = 0.25)\n```\n\n\u003cimg src=\"README/fig-1-1.png\" width=\"100%\" style=\"display: block; margin: auto;\" /\u003e\n\n#### Model fit\n\nThe exponential model is fitted to the data using the function\n`gsl_nls()` by passing the nonlinear model as a two-sided `formula` and\nproviding starting parameters for the model parameters\n![A, \\lambda, b](https://latex.codecogs.com/png.latex?A%2C%20%5Clambda%2C%20b \"A, \\lambda, b\")\nanalogous to a standard `nls()` call.\n\n``` r\nlibrary(gslnls)\n\nex1_fit \u003c- gsl_nls(\n  fn = y ~ A * exp(-lam * x) + b,    ## model formula\n  data = data.frame(x = x, y = y),   ## model fit data\n  start = c(A = 0, lam = 0, b = 0)   ## starting values\n)\n\nex1_fit\n#\u003e Nonlinear regression model\n#\u003e   model: y ~ A * exp(-lam * x) + b\n#\u003e     A   lam     b \n#\u003e 4.893 1.417 1.010 \n#\u003e  residual sum-of-squares: 1.316\n#\u003e \n#\u003e Algorithm: multifit/levenberg-marquardt, (scaling: more, solver: qr)\n#\u003e \n#\u003e Number of iterations to convergence: 9 \n#\u003e Achieved convergence tolerance: 4.441e-16\n```\n\nHere, the nonlinear least squares problem is solved with the\nLevenberg-Marquardt algorithm (default) in the `gsl_multifit_nlinear`\ninterface with `control` parameters:\n\n``` r\n## default control parameters\ngsl_nls_control() |\u003e str()\n#\u003e List of 23\n#\u003e  $ maxiter        : int 100\n#\u003e  $ scale          : chr \"more\"\n#\u003e  $ solver         : chr \"qr\"\n#\u003e  $ fdtype         : chr \"forward\"\n#\u003e  $ factor_up      : num 2\n#\u003e  $ factor_down    : num 3\n#\u003e  $ avmax          : num 0.75\n#\u003e  $ h_df           : num 1.49e-08\n#\u003e  $ h_fvv          : num 0.02\n#\u003e  $ xtol           : num 1.49e-08\n#\u003e  $ ftol           : num 1.49e-08\n#\u003e  $ gtol           : num 1.49e-08\n#\u003e  $ mstart_n       : int 30\n#\u003e  $ mstart_p       : int 5\n#\u003e  $ mstart_q       : int 3\n#\u003e  $ mstart_r       : num 4\n#\u003e  $ mstart_s       : int 2\n#\u003e  $ mstart_tol     : num 0.25\n#\u003e  $ mstart_maxiter : int 10\n#\u003e  $ mstart_maxstart: int 250\n#\u003e  $ mstart_minsp   : int 1\n#\u003e  $ irls_maxiter   : int 50\n#\u003e  $ irls_xtol      : num 0.000122\n```\n\nCheck `?gsl_nls_control` or the [GSL reference\nmanual](https://www.gnu.org/software/gsl/doc/html/nls.html#tunable-parameters)\nfor details on the available tuning parameters to control the trust\nregion algorithms.\n\n#### Object methods\n\nThe fitted model object returned by `gsl_nls()` is of class `\"gsl_nls\"`,\nwhich inherits from class `\"nls\"`. For this reason, generic functions\nsuch as `anova`, `coef`, `confint`, `deviance`, `df.residual`, `fitted`,\n`formula`, `logLik`, `predict`, `print` `profile`, `residuals`,\n`summary`, `vcov`, `hatvalues`, `cooks.distance` and `weights` are also\napplicable for models fitted with `gsl_nls()`.\n\n``` r\n## model summary\nsummary(ex1_fit)\n#\u003e \n#\u003e Formula: y ~ A * exp(-lam * x) + b\n#\u003e \n#\u003e Parameters:\n#\u003e     Estimate Std. Error t value Pr(\u003e|t|)    \n#\u003e A     4.8930     0.1811  27.014  \u003c 2e-16 ***\n#\u003e lam   1.4169     0.1304  10.865 2.61e-10 ***\n#\u003e b     1.0097     0.1092   9.246 4.92e-09 ***\n#\u003e ---\n#\u003e Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n#\u003e \n#\u003e Residual standard error: 0.2446 on 22 degrees of freedom\n#\u003e \n#\u003e Number of iterations to convergence: 9 \n#\u003e Achieved convergence tolerance: 4.441e-16\n\n## asymptotic confidence intervals\nconfint(ex1_fit)\n#\u003e         2.5 %   97.5 %\n#\u003e A   4.5173851 5.268653\n#\u003e lam 1.1464128 1.687314\n#\u003e b   0.7832683 1.236216\n```\n\nThe `predict` method extends the existing `predict.nls` method by\nallowing for calculation of asymptotic confidence and prediction\n(tolerance) intervals in addition to prediction of the expected\nresponse:\n\n``` r\n## asymptotic prediction intervals\npredict(ex1_fit, interval = \"prediction\", level = 0.95)\n#\u003e            fit       lwr      upr\n#\u003e  [1,] 5.902761 5.2670162 6.538506\n#\u003e  [2,] 5.108572 4.5388041 5.678340\n#\u003e  [3,] 4.443289 3.8987833 4.987794\n#\u003e  [4,] 3.885988 3.3479065 4.424069\n#\u003e  [5,] 3.419142 2.8812430 3.957042\n....\n```\n\n\u003cimg src=\"README/fig-2-1.png\" width=\"100%\" style=\"display: block; margin: auto;\" /\u003e\n\nThe new `confintd` method can be used to evaluate asymptotic confidence\nintervals of derived (or transformed) parameters based on the delta\nmethod, i.e. a first-order (Taylor) approximation of the function of the\nparameters:\n\n``` r\n## delta method confidence intervals\nconfintd(ex1_fit, expr = c(\"b\", \"A + b\", \"log(lam)\"), level = 0.95)\n#\u003e                fit       lwr       upr\n#\u003e b        1.0097419 0.7832683 1.2362155\n#\u003e A + b    5.9027612 5.5194278 6.2860945\n#\u003e log(lam) 0.3484454 0.1575657 0.5393251\n```\n\n#### Jacobian calculation\n\nIf the `jac` argument in `gsl_nls()` is undefined, the Jacobian to solve\nthe [trust region\nsubproblem](https://www.gnu.org/software/gsl/doc/html/nls.html#solving-the-trust-region-subproblem-trs)\nis approximated numerically by (forward or centered) finite differences.\nInstead, an analytic Jacobian can be passed to `jac` by defining a\nfunction that returns the\n![(n \\times p)](https://latex.codecogs.com/png.latex?%28n%20%5Ctimes%20p%29 \"(n \\times p)\")-dimensional\nJacobian matrix of the nonlinear model `fn`, where the first argument\nmust be the vector of parameters of length\n![p](https://latex.codecogs.com/png.latex?p \"p\").\n\nIn the above example, the Jacobian is a\n![(50 \\times 3)](https://latex.codecogs.com/png.latex?%2850%20%5Ctimes%203%29 \"(50 \\times 3)\")-matrix\n![\\[\\boldsymbol{J}\\_{ij}\\]\\_{ij}](https://latex.codecogs.com/png.latex?%5B%5Cboldsymbol%7BJ%7D_%7Bij%7D%5D_%7Bij%7D \"[\\boldsymbol{J}_{ij}]_{ij}\")\nwith rows:\n\n![\\boldsymbol{J}\\_i \\\\= \\\\\\left\\[ \\frac{\\partial f_i}{\\partial A}, \\frac{\\partial f_i}{\\partial \\lambda}, \\frac{\\partial f_i}{\\partial b} \\right\\] \\\\= \\\\\\left\\[ \\exp(-\\lambda \\cdot x_i), -A \\cdot \\exp(-\\lambda \\cdot x_i) \\cdot x_i, 1 \\right\\]](https://latex.codecogs.com/png.latex?%5Cboldsymbol%7BJ%7D_i%20%5C%20%3D%20%5C%20%5Cleft%5B%20%5Cfrac%7B%5Cpartial%20f_i%7D%7B%5Cpartial%20A%7D%2C%20%5Cfrac%7B%5Cpartial%20f_i%7D%7B%5Cpartial%20%5Clambda%7D%2C%20%5Cfrac%7B%5Cpartial%20f_i%7D%7B%5Cpartial%20b%7D%20%5Cright%5D%20%5C%20%3D%20%5C%20%5Cleft%5B%20%5Cexp%28-%5Clambda%20%5Ccdot%20x_i%29%2C%20-A%20%5Ccdot%20%5Cexp%28-%5Clambda%20%5Ccdot%20x_i%29%20%5Ccdot%20x_i%2C%201%20%5Cright%5D \"\\boldsymbol{J}_i \\ = \\ \\left[ \\frac{\\partial f_i}{\\partial A}, \\frac{\\partial f_i}{\\partial \\lambda}, \\frac{\\partial f_i}{\\partial b} \\right] \\ = \\ \\left[ \\exp(-\\lambda \\cdot x_i), -A \\cdot \\exp(-\\lambda \\cdot x_i) \\cdot x_i, 1 \\right]\")\n\nwhich can be encoded as:\n\n``` r\n## analytic Jacobian (1)\ngsl_nls(\n  fn = y ~ A * exp(-lam * x) + b,    ## model formula\n  data = data.frame(x = x, y = y),   ## model fit data\n  start = c(A = 0, lam = 0, b = 0),  ## starting values\n  jac = function(par, x) with(as.list(par), cbind(A = exp(-lam * x), lam = -A * x * exp(-lam * x), b = 1)),\n  x = x                              ## argument passed to jac\n)\n#\u003e Nonlinear regression model\n#\u003e   model: y ~ A * exp(-lam * x) + b\n#\u003e     A   lam     b \n#\u003e 4.893 1.417 1.010 \n#\u003e  residual sum-of-squares: 1.316\n#\u003e \n#\u003e Algorithm: multifit/levenberg-marquardt, (scaling: more, solver: qr)\n#\u003e \n#\u003e Number of iterations to convergence: 9 \n#\u003e Achieved convergence tolerance: 6.661e-16\n```\n\nIf the model formula `fn` can be derived with `stats::deriv()`, then the\nanalytic Jacobian in `jac` can be computed automatically using symbolic\ndifferentiation and it suffices to set `jac = TRUE`:\n\n``` r\n## analytic Jacobian (2)\ngsl_nls(\n  fn = y ~ A * exp(-lam * x) + b,    ## model formula\n  data = data.frame(x = x, y = y),   ## model fit data\n  start = c(A = 0, lam = 0, b = 0),  ## starting values\n  jac = TRUE                         ## symbolic derivation\n)\n#\u003e Nonlinear regression model\n#\u003e   model: y ~ A * exp(-lam * x) + b\n#\u003e     A   lam     b \n#\u003e 4.893 1.417 1.010 \n#\u003e  residual sum-of-squares: 1.316\n#\u003e \n#\u003e Algorithm: multifit/levenberg-marquardt, (scaling: more, solver: qr)\n#\u003e \n#\u003e Number of iterations to convergence: 9 \n#\u003e Achieved convergence tolerance: 6.661e-16\n```\n\nAlternatively, a self-starting nonlinear model (see `?selfStart`) can be\npassed to `gsl_nls()`. In this case, the Jacobian matrix is evaluated\nfrom the `\"gradient\"` attribute of the self-starting model object:\n\n``` r\n## self-starting model\nss_fit \u003c- gsl_nls(\n  fn =  y ~ SSasymp(x, Asym, R0, lrc),    ## model formula\n  data = data.frame(x = x, y = y)         ## model fit data\n)\n\nss_fit\n#\u003e Nonlinear regression model\n#\u003e   model: y ~ SSasymp(x, Asym, R0, lrc)\n#\u003e   Asym     R0    lrc \n#\u003e 1.0097 5.9028 0.3484 \n#\u003e  residual sum-of-squares: 1.316\n#\u003e \n#\u003e Algorithm: multifit/levenberg-marquardt, (scaling: more, solver: qr)\n#\u003e \n#\u003e Number of iterations to convergence: 1 \n#\u003e Achieved convergence tolerance: 1.181e-13\n```\n\nThe self-starting model `SSasymp()` uses a different model\nparameterization (`A = R0 - Asym`, `lam = exp(lrc)`, `b = Asym`), but\nthe fitted models are equivalent. Also, when using a *self-starting*\nmodel, no starting values need to be provided.\n\n**Remark**: confidence intervals for the coefficients in the original\nmodel parameterization can be evaluated with the `confintd` method:\n\n``` r\n## delta method confidence intervals\nconfintd(ss_fit, expr = c(\"R0 - Asym\", \"exp(lrc)\", \"Asym\"), level = 0.95)\n#\u003e                fit       lwr      upr\n#\u003e R0 - Asym 4.893019 4.5173851 5.268653\n#\u003e exp(lrc)  1.416863 1.1464128 1.687314\n#\u003e Asym      1.009742 0.7832683 1.236216\n```\n\n#### Multi-start optimization\n\nIn addition to single-start NLS optimization, the `gsl_nls()` function\nhas built-in support for multi-start optimization. For multi-start\noptimization, instead of a list or vector of fixed start parameters,\npass a `list` or `matrix` of start parameter ranges to the argument\n`start`:\n\n``` r\ngsl_nls(\n  fn = y ~ A * exp(-lam * x) + b,    ## model formula\n  data = data.frame(x = x, y = y),   ## model fit data\n  start = list(A = c(-100, 100), lam = c(-5, 5), b = c(-10, 10))   ## multi-start\n)\n#\u003e Nonlinear regression model\n#\u003e   model: y ~ A * exp(-lam * x) + b\n#\u003e     A   lam     b \n#\u003e 4.893 1.417 1.010 \n#\u003e  residual sum-of-squares: 1.316\n#\u003e \n#\u003e Algorithm: multifit/levenberg-marquardt, (scaling: more, solver: qr)\n#\u003e \n#\u003e Number of iterations to convergence: 3 \n#\u003e Achieved convergence tolerance: 8.882e-16\n```\n\nThe multi-start procedure is a modified version of the algorithm\ndescribed in Hickernell and Yuan (1997), see `?gsl_nls` for additional\ndetails. If `start` contains missing (or infinite) values, the\nmulti-start algorithm is executed without fixed parameter ranges for the\nmissing parameters. More precisely, the ranges are initialized to the\nunit interval and dynamically increased or decreased in each major\niteration of the multi-start algorithm.\n\n``` r\ngsl_nls(\n  fn = y ~ A * exp(-lam * x) + b,    ## model formula\n  data = data.frame(x = x, y = y),   ## model fit data\n  start = list(A = NA, lam = NA, b = NA)   ## dynamic multi-start\n)\n#\u003e Nonlinear regression model\n#\u003e   model: y ~ A * exp(-lam * x) + b\n#\u003e     A   lam     b \n#\u003e 4.893 1.417 1.010 \n#\u003e  residual sum-of-squares: 1.316\n#\u003e \n#\u003e Algorithm: multifit/levenberg-marquardt, (scaling: more, solver: qr)\n#\u003e \n#\u003e Number of iterations to convergence: 3 \n#\u003e Achieved convergence tolerance: 2.22e-16\n```\n\n**Remark**: the dynamic multi-start procedure is not expected to always\nreturn a global minimum of the NLS objective. Especially when the\nobjective function contains many local optima, the multi-start algorithm\nmay be unable to select parameter ranges that include the global\nminimizing solution.\n\n#### Robust loss functions\n\nThe `loss` argument in `gsl_nls()` allows to specify a robust loss\nfunction\n![\\rho(x)](https://latex.codecogs.com/png.latex?%5Crho%28x%29 \"\\rho(x)\")\nother than the default squared loss\n![\\rho(x) = \\frac{1}{2}x^2](https://latex.codecogs.com/png.latex?%5Crho%28x%29%20%3D%20%5Cfrac%7B1%7D%7B2%7Dx%5E2 \"\\rho(x) = \\frac{1}{2}x^2\")\nin the optimization objective:\n\n![\\arg \\min\\_{\\boldsymbol{\\theta}} \\sum\\_{i=1}^n \\rho\\left(y_i - f(x_i, \\boldsymbol{\\theta})\\right)](https://latex.codecogs.com/png.latex?%5Carg%20%5Cmin_%7B%5Cboldsymbol%7B%5Ctheta%7D%7D%20%5Csum_%7Bi%3D1%7D%5En%20%5Crho%5Cleft%28y_i%20-%20f%28x_i%2C%20%5Cboldsymbol%7B%5Ctheta%7D%29%5Cright%29 \"\\arg \\min_{\\boldsymbol{\\theta}} \\sum_{i=1}^n \\rho\\left(y_i - f(x_i, \\boldsymbol{\\theta})\\right)\")\n\nThe (MM-)estimates in the robust regression problem are obtained by\niterative reweighted least squares (IRLS), see `?gsl_nls_loss` for the\navailable loss functions and their (optional) tuning parameters.\n\n``` r\n## Huber loss\ngsl_nls(\n  fn = y ~ A * exp(-lam * x) + b,    ## model formula\n  data = data.frame(x = x, y = y),   ## model fit data\n  start = c(A = 0, lam = 0, b = 0),  ## starting values\n  loss = \"huber\"\n)\n#\u003e Nonlinear regression model\n#\u003e   model: y ~ A * exp(-lam * x) + b\n#\u003e     A   lam     b \n#\u003e 4.796 1.463 1.092 \n#\u003e  weighted residual sum-of-squares: 0.8127\n#\u003e \n#\u003e Algorithm: multifit/levenberg-marquardt, (scaling: more, solver: qr)\n#\u003e \n#\u003e Number of IRLS iterations to convergence: 8 \n#\u003e Achieved IRLS tolerance: 0.0001023\n#\u003e \n#\u003e Number of NLS iterations to convergence: 9 \n#\u003e Achieved NLS tolerance: 1.11e-16\n```\n\n### Example 2: Gaussian function\n\n#### Data\n\nThe following code generates\n![n = 50](https://latex.codecogs.com/png.latex?n%20%3D%2050 \"n = 50\")\nnoisy observations\n![y_1,\\ldots,y_n](https://latex.codecogs.com/png.latex?y_1%2C%5Cldots%2Cy_n \"y_1,\\ldots,y_n\")\nfrom a Gaussian function with multiplicative independent Gaussian noise\naccording to the model:\n\n![\\left\\\\\n\\begin{aligned}\nf_i \u0026 = a \\cdot \\exp\\left(-\\frac{(x_i - b)^2}{2c^2}\\right), \u0026 i = 1,\\ldots, n \\\\\ny_i \u0026 = f_i \\cdot \\epsilon_i, \u0026 \\epsilon_i \\overset{\\text{iid}}{\\sim} N(1,\\sigma^2)\n\\end{aligned}\n\\right.](https://latex.codecogs.com/png.latex?%5Cleft%5C%7B%0A%5Cbegin%7Baligned%7D%0Af_i%20%26%20%3D%20a%20%5Ccdot%20%5Cexp%5Cleft%28-%5Cfrac%7B%28x_i%20-%20b%29%5E2%7D%7B2c%5E2%7D%5Cright%29%2C%20%26%20i%20%3D%201%2C%5Cldots%2C%20n%20%5C%5C%0Ay_i%20%26%20%3D%20f_i%20%5Ccdot%20%5Cepsilon_i%2C%20%26%20%5Cepsilon_i%20%5Coverset%7B%5Ctext%7Biid%7D%7D%7B%5Csim%7D%20N%281%2C%5Csigma%5E2%29%0A%5Cend%7Baligned%7D%0A%5Cright. \"\\left\\{\n\\begin{aligned}\nf_i \u0026 = a \\cdot \\exp\\left(-\\frac{(x_i - b)^2}{2c^2}\\right), \u0026 i = 1,\\ldots, n \\\\\ny_i \u0026 = f_i \\cdot \\epsilon_i, \u0026 \\epsilon_i \\overset{\\text{iid}}{\\sim} N(1,\\sigma^2)\n\\end{aligned}\n\\right.\")\n\nThe parameters of the Gaussian model function are set to\n![a = 5](https://latex.codecogs.com/png.latex?a%20%3D%205 \"a = 5\"),\n![b = 0.4](https://latex.codecogs.com/png.latex?b%20%3D%200.4 \"b = 0.4\"),\n![c = 0.15](https://latex.codecogs.com/png.latex?c%20%3D%200.15 \"c = 0.15\"),\nwith noise standard deviation\n![\\sigma = 0.1](https://latex.codecogs.com/png.latex?%5Csigma%20%3D%200.1 \"\\sigma = 0.1\")\n(see also\n\u003chttps://www.gnu.org/software/gsl/doc/html/nls.html#geodesic-acceleration-example-2\u003e).\n\n``` r\nset.seed(1)\nn \u003c- 50\nx \u003c- seq_len(n) / n\nf \u003c- function(a, b, c, x) a * exp(-(x - b)^2 / (2 * c^2))\ny \u003c- f(a = 5, b = 0.4, c = 0.15, x) * rnorm(n, mean = 1, sd = 0.1)\n```\n\n\u003cimg src=\"README/fig-3-1.png\" width=\"100%\" style=\"display: block; margin: auto;\" /\u003e\n\n#### Model fit\n\nUsing the default\n[Levenberg-Marquardt](https://www.gnu.org/software/gsl/doc/html/nls.html#levenberg-marquardt)\nalgorithm (without geodesic acceleration), the nonlinear Gaussian model\ncan be fitted with a call to `gsl_nls()` analogous to the previous\nexample. Here, the `trace` argument is activated in order to trace the\nsum of squared residuals (`ssr`) and parameter estimates (`par`) at each\niteration of the algorithm.\n\n``` r\n## Levenberg-Marquardt (default)\nex2a_fit \u003c- gsl_nls(\n  fn = y ~ a * exp(-(x - b)^2 / (2 * c^2)), ## model formula\n  data = data.frame(x = x, y = y),          ## model fit data\n  start = c(a = 1, b = 0, c = 1),           ## starting values\n  trace = TRUE                              ## verbose output\n)\n#\u003e iter   1: ssr = 174.476, par = (2.1711, 3.31169, -4.12254)\n#\u003e iter   2: ssr = 171.29, par = (1.85555, 2.84851, -5.66642)\n#\u003e iter   3: ssr = 168.703, par = (1.93316, 2.34745, -6.33662)\n#\u003e iter   4: ssr = 167.546, par = (1.84665, 1.24131, -7.399)\n#\u003e iter   5: ssr = 166.799, par = (1.87948, 0.380487, -7.61723)\n#\u003e iter   6: ssr = 165.623, par = (1.90658, -2.00516, -7.19306)\n#\u003e iter   7: ssr = 163.944, par = (2.18675, -3.19622, -5.38805)\n#\u003e iter   8: ssr = 161.679, par = (2.41824, -3.1487, -5.03149)\n#\u003e iter   9: ssr = 159.955, par = (2.67373, -3.18704, -4.20168)\n#\u003e iter  10: ssr = 157.391, par = (3.10831, -2.84066, -3.10573)\n#\u003e iter  11: ssr = 153.131, par = (3.54614, -2.3807, -2.53824)\n#\u003e iter  12: ssr = 150.129, par = (4.028, -1.97822, -1.96376)\n#\u003e iter  13: ssr = 146.735, par = (4.49887, -1.35878, -1.39554)\n#\u003e iter  14: ssr = 142.928, par = (3.14454, -0.573011, -1.08117)\n#\u003e iter  15: ssr = 124.564, par = (2.14741, 0.465371, -0.443323)\n#\u003e iter  16: ssr = 102.171, par = (3.74447, 0.263284, 0.353778)\n#\u003e iter  17: ssr = 35.4992, par = (3.4995, 0.437365, 0.186149)\n#\u003e iter  18: ssr = 9.25455, par = (4.41259, 0.381433, 0.160233)\n#\u003e iter  19: ssr = 3.13721, par = (4.94369, 0.400412, 0.15032)\n#\u003e iter  20: ssr = 2.76231, par = (5.11738, 0.397815, 0.14729)\n#\u003e iter  21: ssr = 2.75831, par = (5.13785, 0.397886, 0.146865)\n#\u003e iter  22: ssr = 2.7583, par = (5.1389, 0.397884, 0.146831)\n#\u003e iter  23: ssr = 2.7583, par = (5.13894, 0.397884, 0.146829)\n#\u003e iter  24: ssr = 2.7583, par = (5.13894, 0.397884, 0.146829)\n#\u003e iter  25: ssr = 2.7583, par = (5.13894, 0.397884, 0.146829)\n#\u003e iter  26: ssr = 2.7583, par = (5.13894, 0.397884, 0.146829)\n#\u003e *******************\n#\u003e summary from method 'multifit/levenberg-marquardt'\n#\u003e number of iterations: 26\n#\u003e initial ssr: 210.146\n#\u003e final ssr: 2.7583\n#\u003e ssr/dof: 0.0586872\n#\u003e ssr achieved tolerance: 1.33227e-15\n#\u003e function evaluations: 124\n#\u003e jacobian evaluations: 0\n#\u003e fvv evaluations: 0\n#\u003e status: success\n#\u003e *******************\n\nex2a_fit\n#\u003e Nonlinear regression model\n#\u003e   model: y ~ a * exp(-(x - b)^2/(2 * c^2))\n#\u003e      a      b      c \n#\u003e 5.1389 0.3979 0.1468 \n#\u003e  residual sum-of-squares: 2.758\n#\u003e \n#\u003e Algorithm: multifit/levenberg-marquardt, (scaling: more, solver: qr)\n#\u003e \n#\u003e Number of iterations to convergence: 26 \n#\u003e Achieved convergence tolerance: 1.332e-15\n```\n\n#### Geodesic acceleration\n\nThe nonlinear model can also be fitted using the Levenberg-Marquardt\nalgorithm with [geodesic\nacceleration](https://www.gnu.org/software/gsl/doc/html/nls.html#levenberg-marquardt-with-geodesic-acceleration)\nby changing the default `algorithm = \"lm\"` to `algorithm = \"lmaccel\"`.\n\n``` r\n## Levenberg-Marquardt w/ geodesic acceleration\nex2b_fit \u003c- gsl_nls(\n  fn = y ~ a * exp(-(x - b)^2 / (2 * c^2)), ## model formula\n  data = data.frame(x = x, y = y),          ## model fit data\n  start = c(a = 1, b = 0, c = 1),           ## starting values\n  algorithm = \"lmaccel\",                    ## algorithm\n  trace = TRUE                              ## verbose output\n)\n#\u003e iter   1: ssr = 158.039, par = (1.58476, 0.502555, 0.511498)\n#\u003e iter   2: ssr = 126.469, par = (1.8444, 0.366374, 0.403898)\n#\u003e iter   3: ssr = 77.0115, par = (2.5025, 0.392374, 0.272717)\n#\u003e iter   4: ssr = 26.4036, par = (3.64063, 0.394564, 0.205618)\n#\u003e iter   5: ssr = 5.35491, par = (4.63506, 0.396865, 0.163659)\n#\u003e iter   6: ssr = 2.82529, par = (5.05578, 0.397909, 0.149333)\n#\u003e iter   7: ssr = 2.75877, par = (5.13246, 0.397896, 0.147057)\n#\u003e iter   8: ssr = 2.7583, par = (5.13862, 0.397885, 0.146843)\n#\u003e iter   9: ssr = 2.7583, par = (5.13893, 0.397884, 0.14683)\n#\u003e iter  10: ssr = 2.7583, par = (5.13894, 0.397884, 0.146829)\n#\u003e iter  11: ssr = 2.7583, par = (5.13894, 0.397884, 0.146829)\n#\u003e iter  12: ssr = 2.7583, par = (5.13894, 0.397884, 0.146829)\n#\u003e *******************\n#\u003e summary from method 'multifit/levenberg-marquardt+accel'\n#\u003e number of iterations: 12\n#\u003e initial ssr: 210.146\n#\u003e final ssr: 2.7583\n#\u003e ssr/dof: 0.0586872\n#\u003e ssr achieved tolerance: 3.19744e-14\n#\u003e function evaluations: 76\n#\u003e jacobian evaluations: 0\n#\u003e fvv evaluations: 0\n#\u003e status: success\n#\u003e *******************\n\nex2b_fit\n#\u003e Nonlinear regression model\n#\u003e   model: y ~ a * exp(-(x - b)^2/(2 * c^2))\n#\u003e      a      b      c \n#\u003e 5.1389 0.3979 0.1468 \n#\u003e  residual sum-of-squares: 2.758\n#\u003e \n#\u003e Algorithm: multifit/levenberg-marquardt+accel, (scaling: more, solver: qr)\n#\u003e \n#\u003e Number of iterations to convergence: 12 \n#\u003e Achieved convergence tolerance: 3.197e-14\n```\n\nWith geodesic acceleration enabled the method converges after 12\niterations, whereas the method without geodesic acceleration required 26\niterations. This indicates that the nonlinear least-squares solver\nbenefits (substantially) from the geodesic acceleration correction.\n\n##### Second directional derivative\n\nBy default, if the `fvv` argument is undefined, the second directional\nderivative\n![D^2_v f](https://latex.codecogs.com/png.latex?D%5E2_v%20f \"D^2_v f\")\nused to calculate the geodesic acceleration correction is approximated\nby forward (or centered) finite differences. To use an analytic\nexpression for\n![D^2_v f](https://latex.codecogs.com/png.latex?D%5E2_v%20f \"D^2_v f\"),\na function returning the\n![n](https://latex.codecogs.com/png.latex?n \"n\")-dimensional vector of\nsecond directional derivatives of the nonlinear model can be passed to\n`fvv`. The first argument of the function must be the vector of\nparameters of length ![p](https://latex.codecogs.com/png.latex?p \"p\")\nand the second argument must be the velocity vector, also of length\n![p](https://latex.codecogs.com/png.latex?p \"p\").\n\nFor the Gaussian model function, the matrix of second partial\nderivatives, i.e. the Hessian, is given by\n(cf. \u003chttps://www.gnu.org/software/gsl/doc/html/nls.html#geodesic-acceleration-example-2\u003e):\n\n![\\boldsymbol{H}\\_{f_i} \\\\= \\\\\n\\left\\[\\begin{matrix} \n\\frac{\\partial^2 f_i}{\\partial a^2} \u0026 \\frac{\\partial^2 f_i}{\\partial a \\partial b} \u0026 \\frac{\\partial^2 f_i}{\\partial a \\partial c} \\\\\n\u0026 \\frac{\\partial^2 f_i}{\\partial b^2} \u0026 \\frac{\\partial^2 f_i}{\\partial b \\partial c} \\\\\n\u0026 \u0026 \\frac{\\partial^2 f_i}{\\partial c^2}\n\\end{matrix}\\right\\] \\\\= \\\\\n\\left\\[\\begin{matrix}\n0 \u0026 \\frac{z_i}{c} e_i \u0026 \\frac{z_i^2}{c} e_i \\\\\n\u0026 -\\frac{a}{c^2} (1 - z_i^2) e_i \u0026 -\\frac{a}{c^2} z_i (2 - z_i^2) e_i \\\\\n\u0026 \u0026 -\\frac{a}{c^2} z_i^2 (3 - z_i^2) e_i \n\\end{matrix}\\right\\]](https://latex.codecogs.com/png.latex?%5Cboldsymbol%7BH%7D_%7Bf_i%7D%20%5C%20%3D%20%5C%20%0A%5Cleft%5B%5Cbegin%7Bmatrix%7D%20%0A%5Cfrac%7B%5Cpartial%5E2%20f_i%7D%7B%5Cpartial%20a%5E2%7D%20%26%20%5Cfrac%7B%5Cpartial%5E2%20f_i%7D%7B%5Cpartial%20a%20%5Cpartial%20b%7D%20%26%20%5Cfrac%7B%5Cpartial%5E2%20f_i%7D%7B%5Cpartial%20a%20%5Cpartial%20c%7D%20%5C%5C%0A%26%20%5Cfrac%7B%5Cpartial%5E2%20f_i%7D%7B%5Cpartial%20b%5E2%7D%20%26%20%5Cfrac%7B%5Cpartial%5E2%20f_i%7D%7B%5Cpartial%20b%20%5Cpartial%20c%7D%20%5C%5C%0A%26%20%26%20%5Cfrac%7B%5Cpartial%5E2%20f_i%7D%7B%5Cpartial%20c%5E2%7D%0A%5Cend%7Bmatrix%7D%5Cright%5D%20%5C%20%3D%20%5C%20%0A%5Cleft%5B%5Cbegin%7Bmatrix%7D%0A0%20%26%20%5Cfrac%7Bz_i%7D%7Bc%7D%20e_i%20%26%20%5Cfrac%7Bz_i%5E2%7D%7Bc%7D%20e_i%20%5C%5C%0A%26%20-%5Cfrac%7Ba%7D%7Bc%5E2%7D%20%281%20-%20z_i%5E2%29%20e_i%20%26%20-%5Cfrac%7Ba%7D%7Bc%5E2%7D%20z_i%20%282%20-%20z_i%5E2%29%20e_i%20%5C%5C%0A%26%20%26%20-%5Cfrac%7Ba%7D%7Bc%5E2%7D%20z_i%5E2%20%283%20-%20z_i%5E2%29%20e_i%20%0A%5Cend%7Bmatrix%7D%5Cright%5D \"\\boldsymbol{H}_{f_i} \\ = \\ \n\\left[\\begin{matrix} \n\\frac{\\partial^2 f_i}{\\partial a^2} \u0026 \\frac{\\partial^2 f_i}{\\partial a \\partial b} \u0026 \\frac{\\partial^2 f_i}{\\partial a \\partial c} \\\\\n\u0026 \\frac{\\partial^2 f_i}{\\partial b^2} \u0026 \\frac{\\partial^2 f_i}{\\partial b \\partial c} \\\\\n\u0026 \u0026 \\frac{\\partial^2 f_i}{\\partial c^2}\n\\end{matrix}\\right] \\ = \\ \n\\left[\\begin{matrix}\n0 \u0026 \\frac{z_i}{c} e_i \u0026 \\frac{z_i^2}{c} e_i \\\\\n\u0026 -\\frac{a}{c^2} (1 - z_i^2) e_i \u0026 -\\frac{a}{c^2} z_i (2 - z_i^2) e_i \\\\\n\u0026 \u0026 -\\frac{a}{c^2} z_i^2 (3 - z_i^2) e_i \n\\end{matrix}\\right]\")\n\nwhere the lower half of the Hessian matrix is omitted since it is\nsymmetric and where we use the notation,\n\n![\\begin{aligned}\nz_i \u0026 \\\\= \\\\\\frac{x_i - b}{c} \\\\\ne_i \u0026 \\\\= \\\\\\exp\\left(-\\frac{1}{2}z_i^2 \\right)\n\\end{aligned}](https://latex.codecogs.com/png.latex?%5Cbegin%7Baligned%7D%0Az_i%20%26%20%5C%20%3D%20%5C%20%5Cfrac%7Bx_i%20-%20b%7D%7Bc%7D%20%5C%5C%0Ae_i%20%26%20%5C%20%3D%20%5C%20%5Cexp%5Cleft%28-%5Cfrac%7B1%7D%7B2%7Dz_i%5E2%20%5Cright%29%0A%5Cend%7Baligned%7D \"\\begin{aligned}\nz_i \u0026 \\ = \\ \\frac{x_i - b}{c} \\\\\ne_i \u0026 \\ = \\ \\exp\\left(-\\frac{1}{2}z_i^2 \\right)\n\\end{aligned}\")\n\nBased on the Hessian matrix, the second directional derivative of\n![f_i](https://latex.codecogs.com/png.latex?f_i \"f_i\"), with\n![i = 1,\\ldots,n](https://latex.codecogs.com/png.latex?i%20%3D%201%2C%5Cldots%2Cn \"i = 1,\\ldots,n\"),\nbecomes:\n\n![\\begin{aligned}\nD_v^2 f_i \u0026 \\\\= \\\\\\sum\\_{j,k} v\\_{\\theta_j}v\\_{\\theta_k} \\frac{\\partial^2 f_i}{\\partial \\theta_j \\partial \\theta_k} \\\\\n\u0026 \\\\= \\\\v_a^2 \\frac{\\partial^2 f_i}{\\partial a^2} + 2 v_av_b\\frac{\\partial^2 f_i}{\\partial a \\partial b} + 2v_av_c\\frac{\\partial^2 f_i}{\\partial a \\partial c} + v_b^2\\frac{\\partial^2 f_i}{\\partial b^2} + 2v_bv_c\\frac{\\partial^2 f_i}{\\partial b \\partial c} + v_c^2\\frac{\\partial^2 f_i}{\\partial c^2} \\\\\n\u0026 \\\\= \\\\2v_a v_b\\frac{z_i}{c} e_i + 2v_av_c \\frac{z_i^2}{c} e_i - v_b^2\\frac{a}{c^2} (1 - z_i^2) e_i - 2v_bv_c \\frac{a}{c^2} z_i (2 - z_i^2) e_i - v_c^2\\frac{a}{c^2} z_i^2 (3 - z_i^2) e_i\n\\end{aligned}](https://latex.codecogs.com/png.latex?%5Cbegin%7Baligned%7D%0AD_v%5E2%20f_i%20%26%20%5C%20%3D%20%5C%20%5Csum_%7Bj%2Ck%7D%20v_%7B%5Ctheta_j%7Dv_%7B%5Ctheta_k%7D%20%5Cfrac%7B%5Cpartial%5E2%20f_i%7D%7B%5Cpartial%20%5Ctheta_j%20%5Cpartial%20%5Ctheta_k%7D%20%5C%5C%0A%26%20%5C%20%3D%20%5C%20v_a%5E2%20%5Cfrac%7B%5Cpartial%5E2%20f_i%7D%7B%5Cpartial%20a%5E2%7D%20%2B%202%20v_av_b%5Cfrac%7B%5Cpartial%5E2%20f_i%7D%7B%5Cpartial%20a%20%5Cpartial%20b%7D%20%2B%202v_av_c%5Cfrac%7B%5Cpartial%5E2%20f_i%7D%7B%5Cpartial%20a%20%5Cpartial%20c%7D%20%2B%20v_b%5E2%5Cfrac%7B%5Cpartial%5E2%20f_i%7D%7B%5Cpartial%20b%5E2%7D%20%2B%202v_bv_c%5Cfrac%7B%5Cpartial%5E2%20f_i%7D%7B%5Cpartial%20b%20%5Cpartial%20c%7D%20%2B%20v_c%5E2%5Cfrac%7B%5Cpartial%5E2%20f_i%7D%7B%5Cpartial%20c%5E2%7D%20%5C%5C%0A%26%20%5C%20%3D%20%5C%202v_a%20v_b%5Cfrac%7Bz_i%7D%7Bc%7D%20e_i%20%2B%202v_av_c%20%5Cfrac%7Bz_i%5E2%7D%7Bc%7D%20e_i%20-%20v_b%5E2%5Cfrac%7Ba%7D%7Bc%5E2%7D%20%281%20-%20z_i%5E2%29%20e_i%20-%202v_bv_c%20%5Cfrac%7Ba%7D%7Bc%5E2%7D%20z_i%20%282%20-%20z_i%5E2%29%20e_i%20-%20v_c%5E2%5Cfrac%7Ba%7D%7Bc%5E2%7D%20z_i%5E2%20%283%20-%20z_i%5E2%29%20e_i%0A%5Cend%7Baligned%7D \"\\begin{aligned}\nD_v^2 f_i \u0026 \\ = \\ \\sum_{j,k} v_{\\theta_j}v_{\\theta_k} \\frac{\\partial^2 f_i}{\\partial \\theta_j \\partial \\theta_k} \\\\\n\u0026 \\ = \\ v_a^2 \\frac{\\partial^2 f_i}{\\partial a^2} + 2 v_av_b\\frac{\\partial^2 f_i}{\\partial a \\partial b} + 2v_av_c\\frac{\\partial^2 f_i}{\\partial a \\partial c} + v_b^2\\frac{\\partial^2 f_i}{\\partial b^2} + 2v_bv_c\\frac{\\partial^2 f_i}{\\partial b \\partial c} + v_c^2\\frac{\\partial^2 f_i}{\\partial c^2} \\\\\n\u0026 \\ = \\ 2v_a v_b\\frac{z_i}{c} e_i + 2v_av_c \\frac{z_i^2}{c} e_i - v_b^2\\frac{a}{c^2} (1 - z_i^2) e_i - 2v_bv_c \\frac{a}{c^2} z_i (2 - z_i^2) e_i - v_c^2\\frac{a}{c^2} z_i^2 (3 - z_i^2) e_i\n\\end{aligned}\")\n\nwhich can be encoded using `gsl_nls()` as follows:\n\n``` r\n## second directional derivative\nfvv \u003c- function(par, v, x) {\n  with(as.list(par), {\n    zi \u003c- (x - b) / c\n    ei \u003c- exp(-zi^2 / 2)\n    2 * v[[\"a\"]] * v[[\"b\"]] * zi / c * ei + 2 * v[[\"a\"]] * v[[\"c\"]] * zi^2 / c * ei - \n      v[[\"b\"]]^2 * a / c^2 * (1 - zi^2) * ei - 2 * v[[\"b\"]] * v[[\"c\"]] * a / c^2 * zi * (2 - zi^2) * ei -\n      v[[\"c\"]]^2 * a / c^2 * zi^2 * (3 - zi^2) * ei\n  })\n}\n\n## analytic fvv (1)\ngsl_nls(\n  fn = y ~ a * exp(-(x - b)^2 / (2 * c^2)), ## model formula\n  data = data.frame(x = x, y = y),          ## model fit data\n  start = c(a = 1, b = 0, c = 1),           ## starting values\n  algorithm = \"lmaccel\",                    ## algorithm\n  trace = TRUE,                             ## verbose output\n  fvv = fvv,                                ## analytic function\n  x = x                                     ## argument passed to fvv\n)\n#\u003e iter   1: ssr = 158.14, par = (1.584, 0.502802, 0.512515)\n#\u003e iter   2: ssr = 126.579, par = (1.84317, 0.365984, 0.404378)\n#\u003e iter   3: ssr = 77.1032, par = (2.50015, 0.392468, 0.272532)\n#\u003e iter   4: ssr = 26.4382, par = (3.63788, 0.394722, 0.205505)\n#\u003e iter   5: ssr = 5.36705, par = (4.63344, 0.396903, 0.16368)\n#\u003e iter   6: ssr = 2.82578, par = (5.05546, 0.39791, 0.149341)\n#\u003e iter   7: ssr = 2.75877, par = (5.13243, 0.397896, 0.147057)\n#\u003e iter   8: ssr = 2.7583, par = (5.13862, 0.397885, 0.146843)\n#\u003e iter   9: ssr = 2.7583, par = (5.13893, 0.397884, 0.14683)\n#\u003e iter  10: ssr = 2.7583, par = (5.13894, 0.397884, 0.146829)\n#\u003e iter  11: ssr = 2.7583, par = (5.13894, 0.397884, 0.146829)\n#\u003e iter  12: ssr = 2.7583, par = (5.13894, 0.397884, 0.146829)\n#\u003e *******************\n#\u003e summary from method 'multifit/levenberg-marquardt+accel'\n#\u003e number of iterations: 12\n#\u003e initial ssr: 210.146\n#\u003e final ssr: 2.7583\n#\u003e ssr/dof: 0.0586872\n#\u003e ssr achieved tolerance: 3.10862e-14\n#\u003e function evaluations: 58\n#\u003e jacobian evaluations: 0\n#\u003e fvv evaluations: 18\n#\u003e status: success\n#\u003e *******************\n#\u003e Nonlinear regression model\n#\u003e   model: y ~ a * exp(-(x - b)^2/(2 * c^2))\n#\u003e      a      b      c \n#\u003e 5.1389 0.3979 0.1468 \n#\u003e  residual sum-of-squares: 2.758\n#\u003e \n#\u003e Algorithm: multifit/levenberg-marquardt+accel, (scaling: more, solver: qr)\n#\u003e \n#\u003e Number of iterations to convergence: 12 \n#\u003e Achieved convergence tolerance: 3.109e-14\n```\n\nIf the model formula `fn` can be derived with `stats::deriv()`, then the\nanalytic Hessian and second directional derivatives in `fvv` can be\ncomputed automatically using symbolic differentiation. Analogous to the\n`jac` argument, to evaluate `fvv` by means of symbolic differentiation,\nset `fvv = TRUE`:\n\n``` r\n## analytic fvv (2)\ngsl_nls(\n  fn = y ~ a * exp(-(x - b)^2 / (2 * c^2)), ## model formula\n  data = data.frame(x = x, y = y),          ## model fit data\n  start = c(a = 1, b = 0, c = 1),           ## starting values\n  algorithm = \"lmaccel\",                    ## algorithm\n  trace = TRUE,                             ## verbose output\n  fvv = TRUE                                ## automatic derivation\n)\n#\u003e iter   1: ssr = 158.14, par = (1.584, 0.502802, 0.512515)\n#\u003e iter   2: ssr = 126.579, par = (1.84317, 0.365984, 0.404378)\n#\u003e iter   3: ssr = 77.1032, par = (2.50015, 0.392468, 0.272532)\n#\u003e iter   4: ssr = 26.4382, par = (3.63788, 0.394722, 0.205505)\n#\u003e iter   5: ssr = 5.36705, par = (4.63344, 0.396903, 0.16368)\n#\u003e iter   6: ssr = 2.82578, par = (5.05546, 0.39791, 0.149341)\n#\u003e iter   7: ssr = 2.75877, par = (5.13243, 0.397896, 0.147057)\n#\u003e iter   8: ssr = 2.7583, par = (5.13862, 0.397885, 0.146843)\n#\u003e iter   9: ssr = 2.7583, par = (5.13893, 0.397884, 0.14683)\n#\u003e iter  10: ssr = 2.7583, par = (5.13894, 0.397884, 0.146829)\n#\u003e iter  11: ssr = 2.7583, par = (5.13894, 0.397884, 0.146829)\n#\u003e iter  12: ssr = 2.7583, par = (5.13894, 0.397884, 0.146829)\n#\u003e *******************\n#\u003e summary from method 'multifit/levenberg-marquardt+accel'\n#\u003e number of iterations: 12\n#\u003e initial ssr: 210.146\n#\u003e final ssr: 2.7583\n#\u003e ssr/dof: 0.0586872\n#\u003e ssr achieved tolerance: 3.10862e-14\n#\u003e function evaluations: 58\n#\u003e jacobian evaluations: 0\n#\u003e fvv evaluations: 18\n#\u003e status: success\n#\u003e *******************\n#\u003e Nonlinear regression model\n#\u003e   model: y ~ a * exp(-(x - b)^2/(2 * c^2))\n#\u003e      a      b      c \n#\u003e 5.1389 0.3979 0.1468 \n#\u003e  residual sum-of-squares: 2.758\n#\u003e \n#\u003e Algorithm: multifit/levenberg-marquardt+accel, (scaling: more, solver: qr)\n#\u003e \n#\u003e Number of iterations to convergence: 12 \n#\u003e Achieved convergence tolerance: 3.109e-14\n```\n\n### Example 3: Branin function\n\nAs a third example, we compare the available trust region methods by\nminimizing the Branin test function, a common optimization test problem.\nFor the Branin test function, the following bivariate expression is\nused:\n\n![\\left\\\\\n\\begin{aligned}\nF(x_1, x_2) \u0026 \\\\= \\\\f_1(x_1, x_2)^2 + f_2(x_1, x_2)^2 \\\\\nf_1(x_1, x_2) \u0026 \\\\= \\\\x_2 + a_1 x_1^2 + a_2 x_1 + a_3 \\\\\nf_2(x_1, x_2) \u0026 \\\\= \\\\\\sqrt{a_4 \\cdot (1 + (1 - a_5) \\cos(x_1))}\n\\end{aligned}\n\\right.](https://latex.codecogs.com/png.latex?%5Cleft%5C%7B%0A%5Cbegin%7Baligned%7D%0AF%28x_1%2C%20x_2%29%20%26%20%5C%20%3D%20%5C%20f_1%28x_1%2C%20x_2%29%5E2%20%2B%20f_2%28x_1%2C%20x_2%29%5E2%20%5C%5C%0Af_1%28x_1%2C%20x_2%29%20%26%20%5C%20%3D%20%5C%20x_2%20%2B%20a_1%20x_1%5E2%20%2B%20a_2%20x_1%20%2B%20a_3%20%5C%5C%0Af_2%28x_1%2C%20x_2%29%20%26%20%5C%20%3D%20%5C%20%5Csqrt%7Ba_4%20%5Ccdot%20%281%20%2B%20%281%20-%20a_5%29%20%5Ccos%28x_1%29%29%7D%0A%5Cend%7Baligned%7D%0A%5Cright. \"\\left\\{\n\\begin{aligned}\nF(x_1, x_2) \u0026 \\ = \\ f_1(x_1, x_2)^2 + f_2(x_1, x_2)^2 \\\\\nf_1(x_1, x_2) \u0026 \\ = \\ x_2 + a_1 x_1^2 + a_2 x_1 + a_3 \\\\\nf_2(x_1, x_2) \u0026 \\ = \\ \\sqrt{a_4 \\cdot (1 + (1 - a_5) \\cos(x_1))}\n\\end{aligned}\n\\right.\")\n\nwith known constants\n![a_1 = -5.1/(4 \\pi^2)](https://latex.codecogs.com/png.latex?a_1%20%3D%20-5.1%2F%284%20%5Cpi%5E2%29 \"a_1 = -5.1/(4 \\pi^2)\"),\n![a_2 = 5/\\pi](https://latex.codecogs.com/png.latex?a_2%20%3D%205%2F%5Cpi \"a_2 = 5/\\pi\"),\n![a_3 = -6](https://latex.codecogs.com/png.latex?a_3%20%3D%20-6 \"a_3 = -6\"),\n![a_4 = 10](https://latex.codecogs.com/png.latex?a_4%20%3D%2010 \"a_4 = 10\"),\n![a_5 = 1 / (8\\pi)](https://latex.codecogs.com/png.latex?a_5%20%3D%201%20%2F%20%288%5Cpi%29 \"a_5 = 1 / (8\\pi)\")\n(cf. \u003chttps://www.gnu.org/software/gsl/doc/html/nls.html#comparing-trs-methods-example\u003e),\nsuch that\n![F(x_1, x_2)](https://latex.codecogs.com/png.latex?F%28x_1%2C%20x_2%29 \"F(x_1, x_2)\")\nhas three local minima in the range\n![(x_1, x_2) \\in \\[-5, 15\\] \\times \\[-5, 15\\]](https://latex.codecogs.com/png.latex?%28x_1%2C%20x_2%29%20%5Cin%20%5B-5%2C%2015%5D%20%5Ctimes%20%5B-5%2C%2015%5D \"(x_1, x_2) \\in [-5, 15] \\times [-5, 15]\").\n\nThe minimization problem can be solved with `gsl_nls()` by considering\nthe cost function\n![F(x_1, x_2)](https://latex.codecogs.com/png.latex?F%28x_1%2C%20x_2%29 \"F(x_1, x_2)\")\nas a sum of squared residuals in which\n![f_1(x_1, x_2)](https://latex.codecogs.com/png.latex?f_1%28x_1%2C%20x_2%29 \"f_1(x_1, x_2)\")\nand\n![f_2(x_1, x_2)](https://latex.codecogs.com/png.latex?f_2%28x_1%2C%20x_2%29 \"f_2(x_1, x_2)\")\nare two residuals relative to two zero responses. Here, instead of\npassing a `formula` to `gsl_nls()`, the nonlinear model is passed\ndirectly as a `function`. The first parameter of the function is the\nvector of parameters,\ni.e. ![(x_1, x_2)](https://latex.codecogs.com/png.latex?%28x_1%2C%20x_2%29 \"(x_1, x_2)\"),\nand the function returns the vector of model evaluations,\ni.e. ![(f_1(x_1, x_2), f_2(x_1, x_2))](https://latex.codecogs.com/png.latex?%28f_1%28x_1%2C%20x_2%29%2C%20f_2%28x_1%2C%20x_2%29%29 \"(f_1(x_1, x_2), f_2(x_1, x_2))\").\nWhen passing a `function` instead of `formula` to `gsl_nls()`, the\nvector of observed responses should be included in the `y` argument. In\nthis example, `y` is set to a vector of zeros. As starting values, we\nuse\n![x_1 = 6](https://latex.codecogs.com/png.latex?x_1%20%3D%206 \"x_1 = 6\")\nand\n![x_2 = 14.5](https://latex.codecogs.com/png.latex?x_2%20%3D%2014.5 \"x_2 = 14.5\")\nequivalent to the example in the GSL reference manual.\n\n``` r\n## Branin model function\nbranin \u003c- function(x) {\n  a \u003c- c(-5.1 / (4 * pi^2), 5 / pi, -6, 10, 1 / (8 * pi))\n  f1 \u003c- x[2] + a[1] * x[1]^2 + a[2] * x[1] + a[3] \n  f2 \u003c- sqrt(a[4] * (1 + (1 - a[5]) * cos(x[1])))\n  c(f1, f2)\n}\n\n## Levenberg-Marquardt minimization\nex3_fit \u003c- gsl_nls(\n  fn = branin,                   ## model function      \n  y = c(0, 0),                   ## response vector \n  start = c(x1 = 6, x2 = 14.5),  ## starting values\n  algorithm = \"lm\"               ## algorithm\n)\n\nex3_fit\n#\u003e Nonlinear regression model\n#\u003e   model: y ~ fn(x)\n#\u003e     x1     x2 \n#\u003e -3.142 12.275 \n#\u003e  residual sum-of-squares: 0.3979\n#\u003e \n#\u003e Algorithm: multifit/levenberg-marquardt, (scaling: more, solver: qr)\n#\u003e \n#\u003e Number of iterations to convergence: 20 \n#\u003e Achieved convergence tolerance: 1.665e-16\n```\n\n**Note**: When using the `function` method of `gsl_nls()`, the returned\nobject no longer has class `\"nls\"`,\n\n``` r\nclass(ex3_fit)\n#\u003e [1] \"gsl_nls\"\n```\n\nHowever, all generics `anova`, `coef`, `confint`, `deviance`,\n`df.residual`, `fitted`, `formula`, `logLik`, `predict`, `print`,\n`residuals`, `summary`, `vcov` and `weights` remain applicable to\nobjects (only) of class `\"gls_nls\"`.\n\n#### Method comparisons\n\nSolving the same minimization problem with all available trust region\nmethods, i.e. `algorithm` set to `\"lm\"`, `\"lmaccel\"`, `\"dogleg\"`,\n`\"ddogleg\"` and `\"subspace2D\"` respectively, and tracing the parameter\nvalues at each iteration, we can visualize the minimization paths\nfollowed by each method.\n\n\u003cimg src=\"README/fig-4-1.png\" width=\"100%\" style=\"display: block; margin: auto;\" /\u003e\n\nAnalogous to the\n[example](https://www.gnu.org/software/gsl/doc/html/nls.html#comparing-trs-methods-example)\nin the GSL reference manual, the standard Levenberg-Marquardt method\nwithout geodesic acceleration converges to the minimum at\n![(-\\pi, 12.275)](https://latex.codecogs.com/png.latex?%28-%5Cpi%2C%2012.275%29 \"(-\\pi, 12.275)\"),\nall other methods converge to the minimum at\n![(\\pi, 2.275)](https://latex.codecogs.com/png.latex?%28%5Cpi%2C%202.275%29 \"(\\pi, 2.275)\").\n\n### Example 4: Large NLS example\n\nTo illustrate the use of `gsl_nls_large()`, we reproduce the large\nnonlinear least-squares example\n(\u003chttps://www.gnu.org/software/gsl/doc/html/nls.html#large-nonlinear-least-squares-example\u003e)\nfrom the GSL reference manual. The nonlinear least squares model is\ndefined as:\n\n![\\left\\\\\n\\begin{aligned}\nf_i \u0026 \\\\= \\sqrt{\\alpha}(\\theta_i + 1), \\quad i = 1,\\ldots,p \\\\\nf\\_{p + 1} \u0026 \\\\= \\Vert \\boldsymbol{\\theta} \\Vert^2  - \\frac{1}{4}\n\\end{aligned}\n\\right.](https://latex.codecogs.com/png.latex?%5Cleft%5C%7B%0A%5Cbegin%7Baligned%7D%0Af_i%20%26%20%5C%20%3D%20%5Csqrt%7B%5Calpha%7D%28%5Ctheta_i%20%2B%201%29%2C%20%5Cquad%20i%20%3D%201%2C%5Cldots%2Cp%20%5C%5C%0Af_%7Bp%20%2B%201%7D%20%26%20%5C%20%3D%20%5CVert%20%5Cboldsymbol%7B%5Ctheta%7D%20%5CVert%5E2%20%20-%20%5Cfrac%7B1%7D%7B4%7D%0A%5Cend%7Baligned%7D%0A%5Cright. \"\\left\\{\n\\begin{aligned}\nf_i \u0026 \\ = \\sqrt{\\alpha}(\\theta_i + 1), \\quad i = 1,\\ldots,p \\\\\nf_{p + 1} \u0026 \\ = \\Vert \\boldsymbol{\\theta} \\Vert^2  - \\frac{1}{4}\n\\end{aligned}\n\\right.\")\n\nwith given constant\n![\\alpha = 10^{-5}](https://latex.codecogs.com/png.latex?%5Calpha%20%3D%2010%5E%7B-5%7D \"\\alpha = 10^{-5}\")\nand unknown parameters\n![\\theta_1,\\ldots, \\theta_p](https://latex.codecogs.com/png.latex?%5Ctheta_1%2C%5Cldots%2C%20%5Ctheta_p \"\\theta_1,\\ldots, \\theta_p\").\nThe residual\n![f\\_{p + 1}](https://latex.codecogs.com/png.latex?f_%7Bp%20%2B%201%7D \"f_{p + 1}\")\nadds an\n![L_2](https://latex.codecogs.com/png.latex?L_2 \"L_2\")-regularization\nconstraint on the parameter vector and makes the model nonlinear. The\n![(p + 1) \\times p](https://latex.codecogs.com/png.latex?%28p%20%2B%201%29%20%5Ctimes%20p \"(p + 1) \\times p\")-dimensional\nJacobian matrix is given by:\n\n![\\boldsymbol{J}(\\boldsymbol{\\theta}) \\\\= \\\\\n\\left\\[ \\begin{matrix} \n\\frac{\\partial f_1}{\\partial \\theta_1} \u0026 \\ldots \u0026 \\frac{\\partial f_1}{\\partial \\theta_p} \\\\\n\\vdots \u0026 \\ddots \u0026 \\vdots \\\\\n\\frac{\\partial f\\_{p+1}}{\\partial \\theta_1} \u0026 \\ldots \u0026 \\frac{\\partial f\\_{p+1}}{\\partial \\theta_p}\n\\end{matrix} \\right\\] \\\\= \n\\left\\[ \\begin{matrix}\n\\sqrt{\\alpha} \\boldsymbol{I}\\_{p \\times p} \\\\\n2 \\boldsymbol{\\theta}'\n\\end{matrix} \\right\\]](https://latex.codecogs.com/png.latex?%5Cboldsymbol%7BJ%7D%28%5Cboldsymbol%7B%5Ctheta%7D%29%20%5C%20%3D%20%5C%20%0A%5Cleft%5B%20%5Cbegin%7Bmatrix%7D%20%0A%5Cfrac%7B%5Cpartial%20f_1%7D%7B%5Cpartial%20%5Ctheta_1%7D%20%26%20%5Cldots%20%26%20%5Cfrac%7B%5Cpartial%20f_1%7D%7B%5Cpartial%20%5Ctheta_p%7D%20%5C%5C%0A%5Cvdots%20%26%20%5Cddots%20%26%20%5Cvdots%20%5C%5C%0A%5Cfrac%7B%5Cpartial%20f_%7Bp%2B1%7D%7D%7B%5Cpartial%20%5Ctheta_1%7D%20%26%20%5Cldots%20%26%20%5Cfrac%7B%5Cpartial%20f_%7Bp%2B1%7D%7D%7B%5Cpartial%20%5Ctheta_p%7D%0A%5Cend%7Bmatrix%7D%20%5Cright%5D%20%5C%20%3D%20%0A%5Cleft%5B%20%5Cbegin%7Bmatrix%7D%0A%5Csqrt%7B%5Calpha%7D%20%5Cboldsymbol%7BI%7D_%7Bp%20%5Ctimes%20p%7D%20%5C%5C%0A2%20%5Cboldsymbol%7B%5Ctheta%7D%27%0A%5Cend%7Bmatrix%7D%20%5Cright%5D \"\\boldsymbol{J}(\\boldsymbol{\\theta}) \\ = \\ \n\\left[ \\begin{matrix} \n\\frac{\\partial f_1}{\\partial \\theta_1} \u0026 \\ldots \u0026 \\frac{\\partial f_1}{\\partial \\theta_p} \\\\\n\\vdots \u0026 \\ddots \u0026 \\vdots \\\\\n\\frac{\\partial f_{p+1}}{\\partial \\theta_1} \u0026 \\ldots \u0026 \\frac{\\partial f_{p+1}}{\\partial \\theta_p}\n\\end{matrix} \\right] \\ = \n\\left[ \\begin{matrix}\n\\sqrt{\\alpha} \\boldsymbol{I}_{p \\times p} \\\\\n2 \\boldsymbol{\\theta}'\n\\end{matrix} \\right]\")\n\nwith\n![\\boldsymbol{I}\\_{p \\times p}](https://latex.codecogs.com/png.latex?%5Cboldsymbol%7BI%7D_%7Bp%20%5Ctimes%20p%7D \"\\boldsymbol{I}_{p \\times p}\")\nthe\n![(p \\times p)](https://latex.codecogs.com/png.latex?%28p%20%5Ctimes%20p%29 \"(p \\times p)\")-dimensional\nidentity matrix.\n\nThe model residuals and Jacobian matrix can be written as a function of\nthe parameter vector\n![\\boldsymbol{\\theta}](https://latex.codecogs.com/png.latex?%5Cboldsymbol%7B%5Ctheta%7D \"\\boldsymbol{\\theta}\")\nas,\n\n``` r\n## model and jacobian\nf \u003c- function(theta) {\n  val \u003c- c(sqrt(1e-5) * (theta - 1), sum(theta^2) - 0.25)\n  attr(val, \"gradient\") \u003c- rbind(diag(sqrt(1e-5), nrow = length(theta)), 2 * t(theta))\n  return(val)\n}\n```\n\nHere, the Jacobian is returned in the `\"gradient\"` attribute of the\nevaluated vector (as in a `selfStart` model) from which it is detected\nautomatically by `gsl_nls()` or `gsl_nls_large()`.\n\nFirst, the least squares objective is minimized with a call to\n`gsl_nls()` analogous to the previous example by passing the nonlinear\nmodel as a `function` and setting the response vector `y` to a vector of\nzeros. The number of parameters is set to\n![p = 500](https://latex.codecogs.com/png.latex?p%20%3D%20500 \"p = 500\")\nand as starting values we use\n![\\theta_1 = 1, \\ldots, \\theta_p = p](https://latex.codecogs.com/png.latex?%5Ctheta_1%20%3D%201%2C%20%5Cldots%2C%20%5Ctheta_p%20%3D%20p \"\\theta_1 = 1, \\ldots, \\theta_p = p\")\nequivalent to the example in the GSL reference manual.\n\n``` r\n## number of parameters\np \u003c- 500\n\n## standard Levenberg-Marquardt\nsystem.time({ \n  ex4_fit_lm \u003c- gsl_nls(\n    fn = f,\n    y = rep(0, p + 1),\n    start = 1:p,\n    control = list(maxiter = 500)\n  )\n})\n#\u003e    user  system elapsed \n#\u003e  36.507   0.140  36.661\n\ncat(\"Residual sum-of-squares:\", deviance(ex4_fit_lm), \"\\n\")\n#\u003e Residual sum-of-squares: 0.004778845\n```\n\nSecond, the same model is fitted with a call to `gsl_nls_large()` using\nthe Steihaug-Toint Conjugate Gradient algorithm, which results in a much\nsmaller runtime:\n\n``` r\n## large-scale Steihaug-Toint \nsystem.time({ \n  ex4_fit_cgst \u003c- gsl_nls_large(\n    fn = f,\n    y = rep(0, p + 1),\n    start = 1:p,\n    algorithm = \"cgst\",\n    control = list(maxiter = 500)\n  )\n})\n#\u003e    user  system elapsed \n#\u003e   1.424   0.348   1.779\n\ncat(\"Residual sum-of-squares:\", deviance(ex4_fit_cgst), \"\\n\")\n#\u003e Residual sum-of-squares: 0.004778845\n```\n\n#### Sparse Jacobian matrix\n\nThe Jacobian matrix\n![\\boldsymbol{J}(\\boldsymbol{\\theta})](https://latex.codecogs.com/png.latex?%5Cboldsymbol%7BJ%7D%28%5Cboldsymbol%7B%5Ctheta%7D%29 \"\\boldsymbol{J}(\\boldsymbol{\\theta})\")\nis very sparse in the sense that it contains only a small number of\nnonzero entries. The `gsl_nls_large()` function also accepts the\ncalculated Jacobian as a sparse matrix of class `\"dgCMatrix\"`,\n`\"dgRMatrix\"` or `\"dgTMatrix\"` (see the\n[Matrix](https://cran.r-project.org/web/packages/Matrix/Matrix.pdf)\npackage). The following updated model function returns the sparse\nJacobian as a `\"dgCMatrix\"` instead of a dense numeric matrix:\n\n``` r\n## model and sparse Jacobian\nfsp \u003c- function(theta) {\n  val \u003c- c(sqrt(1e-5) * (theta - 1), sum(theta^2) - 0.25)\n  attr(val, \"gradient\") \u003c- rbind(Matrix::Diagonal(x = sqrt(1e-5), n = length(theta)), 2 * t(theta))\n  return(val)\n}\n```\n\nAs illustrated by the benchmarks below, besides a slight improvement in\nruntimes, the required amount of memory is significantly smaller for the\nmodel functions returning a sparse Jacobian than the model functions\nreturning a dense Jacobian:\n\n``` r\n## computation times and allocated memory\nbench::mark(\n  \"Dense LM\" = gsl_nls_large(fn = f, y = rep(0, p + 1), start = 1:p, algorithm = \"lm\", control = list(maxiter = 500)),\n  \"Dense CGST\" = gsl_nls_large(fn = f, y = rep(0, p + 1), start = 1:p, algorithm = \"cgst\"),\n  \"Sparse LM\" = gsl_nls_large(fn = fsp, y = rep(0, p + 1), start = 1:p, algorithm = \"lm\", control = list(maxiter = 500)),\n  \"Sparse CGST\" = gsl_nls_large(fn = fsp, y = rep(0, p + 1), start = 1:p, algorithm = \"cgst\"),\n  check = FALSE,\n  min_iterations = 5\n)\n#\u003e # A tibble: 4 × 6\n#\u003e   expression       min   median `itr/sec` mem_alloc `gc/sec`\n#\u003e   \u003cbch:expr\u003e  \u003cbch:tm\u003e \u003cbch:tm\u003e     \u003cdbl\u003e \u003cbch:byt\u003e    \u003cdbl\u003e\n#\u003e 1 Dense LM       7.74s     7.8s     0.128    1.82GB    4.91 \n#\u003e 2 Dense CGST     1.31s    1.32s     0.722    1.02GB   16.0  \n#\u003e 3 Sparse LM      5.65s    5.67s     0.176   33.41MB    0.106\n#\u003e 4 Sparse CGST 149.31ms 158.18ms     6.44    23.04MB    3.86\n```\n\n## NLS test problems\n\nThe R-package currently contains a collection of 59 NLS test problems\noriginating primarily from the [NIST Statistical Reference Datasets\n(StRD)](https://www.itl.nist.gov/div898/strd/nls/nls_main.shtml)\narchive; Bates and Watts (1988); and Moré, Garbow, and Hillstrom (1981).\n\n``` r\n## avalable test problems\nnls_test_list()\n#\u003e                                     name    class  p   n       check\n#\u003e 1                                Misra1a  formula  2  14  p, n fixed\n#\u003e 2                               Chwirut2  formula  3  54  p, n fixed\n#\u003e 3                               Chwirut1  formula  3 214  p, n fixed\n#\u003e 4                               Lanczos3  formula  6  24  p, n fixed\n#\u003e 5                                 Gauss1  formula  8 250  p, n fixed\n#\u003e 6                                 Gauss2  formula  8 250  p, n fixed\n#\u003e 7                                DanWood  formula  2   6  p, n fixed\n#\u003e 8                                Misra1b  formula  2  14  p, n fixed\n#\u003e 9                                 Kirby2  formula  5 151  p, n fixed\n#\u003e 10                                 Hahn1  formula  7 236  p, n fixed\n#\u003e 11                                Nelson  formula  3 128  p, n fixed\n#\u003e 12                                 MGH17  formula  5  33  p, n fixed\n#\u003e 13                              Lanczos1  formula  6  24  p, n fixed\n#\u003e 14                              Lanczos2  formula  6  24  p, n fixed\n#\u003e 15                                Gauss3  formula  8 250  p, n fixed\n#\u003e 16                               Misra1c  formula  2  14  p, n fixed\n#\u003e 17                               Misra1d  formula  2  14  p, n fixed\n#\u003e 18                              Roszman1  formula  4  25  p, n fixed\n#\u003e 19                                  ENSO  formula  9 168  p, n fixed\n#\u003e 20                                 MGH09  formula  4  11  p, n fixed\n#\u003e 21                               Thurber  formula  7  37  p, n fixed\n#\u003e 22                                BoxBOD  formula  2   6  p, n fixed\n#\u003e 23                            Ratkowsky2  formula  3   9  p, n fixed\n#\u003e 24                                 MGH10  formula  3  16  p, n fixed\n#\u003e 25                              Eckerle4  formula  3  35  p, n fixed\n#\u003e 26                            Ratkowsky3  formula  4  15  p, n fixed\n#\u003e 27                              Bennett5  formula  3 154  p, n fixed\n#\u003e 28                         Isomerization  formula  4  24  p, n fixed\n#\u003e 29                             Lubricant  formula  9  53  p, n fixed\n#\u003e 30                         Sulfisoxazole  formula  4  12  p, n fixed\n#\u003e 31                                Leaves  formula  4  15  p, n fixed\n#\u003e 32                              Chloride  formula  3  54  p, n fixed\n#\u003e 33                          Tetracycline  formula  4   9  p, n fixed\n#\u003e 34                     Linear, full rank function  5  10 p \u003c= n free\n#\u003e 35                        Linear, rank 1 function  5  10 p \u003c= n free\n#\u003e 36 Linear, rank 1, zero columns and rows function  5  10 p \u003c= n free\n#\u003e 37                            Rosenbrock function  2   2  p, n fixed\n#\u003e 38                        Helical valley function  3   3  p, n fixed\n#\u003e 39                       Powell singular function  4   4  p, n fixed\n#\u003e 40                     Freudenstein/Roth function  2   2  p, n fixed\n#\u003e 41                                  Bard function  3  15  p, n fixed\n#\u003e 42                   Kowalik and Osborne function  4  11  p, n fixed\n#\u003e 43                                 Meyer function  3  16  p, n fixed\n#\u003e 44                                Watson function  6  31  p, n fixed\n#\u003e 45                     Box 3-dimensional function  3  10 p \u003c= n free\n#\u003e 46                  Jennrich and Sampson function  2  10 p \u003c= n free\n#\u003e 47                      Brown and Dennis function  4  20 p \u003c= n free\n#\u003e 48                             Chebyquad function  9   9 p \u003c= n free\n#\u003e 49                   Brown almost-linear function 10  10 p == n free\n#\u003e 50                             Osborne 1 function  5  33  p, n fixed\n#\u003e 51                             Osborne 2 function 11  65  p, n fixed\n#\u003e 52                              Hanson 1 function  2  16  p, n fixed\n#\u003e 53                              Hanson 2 function  3  16  p, n fixed\n#\u003e 54                             McKeown 1 function  2   3  p, n fixed\n#\u003e 55                             McKeown 2 function  3   4  p, n fixed\n#\u003e 56                             McKeown 3 function  5  10  p, n fixed\n#\u003e 57              Devilliers and Glasser 1 function  4  24  p, n fixed\n#\u003e 58              Devilliers and Glasser 2 function  5  16  p, n fixed\n#\u003e 59                        Madsen example function  2   3  p, n fixed\n```\n\nThe function `nls_test_problem()` fetches the model definition and model\ndata required to solve a specific NLS test problem with `gsl_nls()` (or\n`nls()` if the model is defined as a `formula`). This also returns the\nvector of certified target values corresponding to the *best-available*\nsolutions and a vector of suggested starting values for the parameters:\n\n``` r\n## example regression problem\n(ratkowsky2 \u003c- nls_test_problem(name = \"Ratkowsky2\"))\n#\u003e $data\n#\u003e       y  x\n#\u003e 1  8.93  9\n#\u003e 2 10.80 14\n#\u003e 3 18.59 21\n#\u003e 4 22.33 28\n#\u003e 5 39.35 42\n#\u003e 6 56.11 57\n#\u003e 7 61.73 63\n#\u003e 8 64.62 70\n#\u003e 9 67.08 79\n#\u003e \n#\u003e $fn\n#\u003e y ~ b1/(1 + exp(b2 - b3 * x))\n#\u003e \u003cenvironment: 0x5582f0974200\u003e\n#\u003e \n#\u003e $start\n#\u003e    b1    b2    b3 \n#\u003e 100.0   1.0   0.1 \n#\u003e \n#\u003e $target\n#\u003e         b1         b2         b3 \n#\u003e 72.4622376  2.6180768  0.0673592 \n#\u003e \n#\u003e attr(,\"class\")\n#\u003e [1] \"nls_test_formula\"\n\nwith(ratkowsky2,\n     gsl_nls(\n       fn = fn,\n       data = data,\n       start = start\n     )\n)\n#\u003e Nonlinear regression model\n#\u003e   model: y ~ b1/(1 + exp(b2 - b3 * x))\n#\u003e    data: data\n#\u003e       b1       b2       b3 \n#\u003e 72.46224  2.61808  0.06736 \n#\u003e  residual sum-of-squares: 8.057\n#\u003e \n#\u003e Algorithm: multifit/levenberg-marquardt, (scaling: more, solver: qr)\n#\u003e \n#\u003e Number of iterations to convergence: 10 \n#\u003e Achieved convergence tolerance: 5.151e-14\n\n## example optimization problem\nmadsen \u003c- nls_test_problem(name = \"Madsen\")\nwith(madsen,\n     gsl_nls(\n       fn = fn,\n       y = y,\n       start = start,\n       jac = jac\n     )\n)\n#\u003e Nonlinear regression model\n#\u003e   model: y ~ fn(x)\n#\u003e      x1      x2 \n#\u003e -0.1554  0.6946 \n#\u003e  residual sum-of-squares: 0.7732\n#\u003e \n#\u003e Algorithm: multifit/levenberg-marquardt, (scaling: more, solver: qr)\n#\u003e \n#\u003e Number of iterations to convergence: 42 \n#\u003e Achieved convergence tolerance: 2.22e-16\n```\n\n## Other R-packages\n\nOther CRAN R-packages interfacing with GSL that served as inspiration\nfor this package include:\n\n- [RcppGSL](https://cran.r-project.org/web/packages/RcppGSL/index.html)\n  by Dirk Eddelbuettel and Romain Francois\n- [GSL](https://cran.r-project.org/web/packages/gsl/index.html) by Robin\n  Hankin and others\n- [RcppZiggurat](https://cran.r-project.org/web/packages/RcppZiggurat/index.html)\n  by Dirk Eddelbuettel\n\nExcept for Barron’s family of loss functions, all robust loss functions\nare credited to\n[robustbase](https://cran.r-project.org/web/packages/robustbase/index.html)\nby Martin Maechler and others.\n\n## License\n\nLGPL-3\n\n# References\n\n\u003cdiv id=\"refs\" class=\"references csl-bib-body hanging-indent\"\nentry-spacing=\"0\"\u003e\n\n\u003cdiv id=\"ref-BW88\" class=\"csl-entry\"\u003e\n\nBates, D.M., and D.G. Watts. 1988. *Nonlinear Regression Analysis and\nIts Applications ISBN 0471816434*.\n\n\u003c/div\u003e\n\n\u003cdiv id=\"ref-gsl_manual\" class=\"csl-entry\"\u003e\n\nGalassi, M., J. Davies, J. Theiler, B. Gough, G. Jungman, M. Booth, and\nF. Rossi. 2009. *GNU Scientific Library Reference Manual (3rd Ed.), ISBN\n0954612078*. \u003chttps://www.gnu.org/software/gsl/\u003e.\n\n\u003c/div\u003e\n\n\u003cdiv id=\"ref-HY97\" class=\"csl-entry\"\u003e\n\nHickernell, F.J., and Y. Yuan. 1997. “A Simple Multistart Algorithm for\nGlobal Optimization.” *OR Transactions* 1(2).\n\n\u003c/div\u003e\n\n\u003cdiv id=\"ref-MGH81\" class=\"csl-entry\"\u003e\n\nMoré, J.J., B.S. Garbow, and K.E. Hillstrom. 1981. “Testing\nUnconstrained Optimization Software.” *ACM Transactions on Mathematical\nSoftware (TOMS)* 7(1).\n\n\u003c/div\u003e\n\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjorischau%2Fgslnls","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjorischau%2Fgslnls","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjorischau%2Fgslnls/lists"}