Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/yixuan/LBFGSpp

A header-only C++ library for L-BFGS and L-BFGS-B algorithms
https://github.com/yixuan/LBFGSpp

l-bfgs l-bfgs-b lbfgs-solver lbfgsb-solver limited-memory optimization

Last synced: 27 days ago
JSON representation

A header-only C++ library for L-BFGS and L-BFGS-B algorithms

Lists

README 

        
# LBFGS++ LBFGS++



> **UPDATE on 2020-03-06**: **LBFGS++** now includes a new L-BFGS-B solver for

> box-constrained optimization problems. Check the example below for its usage.



**LBFGS++** is a header-only C++ library that implements the Limited-memory

BFGS algorithm (L-BFGS) for unconstrained minimization problems, and a modified

version of the L-BFGS-B algorithm for box-constrained ones.



The code for the L-BFGS solver is derived and modified from the

[libLBFGS](https://github.com/chokkan/liblbfgs)

library developed by [Naoaki Okazaki](http://www.chokkan.org/).



**LBFGS++** is implemented as a header-only C++ library, whose only dependency,

[Eigen](http://eigen.tuxfamily.org/), is also header-only.



## A Quick Example



To use **LBFGS++**, one needs to first define a functor to represent the

multivariate function to be minimized. It should return the objective function

value on a vector `x` and overwrite the vector `grad` with the gradient

evaluated on `x`. For example we could define the

[Rosenbrock function](https://en.wikipedia.org/wiki/Rosenbrock_function) in the

following way:



```cpp

#include 

#include 

#include 



using Eigen::VectorXd;

using namespace LBFGSpp;



class Rosenbrock

{

private:

    int n;

public:

    Rosenbrock(int n_) : n(n_) {}

    double operator()(const VectorXd& x, VectorXd& grad)

    {

        double fx = 0.0;

        for(int i = 0; i < n; i += 2)

        {

            double t1 = 1.0 - x[i];

            double t2 = 10 * (x[i + 1] - x[i] * x[i]);

            grad[i + 1] = 20 * t2;

            grad[i]     = -2.0 * (x[i] * grad[i + 1] + t1);

            fx += t1 * t1 + t2 * t2;

        }

        return fx;

    }

};

```



Then we just need to set up parameters, create solver object,

provide initial guess, and then run the minimization function.



```cpp

int main()

{

    const int n = 10;

    // Set up parameters

    LBFGSParam param;

    param.epsilon = 1e-6;

    param.max_iterations = 100;



    // Create solver and function object

    LBFGSSolver solver(param);

    Rosenbrock fun(n);



    // Initial guess

    VectorXd x = VectorXd::Zero(n);

    // x will be overwritten to be the best point found

    double fx;

    int niter = solver.minimize(fun, x, fx);



    std::cout << niter << " iterations" << std::endl;

    std::cout << "x = \n" << x.transpose() << std::endl;

    std::cout << "f(x) = " << fx << std::endl;



    return 0;

}

```



The example can then be compiled and run.



```bash

$ g++ -I/path/to/eigen -I/path/to/lbfgspp/include -O2 example.cpp

$ ./a.out

23 iterations

x =

1 1 1 1 1 1 1 1 1 1

f(x) = 1.87948e-19

```



You can also use a different line search algorithm by providing a second template parameter

to `LBFGSSolver`. For example, the code below illustrates the bracketing line search algorithm

(contributed by [@DirkToewe](https://github.com/DirkToewe)).



```cpp

int main()

{

    const int n = 10;

    // Set up parameters

    LBFGSParam param;

    param.epsilon = 1e-6;

    param.max_iterations = 100;



    // Create solver and function object

    LBFGSSolver solver(param);

    Rosenbrock fun(n);



    // Initial guess

    VectorXd x = VectorXd::Zero(n);

    // x will be overwritten to be the best point found

    double fx;

    int niter = solver.minimize(fun, x, fx);



    std::cout << niter << " iterations" << std::endl;

    std::cout << "x = \n" << x.transpose() << std::endl;

    std::cout << "f(x) = " << fx << std::endl;



    return 0;

}

```



## Box-constrained Problem



If the parameters to be optimized have simple bounds, then the

L-BFGS-**B** solver class `LBFGSBSolver` can be used.

The code is very similar to that of `LBFGSSolver`. Below is the same Rosenbrock

example, but we require that all variables should be between 2 and 4.



```cpp

#include 

#include 

#include   // Note the different header file



using Eigen::VectorXd;

using namespace LBFGSpp;



class Rosenbrock

{

private:

    int n;

public:

    Rosenbrock(int n_) : n(n_) {}

    double operator()(const VectorXd& x, VectorXd& grad)

    {

        double fx = 0.0;

        for(int i = 0; i < n; i += 2)

        {

            double t1 = 1.0 - x[i];

            double t2 = 10 * (x[i + 1] - x[i] * x[i]);

            grad[i + 1] = 20 * t2;

            grad[i]     = -2.0 * (x[i] * grad[i + 1] + t1);

            fx += t1 * t1 + t2 * t2;

        }

        return fx;

    }

};



int main()

{

    const int n = 10;

    // Set up parameters

    LBFGSBParam param;  // New parameter class

    param.epsilon = 1e-6;

    param.max_iterations = 100;



    // Create solver and function object

    LBFGSBSolver solver(param);  // New solver class

    Rosenbrock fun(n);



    // Bounds

    VectorXd lb = VectorXd::Constant(n, 2.0);

    VectorXd ub = VectorXd::Constant(n, 4.0);



    // Initial guess

    VectorXd x = VectorXd::Constant(n, 3.0);



    // x will be overwritten to be the best point found

    double fx;

    int niter = solver.minimize(fun, x, fx, lb, ub);



    std::cout << niter << " iterations" << std::endl;

    std::cout << "x = \n" << x.transpose() << std::endl;

    std::cout << "f(x) = " << fx << std::endl;



    return 0;

}

```



Note that we also allow infinite values for the lower and upper bounds.

In such cases one can define `ub[i] = std::numeric_limits::infinity()`,

for example.



## Documentation



The [API reference](https://lbfgspp.statr.me/doc/) page contains the documentation

of **LBFGS++** generated by [Doxygen](https://www.doxygen.nl/).



## License



**LBFGS++** is an open source project under the MIT license.