{"id":15358092,"url":"https://github.com/antonior92/ip-nonlinear-solver","last_synced_at":"2025-06-24T08:04:16.192Z","repository":{"id":73581241,"uuid":"92684946","full_name":"antonior92/ip-nonlinear-solver","owner":"antonior92","description":"A trust-region interior-point method for general nonlinear programing problems (GSoC 2017).","archived":false,"fork":false,"pushed_at":"2018-04-24T14:45:21.000Z","size":753,"stargazers_count":36,"open_issues_count":0,"forks_count":13,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-15T07:23:56.547Z","etag":null,"topics":["constraints","gsoc","gsoc-2017","interior-point-method","nonlinear-programming-algorithms","optimization","scipy","sequential-quadratic-programming"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/antonior92.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2017-05-28T20:20:51.000Z","updated_at":"2024-10-07T06:44:20.000Z","dependencies_parsed_at":null,"dependency_job_id":"03606f62-f4bc-4ff0-a277-51af68ecb0bc","html_url":"https://github.com/antonior92/ip-nonlinear-solver","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/antonior92/ip-nonlinear-solver","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antonior92%2Fip-nonlinear-solver","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antonior92%2Fip-nonlinear-solver/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antonior92%2Fip-nonlinear-solver/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antonior92%2Fip-nonlinear-solver/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/antonior92","download_url":"https://codeload.github.com/antonior92/ip-nonlinear-solver/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antonior92%2Fip-nonlinear-solver/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261632138,"owners_count":23187269,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["constraints","gsoc","gsoc-2017","interior-point-method","nonlinear-programming-algorithms","optimization","scipy","sequential-quadratic-programming"],"created_at":"2024-10-01T12:39:55.801Z","updated_at":"2025-06-24T08:04:16.179Z","avatar_url":"https://github.com/antonior92.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ip-nonlinear-solver\n\n**IMPORTANT: These algorithms have been integrated to SciPy library (release 1.1 or above) under the name of ``trust-constr``. The version available there is more complete and better mantained.**\n\n[![Join the chat at https://gitter.im/ip-nonlinear-solver/GSoC2017](https://badges.gitter.im/ip-nonlinear-solver/GSoC2017.svg)](https://gitter.im/ip-nonlinear-solver/GSoC2017?utm_source=badge\u0026utm_medium=badge\u0026utm_campaign=pr-badge\u0026utm_content=badge)\n\n\n\nA trust-region interior-point method for general nonlinear programing problems. The implemetation\nis part my GSoC project for Scipy. A series of blog post describe different aspects of the algorithm\nand its use  ([link](https://antonior92.github.io/tags/#gsoc-2017)).\n\nThe implementation is being\nintegrated to SciPy through the pull request:\n\n[https://github.com/scipy/scipy/pull/7729](https://github.com/scipy/scipy/pull/7729)\n\n\n## Instalation Guide\n\n1) **Download repository from github**\n\n```bash\ngit clone git@github.com:antonior92/ip-nonlinear-solver.git\n# or, alternatively:\n# git clone https://github.com/antonior92/ip-nonlinear-solver.git\n```\n\n2) **Install package**\n\nMove into the downloaded directory and install requirements with:\n```bash\npip install -r requirements.txt\n```\n\nIn sequence, install package with:\n```bash\npython setup.py install\n```\n\n3) **Test instalation**\n\nThe instalation can be tested by running:\n```bash\npython setup.py test\n```\ninside the downloaded directory.\n\n## Usage Example\nConsider the following minimization problem:\n```\nmin (1/2)*(x -2)**2 + (1/2)*(y - 1/2)**2, \nsubject to: 1/(x + 1) - y \u003e= 1/4;\n            x \u003e= 0\n            y \u003e= 0 \n```\n\nThe code for solving this problem is presented bellow:\n\n```Python\n# Example\nfrom __future__ import division\nimport numpy as np\nfrom ipsolver import minimize_constrained, NonlinearConstraint, BoxConstraint\n\n# Define objective function and derivatives\nfun = lambda x: 1/2*(x[0] - 2)**2 + 1/2*(x[1] - 1/2)**2\ngrad = lambda x: np.array([x[0] - 2, x[1] - 1/2])\nhess =  lambda x: np.eye(2)\n# Define nonlinear constraint\nc = lambda x: np.array([1/(x[0] + 1) - x[1],])\nc_jac = lambda x: np.array([[-1/(x[0] + 1)**2, -1]])\nc_hess = lambda x, v: 2*v[0]*np.array([[1/(x[0] + 1)**3, 0], [0, 0]])\nnonlinear = NonlinearConstraint(c, ('greater', 1/4), c_jac, c_hess)\n# Define box constraint\nbox = BoxConstraint((\"greater\",))\n\n# Define initial point\nx0 = np.array([0, 0])\n# Apply solver\nresult = minimize_constrained(fun, x0, grad, hess, (nonlinear, box))\n\n# Print result\nprint(result.x)\n```\n\n## Documentation\n\nThe function [``minimize_constrained``](https://github.com/antonior92/ip-nonlinear-solver#minimize_constrained) solves nonlinear programming problems.\nThis functions minimizes an object function subject to constraints. These constraints\ncan be specified using the classes [``NonlinearConstraint``](https://github.com/antonior92/ip-nonlinear-solver#nonlinearconstraint),\n[``LinearConstraint``](https://github.com/antonior92/ip-nonlinear-solver#linearconstraint) and [``BoxConstraint``](https://github.com/antonior92/ip-nonlinear-solver#BoxConstraint).\n\n\n### ``minimize_constrained``\n\n```\nminimize_constrained(fun, x0, grad, hess='2-point', constraints=(),\n                     method=None, xtol=1e-8, gtol=1e-8, sparse_jacobian=None,\n                     options={}, callback=None, max_iter=1000, verbose=0)\n\n    Minimize scalar function subject to constraints.\n\n    Parameters\n    ----------\n    fun : callable\n        The objective function to be minimized.\n\n            fun(x) -\u003e float\n\n        where x is an array with shape (n,).\n    x0 : ndarray, shape (n,)\n        Initial guess. Array of real elements of size (n,),\n        where ``n`` is the number of independent variables.\n    grad : callable\n        Gradient of the objective function:\n\n            grad(x) -\u003e array_like, shape (n,)\n\n        where x is an array with shape (n,).\n    hess : {callable, '2-point', '3-point', 'cs', None}, optional\n        Method for computing the Hessian matrix. The keywords\n        select a finite difference scheme for numerical\n        estimation. The scheme '3-point' is more accurate, but requires\n        twice as much operations compared to '2-point' (default). The\n        scheme 'cs' uses complex steps, and while potentially the most\n        accurate, it is applicable only when `fun` correctly handles\n        complex inputs and can be analytically continued to the complex\n        plane. If it is a callable, it should return the \n        Hessian matrix of `dot(fun, v)`:\n\n            hess(x, v) -\u003e {LinearOperator, sparse matrix, ndarray}, shape (n, n)\n\n        where x is a (n,) ndarray and v is a (m,) ndarray. When ``hess``\n        is None it considers the hessian is an matrix filled with zeros.\n    constraints : Constraint or List of Constraint's, optional\n        A single object or a list of objects specifying\n        constraints to the optimization problem.\n        Available constraints are:\n\n            - `BoxConstraint`\n            - `LinearConstraint`\n            - `NonlinearConstraint`\n\n    method : {str, None}, optional\n        Type of solver. Should be one of:\n\n            - 'equality-constrained-sqp'\n            - 'tr-interior-point'\n\n        When None the more appropriate method is choosen.\n        'equality-constrained-sqp' is chosen for problems that\n        only have equality constraints and 'tr-interior-point'\n        for general optimization problems.\n    xtol : float, optional\n        Tolerance for termination by the change of the independent variable.\n        The algorithm will terminate when ``delta \u003c xtol``, where ``delta``\n        is the algorithm trust-radius. Default is 1e-8.\n    gtol : float, optional\n        Tolerance for termination by the norm of the Lagrangian gradient.\n        The algorithm will terminate when both the infinity norm (i.e. max\n        abs value) of the Lagrangian gradient and the constraint violation\n        are smaller than ``gtol``. Default is 1e-8.\n    sparse_jacobian : {bool, None}\n        The algorithm uses a sparse representation of the Jacobian if True\n        and a dense representation if False. When sparse_jacobian is None\n        the algorithm uses the more convenient option, using a sparse\n        representation if at least one of the constraint Jacobians are sparse\n        and a dense representation when they are all dense arrays.\n    options : dict, optional\n        A dictionary of solver options. Available options include:\n\n            initial_trust_radius: float\n                Initial trust-region radius. By defaut uses 1.0, as\n                suggested in [1]_, p.19, immediatly after algorithm III.\n            initial_penalty : float\n                Initial penalty for merit function. By defaut uses 1.0, as\n                suggested in [1]_, p.19, immediatly after algorithm III.\n            initial_barrier_parameter: float\n                Initial barrier parameter. Exclusive for 'tr_interior_point'\n                method. By default uses 0.1, as suggested in [1]_ immediatly\n                after algorithm III, p. 19.\n            initial_tolerance: float\n                Initial subproblem tolerance. Exclusive for\n                'tr_interior_point' method. By defaut uses 0.1,\n                as suggested in [1]_ immediatly after algorithm III, p. 19.\n            return_all : bool, optional\n                When True return the list of all vectors\n                through the iterations.\n            factorization_method : string, optional\n                Method used for factorizing the jacobian matrix.\n                Should be one of:\n\n                - 'NormalEquation': The operators\n                   will be computed using the\n                   so-called normal equation approach\n                   explained in [1]_. In order to do\n                   so the Cholesky factorization of\n                   ``(A A.T)`` is computed. Exclusive\n                   for sparse matrices. Requires\n                   scikit-sparse installed.\n                - 'AugmentedSystem': The operators\n                   will be computed using the\n                   so-called augmented system approach\n                   explained in [1]_. It perform the\n                   LU factorization of an augmented\n                   system. Exclusive for sparse matrices.\n                - 'QRFactorization': Compute projections\n                   using QR factorization. Exclusive for\n                   dense matrices.\n                - 'SVDFactorization': Compute projections\n                   using SVD factorization. Exclusive for\n                   dense matrices.\n\n                The factorization methods 'NormalEquation' and\n                'AugmentedSystem' should be used only when\n                ``sparse_jacobian=True``. They usually provide\n                similar results. The methods 'QRFactorization'\n                and 'SVDFactorization' should be used when\n                ``sparse_jacobian=False``. By default uses\n                'QRFactorization' for  dense matrices.\n                The 'SVDFactorization' method can cope\n                with Jacobian matrices with deficient row\n                rank and will be used whenever other\n                factorization methods fails (which may\n                imply the conversion to a dense format).\n\n    callback : callable, optional\n        Called after each iteration:\n\n            callback(OptimizeResult state) -\u003e bool\n\n        If callback returns True the algorithm execution is terminated.\n        ``state`` is an `OptimizeResult` object, with the same fields\n        as the ones from the return.\n    max_iter : int, optional\n        Maximum number of algorithm iterations. By default ``max_iter=1000``\n    verbose : {0, 1, 2}, optional\n        Level of algorithm's verbosity:\n\n            * 0 (default) : work silently.\n            * 1 : display a termination report.\n            * 2 : display progress during iterations.\n\n    Returns\n    -------\n    `OptimizeResult` with the following fields defined:\n    x : ndarray, shape (n,)\n        Solution found.\n    s : ndarray, shape (n_ineq,)\n        Slack variables at the solution. ``n_ineq`` is the total number\n        of inequality constraints.\n    v : ndarray, shape (n_ineq + n_eq,)\n        Estimated Lagrange multipliers at the solution. ``n_ineq + n_eq``\n        is the total number of equality and inequality constraints.\n    niter : int\n        Total number of iterations.\n    nfev : int\n        Total number of objective function evaluations.\n    ngev : int\n        Total number of objective function gradient evaluations.\n    nhev : int\n        Total number of Lagragian Hessian evaluations. Each time the\n        Lagrangian Hessian is evaluated the objective function\n        Hessian and the constraints Hessians are evaluated\n        one time each.\n    ncev : int\n        Total number of constraint evaluations. The same couter\n        is used for equality and inequality constraints, because\n        they always are evaluated the same number of times.\n    njev : int\n        Total number of constraint Jacobian matrix evaluations.\n        The same couter is used for equality and inequality\n        constraint Jacobian matrices, because they always are\n        evaluated the same number of times.\n    cg_niter : int\n        Total number of CG iterations.\n    cg_info : Dict\n        Dictionary containing information about the latest CG iteration:\n\n            - 'niter' : Number of iterations.\n            - 'stop_cond' : Reason for CG subproblem termination:\n\n                1. Iteration limit was reached;\n                2. Reached the trust-region boundary;\n                3. Negative curvature detected;\n                4. Tolerance was satisfied.\n\n            - 'hits_boundary' : True if the proposed step is on the boundary\n              of the trust region.\n\n    execution_time : float\n        Total execution time.\n    trust_radius : float\n        Trust radius at the last iteration.\n    penalty : float\n        Penalty function at last iteration.\n    tolerance : float\n        Tolerance for barrier subproblem at the last iteration.\n        Exclusive for 'tr_interior_point'.\n    barrier_parameter : float\n        Barrier parameter at the last iteration. Exclusive for\n        'tr_interior_point'.\n    status : {0, 1, 2, 3}\n        Termination status:\n\n            * 0 : The maximum number of function evaluations is exceeded.\n            * 1 : `gtol` termination condition is satisfied.\n            * 2 : `xtol` termination condition is satisfied.\n            * 3 : `callback` function requested termination.\n\n    message : str\n        Termination message.\n    method : {'equality_constrained_sqp', 'tr_interior_point'}\n        Optimization method used.\n    constr_violation : float\n        Constraint violation at last iteration.\n    optimality : float\n        Norm of the Lagrangian gradient at last iteration.\n    fun : float\n        For the 'equality_constrained_sqp' method this is the objective\n        function evaluated at the solution and for the 'tr_interior_point'\n        method this is the barrier function evaluated at the solution.\n    grad : ndarray, shape (n,)\n        For the 'equality_constrained_sqp' method this is the gradient of the\n        objective function evaluated at the solution and for the\n        'tr_interior_point' method  this is the gradient of the barrier\n        function evaluated at the solution.\n    constr : ndarray, shape (n_ineq + n_eq,)\n        For the 'equality_constrained_sqp' method this is the equality\n        constraint evaluated at the solution and for the 'tr_interior_point'\n        method this are the equality and inequality constraints evaluated at\n        a given point (with the inequality constraints incremented by the\n        value of the slack variables).\n    jac : {ndarray, sparse matrix}, shape (n_ineq + n_eq, n)\n        For the 'equality_constrained_sqp' method this is the Jacobian\n        matrix of the equality constraint evaluated at the solution and\n        for the tr_interior_point' method his is scaled augmented Jacobian\n        matrix, defined as ``\\hat(A)`` in equation (19.36), reference [2]_,\n        p. 581.\n\n    Notes\n    -----\n    Method 'equality_constrained_sqp' is an implementation of\n    Byrd-Omojokun Trust-Region SQP method described [3]_ and\n    in [2]_, p. 549. It solves equality constrained equality\n    constrained optimization problems by solving, at each substep,\n    a trust-region QP subproblem. The inexact solution of these\n    QP problems using projected CG method makes this method\n    appropriate for large-scale problems.\n\n    Method 'tr_interior_point' is an implementation of the\n    trust-region interior point method described in [1]_.\n    It solves general nonlinear by introducing slack variables\n    and solving a sequence of equality-constrained barrier problems\n    for progressively smaller values of the barrier parameter.\n    The previously described equality constrained SQP method is used\n    to solve the subproblems with increasing levels of accuracy as\n    the iterate gets closer to a solution. It is also an\n    appropriate method for large-scale problems.\n\n    References\n    ----------\n    .. [1] Byrd, Richard H., Mary E. Hribar, and Jorge Nocedal.\n           \"An interior point algorithm for large-scale nonlinear\n           programming.\" SIAM Journal on Optimization 9.4 (1999): 877-900.\n    .. [2] Nocedal, Jorge, and Stephen J. Wright. \"Numerical optimization\"\n           Second Edition (2006).\n    .. [3] Lalee, Marucha, Jorge Nocedal, and Todd Plantenga. \"On the\n           implementation of an algorithm for large-scale equality\n           constrained optimization.\" SIAM Journal on\n           Optimization 8.3 (1998): 682-706.\n```\n\n### ``NonlinearConstraint``\n\n```\nNonlinearConstraint(fun, kind, jac, hess='2-point', enforce_feasibility=False)\n\n   Nonlinear constraint\n\n    Parameters\n    ----------\n    fun : callable\n        The function defining the constraint.\n\n            fun(x) -\u003e array_like, shape (m,)\n\n        where x is a (n,) ndarray and ``m``\n        is the number of constraints.\n    kind : {str, tuple}\n        Specifies the type of contraint. Options for this\n        parameter are:\n\n            - ('interval', lb, ub) for a constraint of the type:\n                lb \u003c= fun(x) \u003c= ub\n            - ('greater', lb) for a constraint of the type:\n                fun(x) \u003e= lb\n            - ('less', ub) for a constraint of the type:\n                fun(x) \u003c= ub\n            - ('equals', c) for a constraint of the type:\n                fun(x) == c\n            - ('greater',) for a constraint of the type:\n                fun(x) \u003e= 0\n            - ('less',) for a constraint of the type:\n                fun(x) \u003c= 0\n            - ('equals',) for a constraint of the type:\n                fun(x) == 0\n\n        where ``lb``,  ``ub`` and ``c`` are (m,) ndarrays or\n        scalar values. In the latter case, the same value\n        will be repeated for all the constraints.\n    jac : callable\n        Jacobian Matrix:\n\n            jac(x) -\u003e {ndarray, sparse matrix}, shape (m, n)\n\n        where x is a (n,) ndarray.\n    hess : {callable, '2-point', '3-point', 'cs', None}\n        Method for computing the Hessian matrix. The keywords\n        select a finite difference scheme for numerical\n        estimation. The scheme '3-point' is more accurate, but requires\n        twice as much operations compared to '2-point' (default). The\n        scheme 'cs' uses complex steps, and while potentially the most\n        accurate, it is applicable only when `fun` correctly handles\n        complex inputs and can be analytically continued to the complex\n        plane. If it is a callable, it should return the \n        Hessian matrix of `dot(fun, v)`:\n\n            hess(x, v) -\u003e {LinearOperator, sparse matrix, ndarray}, shape (n, n)\n\n        where x is a (n,) ndarray and v is a (m,) ndarray. When ``hess``\n        is None it considers the hessian is an matrix filled with zeros.\n    enforce_feasibility : {list of boolean, boolean}, optional\n        Specify if the constraint must be feasible along the iterations.\n        If ``True``  all the iterates generated by the optimization\n        algorithm need to be feasible in respect to a constraint. If ``False``\n        this is not needed. A list can be passed to specify element-wise\n        each constraints needs to stay feasible along the iterations and\n        each does not. Alternatively, a single boolean can be used to\n        specify the feasibility required of all constraints. By default it\n        is False.\n```\n\n### ``LinearConstraint``\n\n```\nLinearConstraint(A, kind, enforce_feasibility=False)\n\n    Linear constraint.\n\n    Parameters\n    ----------\n    A : {ndarray, sparse matrix}, shape (m, n)\n        Matrix for the linear constraint.\n    kind : {str, tuple}\n        Specifies the type of contraint. Options for this\n        parameter are:\n\n            - ('interval', lb, ub) for a constraint of the type:\n                lb \u003c= A x \u003c= ub\n            - ('greater', lb) for a constraint of the type:\n                A x \u003e= lb\n            - ('less', ub) for a constraint of the type:\n                A x \u003c= ub\n            - ('equals', c) for a constraint of the type:\n                A x == c\n            - ('greater',) for a constraint of the type:\n                A x \u003e= 0\n            - ('less',) for a constraint of the type:\n                A x \u003c= 0\n            - ('equals',) for a constraint of the type:\n                A x == 0\n\n        where ``lb``,  ``ub`` and ``c`` are (m,) ndarrays or\n        scalar values. In the latter case, the same value\n        will be repeated for all the constraints.\n    enforce_feasibility : {list of boolean, boolean}, optional\n        Specify if the constraint must be feasible along the iterations.\n        If ``True``  all the iterates generated by the optimization\n        algorithm need to be feasible in respect to a constraint. If ``False``\n        this is not needed. A list can be passed to specify element-wise\n        each constraints needs to stay feasible along the iterations and\n        each does not. Alternatively, a single boolean can be used to\n        specify the feasibility required of all constraints. By default it\n        is False.\n```\n\n### \u003ca name=\"BoxConstraint\"\u003e\u003c/a\u003e ``BoxConstraint``\n\n```\nBoxConstraint(kind, enforce_feasibility=False)\n\n    Box constraint.\n\n    Parameters\n    ----------\n    kind : tuple\n        Specifies the type of contraint. Options for this\n        parameter are:\n\n            - ('interval', lb, ub) for a constraint of the type:\n                lb \u003c= x \u003c= ub\n            - ('greater', lb) for a constraint of the type:\n                x \u003e= lb\n            - ('less', ub) for a constraint of the type:\n                x \u003c= ub\n            - ('equals', c) for a constraint of the type:\n                x == c\n            - ('greater',) for a constraint of the type:\n                x \u003e= 0\n            - ('less',) for a constraint of the type:\n                x \u003c= 0\n            - ('equals',) for a constraint of the type:\n                x == 0\n\n        where ``lb``,  ``ub`` and ``c`` are (m,) ndarrays or\n        scalar values. In the latter case, the same value\n        will be repeated for all the constraints.\n    enforce_feasibility : {list of boolean, boolean}, optional\n        Specify if the constraint must be feasible along the iterations.\n        If ``True``  all the iterates generated by the optimization\n        algorithm need to be feasible in respect to a constraint. If ``False``\n        this is not needed. A list can be passed to specify element-wise\n        each constraints needs to stay feasible along the iterations and\n        each does not. Alternatively, a single boolean can be used to\n        specify the feasibility required of all constraints. By default it\n        is False.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantonior92%2Fip-nonlinear-solver","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fantonior92%2Fip-nonlinear-solver","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantonior92%2Fip-nonlinear-solver/lists"}