{"id":16271192,"url":"https://github.com/ehsanmok/spark-lp","last_synced_at":"2026-03-07T12:02:26.322Z","repository":{"id":95999245,"uuid":"73751736","full_name":"ehsanmok/spark-lp","owner":"ehsanmok","description":"Distributed Linear Programming Solver on top of Apache Spark","archived":false,"fork":false,"pushed_at":"2021-01-04T19:07:27.000Z","size":39,"stargazers_count":75,"open_issues_count":3,"forks_count":23,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-01-06T17:14:31.719Z","etag":null,"topics":["distributed-computing","distributed-optimization","high-performance","linear-programming","scala","spark"],"latest_commit_sha":null,"homepage":"https://open.library.ubc.ca/cIRcle/collections/ubctheses/24/items/1.0340337","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ehsanmok.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-11-14T22:15:14.000Z","updated_at":"2024-09-12T04:44:21.000Z","dependencies_parsed_at":"2023-04-18T19:08:13.781Z","dependency_job_id":null,"html_url":"https://github.com/ehsanmok/spark-lp","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ehsanmok%2Fspark-lp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ehsanmok%2Fspark-lp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ehsanmok%2Fspark-lp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ehsanmok%2Fspark-lp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ehsanmok","download_url":"https://codeload.github.com/ehsanmok/spark-lp/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":233413511,"owners_count":18672692,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed-computing","distributed-optimization","high-performance","linear-programming","scala","spark"],"created_at":"2024-10-10T18:12:49.579Z","updated_at":"2025-09-17T18:30:42.177Z","avatar_url":"https://github.com/ehsanmok.png","language":"Scala","funding_links":[],"categories":["算法库"],"sub_categories":["文件同步"],"readme":"# spark-lp\n\n![Project unmaintained](https://img.shields.io/badge/project-unmaintained-red.svg)\n\nThis package offers an implementation of [Mehrohra's predictor-corrector interior point algorithm](https://en.wikipedia.org/wiki/Mehrotra_predictor%E2%80%93corrector_method), described in my thesis [Distributed linear programming with Apache Spark](https://open.library.ubc.ca/cIRcle/collections/ubctheses/24/items/1.0340337), to solve **large-scale** [linear programming](https://en.wikipedia.org/wiki/Linear_programming) problems at the *lowest cost* using Apache Spark.\n\nLinear programming has the following standard form: \n\n\tminimize c^T x \n\tsubject to Ax=b and x \u003e= 0\n\nwhere `c, b` are given vectors ((.)^T is the traspose operation), `A` is a given `m` by `n` matrix and `x` is the objective vector. We assume that in `A` the number of rows (equations) is\nat most equal to the number of columns (unknowns) (`m \u003c= n`) and `A` has full row rank, thus `AA^T` is invertible.\n\n## Example\n\nThe following is an example of using spark-lp *locally* to solve a linear programming problem in parallel with 2 cores and 2 partitions:\n\n\timport org.apache.spark.{SparkConf, SparkContext}\n\timport org.apache.spark.mllib.linalg.{DenseVector, Vector, Vectors}\n\timport org.apache.spark.mllib.optimization.lp.VectorSpace._\n\timport org.apache.spark.mllib.optimization.lp.vs.dvector.DVectorSpace\n\timport org.apache.spark.mllib.optimization.lp.vs.vector.DenseVectorSpace\n\timport org.apache.spark.mllib.optimization.lp.LP\n\n\tval sparkConf = new SparkConf().setMaster(\"local[2]\").setAppName(\"TestLPSolver\")\n\tval sc = new SparkContext(sparkConf)\n\tval numPartitions = 2\n\tval cArray = Array(2.0, 1.5, 0.0, 0.0, 0.0, 0.0, 0.0)\n\tval BArray = Array(\n    \tArray(12.0, 16.0, 30.0, 1.0, 0.0),\n    \tArray(24.0, 16.0, 12.0, 0.0, 1.0),\n    \tArray(-1.0, 0.0, 0.0, 0.0, 0.0),\n    \tArray(0.0, -1.0, 0.0, 0.0, 0.0),\n    \tArray(0.0, 0.0, -1.0, 0.0, 0.0),\n    \tArray(0.0, 0.0, 0.0, 1.0, 0.0),\n    \tArray(0.0, 0.0, 0.0, 0.0, 1.0))\n\tval bArray = Array(120.0, 120.0, 120.0, 15.0, 15.0)\n\n\tval c: DVector = sc.parallelize(cArray, numPartitions).glom.map(new DenseVector(_))\n\tval rows: DMatrix = sc.parallelize(BArray, numPartitions).map(Vectors.dense(_))\n\tval b: DenseVector = new DenseVector(bArray)\n\n\tval (v, x): (Double, DVector) = LP.solve(c, rows, b, sc=sc)\n\tval xx = Vectors.dense(x.flatMap(_.toArray).collect())\n\tprintln(s\"optimial vector is $xx\")\n\tprintln(\"optimal min value: \" + v)\n\n## Software Architecture Overview\n\nDetailed descriptions of our design is described in chapter 4 of the [thesis](https://open.library.ubc.ca/cIRcle/collections/ubctheses/24/items/1.0340337).\n\n## Advantages\n\n* spark-lp is unique because it is **open-source** and it can solve large-scale LP problems in a distributed way with **fault-tolerance** over **commodity clusters** of machines. Thus, it provides the *lowest cost* opportunity for such applications. See page 42 for cluster results [here](https://open.library.ubc.ca/cIRcle/collections/ubctheses/24/items/1.0340337).\n\n* spark-lp is at least ~10X *faster* and more accurate than spark-tfocs for solving large-scale LP problems. See page 38 for local results [here](https://open.library.ubc.ca/cIRcle/collections/ubctheses/24/items/1.0340337). Our benchmark shows that spark-tfocs is *not* suitable even for small LP problems.\n\n## Future plans:\n\n* Add preprocessing to capture more general LP formats.\n* Add infeasibility detection.\n* Extend to QP solver.\n* Add GPU support, as described in page 47 [here](https://open.library.ubc.ca/cIRcle/collections/ubctheses/24/items/1.0340337), using INDArray provided in [ND4J](http://nd4j.org/) library.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fehsanmok%2Fspark-lp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fehsanmok%2Fspark-lp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fehsanmok%2Fspark-lp/lists"}