https://github.com/alipay/jpmml-sparkml-lightgbm
JPMML-SparkML plugin for converting LightGBM-Spark models to PMML
https://github.com/alipay/jpmml-sparkml-lightgbm
apache-spark lightgbm machine-learning pmml sparkml
Last synced: 10 days ago
JSON representation
JPMML-SparkML plugin for converting LightGBM-Spark models to PMML
- Host: GitHub
- URL: https://github.com/alipay/jpmml-sparkml-lightgbm
- Owner: alipay
- Created: 2018-09-12T04:08:20.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2021-10-23T11:41:02.000Z (almost 4 years ago)
- Last Synced: 2025-04-25T05:36:44.450Z (6 months ago)
- Topics: apache-spark, lightgbm, machine-learning, pmml, sparkml
- Language: Java
- Homepage:
- Size: 9.77 KB
- Stars: 41
- Watchers: 10
- Forks: 9
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
JPMML-SparkML-LightGBM
=====================
JPMML-SparkML plugin for converting [LightGBM-Spark](https://github.com/Azure/mmlspark/blob/master/docs/lightgbm.md) models to PMML.
# Prerequisites #
* [Apache Spark](http://spark.apache.org/) 2.4.x
* [LightGBM-Spark](https://github.com/Azure/mmlspark) 0.18.1.
# Installation #
Enter the project root directory and build using [Apache Maven](http://maven.apache.org/):
```
mvn clean install
```
The build installs JPMML-SparkML-LightGBM library into local repository using coordinates `org.jpmml:jpmml-sparkml-lightgbm:1.0-SNAPSHOT`.
# Usage #
The JPMML-SparkML-LightGBM library extends the [JPMML-SparkML](https://github.com/jpmml/jpmml-sparkml) library with support for `com.microsoft.ml.spark.lightgbm.LightGBMClassificationModel` prediction model classes.
add `org.jpmml:jpmml-sparkml-lightgbm:1.0-SNAPSHOT.jar` to CLASSPATH
```python
import mmlspark
import mmlspark.train
from pyspark.ml import PipelineModel
df = spark.sql("select * from algo_dc_ml_split_data")
model = PipelineModel.load("/user/turing/lightgbm_spark")
from pyspark2pmml import PMMLBuilder
pmmlBuilder = PMMLBuilder(spark.sparkContext, df, model)
pmmlBuilder.buildFile("algo_dc_ml_2c_lightgbm_spark.xml")
hdfs_client.upload("algo_dc_ml_2c_lightgbm_spark.xml","${hdfs_path}")
```
# License #
JPMML-SparkML-LightGBM is licensed under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0). Other licenses are available on request.