https://github.com/redislabs/spark-redis-ml
A spark package for loading Spark ML models to Redis-ML
https://github.com/redislabs/spark-redis-ml
Last synced: 11 months ago
JSON representation
A spark package for loading Spark ML models to Redis-ML
- Host: GitHub
- URL: https://github.com/redislabs/spark-redis-ml
- Owner: RedisLabs
- License: other
- Created: 2016-10-05T16:54:05.000Z (almost 10 years ago)
- Default Branch: master
- Last Pushed: 2019-06-22T07:00:48.000Z (about 7 years ago)
- Last Synced: 2025-04-06T17:02:53.970Z (about 1 year ago)
- Language: Scala
- Homepage: http://redisml.io
- Size: 33.2 KB
- Stars: 63
- Watchers: 15
- Forks: 14
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
---
#### Notice RedisML is planned to be replaced by [RedisAI](http://redisai.io), adding support for deep learning.
***
# Spark-Redis-ML
### A spark package for loading Spark ML models to [Redis-ML](https://github.com/RedisLabsModules/redis-ml "Redis-ML")
## Requirments:
Apache [Spark](https://github.com/apache/spark) 2.0 or later
[Redis](https://github.com/antirez/redis) build from unstable branch
[Jedis](https://github.com/xetorthio/jedis)
[Jedis-ml](https://github.com/RedisLabs/jedis-ml)
## Installation:
```sh
#get and build redis-ml
git clone https://github.com/RedisLabsModules/redis-ml.git
cd redis-ml/src
make
#get and build jedis
git clone https://github.com/xetorthio/jedis.git
cd jedis
mvn package -Dmaven.test.skip=true
#get and build jedis-ml
cd..
git clone https://github.com/RedisLabs/jedis-ml.git
cd jedis-ml
mkdir lib
cp ../jedis/target/jedis-3.0.0-SNAPSHOT.jar lib/
mvn install
#get and build spark-jedis-ml
cd..
git clone https://github.com/RedisLabs/spark-redis-ml.git
cd spark-redis-ml
cp ../jedis/target/jedis-3.0.0-SNAPSHOT.jar lib/
cp ../jedis-ml/target/jedis-ml-1.0-SNAPSHOT.jar lib/
sbt assembly
```
### Usage:
Run Redis server with redis-ml module:
```sh
/path/to/redis-server --loadmodule ./redis-ml.so
```
From Spark root directory, Run Spark shell with the required jars:
```sh
./bin/spark-shell --jars ../spark-redis-ml/target/scala-2.11/spark-redis-ml-assembly-0.1.0.jar,../spark-redis-ml/lib/jedis-3.0.0-SNAPSHOT.jar,../spark-redis-ml/lib/jedis-ml-1.0-SNAPSHOT.jar
```
On Spark shell:
```sh
scala> :load "../spark-redis-ml/scripts/forest-example.scala"
scala> benchmark(10)
```
###