Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/yaooqinn/spark-ranger

已经合入(apache/incubator-kyuubi) ACL Management for Apache Spark SQL with Apache Ranger.
https://github.com/yaooqinn/spark-ranger

acl authorization data-masking ranger row-level-security spark sparksql

Last synced: 3 days ago
JSON representation

已经合入(apache/incubator-kyuubi) ACL Management for Apache Spark SQL with Apache Ranger.

Host: GitHub
URL: https://github.com/yaooqinn/spark-ranger
Owner: yaooqinn
License: apache-2.0
Archived: true
Created: 2019-08-15T08:06:29.000Z (over 5 years ago)
Default Branch: master
Last Pushed: 2021-11-11T08:35:44.000Z (about 3 years ago)
Last Synced: 2023-10-20T23:55:01.095Z (over 1 year ago)
Topics: acl, authorization, data-masking, ranger, row-level-security, spark, sparksql
Language: Scala
Homepage: https://yaooqinn.github.io/spark-ranger/
Size: 116 KB
Stars: 51
Watchers: 11
Forks: 52
Open Issues: 30
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

README

# Notice:

This library has been contribute to https://github.com/apache/submarine as a sub-module,
and that module can still be used individually.

The project here will no longer be updated.

If you have any questions please go to

https://github.com/apache/submarine/tree/master/docs/submarine-security/spark/README.md

to learn how to use and give feedback to the apache submarine community by following
https://submarine.apache.org/community/contributors.html

# Spark SQL Ranger Security Plugin [![License](https://img.shields.io/badge/license-Apache%202-4EB1BA.svg)](https://www.apache.org/licenses/LICENSE-2.0.html) [![](https://tokei.rs/b1/github/yaooqinn/spark-ranger)](https://github.com/yaooqinn/spark-ranger) [![codecov](https://codecov.io/gh/yaooqinn/spark-ranger/branch/master/graph/badge.svg)](https://codecov.io/gh/yaooqinn/spark-ranger) [![Build Status](https://travis-ci.com/yaooqinn/spark-ranger.svg?branch=master)](https://travis-ci.com/yaooqinn/spark-ranger) [![HitCount](http://hits.dwyl.io/yaooqinn/spark-ranger.svg)](http://hits.dwyl.io/yaooqinn/spark-ranger)

ACL Management for Apache Spark SQL with Apache Ranger, enabling:

- Table/Column level authorization
- Row level filtering
- Data masking

## Build
Spark SQL Ranger Security Plugin is built based on [Apache Maven](http://maven.apache.org),

```bash
mvn clean package -Pspark-2.3 -Pranger-1.0 -DskipTests
```

Currently, available profiles are:

Spark: -Pspark-2.3, -Pspark-2.4

Ranger: -Pranger-1.0, -Pranger-1.1, -Pranger-1.2 -Pranger-2.0

## Usage

### Installation

Place the spark-ranger-<version>.jar into $SPARK_HOME/jars.

### Installation Addons

You can find some tips and known problems about this library [here](docs/installation-addons.md).

### Configurations

#### Ranger admin client configurations

Create ranger-spark-security.xml in $SPARK_HOME/conf and add the following configurations for pointing to the right ranger admin server

```xml

ranger.plugin.spark.policy.rest.url
ranger admin address like http://ranger-admin.org:6080

ranger.plugin.spark.service.name
a ranger hive service name

ranger.plugin.spark.policy.cache.dir
./a ranger hive service name/policycache

ranger.plugin.spark.policy.pollIntervalMs
5000

ranger.plugin.spark.policy.source.impl
org.apache.ranger.admin.client.RangerAdminRESTClient

```

Create ranger-spark-audit.xml in $SPARK_HOME/conf and add the following configurations to enable/disable auditing.

```xml

xasecure.audit.is.enabled
true

xasecure.audit.destination.db
false

xasecure.audit.destination.db.jdbc.driver
com.mysql.jdbc.Driver

xasecure.audit.destination.db.jdbc.url
jdbc:mysql://10.171.161.78/ranger

xasecure.audit.destination.db.password
rangeradmin

xasecure.audit.destination.db.user
rangeradmin

```

#### Enable plugin via spark extensions

spark.sql.extensions=org.apache.ranger.authorization.spark.authorizer.RangerSparkSQLExtension