https://github.com/xskipper-io/xskipper

An Extensible Data Skipping Framework
https://github.com/xskipper-io/xskipper

data-skipping indexing scala spark

Last synced: 3 months ago
JSON representation

An Extensible Data Skipping Framework

• Host: GitHub
• URL: https://github.com/xskipper-io/xskipper
• Owner: xskipper-io
• License: apache-2.0
• Created: 2021-01-20T15:34:11.000Z (almost 4 years ago)
• Default Branch: master
• Last Pushed: 2023-07-21T18:25:00.000Z (over 1 year ago)
• Last Synced: 2024-09-29T04:42:04.286Z (3 months ago)
• Topics: data-skipping, indexing, scala, spark
• Language: Scala
• Homepage: https://xskipper.io
• Size: 47.7 MB
• Stars: 42
• Watchers: 5
• Forks: 13
• Open Issues: 10
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE

Awesome Lists containing this project

README

        


![Xskipper](site/docs/img/logo_readme.png)

![Build Status](https://github.com/xskipper-io/xskipper/workflows/build/badge.svg)

Xskipper is an Extensible Data Skipping Framework for [Apache Spark](https://spark.apache.org/).

To get started, see the [Quick Start Guide](https://xskipper.io/getting-started/quick-start-guide/) .

See Xskipper [site](https://xskipper.io) for more info. 

# Run as a project

To build a project using the Xskipper binaries from the Maven Central Repository, use the following Maven coordinates:

## Maven

Include Xskipper in a Maven project by adding it as a dependency in the project's POM file. Xskipper should be compiled with Scala 2.12.

```XML

  io.xskipper

  xskipper-core_2.12

  1.3.0

```

## SBT

Include Xskipper in an SBT project by adding the following line to its build.sbt file:

```Scala

libraryDependencies += "io.xskipper" %% "xskipper-core" % "1.3.0"

```

# Building

Xskipper is compiled using [SBT](https://www.scala-sbt.org/1.x/docs/Command-Line-Reference.html).

To compile, run

    build/sbt compile

To generate artifacts, run

    build/sbt package

To execute tests, run

    build/sbt test

Refer to [SBT docs](https://www.scala-sbt.org/1.x/docs/Command-Line-Reference.html) for more commands.

# Collaboration

Xskipper tracks issues in GitHub and prefers to receive contributions as pull requests.

# Compatibility

Xskipper is compatible with Spark according to the following table:

| Xskipper version | Spark Version        |

| --------------- | --------------------- |

| 1.4.x           | 3.3.x   |

| 1.3.x           | 3.2.x   |

| 1.2.x           | 3.0.x   | 

| 1.1.x           | 2.4.x   | 

| 1.0.x           | 2.3.x   |

# See Also

- [IEEE Big Data 2020 paper - Extensible Data Skipping](https://arxiv.org/abs/2009.08150) (arxiv version)

# License

Apache License 2.0, see [LICENSE](LICENSE).

# Acknowledgements

This software has been developed under the [BigDataStack project](https://bigdatastack.eu/the-bigdatastack-solution), as part of the holistic solution for big data applications and operations. 

BigDataStack has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 779747.