https://github.com/shunfei/indexr

An open-source columnar data format designed for fast & realtime analytic with big data.
https://github.com/shunfei/indexr

columnar-storage datawarehouse indexr olap realtime

Last synced: 2 months ago
JSON representation

An open-source columnar data format designed for fast & realtime analytic with big data.

• Host: GitHub
• URL: https://github.com/shunfei/indexr
• Owner: shunfei
• License: apache-2.0
• Created: 2016-12-23T04:10:04.000Z (over 8 years ago)
• Default Branch: master
• Last Pushed: 2022-11-16T05:23:54.000Z (over 2 years ago)
• Last Synced: 2024-10-28T04:23:43.488Z (7 months ago)
• Topics: columnar-storage, datawarehouse, indexr, olap, realtime
• Language: Java
• Homepage:
• Size: 2.32 MB
• Stars: 453
• Watchers: 65
• Forks: 132
• Open Issues: 11
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# IndexR

![IndexR Logo](images/indexr-logo-150x150.png)

**IndexR** is a super fast columnar data format on HDFS, which focus on fast analytic, both for massive static(historical) data and rapidly ingesting realtime data. IndexR is designed for OLAP. IndexR is greatly suitable for building data warehouse based on Hadoop ecosystem.

* Super fast, 2~4x read speed of Parquet.

* 3 levels indices supported. Say goodbye to full scan.

* Support realtime ingestion. No more wait, analyse anything right after they happen.

* Hardware efficiency, anyone can use.

* Features like realtime and offline pre-aggregation, online schema update, 100% accurate, etc.

* Deep integration with Hadoop ecosystem. Adapted with popular query engines like Apache Drill, Apache Hive, etc.

#### Getting started

* Installation

  * First [Compile from source](https://github.com/shunfei/indexr/wiki/Compilation) or download a pre-compiled package directly from [release page](https://github.com/shunfei/indexr/releases).

  * Then [Set up a cluster](https://github.com/shunfei/indexr/wiki/Deployment).

* User manual - Check [here](https://github.com/shunfei/indexr/wiki/User-Guide).

* Any problems? - Found an [issue](https://github.com/shunfei/indexr/issues).

#### Documentation

[https://github.com/shunfei/indexr/wiki](https://github.com/shunfei/indexr/wiki)

#### Useful Links

* [IndexR 技术白皮书](https://github.com/shunfei/sfmind/blob/master/indexr_white_paper/indexr_white_paper.md)

* [IndexR introduction](https://github.com/shunfei/sfmind/blob/master/indexr.about.en.md)

Please feel free to file any issues.

## Contact

* WeChat: xilyflow

* QQ Group: 606666586 (IndexR讨论组)

## License

    Copyright 2016 Sunteng Tech.

    Licensed under the Apache License, Version 2.0 (the "License"); you may not

    use this file except in compliance with the License. You may obtain a copy of

    the License at

    http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software

    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT

    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the

    License for the specific language governing permissions and limitations under

    the License.