Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/redis-field-engineering/redis-sql-trino

Real-time Indexed SQL Queries for Redis
https://github.com/redis-field-engineering/redis-sql-trino

jdbc redis redisearch sql tableau

Last synced: about 1 month ago
JSON representation

Real-time Indexed SQL Queries for Redis

Awesome Lists containing this project

README 

        
:linkattrs:

:project-owner:   redis-field-engineering

:project-name:    redis-sql-trino

:project-group:   com.redis

:project-version: 0.3.4

:name:            Redis SQL Trino

:toc:

:toc-title:

:toc-placement!:



++++



  Redis SQL Trino



  


    Redis SQL Trino is a SQL interface for Redis Stack, Redis Cloud, and Redis Enterprise.

  



++++



'''



image:https://github.com/{project-owner}/{project-name}/actions/workflows/early-access.yml/badge.svg["Build Status", link="https://github.com/{project-owner}/{project-name}/actions/workflows/early-access.yml"]

image:https://codecov.io/gh/{project-owner}/{project-name}/branch/master/graph/badge.svg?token={codecov-token}["Coverage", link="https://codecov.io/gh/{project-owner}/{project-name}"]



{name} lets you easily integrate with visualization frameworks — like Tableau and SuperSet — and platforms that support JDBC-compatible databases (e.g., Mulesoft).

Query support includes SELECT statements across secondary indexes on both Redis hashes & JSON, aggregations (e.g., count, min, max, avg), ordering, and more.



Trino is a distributed SQL engine designed to query large data sets across one or more heterogeneous data sources.

Though Trino does support a https://trino.io/docs/current/connector/redis.html[Redis OSS connector], this connector is limited to SCAN and subsequent HGET operations, which do not scale well in high-throughput scenarios.

However, that is where {name} shines since it can push the entire query to the data atomically. This eliminates the waste of many network hops and subsequent operations.



[discrete]

== Table of Contents

toc::[]



== Background



Redis is an in-memory data store designed to serve data with the fastest possible response times.

For this reason, Redis is frequently used for caching OLTP-style application queries and as a serving layer in data pipeline architectures (e.g., lambda architectures, online feature stores, etc.).

Redis Stack is an extension to Redis that, among other things, lets you index your data on secondary attributes and then efficiently query it using a custom query language.



We built the {name} connector so that you can query Redis using SQL.

This is useful for any application compatible with JDBC.

For example, {name} lets you query and visualize your Redis data from Tableau.



== Requirements



{name} requires a Redis deployment that includes https://redis.io/docs/stack/search/https://redis.io/docs/stack/search/[RediSearch].

RediSearch is a https://redis.io/docs/reference/modules/[Redis module] that adds querying and secondary indexing to Redis.



Redis deployments that bundle RediSearch include:



* https://redis.com/try-free/[Redis Cloud]: Fully-managed, enterprise-grade Redis deployed on AWS, Azure, or GCP.

* https://redis.com/redis-enterprise-software/overview/[Redis Enterprise]: Enterprise-grade Redis for on-premises and private cloud deployment.

* https://redis.io/docs/stack/[Redis Stack]: Redis distribution that includes RediSearch, RedisJSON, RedisGraph, RedisTimeSeries, and RedisBloom.



== Quick start



To understand how {name} works, it's best to try it for yourself.

View the screen recording or follow the steps below:



image::https://asciinema.org/a/526185.svg[asciicast,link=https://asciinema.org/a/526185]



First, clone this git repository:

[source,console,subs="verbatim,attributes"]

----

git clone https://github.com/{project-owner}/{project-name}.git

cd {project-name}

----



Next, use Docker Compose to launch containers for Trino and Redis Stack:

[source,console]

----

docker compose up

----



This example includes a preloaded data set describing a collection of beers.



Each beer is represented as a Redis hash.

Start the Redis CLI to examine this data.

For example, here's how you can view the "Beer Town Brown" record:

[source,console]

----

docker exec -it redis /opt/redis-stack/bin/redis-cli

127.0.0.1:6379> hgetall beer:190

----



Now let's query the same data using SQL statements through Trino.

Start the Trino CLI:

[source,console]

----

docker exec -it trino trino --catalog redisearch --schema default

----



View "Beer Town Brown" using SQL:

[source,console]

----

trino:default> select * from beers where id = '190';

----



Show all beers with an ABV greater than 3.2%:

[source,console]

----

trino:default> select * from beers where abv > 3.2 order by abv desc;

----



== Installation



To run {name} in production, you'll need:



* Trino

* A copy of the {name} connector

* A https://redis.io/docs/stack/[Redis Stack], https://redis.com/redis-enterprise-cloud/overview/[Redis Cloud], and https://redis.com/redis-enterprise-software/overview/[Redis Enterprise] deployment



=== Trino



First, you'll need a working Trino installation.



See the https://trino.io/docs/current/installation.html[Trino installation and deployment guide] for details.

Trino recommends a https://trino.io/docs/current/installation/containers.html[container-based deployment] using your orchestration platform of choice.

If you run Kubernetes, see the https://trino.io/docs/current/installation/kubernetes.html[Trino Helm chart].



=== {name} Connector



Next, you'll need to install the {name} plugin and configure it.

See our documentation for https://{project-owner}.github.io/{project-name}/#redisearch-connector[plugin installation] and https://{project-owner}.github.io/{project-name}/#configuration[plugin configuration].



=== Redis installation



For a self-managed deployment, or for testing locally, install https://redis.io/docs/stack/[Redis Stack] or spin up a free https://redis.com/try-free/[Redis Cloud] instance.

If you need a fully-managed, cloud-based deployment of Redis on AWS, GCP, or Azure, see all of the https://redis.com/redis-enterprise-cloud/overview/[Redis Cloud] offerings.

For deployment in your own private cloud or data center, consider https://redis.com/redis-enterprise-software/overview/[Redis Enterprise].



== Documentation



{name} documentation is available at https://{project-owner}.github.io/{project-name}



== Usage



The example above uses the https://trino.io/docs/current/client/cli.html[Trino CLI] to access your data.



Most real world applications will use the https://trino.io/docs/current/client/jdbc.html[Trino JDBC driver] to issue queries.

See the https://{project-owner}.github.io/{project-name}/#clients[{name}] documentation for details.



== Support



{name} is supported by Redis, Inc. on a good faith effort basis.

To report bugs, request features, or receive assistance, please https://github.com/{project-owner}/{project-name}/issues[file an issue].



== License



{name} is licensed under the MIT License.

Copyright (C) 2023 Redis, Inc.