{"id":14982333,"url":"https://github.com/redislabs/spark-redis","last_synced_at":"2025-05-14T06:12:34.600Z","repository":{"id":36499569,"uuid":"40805228","full_name":"RedisLabs/spark-redis","owner":"RedisLabs","description":"A connector for Spark that allows reading and writing to/from Redis cluster","archived":false,"fork":false,"pushed_at":"2024-10-22T13:13:21.000Z","size":2263,"stargazers_count":947,"open_issues_count":141,"forks_count":370,"subscribers_count":70,"default_branch":"master","last_synced_at":"2025-04-03T14:07:02.514Z","etag":null,"topics":["dataframe","java","redis","spark"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RedisLabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-08-16T08:39:33.000Z","updated_at":"2025-03-27T02:38:41.000Z","dependencies_parsed_at":"2023-02-10T14:30:43.583Z","dependency_job_id":"4bda0512-f4c3-44dd-9ae2-602ddc481289","html_url":"https://github.com/RedisLabs/spark-redis","commit_stats":{"total_commits":609,"total_committers":23,"mean_commits":26.47826086956522,"dds":0.5747126436781609,"last_synced_commit":"ed602b698b589b2698ad7e18b83e15f786f98d2c"},"previous_names":[],"tags_count":21,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RedisLabs%2Fspark-redis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RedisLabs%2Fspark-redis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RedisLabs%2Fspark-redis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RedisLabs%2Fspark-redis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RedisLabs","download_url":"https://codeload.github.com/RedisLabs/spark-redis/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248313528,"owners_count":21082873,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dataframe","java","redis","spark"],"created_at":"2024-09-24T14:05:13.263Z","updated_at":"2025-04-10T23:24:32.742Z","avatar_url":"https://github.com/RedisLabs.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Integration](https://github.com/RedisLabs/spark-redis/actions/workflows/integration.yml/badge.svg?branch=master)](https://github.com/RedisLabs/spark-redis/actions/workflows/integration.yml)\n[![license](https://img.shields.io/github/license/RedisLabs/spark-redis.svg)](https://github.com/RedisLabs/spark-redis)\n[![Release](https://img.shields.io/github/release/RedisLabs/spark-redis.svg?sort=semver)](https://github.com/RedisLabs/spark-redis/releases/latest)\n[![Maven Central](https://img.shields.io/maven-central/v/com.redislabs/spark-redis_2.12?logo=redis)](https://maven-badges.herokuapp.com/maven-central/com.redislabs/spark-redis_2.12)\n[![Javadocs](https://www.javadoc.io/badge/com.redislabs/spark-redis_2.12.svg)](https://www.javadoc.io/doc/com.redislabs/spark-redis_2.12)\n[![Codecov](https://codecov.io/gh/RedisLabs/spark-redis/branch/master/graph/badge.svg)](https://codecov.io/gh/RedisLabs/spark-redis)\n\n[![Discord](https://img.shields.io/discord/697882427875393627.svg?style=social\u0026logo=discord)](https://discord.gg/redis)\n[![Twitch](https://img.shields.io/twitch/status/redisinc?style=social)](https://www.twitch.tv/redisinc)\n[![YouTube](https://img.shields.io/youtube/channel/views/UCD78lHSwYqMlyetR0_P4Vig?style=social)](https://www.youtube.com/redisinc)\n[![Twitter](https://img.shields.io/twitter/follow/redisinc?style=social)](https://twitter.com/redisinc)\n# Spark-Redis\nA library for reading and writing data in [Redis](http://redis.io) using [Apache Spark](http://spark.apache.org/).\n\nSpark-Redis provides access to all of Redis' data structures - String, Hash, List, Set and Sorted Set - from Spark as RDDs. It also supports reading and writing with DataFrames and Spark SQL syntax.\n\nThe library can be used both with Redis stand-alone as well as clustered databases. When used with Redis cluster, Spark-Redis is aware of its partitioning scheme and adjusts in response to resharding and node failure events.\n\nSpark-Redis also supports Spark Streaming (DStreams) and Structured Streaming.\n\n## Version compatibility and branching\n\nThe library has several branches, each corresponds to a different supported Spark version. For example, 'branch-2.3' works with any Spark 2.3.x version.\nThe master branch contains the recent development for the next release.\n\n| Spark-Redis                                                               | Spark | Redis            | Supported Scala Versions | \n|---------------------------------------------------------------------------|-------| ---------------- | ------------------------ |\n| [master](https://github.com/RedisLabs/spark-redis/)                       | 3.2.x | \u003e=2.9.0          | 2.12                     | \n| [3.0](https://github.com/RedisLabs/spark-redis/tree/branch-3.0)           | 3.0.x | \u003e=2.9.0          | 2.12                     | \n| [2.4, 2.5, 2.6](https://github.com/RedisLabs/spark-redis/tree/branch-2.4) | 2.4.x | \u003e=2.9.0          | 2.11, 2.12               | \n| [2.3](https://github.com/RedisLabs/spark-redis/tree/branch-2.3)           | 2.3.x | \u003e=2.9.0          | 2.11                     | \n| [1.4](https://github.com/RedisLabs/spark-redis/tree/branch-1.4)           | 1.4.x |                  | 2.10                     | \n\n\n## Known limitations\n\n* Java, Python and R API bindings are not provided at this time\n\n## Additional considerations\nThis library is a work in progress so the API may change before the official release.\n\n## Documentation\n\nPlease make sure you use documentation from the correct branch ([2.4](https://github.com/RedisLabs/spark-redis/tree/branch-2.4#documentation), [2.3](https://github.com/RedisLabs/spark-redis/tree/branch-2.3#documentation), etc). \n\n  - [Getting Started](doc/getting-started.md)\n  - [RDD](doc/rdd.md)\n  - [Dataframe](doc/dataframe.md)\n  - [Streaming](doc/streaming.md)\n  - [Structured Streaming](doc/structured-streaming.md)\n  - [Cluster](doc/cluster.md)\n  - [Java](doc/java.md)\n  - [Python](doc/python.md)\n  - [Configuration](doc/configuration.md)\n  - [Dev environment](doc/dev.md)\n\n## Contributing\n\nYou're encouraged to contribute to the Spark-Redis project. \n\nThere are two ways you can do so:\n\n### Submit Issues\n\nIf you encounter an issue while using the library, please report it via the project's [issues tracker](https://github.com/RedisLabs/spark-redis/issues).\n\n### Author Pull Requests\n\nCode contributions to the Spark-Redis project can be made using [pull requests](https://github.com/RedisLabs/spark-redis/pulls). To submit a pull request:\n\n 1. Fork this project.\n 2. Make and commit your changes.\n 3. Submit your changes as a pull request.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fredislabs%2Fspark-redis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fredislabs%2Fspark-redis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fredislabs%2Fspark-redis/lists"}