https://github.com/jgroups-extras/jgroups-aws

Discovery protocol for JGroups using AWS S3 buckets as cluster information store
https://github.com/jgroups-extras/jgroups-aws

Last synced: 3 months ago
JSON representation

Discovery protocol for JGroups using AWS S3 buckets as cluster information store

• Host: GitHub
• URL: https://github.com/jgroups-extras/jgroups-aws
• Owner: jgroups-extras
• License: apache-2.0
• Created: 2017-02-10T10:00:46.000Z (over 9 years ago)
• Default Branch: main
• Last Pushed: 2026-02-27T17:23:54.000Z (3 months ago)
• Last Synced: 2026-02-27T20:22:30.849Z (3 months ago)
• Language: Java
• Homepage: http://jgroups.org
• Size: 796 KB
• Stars: 16
• Watchers: 6
• Forks: 12
• Open Issues: 18
• Metadata Files:
  • Readme: README.adoc
  • License: LICENSE
  • Codeowners: .github/CODEOWNERS
  • Dco: dco.txt

Awesome Lists containing this project

README

          
= JGroups AWS – `aws.S3_PING` image:https://github.com/jgroups-extras/jgroups-aws/workflows/CI/badge.svg[CI,link=https://github.com/jgroups-extras/jgroups-aws/actions] image:https://img.shields.io/maven-central/v/org.jgroups.aws/jgroups-aws?logo=apache-maven[Version,link=https://central.sonatype.com/artifact/org.jgroups.aws/jgroups-aws]

:toc:

Projects JGroups AWS provides an implementation of `aws.S3_PING` discovery protocol using AWS S3 buckets as cluster information store.

It is based on the https://github.com/jwegne/jgroups-native-s3-ping[original code] written by Tobias Sarnowski at Zalando.

It uses the https://docs.aws.amazon.com/s3/[official AWS SDK] and does not implement the HTTP protocol on its own.

The benefit is a more stable connection as well as usage of IAM server profiles and AWS standardized credential distribution.

== Maven Coordinates

[source,xml]

----

    org.jgroups.aws

    jgroups-aws

    4.0.0.Final

----

== Protocol Configuration

This library implements a JGroups discovery protocol which replaces protocols like `MPING` or `TCPPING`.

[source,xml]

----

----

=== Configurations Properties

// n.b. The order of the following list here is by relevance and related/coupled properties are also grouped.

[align="left",cols="3,1,10",options="header"]

|===

| Attribute Name +

System Property +

Environment variable

| Default

| Description

| `region_name` +

System property: `jgroups.aws.s3.region_name` +

Environment variable: `JGROUPS_AWS_S3_REGION_NAME`

| *required*

| The AWS region with which to communicate.

| `bucket_name` +

System property: `jgroups.aws.s3.bucket_name` +

Environment variable: `JGROUPS_AWS_S3_BUCKET_NAME`

| *required*

| The AWS S3 bucket name to use.

| `bucket_prefix` +

System property: `jgroups.aws.s3.bucket_prefix` +

Environment variable: `JGROUPS_AWS_S3_BUCKET_PREFIX`

|

| The prefix to prefix all AWS S3 paths with, e.g. `jgroups/`.

| `check_if_bucket_exists` +

System property: `jgroups.aws.s3.check_if_bucket_exists` +

Environment variable: `JGROUPS_AWS_S3_CHECK_IF_BUCKET_EXISTS`

| `true`

| Whether to check if the bucket exists in AWS S3 and create a new one if it does not exist yet.

| `endpoint` +

System property: `jgroups.aws.s3.endpoint` +

Environment variable: `JGROUPS_AWS_S3_ENDPOINT`

|

| The AWS endpoint with which to communicate.

| `path_style_access_enabled` +

System property: `jgroups.aws.s3.path_style_access_enabled` +

Environment variable: `JGROUPS_AWS_S3_PATH_STYLE_ACCESS_ENABLED`

| `false`

| Forces the AWS S3 client to use path-style addressing for buckets.

| `kms_key_id` +

System property: `jgroups.aws.s3.kms_key_id` +

Environment variable: `JGROUPS_AWS_S3_KMS_KEY_ID`

|

| KMS key to use for enabling KMS server-side encryption (SSE-KMS) for AWS S3.

| `acl_grant_bucket_owner_full_control` +

System property: `jgroups.aws.s3.acl_grant_bucket_owner_full_control` +

Environment variable: `JGROUPS_AWS_S3_ACL_GRANT_BUCKET_OWNER_FULL_CONTROL`

| `false`

| Whether to grant the bucket owner full control over the bucket on each update. This is useful in multi-region deployments where each region exists in its own AWS account.

| System property: `s3ping.magic_number`

| `790`

| The protocol automatically registers itself to JGroups with the magic number `790`. This can be overwritten by setting the system property to different number.

|===

NOTE: This protocol implementation extends `FILE_PING` and inherits its configuration, however, since version 4.0.1.Final, the `register_shutdown_hook` is set to `false` by default.

NOTE: System properties are supported since JGroups AWS release version 3.0.1.Final.

      To use properties in the previous version, the property must be explicitly defined in the configuration, e.g. `"`.

NOTE: Be mindful of how JGroups resolves values at runtime when using system properties or environment variables.

      For example, consider the configuration ``.

      JGroups will first attempt to resolve the `my.bucket.property` property.

      If it is not defined, the default value `myBucketDefaultValue` will be used.

      If no default value is specified, JGroups will then attempt to resolve `jgroups.aws.s3.bucket_name` system property,

      followed by the environment variable `JGROUPS_AWS_S3_BUCKET_NAME`.

=== Example Stack

[source,xml]

----

    

    

    

    

    

    

    

    

    

    

    

    

    

    

    

----

== Testing

Running the automated tests requires having AWS credentials setup with appropriate permissions

along with setting the region name and a bucket name.

[source,shell]

----

declare -x AWS_ACCESS_KEY_ID="..."

declare -x AWS_SECRET_ACCESS_KEY="..."

declare -x JGROUPS_AWS_S3_REGION_NAME="eu-central-1"

declare -x JGROUPS_AWS_S3_BUCKET_NAME="jgroups"

./mvnw verify

----

If the required AWS credentials are not specified, testing with AWS S3 will be skipped (uses `org.junit.jupiter.api.Assumptions`).

In case credentials are not provided and running on Linux, tests will be run against mock containerized S3 instance.

These require a functioning podman or Docker environment.

== Reporting Issues

Project JGroups AWS uses GitHub Issues for tracking:

https://github.com/jgroups-extras/jgroups-aws/issues

== Support Matrix

|===

|Version/Branch |JGroups Version |AWS SDK Version |Base JDK Version

|`4.x`/`main` |`5.5.x`           |`2.x` |17

|`3.1.x`      |`5.4.x`           |`2.x` |11

|`3.0.x`      |`5.2.x` – `5.3.x` |`2.x` |11

|`2.x`        |`5.2.x`           |`1.x` |11

|`1.x`        |`4.2.x`           |`1.x` |8

|===