https://github.com/risdenk/hbase-custom-replication-endpoint-example

hbase-custom-replication-endpoint-example
https://github.com/risdenk/hbase-custom-replication-endpoint-example

Last synced: about 1 month ago
JSON representation

hbase-custom-replication-endpoint-example

• Host: GitHub
• URL: https://github.com/risdenk/hbase-custom-replication-endpoint-example
• Owner: risdenk
• Created: 2015-01-14T16:16:48.000Z (almost 10 years ago)
• Default Branch: master
• Last Pushed: 2015-01-15T03:42:38.000Z (almost 10 years ago)
• Last Synced: 2024-05-01T20:37:36.126Z (7 months ago)
• Language: Java
• Size: 215 KB
• Stars: 2
• Watchers: 2
• Forks: 3
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# HBase Custom Replication Endpoint Example

## Overview

HBase 0.98.9 added the ability to create custom replication endpoints. This is an example about how to create and use a custom replication endpoint.

## Usage

### Cluster

1. `mvn clean package`

2. Add hbase.replication=true to hbase-site.xml

3. Copy jar to lib directory on each region server

4. Restart region servers

5. `java -cp $(hbase classpath):target/hbase-custom-replication-endpoint-example-1.0-SNAPSHOT.jar com.avalonconsult.hbase.utility.AddCustomReplicationEndpoint --peer-name customreplicationendpoint --class-name com.avalonconsult.hbase.CustomReplicationEndpoint`

Using the `hbase shell` enter the following commands:

```

create 'test-table', {NAME => 'cf', REPLICATION_SCOPE=>'1'}

put 'test-table', 'a', 'cf:q', 'v'

disable 'test-table'

drop 'test-table'

```

### Unit Tests

1. `mvn clean test`

## Adding a Custom Replication Endpoint

### HBase Shell

Currently not supported.

### Java

```java

try(ReplicationAdmin replicationAdmin = new ReplicationAdmin(utility.getConfiguration())) {

    ReplicationPeerConfig peerConfig = new ReplicationPeerConfig()

          .setClusterKey(ZKUtil.getZooKeeperClusterKey(utility.getConfiguration()))

          .setReplicationEndpointImpl(TestWrapperCustomReplicationEndpoint.class.getName());

    Map> tableCfs = new HashMap<>();

    List cfs = new ArrayList<>();

    cfs.add(COLUMN_FAMILY);

    tableCfs.put(TABLE_NAME, cfs);

    replicationAdmin.addPeer(PEER_NAME, peerConfig, tableCfs);

}

```

## Notes

* addPeer methods that are deprecated are currently used by `hbase shell`

* Why is there a tableCFs map? Why not stick with the string?

* This method currently calls `getTableCfsStr(tableCfs)` to convert the tableCfs map to a string.

```java

public void addPeer(String id, ReplicationPeerConfig peerConfig, Map> tableCfs) throws ReplicationException

```

* ReplicationAdmin should have a method signature:

```java

public void addPeer(String id, ReplicationPeerConfig peerConfig, String tableCFs) throws ReplicationException

```

* There could be a ReplicationAdmin method like the following:

```java

public void addPeer(String id, String clusterKey, String replicationEndpointClassname, String tableCFs) throws ReplicationException {

  this.replicationPeers.addPeer(

    id,

    new ReplicationPeerConfig()

      .setClusterKey(clusterKey)

      .setReplicationEndpointImpl(replicationEndpointClassname),

    tableCFs

  );

}

```

* This would enable an easy modification of HBase shell add_peer command to support adding a Custom Replication Endpoint peer.

## References

* https://issues.apache.org/jira/browse/HBASE-11367

* https://issues.apache.org/jira/browse/HBASE-11992

* https://issues.apache.org/jira/browse/HBASE-12254