https://github.com/kong/astronode-broadcaster
A TCP replication server, or broadcaster, that replicates TCP commands to other TCP servers
https://github.com/kong/astronode-broadcaster
Last synced: 5 months ago
JSON representation
A TCP replication server, or broadcaster, that replicates TCP commands to other TCP servers
- Host: GitHub
- URL: https://github.com/kong/astronode-broadcaster
- Owner: Kong
- Created: 2013-11-14T08:14:04.000Z (almost 12 years ago)
- Default Branch: master
- Last Pushed: 2021-09-15T14:36:10.000Z (about 4 years ago)
- Last Synced: 2025-05-08T22:06:59.101Z (5 months ago)
- Language: Java
- Homepage: https://mashape.com
- Size: 11 MB
- Stars: 30
- Watchers: 31
- Forks: 10
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Astronode Broadcaster [](https://travis-ci.org/Mashape/Astronode-Broadcaster)
The Astronode Broadcaster is a TCP replication server, or broadcaster, that replicates TCP commands to other TCP servers. Built with Java and Netty, it's super fast, and enterprise ready.
Created with love by [thefosk](https://github.com/thefosk) @ [mashape.com](https://mashape.com)
### Features
* Offload TCP connections from your client apps to the broadcaster. It can handle hundreds of backend connections, simplifying broadcasting data to servers just by opening one connection client-side.
* Automatically handles backend connection downtimes, network failures and reconnections.
* Optionally **auto-update** backend servers from one single configurable HTTP endpoint, useful for hot configuring the broadcaster remotely without restarting it.
* Forwards back the responses from all the backend servers, or optionally skips them all: useful in *write-and-forget* scenarios.
* Tunable network and timeout settings.
* Very fast, and easy to use.
## Usage
```
java -jar astronode-broadcaster.jar -c ./configuration
```
## Download
You can download the Astronode-Broadcaster from: [https://github.com/Mashape/Astronode-Broadcaster/raw/master/astronode-broadcaster.zip](https://github.com/Mashape/Astronode-Broadcaster/raw/master/astronode-broadcaster.zip)
### Generate JAR from source
You can generate the JAR file using [Maven](http://maven.apache.org/), by executing:
```
git clone https://github.com/Mashape/Astronode-Broadcaster
cd Astronode-Broadcaster
mvn clean assembly:assembly
```
This will generate two `jar` files in the `target` folder. Use the `astronode-broadcaster-1.0-SNAPSHOT-jar-with-dependencies.jar` file (and not `astronode-broadcaster-1.0-SNAPSHOT.jar`).
This means you can run the Astronode-Broadcaster by executing:
```
java -jar target/astronode-broadcaster-1.0-SNAPSHOT-jar-with-dependencies.jar -c ./configuraton
```
## Use Cases
There are many different use case scenarios where the Astronode Broadcaster can work well. It can be used to create eventual consistent distributed clusters for those services that don't support clustering. For example you can use it with in-memory stores like Redis or Memcached to replicate any kind of data, counters, or implement an eventual consistent distributed caching cluster across a LAN/WAN.
### Distributed Redis cluster
For example:
* You have two application servers in two different world regions: `app.us` and `app.eu`
* You also have one Redis server for each region: `redis.us:6379` and `redis.eu:6379`
* The application servers need to talk with Redis to load cached content. `app.us` will read from `redis.us` and `app.eu` will read from `redis.eu` to reduce network latency
* Every application reads data from the closest Redis server, but broadcasts the writes to the other region by using Astronode Broadcaster
* You can setup two Astronode Broadcasters in each region: `broadcaster.us:6379` and `broadcaster.eu:6379`
* You configure both broadcasters with the following backend servers: `redis.us:6379` and `redis.eu:6379`
With this configuration, any command sent to `broadcaster.us:6379` is broadcasted to both `redis1.us:6379` and `redis2.eu:6379`. The same happens for every command sent to `broadcaster.eu:6379`.
The server `app.us` is in the same region of `redis.us`. Now you could execute all the reads directly from `redis.us:6379` and submit all the writes to `broadcaster.us:6379`, so that an other server `app.eu` could read the same data in his own region from `redis.eu:6379`. Basically the app servers are using Redis for reading, and Astronode Broadcaster for writing.
If you need to broadcast data to, let's say, 100+ Redis machines, open just one connection to the broadcaster and let him do the hard work for you. And with the auto-update feature, you can hot-configure the broadcaster remotely without reloading it.
# Configuration
```
# Astronode Broadcaster configuration file
# Backend Servers configuration
#
# You can explicitly set the backend servers with the "servers" property.
# both the host and the port are mandatory.
#
# the servers must be comma separated "host:port" values, like: "servers=127.0.0.1:8000, 127.0.0.2:8001"
servers=127.0.0.1:6379
################################ AUTO-UPDATE #################################
# Instead, or in addition, of explicitly set the backend servers, you can also set an HTTP endpoint that will load
# the backend servers. To enable this functionality set "servers_autoupdate=true"
#
# you can specify the HTTP method, the URL and any additional parameter to be sent with the request
#
# the servers will be automatically refreshed by the seconds timeout specified in "servers_autoupdate_refresh"
# or not refreshed at all if the property is set to "0" zero.
servers_autoupdate=false
# The HTTP methods available are GET, POST, PUT, PATCH, DELETE, although only GET and POST are strongly reccomended.
servers_autoupdate_method=POST
# The URL that will return a JSON array containing the servers addresses, like:
# [ "127.0.0.1:10000", "127.0.0.1:20000", "127.0.0.1:30000" ]
servers_autoupdate_url=http://example.com
# Any additional parameters to send along with the request, for example:
# servers_autoupdate_parameters=username=admin&password=hello&role=admin%20user
#
# Be sure to properly encode the parameters
servers_autoupdate_parameters=
# The amount of seconds after which the servers will be repeatedly refreshed by making the HTTP request
#
# set to "0" zero to never refresh the servers after the first request.
servers_autoupdate_refresh=10
############################## GENERAL SETTINGS ###############################
# The host and port where the server should start
host=localhost
# Check that you have permissions to run on the specified port
port=9000
# Set the log level, available values: TRACE, INFO, ERROR
log_level=INFO
############################## NETWORK SETTINGS ###############################
backlog_size=128
reuse_addr=true
# By default Astronode Broadcaster returns all the responses from all the servers
# to skip the responses if not needed, and increase system performance, set this property to "discard_responses=true"
discard_responses=false
############################## TIMEOUT SETTINGS ###############################
# The connection timeout to a backend server in seconds
connect_timeout=5
# The time to wait in seconds before re-attempting the connection
reconnect_delay_timeout=10
# If no data was received from the connected client during the "read_idle_timeout",
# the connection with the client will be closed.
read_idle_timeout=60
```