Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/gptankit/serviceq

Super fault-tolerant HTTP load balancer & queue. White paper for reference - https://github.com/gptankit/serviceq-paper
https://github.com/gptankit/serviceq

go golang http-proxy load-balancer request-queuing

Last synced: 3 months ago
JSON representation

Super fault-tolerant HTTP load balancer & queue. White paper for reference - https://github.com/gptankit/serviceq-paper

Awesome Lists containing this project

README 

        
# ServiceQ [![Build Status](https://travis-ci.com/gptankit/serviceq.svg?branch=master)](https://travis-ci.com/gptankit/serviceq) [![GoDoc](https://godoc.org/github.com/gptankit/serviceq?status.svg)](https://pkg.go.dev/github.com/gptankit/serviceq?tab=subdirectories)



ServiceQ is a fault-tolerant gateway for HTTP clusters. It employs probabilistic routing to distribute load during partial cluster shutdown (k/n nodes experiencing downtimes, timeouts, connection loss etc) and queues requests during total cluster shutdown (n nodes down). The queued requests are forwarded in FIFO order when the cluster is available next.



Below graph shows the routing probability (P) on a down node (D) in a 8-node cluster with respect to number of requests (r). Notice how quickly the routing probability on D reduces as the requests on D start to fail. Depending on the rate of request, it will only take a few seconds (sometime even milliseconds) to move all requests away from D, thus ensuring more requests are routed to healthier nodes.





 




Note that, even when requests keep failing on D (however less), ServiceQ retries them on other nodes until they succeed. If they do not succeed on any of the nodes, they are queued and periodically retried on the cluster (using the same approach above), until they succeed.



Noticeable features



* HTTP Load Balancing


* Probabilistic node selection based on error feedback


* Failed request queueing and deferred forwarding


* Upfront request queueing


* Request retries


* Concurrent connections limit


* Complete TLS/SSL support (automatic and manual)



Here are the steps to run ServiceQ - 



Download



Clone the project into any directory in your workspace 




$ git clone https://github.com/gptankit/serviceq




Change into directory serviceq



How to Build



$ make ('make build' will also work)



Optional: make with debug symbols removed (~25% size reduction)



$ make build-nodbg



This will create a Go binary serviceq in the current directory



How to Install



Make sure the current user has root privileges, then - 



$ make install



This will create a folder serviceq in /usr/local directory and copy the serviceq binary (generated in the build step) to /usr/local/serviceq and sq.properties (serviceq configuration file) to /usr/local/serviceq/config.



How to Run



Before running, make sure the mandatory configurations in /usr/local/serviceq/config/sq.properties are set (LISTENER_PORT, PROTO, ENDPOINTS, CONCURRENCY_PEAK). The configuration file closely resembles a typical INI file so its fairly easy to understand and make changes -




#sq.properties



#Port on which serviceq listens on

LISTENER_PORT=5252



#Protocol the endpoints listens on -- 'http' for both http/https

PROTO=http



#Endpoints seperated by comma (,) -- no spaces allowed, can be a combination of http/https

ENDPOINTS=http://my.server1.com:8080,http://my.server2.com:8080,http://my.server3.com:8080



#Concurrency peak defines how many max concurrent connections are allowed to the cluster

CONCURRENCY_PEAK=2048




Also, verify timeout value (default is set to 5s). Low value is preferable as it allows retries to be faster -




#Timeout (s) is added to each outgoing request to endpoints, the existing timeouts are overriden, value of -1 means no timeout

OUTGOING_REQUEST_TIMEOUT=5




By default deferred queue is enabled with all methods and routes allowed. These options can be controlled as -




#Enable deferred queue for requests on final failures (cluster down)

ENABLE_DEFERRED_Q=true



#Request format allows given method/route on deferred queue -- picked up if ENABLE_DEFERRED_Q is true

#Q_REQUEST_FORMATS=POST /orders,PUT,PATCH,DELETE

#Q_REQUEST_FORMATS=ALL

Q_REQUEST_FORMATS=POST,PUT,PATCH,DELETE




(Note that Q_REQUEST_FORMATS is also considered if ENABLE_UPFRONT_Q is true)



After all is set - 



$ sudo /usr/local/serviceq/serviceq



Refer wiki for more details: https://github.com/gptankit/serviceq/wiki



Feel free to play around and post feedbacks