{"id":13645724,"url":"https://github.com/gptankit/serviceq","last_synced_at":"2025-04-21T17:31:16.653Z","repository":{"id":31389801,"uuid":"128976892","full_name":"gptankit/serviceq","owner":"gptankit","description":"Super fault-tolerant HTTP load balancer \u0026 queue. White paper for reference - https://github.com/gptankit/serviceq-paper","archived":false,"fork":false,"pushed_at":"2025-04-15T10:55:37.000Z","size":140,"stargazers_count":76,"open_issues_count":2,"forks_count":8,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-04-15T11:45:14.615Z","etag":null,"topics":["go","golang","http-proxy","load-balancer","request-queuing"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gptankit.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-04-10T18:16:33.000Z","updated_at":"2025-04-15T10:55:36.000Z","dependencies_parsed_at":"2024-01-14T09:57:25.333Z","dependency_job_id":"177e4491-a9aa-4db4-b8ef-a1d73c326e39","html_url":"https://github.com/gptankit/serviceq","commit_stats":null,"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gptankit%2Fserviceq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gptankit%2Fserviceq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gptankit%2Fserviceq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gptankit%2Fserviceq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gptankit","download_url":"https://codeload.github.com/gptankit/serviceq/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250100357,"owners_count":21374925,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","golang","http-proxy","load-balancer","request-queuing"],"created_at":"2024-08-02T01:02:40.572Z","updated_at":"2025-04-21T17:31:16.424Z","avatar_url":"https://github.com/gptankit.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# ServiceQ [![Build Status](https://travis-ci.com/gptankit/serviceq.svg?branch=master)](https://travis-ci.com/gptankit/serviceq) [![GoDoc](https://godoc.org/github.com/gptankit/serviceq?status.svg)](https://pkg.go.dev/github.com/gptankit/serviceq?tab=subdirectories)\n\nServiceQ is a fault-tolerant gateway for HTTP clusters. It employs probabilistic routing to distribute load during partial cluster shutdown (k/n nodes experiencing downtimes, timeouts, connection loss etc) and queues requests during total cluster shutdown (n nodes down). The queued requests are forwarded in FIFO order when the cluster is available next.\n\nBelow graph shows the routing probability (P) on a down node (D) in a 8-node cluster with respect to number of requests (r). Notice how quickly the routing probability on D reduces as the requests on D start to fail. Depending on the rate of request, it will only take a few seconds (sometime even milliseconds) to move all requests away from D, thus ensuring more requests are routed to healthier nodes.\n\n\u003cp\u003e\n\u003cimg src=\"https://github.com/gptankit/illustrations/blob/master/serviceq/prob-8.png?raw=true\" style=\"width:50%\"/\u003e \n\u003c/p\u003e\n\nNote that, even when requests keep failing on D (however less), ServiceQ retries them on other nodes until they succeed. If they do not succeed on any of the nodes, they are queued and periodically retried on the cluster (using the same approach above), until they succeed.\n\n\u003cb\u003eNoticeable features\u003c/b\u003e\n\n* HTTP Load Balancing\u003cbr/\u003e\n* Probabilistic node selection based on error feedback\u003cbr/\u003e\n* Failed request queueing and deferred forwarding\u003cbr/\u003e\n* Upfront request queueing\u003cbr/\u003e\n* Request retries\u003cbr/\u003e\n* Concurrent connections limit\u003cbr/\u003e\n* Complete TLS/SSL support (automatic and manual)\n\nHere are the steps to run ServiceQ - \u003c/br\u003e\n\n\u003cb\u003eDownload\u003c/b\u003e\n\nClone the project into any directory in your workspace \u003cbr/\u003e\n\n\u003cpre\u003e\n$ git clone https://github.com/gptankit/serviceq\n\u003c/pre\u003e\n\nChange into directory \u003ci\u003eserviceq\u003c/i\u003e\u003cbr/\u003e\n\n\u003cb\u003eHow to Build\u003c/b\u003e\n\n\u003cpre\u003e$ make ('make build' will also work)\u003c/pre\u003e\n\nOptional: \u003ci\u003emake\u003c/i\u003e with debug symbols removed (~25% size reduction)\n\n\u003cpre\u003e$ make build-nodbg\u003c/pre\u003e\n\nThis will create a Go binary \u003ci\u003eserviceq\u003c/i\u003e in the current directory\n\n\u003cb\u003eHow to Install\u003c/b\u003e\n\nMake sure the current user has root privileges, then - \u003c/br\u003e\n\n\u003cpre\u003e$ make install\u003c/pre\u003e\n\nThis will create a folder \u003ci\u003eserviceq\u003c/i\u003e in \u003ci\u003e/usr/local\u003c/i\u003e directory and copy the \u003ci\u003eserviceq\u003c/i\u003e binary (generated in the build step) to \u003ci\u003e/usr/local/serviceq\u003c/i\u003e and \u003ci\u003esq.properties\u003c/i\u003e (serviceq configuration file) to \u003ci\u003e/usr/local/serviceq/config\u003c/i\u003e.\u003cbr/\u003e\n\n\u003cb\u003eHow to Run\u003c/b\u003e\n\nBefore running, make sure the mandatory configurations in \u003ci\u003e/usr/local/serviceq/config/sq.properties\u003c/i\u003e are set (\u003cb\u003eLISTENER_PORT\u003c/b\u003e, \u003cb\u003ePROTO\u003c/b\u003e, \u003cb\u003eENDPOINTS\u003c/b\u003e, \u003cb\u003eCONCURRENCY_PEAK\u003c/b\u003e). The configuration file closely resembles a typical \u003cb\u003eINI\u003c/b\u003e file so its fairly easy to understand and make changes -\u003c/br\u003e\n\n\u003cpre\u003e\n#sq.properties\n\n#Port on which serviceq listens on\nLISTENER_PORT=5252\n\n#Protocol the endpoints listens on -- 'http' for both http/https\nPROTO=http\n\n#Endpoints seperated by comma (,) -- no spaces allowed, can be a combination of http/https\nENDPOINTS=http://my.server1.com:8080,http://my.server2.com:8080,http://my.server3.com:8080\n\n#Concurrency peak defines how many max concurrent connections are allowed to the cluster\nCONCURRENCY_PEAK=2048\n\u003c/pre\u003e\n\nAlso, verify timeout value (default is set to 5s). Low value is preferable as it allows retries to be faster -\u003c/br\u003e\n\n\u003cpre\u003e\n#Timeout (s) is added to each outgoing request to endpoints, the existing timeouts are overriden, value of -1 means no timeout\nOUTGOING_REQUEST_TIMEOUT=5\n\u003c/pre\u003e\n\nBy default deferred queue is enabled with all methods and routes allowed. These options can be controlled as -\u003c/br\u003e\n\n\u003cpre\u003e\n#Enable deferred queue for requests on final failures (cluster down)\nENABLE_DEFERRED_Q=true\n\n#Request format allows given method/route on deferred queue -- picked up if ENABLE_DEFERRED_Q is true\n#Q_REQUEST_FORMATS=POST /orders,PUT,PATCH,DELETE\n#Q_REQUEST_FORMATS=ALL\nQ_REQUEST_FORMATS=POST,PUT,PATCH,DELETE\n\u003c/pre\u003e\n\n(Note that Q_REQUEST_FORMATS is also considered if ENABLE_UPFRONT_Q is true)\n\nAfter all is set - \u003c/br\u003e\n\n\u003cpre\u003e$ sudo /usr/local/serviceq/serviceq\u003c/pre\u003e\n\nRefer wiki for more details: https://github.com/gptankit/serviceq/wiki\n\nFeel free to play around and post feedbacks\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgptankit%2Fserviceq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgptankit%2Fserviceq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgptankit%2Fserviceq/lists"}