{"id":24545033,"url":"https://github.com/tkxwaweru/load_balancer","last_synced_at":"2025-04-15T15:05:37.904Z","repository":{"id":236517517,"uuid":"792756655","full_name":"tkxwaweru/load_balancer","owner":"tkxwaweru","description":"This project features the creation of a load balancer intended to manage situations when servers are bombarded with a very large number of user resource requests simultaneously.","archived":false,"fork":false,"pushed_at":"2024-06-12T05:48:31.000Z","size":113,"stargazers_count":1,"open_issues_count":0,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-15T15:05:33.847Z","etag":null,"topics":["distributed-systems","docker","docker-compose","flask","python","servers"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tkxwaweru.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-27T13:44:53.000Z","updated_at":"2025-02-03T14:22:29.000Z","dependencies_parsed_at":"2024-04-27T15:26:04.870Z","dependency_job_id":"6fc61641-49cf-4d93-b065-59015d1bfed3","html_url":"https://github.com/tkxwaweru/load_balancer","commit_stats":null,"previous_names":["tkxwaweru/load_balancer"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tkxwaweru%2Fload_balancer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tkxwaweru%2Fload_balancer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tkxwaweru%2Fload_balancer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tkxwaweru%2Fload_balancer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tkxwaweru","download_url":"https://codeload.github.com/tkxwaweru/load_balancer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249094931,"owners_count":21211837,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed-systems","docker","docker-compose","flask","python","servers"],"created_at":"2025-01-22T21:17:46.034Z","updated_at":"2025-04-15T15:05:37.887Z","avatar_url":"https://github.com/tkxwaweru.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003clink rel=\"stylesheet\" href=\"https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.15.3/css/all.min.css\"\u003e\n\n# IMPLEMENTING A LOAD BALANCER \n\n## ICS 4B\n### Group Members:\n1. 137931 Trevor Waweru \n2. 140801 Kyla Arunga\n3. 138551 Ruweida Ismail\n4. 145644 Fardowsa Hassan\n\n## Quick Links\n- [Introduction](https://github.com/tkxwaweru/load_balancer?tab=readme-ov-file#introduction)\n\n- [Running the project](https://github.com/tkxwaweru/load_balancer?tab=readme-ov-file#running-the-project)\n\n- [Analysis](https://github.com/tkxwaweru/load_balancer?tab=readme-ov-file#analysis)\n\n## Introduction\nThis project deals with the implementation of a load balancer that is meant to divide a load of requests among N server replicas (for instance N = 3 replicas). The purpose of the load balancer is to prevent overwhelming a single server with a large number of requests which could lead to slow server responses and probable server failure.\n\nThe system is capable of adding server replicas into the environment to increase the number servers the load balancer has to work with in terms of load distribution. The system utilizes consistent hashing and quadratic probing to guide the load distribution algorithm. \n\n## Running the project\n### 1. Requirements:\n1. Windows OS (Windows 10 or newer)\n2. Docker version 26.1.1\n3. Docker Compose version v2.27.0\n4. Python version 3.11.0 or newer\n5. Flask version 3.0.3 or newer\n6. Docker desktop\n\n### 2. Installation\n\n\u003cb\u003eDisclaimer\u003c/b\u003e: Modify the commands appropriately for your case.\n\n1. Clone this repository into your machine \n  ```{code}\n  git clone https://github.com/tkxwaweru/load_balancer.git load_balancer\n  ```\n2. Build the server containers:\n  ```{code}\n  docker-compose build\n  ```\n3. Run the containers in the background:\n  ```{code}\n  docker-compose up -d\n  ```\n4. View the logs (you can also monitor the logs via docker desktop):\n  ```{code}\n  docker-compose logs\n  ```\n5. The system makes use of batch files (.bat) to ease certain operations within the system. They shall be discussed in the analysis section.\n## Analysis\n\n### Test 1: Request handling on N = 3 server containers\nLaunch 10000 async requests on N = 3 server containers and report the request count handled by each server instance in a bar chart. Explain your observations in the graph and your view on the performance.\n\n#### \u003cu\u003eProcess\u003c/u\u003e\nBy default, the system starts with N = 3 active server replicas:\n```\nC:\\Users\\user\\load_balancer\\server\u003ereplicas.bat\n{\n  \"message\": {\n    \"N\": 3,\n    \"replicas\": [\n      \"1\",\n      \"2\",\n      \"3\"\n    ]\n  },\n  \"status\": \"successful\"\n}\n```\nTo perform this task we utilised a batch file: 10K_load.bat which makes use of the request load balancer endpoint.\n```\n@echo off\nrem Send a payload of 10000 asynchronous requests to the load balancer for server distribution\n\nFOR /L %%i IN (1,1,10000) DO (\n    echo Sending request with request_id=%%i\n    curl http://localhost:5000/request?request_id=%%i\n)\npause\n```\nAfter performing this task we used a batch file (request_count.bat) to count requests handled by each of the 3 server replicas then plotted a bar graph showing request count per server.\n\nAfter running request_count.bat:\n```\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 36136  100 36136    0     0   550k      0 --:--:-- --:--:-- --:--:--  551k\nNumber of requests for Server 1: 3650\n\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 33801  100 33801    0     0   280k      0 --:--:-- --:--:-- --:--:--  282k\nNumber of requests for Server 2: 3412\n\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 29104  100 29104    0     0  1032k      0 --:--:-- --:--:-- --:--:-- 1052k\nNumber of requests for Server 3: 2938\n```\nBar graph:\n\n![Bar graph](https://github.com/tkxwaweru/load_balancer/blob/main/images/test1_bar-graph.png)\n\n\u003ci\u003eFurther information on the plotting of the graph can be found in \u003cb\u003esheets/Analysis.xlsx\u003c/b\u003e.\u003c/i\u003e\n\n#### \u003cu\u003eObservations\u003c/u\u003e\n\nThe distribution of requests among the servers seems relatively balanced. Although the numbers aren't exactly equal, they are in the same ballpark, indicating that the load balancer is distributing requests somewhat evenly across the servers.\n\nOverall, based on the observations, the load balancer seems to be performing reasonably well in distributing the load among the available servers.\n\n### Test 2: Request handling while incrementing server containers from N = 2 to N = 6\nNext, increment N from 2 to 6 and launch 10000 requests on each such increment. Report the average load of the servers at each run in a line chart. Explain your observations in the graph and your view on the scalability of the load balancer implementation.\n\n#### \u003cu\u003eProcess\u003c/u\u003e\n\nThis test involved a repetititive process of sending a payload of 10000 request to N servers to observe the load distribution starting from N = 2 to N = 6 server replicas. \n\nThis process involved the use of various batch files to add servers dynamically into the system (server3_add.bat, server4_add.bat etc.) a batch file to send the load (10K_load.bat) and finally a batch file to view the load count after each iteration (request_count.bat).\n\nHere is the output from each iteration after running request_count.bat:\n\n1. N = 2:\n```\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 53543  100 53543    0     0  1200k      0 --:--:-- --:--:-- --:--:-- 1216k\nNumber of requests for Server 1: 5412\n\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 45449  100 45449    0     0   723k      0 --:--:-- --:--:-- --:--:--  727k\nNumber of requests for Server 2: 4588\n\n```\n2. N = 3:\n```\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 36136  100 36136    0     0  1012k      0 --:--:-- --:--:-- --:--:-- 1037k\nNumber of requests for Server 1: 3650\n\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 33801  100 33801    0     0  1092k      0 --:--:-- --:--:-- --:--:-- 1100k\nNumber of requests for Server 2: 3412\n\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 29104  100 29104    0     0  1069k      0 --:--:-- --:--:-- --:--:-- 1093k\nNumber of requests for Server 3: 2938\n```\n3. N = 4:\n```\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 25802  100 25802    0     0   768k      0 --:--:-- --:--:-- --:--:--  787k\nNumber of requests for Server 1: 2605\n\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 25652  100 25652    0     0   948k      0 --:--:-- --:--:-- --:--:--  963k\nNumber of requests for Server 2: 2588\n\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 21528  100 21528    0     0   353k      0 --:--:-- --:--:-- --:--:--  356k\nNumber of requests for Server 3: 2171\n\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 26108  100 26108    0     0   893k      0 --:--:-- --:--:-- --:--:--  910k\nNumber of requests for Server 4: 2636\n\n```\n4. N = 5:\n```\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 21287  100 21287    0     0   491k      0 --:--:-- --:--:-- --:--:--  494k\nNumber of requests for Server 1: 2148\n\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 20851  100 20851    0     0   609k      0 --:--:-- --:--:-- --:--:--  617k\nNumber of requests for Server 2: 2104\n\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 17592  100 17592    0     0   573k      0 --:--:-- --:--:-- --:--:--  592k\nNumber of requests for Server 3: 1773\n\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 19597  100 19597    0     0   702k      0 --:--:-- --:--:-- --:--:--  708k\nNumber of requests for Server 4: 1976\n\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 19812  100 19812    0     0   391k      0 --:--:-- --:--:-- --:--:--  394k\nNumber of requests for Server 5: 1999\n```\n5. N = 6:\n```\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 16035  100 16035    0     0   517k      0 --:--:-- --:--:-- --:--:--  521k\nNumber of requests for Server 1: 1616\n\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 17774  100 17774    0     0   627k      0 --:--:-- --:--:-- --:--:--  642k\nNumber of requests for Server 2: 1793\n\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 15271  100 15271    0     0   462k      0 --:--:-- --:--:-- --:--:--  466k\nNumber of requests for Server 3: 1538\n\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 16141  100 16141    0     0   585k      0 --:--:-- --:--:-- --:--:--  606k\nNumber of requests for Server 4: 1627\n\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 17444  100 17444    0     0   752k      0 --:--:-- --:--:-- --:--:--  774k\nNumber of requests for Server 5: 1760\n\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 16523  100 16523    0     0   303k      0 --:--:-- --:--:-- --:--:--  304k\nNumber of requests for Server 6: 1666\n```\n\nBelow is a line graph that shows the trend of average requests per server as we increment N.\n\n![Line graph](https://github.com/tkxwaweru/load_balancer/blob/main/images/test2_line-graph.png)\n\n\u003ci\u003eFurther information on the plotting of the graph can be found in \u003cb\u003esheets/Analysis.xlsx\u003c/b\u003e.\u003c/i\u003e\n\n#### \u003cu\u003eObservations\u003c/u\u003e\n\nAs the number of server replicas (N) increases, the average load per server decreases. This trend suggests that the load balancer is effectively distributing the load more evenly as the number of servers increases. This is a positive indication of the load balancer's ability to scale and handle increased traffic by distributing it across a larger number of servers.\n\nThe load balancer implementation demonstrates scalability as evidenced by its ability to handle an increasing number of server replicas while maintaining a balanced load distribution. As N increases from 2 to 6, the average load per server decreases proportionally, indicating that the load balancer can efficiently utilize additional server resources to accommodate higher traffic loads.\n\n### Test 3: Testing of endpoints and server failure handling\nTest all endpoints of the load balancer and show that in case of server failure, the load balancer spawns a new instance quickly to handle the load.\n\n1. /rep endpoint:\n\nHere we can make use of the replicas.bat file to demonstrate the functionality of this endpoint:\n\nreplicas.bat\n```\n@echo off\nrem Display available replicas of servers in the system\n\ncurl http://localhost:5000/rep\n```\n\nSample output:\n```\n{\n  \"message\": {\n    \"N\": 3,\n    \"replicas\": [\n      \"2\",\n      \"3\",\n      \"4\"\n    ]\n  },\n  \"status\": \"successful\"\n}\n```\n\n2. /rm endpoint:\n\nHere we can make use of a server removal batch file (server3_rm.bat) to demonstrate the functionality of this endpoint by removing server 3:\n\nserver3_rm.bat\n```\n@echo off\n\nrem Dynamically remove server 3\ncurl -X DELETE -H \"Content-Type: application/json\" -d \"{\\\"n\\\": 1, \\\"hostnames\\\": [\\\"3\\\"]}\" http://localhost:5000/rm\n```\n\nOutput:\n```\n{\n  \"message\": {\n    \"N\": 2,\n    \"replicas\": [\n      \"2\",\n      \"4\"\n    ]\n  },\n  \"status\": \"successful\"\n}\n```\n\n3. /add endpoint:\n\nHere we can make use of a server addition batch file (server3_add.bat) to demonstrate the functionality of this endpoint by adding server 3 back which we removed above:\n\nserver3_add.bat\n```\n@echo off\n\nrem Dynamically add server 3\ncurl -X POST -H \"Content-Type: application/json\" -d \"{\\\"n\\\": 1, \\\"hostnames\\\": [\\\"3\\\"]}\" http://localhost:5000/add\n```\n\nSample output:\n```\n{\n  \"message\": {\n    \"N\": 3,\n    \"replicas\": [\n      \"2\",\n      \"4\",\n      \"3\"\n    ]\n  },\n  \"status\": \"successful\"\n}\n```\n\n4. /request endpoint:\nThe functionality of this endpoint can be demonstrated using the payload batch files e.g. 100_load.bat which make use of the endpoint to send requests to the load balancer for distribution to server replicas.\n\n100_load.bat\n```\n@echo off\nrem Send a minor payload of 100 requests to the load balancer for server distribution\n\nFOR /L %%i IN (1,1,100) DO (\n    echo Sending request with request_id=%%i\n    curl http://localhost:5000/request?request_id=%%i\n)\npause\n```\n\nSample output:\n```\nSending request with request_id=1\n{\n  \"response\": {\n    \"message\": \"Hello from Server: 4\",\n    \"status\": \"Successful\"\n  },\n  \"server_id\": \"4\"\n}\nSending request with request_id=2\n{\n  \"response\": {\n    \"message\": \"Hello from Server: 2\",\n    \"status\": \"Successful\"\n  },\n  \"server_id\": \"2\"\n}\nSending request with request_id=3\n{\n  \"response\": {\n    \"message\": \"Hello from Server: 4\",\n    \"status\": \"Successful\"\n  },\n  \"server_id\": \"4\"\n}\n```\n\n5. Server failure simulation\n\nHere, a batch file (simulate_failure.bat) is used to simulate the failure of server 1. Upon failure of the server, a new replica, server 4 is added so that we have N = 3 replicas to help distribute the load. The batch file then sends a payload of 1000 request to the 3 replicas and outputs the count of requests per server.\n\nsimulate_failure.bat:\n```\n@echo off\nsetlocal enabledelayedexpansion\n\nrem Step 1: Remove server 1 to simulate failure of the server and inability to handle requests\necho Removing server 1...\ncurl -X DELETE -H \"Content-Type: application/json\" -d \"{\\\"n\\\": 1, \\\"hostnames\\\": [\\\"1\\\"]}\" http://localhost:5000/rm \ntimeout /t 5\n\nrem Step 2: Add server 4 - new replica to handle load\necho Adding server 4...\ncurl -X POST -H \"Content-Type: application/json\" -d \"{\\\"n\\\": 1, \\\"hostnames\\\": [\\\"4\\\"]}\" http://localhost:5000/add\ntimeout /t 5\n\nrem Step 3: Display active replicas\necho Displaying active replicas...\ncurl http://localhost:5000/rep\ntimeout /t 5\n\nrem Step 4: Send a payload of 1000 requests for testing\necho Sending a payload of 1000 requests...\nFOR /L %%i IN (1,1,1000) DO (\n    echo Sending request with request_id=%%i\n    curl http://localhost:5000/request?request_id=%%i\n)\ntimeout /t 5\n\nrem Step 5: Display count of requests per active server\necho Displaying count of requests per server...\nrem Define the base URL\nset BASE_URL=http://localhost:5000\n\nrem Define the active servers (update this list as needed)\nset ACTIVE_SERVERS=2 3 4 \n\nrem Temporary file to store curl output\nset TEMP_FILE=temp_output.txt\n\nrem Iterate over each server and get the requests\nfor %%s in (%ACTIVE_SERVERS%) do (\n    rem Fetch the requests for the current server and save to temporary file\n    curl %BASE_URL%/requests/%%s \u003e %TEMP_FILE%\n\n    rem Count the number of request IDs\n    set COUNT=0\n    for /f %%A in ('findstr /r /c:\"[0-9]\" %TEMP_FILE% ^| find /c /v \"\"') do (\n        set COUNT=%%A\n    )\n\n    rem Display the count\n    echo Number of requests for Server %%s: !COUNT!\n    echo.\n)\n\nrem Clean up the temporary file\ndel %TEMP_FILE%\n\nendlocal\n```\n\nSimply run the batch file to observe the simulation:\n```\nC:\\Users\\user\\load_balancer\\server\u003esimulate_failure.bat\n```\n\nLoad count after simulation:\n```\nDisplaying count of requests per server...\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100  3015  100  3015    0     0  99927      0 --:--:-- --:--:-- --:--:--   98k\nNumber of requests for Server 2: 333\n\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100  2769  100  2769    0     0  28131      0 --:--:-- --:--:-- --:--:-- 28255\nNumber of requests for Server 3: 306\n\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100  3256  100  3256    0     0   109k      0 --:--:-- --:--:-- --:--:--  109k\nNumber of requests for Server 4: 361\n```\n\n### Hash function modification\nFinally, modify the hash functions H(i), Φ(i, j) and report the observations from Test 1 and Test 2.\n\nFor this case, the load balancer has demonstrated efficiency in load distribution. \n\nIt is capable of nearly evenly distributing load among available server replicas and can continue doing so as the number of replicas increase thus also demonstarting scalability. As such the hashing algorithm need not be modified.\n\n## Conclusion\nThe load balancer developed has proven successful. It is capable of close to evenly distributing load across available active server replicas. This can be observed in the analysis section. Moreover, the system is capable of managing it's active server replicas through processes such as adding replicas, removing replicas and routing requests to available replicas using a consistent hashing algorithm.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftkxwaweru%2Fload_balancer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftkxwaweru%2Fload_balancer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftkxwaweru%2Fload_balancer/lists"}