{"id":21557297,"url":"https://github.com/sweeticelolly/multithreaded-http-server","last_synced_at":"2025-04-10T10:34:17.292Z","repository":{"id":159225867,"uuid":"365022019","full_name":"SweetIceLolly/multithreaded-http-server","owner":"SweetIceLolly","description":"A good performance multithreaded HTTP REST server","archived":false,"fork":false,"pushed_at":"2021-05-07T04:59:08.000Z","size":259,"stargazers_count":12,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-24T09:21:38.429Z","etag":null,"topics":["cpp","http","http-server","jmeter","json","json-request","multithreading","restserver","socket","thread-pool"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SweetIceLolly.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-05-06T19:53:50.000Z","updated_at":"2024-11-21T10:10:30.000Z","dependencies_parsed_at":"2023-07-29T16:15:52.547Z","dependency_job_id":null,"html_url":"https://github.com/SweetIceLolly/multithreaded-http-server","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SweetIceLolly%2Fmultithreaded-http-server","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SweetIceLolly%2Fmultithreaded-http-server/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SweetIceLolly%2Fmultithreaded-http-server/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SweetIceLolly%2Fmultithreaded-http-server/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SweetIceLolly","download_url":"https://codeload.github.com/SweetIceLolly/multithreaded-http-server/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248199245,"owners_count":21063641,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp","http","http-server","jmeter","json","json-request","multithreading","restserver","socket","thread-pool"],"created_at":"2024-11-24T08:11:54.135Z","updated_at":"2025-04-10T10:34:17.279Z","avatar_url":"https://github.com/SweetIceLolly.png","language":"C++","readme":"# Pickles\nGood performance multithreaded HTTP REST server written in C++\n\n# Description\nThere are two major components in this tiny project: The HTTP server and the thread pool. Both of the components can function independently.\n\nIn the RESTserver folder: `mongoose.c` and `mongoose.h` are Cesanta Mongoose, with some minor modifications to make them suit for this project. The `RESTserver` class is the encapsulation of Mongoose using C++ classes, allowing router and handlers of different request methods.\n\nIn the ThreadPool folder: The ThreadPool class is the encapsulation of `std::thread` and provides the basic functionality of a thread pool.\n\n# \"Boast\"\n\n## Cross-platform\nCompiled successfully on Windows using MSVC and on Linux using g++.\n\n## Intuitive and Elegant\nWith just several lines, you can build your own REST server in C++!\n```cpp\n#include \"RESTserver/RESTserver.hpp\"\nRESTserver server;\nint main() {\n    server.addHandler(\"GET\", \"/hello\",\n        [](struct mg_connection *connection, int ev, mg_http_message *ev_data, void *fn_data) {\n            mg_http_reply(connection, 200, NULL, \"Hello world!\");\n        }\n    );\n    server.startServer(\"localhost:8000\", 1000, NULL);\n    return 0;\n}\n```\nCompile with: `g++ -Wall -O2 example.cpp RESTserver/mongoose.c RESTserver/RESTserver.cpp -o example`\n\n## Good Performance\nI can't say it's high performance but the performance is not bad :D\n\nTested on my laptop:\n\nOS: Deepin 20.2 Linux version 5.10.18-amd64-desktop\n\nRAM: 8GB DDR4 2667MHz\n\nCPU: Intel(R) Core(TM) i5-8265U @ 1.60GHz, 4 cores, 8 Threads\n\nBenchmark software: Apache JMeter 5.4.1\n\n### Test 1: Hello World example\nThis test uses the above compiled Hello World example.\n\nIn Apache JMeter, I used a thread group with 8 threads and infinite loops to request `localhost:8000/hello`. Below is the summary report when the test has been ran for 1 minute:\n![image](https://user-images.githubusercontent.com/47358542/117364463-ec1ddf80-ae8b-11eb-9d95-0c89411e3992.png)\nThe Hello World example, with `-O2` optimization handled 4940000+ requests in a minute with average speed 82000+ requests per second. Below is the peak speed with 5 seconds test time:\n![image](https://user-images.githubusercontent.com/47358542/117364685-369f5c00-ae8c-11eb-8150-271e56042bfd.png)\nThe server processed 510000+ requests in 5 seconds with peak speed reached 111000+ requests per second. This is not bad at all for a Hello World example :)\n\n### Test 2: HTTP Requests with JSONs\nThis test is using the POST method to request and return JSONs. Below is the code used, compiled with `-O2` optimization.\n\n```cpp\n#include \"RESTserver/RESTserver.hpp\"\n#include \"utils/json.hpp\"\n\nRESTserver server;\nusing json = nlohmann::json;\n\nint main() {\n    server.addHandler(\"POST\", \"/json\",\n        [](struct mg_connection *connection, int ev, mg_http_message *ev_data, void *fn_data) {\n            char *buffer = new char[ev_data-\u003ebody.len + 1];\n            strncpy(buffer, ev_data-\u003ebody.ptr, ev_data-\u003ebody.len);\n            buffer[ev_data-\u003ebody.len] = '\\0';\n            json j = json::parse(buffer);\n            int val = j.value(\"value\", 0);\n            json res = {\n                { \"plus\", val + 20 },\n                { \"minus\", val - 69 },\n                { \"plus_float\", val + 3.14 },\n                { \"list\", { val, val * 2, val * 3.14 } },\n                { \"object\", {\n                    { \"item1\", val },\n                    { \"item2\", true },\n                    { \"item3\", val - 3.14 },\n                    { \"anotherList\", { val - 1, val - 2.5, \"somestr\", false, val } }\n                }}\n            };\n            mg_http_reply(connection, 200, NULL, res.dump().c_str());\n            delete buffer;\n        }\n    );\n\n    server.startServer(\"localhost:8000\", 1000, NULL);\n    return 0;\n}\n```\nThe program first parses the requested json and obtain the `value` entry, do some arithmetics and form the new JSON data, then seralizes the new JSON data into string and respond. Data types included objects, lists, integers, floating point numbers, and boolean values. Here is a sample request and response:\n\n(request)\n```json\n{\n\t\"something\": \"unrelated\",\n\t\"someobject\": {\n\t\t\"useless\": { \"ha\": 123 },\n\t\t\"blah\": false,\n\t\t\"bepsu\": [1, 2.5, false, {}]\n\t},\n\t\"somelist\": [[], {}, [123, 567], true, \"sss\", 1],\n\t\"value\": -54303,\n\t\"someother\": \"things\"\n}\n```\n\n(response)\n```json\n{\n   \"list\":[\n      -54303,\n      -108606,\n      -170511.42\n   ],\n   \"minus\":-54372,\n   \"object\":{\n      \"anotherList\":[\n         -54304,\n         -54305.5,\n         \"somestr\",\n         false,\n         -54303\n      ],\n      \"item1\":-54303,\n      \"item2\":true,\n      \"item3\":-54306.14\n   },\n   \"plus\":-54283,\n   \"plus_float\":-54299.86\n}\n```\nIn Apache JMeter, I used a thread group with 8 threads and infinite loops to request `localhost:8000/json`. I added a random variable to generate random values for the `value` entry for every request. Below is the summary report when the test has been ran for 1 minute:\n![image](https://user-images.githubusercontent.com/47358542/117379592-c56ca280-aea5-11eb-8f20-0cf97dbbd882.png)\nThe program handled 231000+ requests in a minute with average speed 38000+ requests per second. Below is the peak speed with 5 seconds test time:\n![image](https://user-images.githubusercontent.com/47358542/117379779-3e6bfa00-aea6-11eb-8bf4-c69494e591e1.png)\nThe server processed 200000+ requests in 5 seconds with peak speed reached 44000+ requests per second.\n\n## Low RAM Usage\n\nIf you code carefully, the server won't have any memory leak and maintain low RAM usage. The examples in this document seems doesn't have any memory leak.\n\n- The Hello World example uses 144KB of RAM after it started up. After it has processed millions of requests and still processing requests at about 80000reqs/sec, it maintains RAM usage at 144KB.\n\n- The JSON request example uses 144KB of RAM after it started up. After it has processed millions of requests and still processing requests at about 35000reqs/sec, it maintains RAM usage at 144KB.\n\n- The thread pool with calculation-intensive tasks example uses 152KB of RAM after it started up. After it has processed hundreds of requests and still processing requests at about 8reqs/sec, it maintains RAM usage at 152KB.\n\n# Weakness\n\n## Bad Performance with Intensive Tasks in Threads\n### Test 3: Thread Pool with Calculation-intensive Tasks\nThis test simulates calculation-intensive situations, in which the tasks will be carried out in alternate threads so that the main thread (which handles new requests) is not blocked. Below is the code used, compiled with `g++ -Wall -O2 tpexample.cpp RESTserver/mongoose.c RESTserver/RESTserver.cpp ThreadPool/ThreadPool.cpp -lpthread -o tpexample`.\n\n```cpp\n#include \"RESTserver/RESTserver.hpp\"\n#include \"ThreadPool/ThreadPool.hpp\"\n#include \u003cmath.h\u003e\n\nRESTserver server;\nThreadPool threadPool;\n\ntypedef struct _response {\n    char *data;\n    int httpCode;\n    char *headers;\n} response;\n\ntypedef struct _calc_param {\n    double  val;\n    int     socket;\n} calc_param;\n\nstatic void handleCalc(void *calcParam) {\n    calc_param *param = (calc_param*)calcParam;\n    response res;\n    double tmp;\n\n    tmp = 0;\n    for (int i = 0; i \u003c 5000000; i++) {\n        tmp += pow(-1, i) * pow(param-\u003eval, i + 1.0) / (i + 1.0);\n    }\n\n    res.data = strdup(std::to_string(tmp).c_str());\n    res.headers = strdup(\"\");\n    res.httpCode = 200;\n    send(param-\u003esocket, (char *)\u0026res, sizeof(res), 0);\n    close(param-\u003esocket);\n}\n\nint main() {\n    server.setPollHandler(\n        [](struct mg_connection *connection, int ev, mg_http_message *ev_data, void *fn_data) {\n            if (connection-\u003esocketpair_socket != 0) {\n                response res = { 0 };\n                if (recv(connection-\u003esocketpair_socket, (char *)\u0026res, sizeof(res), 0) == sizeof(res)) {\n                    mg_http_reply(connection, res.httpCode, res.headers, res.data);\n                    close(connection-\u003esocketpair_socket);\n                    connection-\u003esocketpair_socket = 0;\n                    free(res.data);\n                    free(res.headers);\n                }\n            }\n        }\n    );\n\n    server.addHandler(\"GET\", \"/calc\",\n        [](struct mg_connection *connection, int ev, mg_http_message *ev_data, void *fn_data) {\n            char buf[10];\n            int len = mg_http_get_var(\u0026ev_data-\u003equery, \"value\", buf, 10);\n            double n = atof(buf);\n            int blocking = -1, non_blocking = -1;\n\n            mg_socketpair(\u0026blocking, \u0026non_blocking);\n            static calc_param param;\n            param.val = n;\n            param.socket = blocking;\n            connection-\u003esocketpair_socket = non_blocking;\n            threadPool.addJob(job(handleCalc, (void *)\u0026param));\n        }\n    );\n\n    threadPool.init(8);\n    server.startServer(\"localhost:8000\", 50, NULL);\n    threadPool.shutdown();\n\n    return 0;\n}\n```\nThe program uses socket pair technique (which comes together with Mongoose). When the main thread receives request, it will create a socket pair, pass the job parameters and the socket to the thread, and return. When the job is finished, the thread will store the response in a static variable (notice that the thread function `handleCalc` is static), send the address to the response data to the paried socket and return. In each poll event, the paired socket will check if there are finished requests and respond them to the client. The requests will be `GET` requests with a parameter `value`. The value `x` will be a number between -1 and 1. The calculation task is to calculate ![image](https://user-images.githubusercontent.com/47358542/117383688-5431ed00-aeaf-11eb-9776-6aab18ee0f68.png) , which will converge to `ln(x+1)`. With a quick decompilation, we can see the compiler didn't optimize the calculation:\n\n![image](https://user-images.githubusercontent.com/47358542/117399840-6a4fa580-aecf-11eb-880c-b30331bcec4f.png)\n\nSample request URL: `http://localhost:8000/calc?value=0.605690`\n\nSample response: `0.473554`\n\nIn Apache JMeter, I used a thread group with 8 threads and infinite loops to request `localhost:8000/calc`. I added a random variable to generate random values for the `value` parameter for every request. The test has ran for 10 seconds and the result is quite disappointing:\n\n![image](https://user-images.githubusercontent.com/47358542/117394838-e85a7f00-aec4-11eb-898f-36f31709638b.png)\n\nThe server handled 94 requests in a 10 seconds which is about 9 requests per second.\n\nWhy is it so slow? I found out the program didn't utilize the full CPU resource - the CPU utlization is only about 12%. This is unexpected because the use of thread pool is meant to utilize all CPU resource. I am not sure why this happens and will investigate on it.\n\n## Not that Elegant\n- Doesn't support URL regex match - since this program directly maps URL string to handler functions\n\n- Unfriendly combination of thread pool and the server. As you can see, the above example of thread pool is not elegant. The user has to write a lot of additional code to make the thread pool function together with the server. This should be improved.\n\n- A lot of code is written in C style without the use of C++ features. For example, the thread pool library can be more elegant. This is my first time implementing a thread pool, so don't be too hard on me :D\n\n- Poor encapsulation of the server. A lot of function is not encapsulated the RESTserver class, resulted in the user have to call a lot of functions directly.\n\n## Bad Performance on Windows Compared to Linux\n\nWhen the same tests are carried out on Windows, the speed is much worse. The Hello World example reached about 6000reqs/sec, the JSON example reached about 3500reqs/sec. I believe this is due to the operating system difference in the underlying syscalls like socket, memory allocation, etc.\n\nHowever, RAM usage remains good on Windows. These examples are able to maintain a stable RAM usage on Windows and seems doesn't have memory leaks.\n\n## May Not be Capable as a Webpage Server\n\nThis program is intended to be used to implement an RESTful server to handle POST/GET requests. You may be able to serve simple webpages using this program, but I am not sure if it will work well.\n\n# Conclusion\n\nThis server is expected to perform well in normal tasks (e.g. JSON parsing and serializing, database operations). It is also capable to handle long operations (e.g. operations that take 3 seconds) by putting them into the thread pool. However, if there's a lot of long operations (especially when they come with high-concurrency), this program could be overwhelmed and probably crash (rare but happened). It is worth to mention that, with the use of thread pool, this server can handle other requests while processing long requests. In other words, one request will not block other requests.\n\n# Hopefully...\n\n- I will perform more tests and make improvements (like exceptions handling) to make sure this server is robust\n\n- I will try to improve the thread pool, and probably introduce a priority queue to prioritize jobs\n\n- I will try to combine this server with database operations to see if it can perform well\n\n- I will try to improve the flexibility of the RESTserver class (e.g. URL regex support)\n\n- I will try to improve the encapsulation of the RESTserver class (e.g. have functions to use socket pair, extraction of request arguments, extraction of request body, etc.)\n\n# Thanks\n- Cesanta Mongoose, the heart and soul of this project! https://github.com/cesanta/mongoose\n- nlohmann's json library for C++, provides fast and memory-efficient JSON functionalities: https://github.com/nlohmann/json\n- John's Blog - Thread pool in C, helped me implement the thread pool: https://nachtimwald.com/2019/04/12/thread-pool-in-c/\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsweeticelolly%2Fmultithreaded-http-server","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsweeticelolly%2Fmultithreaded-http-server","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsweeticelolly%2Fmultithreaded-http-server/lists"}