{"id":15352324,"url":"https://github.com/tmaret/fibers-server","last_synced_at":"2025-04-06T13:42:58.938Z","repository":{"id":49201243,"uuid":"252771310","full_name":"tmaret/fibers-server","owner":"tmaret","description":"Server based on Jetty to evaluate Java virtual threads (fibers) provided by Early-Access builds of OpenJDK project Loom.","archived":false,"fork":false,"pushed_at":"2021-06-23T21:00:23.000Z","size":3630,"stargazers_count":3,"open_issues_count":2,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-02-12T19:48:35.505Z","etag":null,"topics":["fibers","jetty","jetty-server","loom"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tmaret.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-04-03T15:30:13.000Z","updated_at":"2023-06-25T16:43:21.000Z","dependencies_parsed_at":"2022-09-22T01:11:49.909Z","dependency_job_id":null,"html_url":"https://github.com/tmaret/fibers-server","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmaret%2Ffibers-server","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmaret%2Ffibers-server/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmaret%2Ffibers-server/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmaret%2Ffibers-server/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tmaret","download_url":"https://codeload.github.com/tmaret/fibers-server/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247492464,"owners_count":20947541,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fibers","jetty","jetty-server","loom"],"created_at":"2024-10-01T12:09:14.850Z","updated_at":"2025-04-06T13:42:58.907Z","avatar_url":"https://github.com/tmaret.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# fibers-server\n\n`fibers-server` is an evaluation server based on [Jetty](https://www.eclipse.org/jetty) for virtual threads (fibers) provided by [Project Loom](https://wiki.openjdk.java.net/display/loom/Main)\nEarly-Access [builds](https://jdk.java.net/loom/). The server provides a sync and an async servlet handlers that can be configured to serve requests including more or less waiting (IO and idle) and service time (CPU). The server can be started with a queued or unbounded thread pool. Both pools can be backed by virtual or kernel threads.\n\n\n# Setup\n\nSetup Project Loom early-access build by following the [instructions](https://jdk.java.net/loom/).\n\n```\n$ java -version\nopenjdk version \"16-loom\" 2021-03-16\nOpenJDK Runtime Environment (build 16-loom+4-56)\nOpenJDK 64-Bit Server VM (build 16-loom+4-56, mixed mode, sharing)\n```\n\n# Build\n\n```\nmvn clean:install\n```\n\n# Run\n\nPrint help usage via\n\n```\ndocker run -p 8080:8080 -i tmaretdotio/fibers-server:0.0.2 -h\n```\n\nRun a server using kernel threads and unbounded pool\n\n```\ndocker run -p 8080:8080 -i tmaretdotio/fibers-server:0.0.2 -t kernel\n```\n\nRun the server using lightweight threads (fibers) and unbounded pool\n\n```\ndocker run -p 8080:8080 -i tmaretdotio/fibers-server:0.0.2 -t fibers\n```\n\nRun the server with bounded pool (400) and lightweight threads\n\n```\ndocker run -p 8080:8080 -i tmaretdotio/fibers-server:0.0.2 -t fibers -c 400\n```\n\n# Use\n\nRequest served by a sync servlet\n\n```\ncurl http://localhost:8080/sync\n```\n\nRequest served by an async servlet\n\n```\ncurl http://localhost:8080/async\n```\n\nSpecify the amount of work performed by the servlet before returning the response.\n\n```\ncurl 'http://localhost:8080/sync?cpuIterations=10000\u0026idleDelay=0\u0026fileLength=100000'\n```\n\n| Request Parameter | Definition |\n| :---------------: | ---------- |\n| `cpuIterations`   | The number of SHA256 iterations on a 256 byte array. Default is 10'000 iterations and corresponds to ~ 3ms on a 2.3 GHz Intel Core i9. |\n| `idleDelay`       | The time to sleep in ms. Default is 0ms. |\n| `fileLength`      | The length in bytes of the file returned. The file is auto-generated and served from disk. Default length is 100KB.|\n\n# Benchmark\n\nThe following table summarises the `fibers-server` throughput under load generated by [wrk2](https://github.com/giltene/wrk2).\n\nThroughputs values labeled `fibers` refer to the `fibers-server` running with an unbounded pool of virtual threads.\n\nThroughput values labeled `kernel` refer to the `fibers-server` running with a queued pool of 1000 kernel threads.\n\n\n## Experiment #1 — 95% waiting time local sync servlet\n\nWorkload consists of 80% sleep, 15% IO and 5% CPU. The raw results are [here](./benchmark/experiment-1).\n\nThe server and load generator are deployed as two containers inside a single Kubernetes Pod communicating via localhost on the shared network interface.\n\nThe load generator was allocated 8 cores and 16GB memory.\nThe server was allocated 4 cores and 3GB memory. :hand: The server %CPU topped 400% for higher concurrency and was thus limiting the throughput.\n\n### Throughput for 10ms requests\n\n|Concurrency (threads)|fibers req/s|kernel req/s|\n|---------------------|------------|------------|\n|100                  |3411        |2183        |\n|250                  |3272        |2150        |\n|500                  |3350        |2012        |\n|1000                 |3187        |1828        |\n|2000                 |3265        |1863        |\n|4000                 |3055        |1839        |\n\n![Throughput](./benchmark/experiment-1/graph/throughput-10ms.svg)\n\n\n### Throughput for 100ms requests\n\n|Concurrency (threads)|fibers req/s|kernel req/s|\n|---------------------|------------|------------|\n|100                  |330         |256         |\n|250                  |316         |248         |\n|500                  |334         |240         |\n|1000                 |323         |186         |\n|2000                 |322         |173         |\n|4000                 |325         |177         |\n\n![Throughput](./benchmark/experiment-1/graph/throughput-100ms.svg)\n\n### Throughput for 1s requests\n\nThe N/A values indicates the test could not complete.\n\n|Concurrency (threads)|fibers req/s|kernel req/s|\n|---------------------|------------|------------|\n|100                  |31          |24          |\n|250                  |30          |19          |\n|500                  |30          |13          |\n|1000                 |31          |N/A         |\n|2000                 |32          |N/A         |\n|4000                 |29          |N/A         |\n\n![Throughput](./benchmark/experiment-1/graph/throughput-1s.svg)\n\n## Experiment #2 — 99% waiting time local sync servlet\n\nWorkload consists of 95% sleep, 4% IO and 1% CPU. The raw results are [here](./benchmark/experiment-2).\n\nThe server and load generator are deployed as two containers inside a single Kubernetes Pod communicating via localhost on the shared network interface.\n\nThe load generator was allocated 8 cores and 16GB memory.\nThe server was allocated 4 cores and 3GB memory. :hand: The server %CPU topped 400% for higher concurrency and was thus limiting the throughput.\n\n### Throughput for 10ms requests\n\n|Concurrency (threads)|fibers req/s|kernel req/s|\n|---------------------|------------|------------|\n|100                  |9723        |9193        |\n|250                  |13735       |10047       |\n|500                  |13942       |9683        |\n|1000                 |13754       |9355        |\n|2000                 |14350       |10190       |\n|4000                 |12973       |10045       |\n\n\n![Throughput](./benchmark/experiment-2/graph/throughput-10ms.svg)\n\n### Throughput for 100ms requests\n\n|Concurrency (threads)|fibers req/s|kernel req/s|\n|---------------------|------------|------------|\n|100                  |1012        |1025        |\n|250                  |1925        |1850        |\n|500                  |1944        |1797        |\n|1000                 |1953        |1523        |\n|2000                 |1871        |1623        |\n|4000                 |1918        |1570        |\n\n\n![Throughput](./benchmark/experiment-2/graph/throughput-100ms.svg)\n\n### Throughput for 1s requests\n\n|Concurrency (threads)|fibers req/s|kernel req/s|\n|---------------------|------------|------------|\n|100                  |100         |99          |\n|250                  |206         |197         |\n|500                  |206         |221         |\n|1000                 |198         |175         |\n|2000                 |199         |173         |\n|4000                 |198         |169         |\n\n![Throughput](./benchmark/experiment-2/graph/throughput-1s.svg)\n\n## Experiment #3 — 99% waiting time remote sync servlet\n\nWorkload consists of 95% sleep, 4% IO and 1% CPU. The raw results are [here](./benchmark/experiment-3).\n\nThe server and load generator are deployed as two Kubernetes Pods communicating using Kubernetes [cluster routing](https://kubernetes.io/docs/concepts/cluster-administration/networking).\n\nThe load generator was allocated 8 cores and 16GB memory.\nThe server was allocated 4 cores and 3GB memory. :hand: The server %CPU topped 400% for higher concurrency and was thus limiting the throughput.\n\n### Throughput for 10ms requests\n\n|Concurrency (threads)|fibers req/s|kernel req/s|\n|---------------------|------------|------------|\n|100                  |9633        |9214        |\n|250                  |14181       |9444        |\n|500                  |14618       |9007        |\n|1000                 |13372       |8415        |\n|2000                 |13406       |9485        |\n|4000                 |12574       |9643        |\n\n\n![Throughput](./benchmark/experiment-3/graph/throughput-10ms.svg)\n\n\n### Throughput for 100ms requests\n\n|Concurrency (threads)|fibers req/s|kernel req/s|\n|---------------------|------------|------------|\n|100                  |1027        |1022        |\n|250                  |1980        |1663        |\n|500                  |1973        |1592        |\n|1000                 |1938        |1530        |\n|2000                 |1885        |1554        |\n|4000                 |2056        |1577        |\n\n![Throughput](./benchmark/experiment-3/graph/throughput-100ms.svg)\n\n### Throughput for 1s requests\n\n|Concurrency (threads)|fibers req/s|kernel req/s|\n|---------------------|------------|------------|\n|100                  |100         |97          |\n|250                  |202         |172         |\n|500                  |206         |176         |\n|1000                 |203         |156         |\n|2000                 |197         |151         |\n|4000                 |212         |168         |\n\n![Throughput](./benchmark/experiment-3/graph/throughput-1s.svg)\n\n### Throughput for 10s requests\n\nThe N/A values indicates the test could not complete.\n\n|Concurrency (threads)|fibers req/s|kernel req/s|\n|---------------------|------------|------------|\n|100                  |8           |4           |\n|250                  |12          |10          |\n|500                  |8           |0           |\n|1000                 |11          |N/A         |\n|2000                 |10          |N/A         |\n|4000                 |11          |N/A         |\n\n![Throughput](./benchmark/experiment-3/graph/throughput-10s.svg)\n\n## Experiment #4 — 99.5% waiting time remote sync servlet\n\nWorkload consists of 95.5% sleep, 4% IO and 0.5% CPU. The raw results are [here](./benchmark/experiment-4).\n\nThe server and load generator are deployed as two Kubernetes Pods communicating using Kubernetes [cluster routing](https://kubernetes.io/docs/concepts/cluster-administration/networking).\n\nThe load generator was allocated 8 cores and 16GB memory.\nThe server was allocated 8 cores and 3GB memory.\n\n:+1: Throughput was _not_ limited by server 8 cores. %CPU usage remained \u003c 800% during the test.\n\n### Throughput for 10ms requests\n\n|Concurrency (threads)|fibers req/s|kernel req/s|\n|---------------------|------------|------------|\n|100                  |10262       |10342       |\n|250                  |25468       |24383       |\n|500                  |32888       |24898       |\n|1000                 |34615       |24300       |\n|2000                 |34818       |25245       |\n|4000                 |35678       |24589       |\n|8000                 |36942       |29906       |\n\n\n![Throughput](./benchmark/experiment-4/graph/throughput-10ms.svg)\n\n\n### Throughput for 100ms requests\n\n|Concurrency (threads)|fibers req/s|kernel req/s|\n|---------------------|------------|------------|\n|100                  |1006        |1008        |\n|250                  |2523        |2515        |\n|500                  |4926        |4683        |\n|1000                 |5387        |4423        |\n|2000                 |5384        |4551        |\n|4000                 |5355        |4336        |\n|8000                 |6235        |4923        |\n\n\n\n![Throughput](./benchmark/experiment-4/graph/throughput-100ms.svg)\n\n### Throughput for 1s requests\n\n|Concurrency (threads)|fibers req/s|kernel req/s|\n|---------------------|------------|------------|\n|100                  |98          |97          |\n|250                  |239         |230         |\n|500                  |398         |315         |\n|1000                 |406         |270         |\n|2000                 |410         |257         |\n|4000                 |402         |285         |\n|8000                 |452         |325         |\n\n![Throughput](./benchmark/experiment-4/graph/throughput-1s.svg)\n\n## Monitoring resources\n\nUsing `top` while benchmarking the server with 4000 concurrency shows ~ 1000 kernel threads to support the `kernel` server mode\n\n![threads](./docs/top-kernel.png)\n\nand ~ 10 kernel threads to support the `fibers` server mode\n\n![threads](./docs/top-fibers.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftmaret%2Ffibers-server","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftmaret%2Ffibers-server","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftmaret%2Ffibers-server/lists"}