https://github.com/joeltadeu/web-frameworks-benchmark
https://github.com/joeltadeu/web-frameworks-benchmark
Last synced: 4 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/joeltadeu/web-frameworks-benchmark
- Owner: joeltadeu
- Created: 2023-07-10T18:57:45.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2023-07-27T00:00:59.000Z (almost 2 years ago)
- Last Synced: 2025-01-07T12:45:13.349Z (5 months ago)
- Language: Java
- Size: 8.1 MB
- Stars: 1
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Comparing Web Frameworks and Testing Performance
## Overview
The objective of this project was to compare the best known frameworks for java language such as Spring Boot, Micronaut and Quarkus, C# with .Net/API Core, Rust with Actix and Rocket
creating an api for querying books using different forms of persistence such as JDBC, Hibernate and Reactive, Entity Framework, Diesel.
It was also possible to carry out an extensive performance test using the Gatling tool simulating the simultaneous access of 100 to 500 users for 10 minutes.
Below is the list of created microservices:
| Microservice | Language | Framework | Persistence |
|------------------------------------------------------------------------|:---------|:-------------:|------------------|
| [ms-book-micronaut-reactive](ms-book-micronaut-reactive) | Java | Micronaut | Reactive |
| [ms-book-micronaut-rest-jdbc](ms-book-micronaut-rest-jdbc) | Java | Micronaut | JDBC |
| [ms-book-quarkus-reactive](ms-book-quarkus-reactive) | Java | Quarkus | Reactive |
| [ms-book-quarkus-rest-hibernate](ms-book-quarkus-rest-hibernate) | Java | Quarkus | Hibernate |
| [ms-book-quarkus-rest-jdbc](ms-book-quarkus-rest-jdbc) | Java | Quarkus | JDBC |
| [ms-book-springboot-reactive](ms-book-springboot-reactive) | Java | Spring Boot | Reactive |
| [ms-book-springboot-rest-hibernate](ms-book-springboot-rest-hibernate) | Java | Spring Boot | Hibernate |
| [ms-book-springboot-rest-jdbc](ms-book-springboot-rest-jdbc) | Java | Spring Boot | JDBC |
| [ms-book-dotnet-entity-framework](ms-book-dotnet-entity-framework) | C# | .Net/API Core | Entity Framework |
| [ms-book-rust-actix-diesel](ms-book-rust-actix-diesel) | Rust | Actix | Diesel |
| [ms-book-rust-rocket-diesel](ms-book-rust-rocket-diesel) | Rust | Rocket | Diesel |
Each project has its own docker-compose and can run independently.
Projects were also created to assist in the creation of the database, in the execution of performance tests and in the display of consolidated results.
Below is the list of projects and a brief description:
| Project | Description |
|--------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------|
| [book-database](book-database) | Project responsible for creating the database used by microservices |
| [book-deploy](book-deploy) | Project containing common files for prometheus and grafana configuration |
| [book-metabase](book-metabase) | Project responsible for creating the container with the metabase tool to display the comparative dashboard between the frameworks |
| [book-performance-tests](book-performance-tests) | Project responsible for running the performance tests using the Gatling tool |
## Comparing Analysis
### Compiled size
The .Net microservice with Entity Framework is the smallest compiled at 17.2 MB. The spring boot microservice with hibernate has the largest compiled with 43.66 MB, approximately 154% larger than the smallest.
### Container Size
The .Net microservice with Entity Framework has the smallest container size at 118 MB. The quarkus microservice with hibernate has the largest container with 444 MB, approximately 276% larger than the smallest.
### Used Memory - start container
In the graph below we have a comparison of the memory used when uploading the container of each microservice. .Net with Entity Framework has the lowest initial memory footprint at 66 MB. The spring boot microservice with hibernate had the highest memory consumption with 264.3 MB, approximately 300% higher than the lowest.
### Used Memory - 100 users in 10 min
In the graph below we have a comparison of memory usage simulating the access of 100 simultaneous users for 10 minutes. Spring boot with JDBC had the lowest memory footprint at 690 MB. The quarkus microservice with hibernate had the highest memory consumption with 4.63 GB, approximately 571% higher than the lowest consumption.
### Used Memory - 500 users in 10 min
In the graph below we have a comparison of memory usage simulating the access of 500 simultaneous users for 10 minutes. Quarkus with Reactive had the lowest memory consumption at 400 MB but that's because it wasn't able to respond to requests. Thus, we consider that the Micronaut Reactive microservice had the lowest memory consumption with 2.19 GB. The quarkus microservice with jdbc had the highest memory consumption with 4.62 GB, approximately 110% higher than the lowest consumption.
## Performance Tests
### Total of requests - 100 users in 10 min
In the graph below we have a comparison of the number of requests that each microservice was able to respond with 100 simultaneous users for 10 minutes. Spring-boot JDBC was able to respond to the most requests with a total of 12,566 while Micronaut Reactive was able to respond to the fewest requests with a total of 11,144. A difference of approximately 13%.
### Total of requests - 500 users in 10 min
In the graph below we have a comparison of the number of requests that each microservice was able to respond with 500 simultaneous users for 10 minutes. Spring-boot hibernate was able to respond to the most requests with a total of 57,868 while Quarkus Reactive was able to respond to the fewest requests with a total of 19,214. A difference of approximately 201%.
### Requests per second - 100 users in 10 min
In the graph below we have a comparison of the number of requests answered in one second simulating the access of 100 simultaneous users for 10 minutes. Spring-boot JDBC had the highest number of requests per second with 18.76 while micronaut reactive had the lowest with 16.61. A difference of approximately 13%.
### Requests per second - 500 users in 10 min
In the graph below we have a comparison of the number of requests answered in one second simulating the access of 500 simultaneous users for 10 minutes. Spring-boot hibernate had the highest number of requests per second with 86.24 while quarkus reactive had the lowest with 28.13. A difference of approximately 207%.
### Response time - 100 users in 10 min
In the graph below we have a comparison of the response time of each microservice simulating the access of 100 simultaneous users for 10 minutes. Spring-boot JDBC had the best average with 40 milliseconds while micronaut reactive had the worst average with 681 milliseconds. A difference of 1,603%.
### Response time - 500 users in 10 min
In the graph below we have a comparison of the response time of each microservice simulating the access of 500 simultaneous users for 10 minutes. Spring-boot Hibernate had the best average with 424 milliseconds while quarkus reactive had the worst average with 7,920 milliseconds. A difference of 1,768%.
