Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/dvassallo/s3-benchmark
Measure Amazon S3's performance from any location.
https://github.com/dvassallo/s3-benchmark
aws benchmark performance-analysis s3
Last synced: 5 days ago
JSON representation
Measure Amazon S3's performance from any location.
- Host: GitHub
- URL: https://github.com/dvassallo/s3-benchmark
- Owner: dvassallo
- License: mit
- Created: 2019-04-02T19:07:55.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2024-01-19T16:53:48.000Z (about 1 year ago)
- Last Synced: 2025-01-10T03:27:24.774Z (12 days ago)
- Topics: aws, benchmark, performance-analysis, s3
- Language: Go
- Homepage:
- Size: 80.5 MB
- Stars: 833
- Watchers: 23
- Forks: 135
- Open Issues: 19
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-list - s3-benchmark
README
# S3 Benchmark
Your [Amazon S3](https://aws.amazon.com/s3/) performance depends on 3 things:
1. Your distance to the S3 endpoint.
2. The size of your objects.
3. The number of parallel transfers you can make.
With this tool you can measure S3's performance from any location, using different thread counts and object sizes. Here's an example from a c5.4xlarge EC2 instance:
## Usage
### Download
#### macOS
```
curl -OL https://github.com/dvassallo/s3-benchmark/raw/master/build/darwin-amd64/s3-benchmark
```
#### Linux 64-bit x86
```
curl -OL https://github.com/dvassallo/s3-benchmark/raw/master/build/linux-amd64/s3-benchmark
```
#### Linux 64-bit ARM
```
curl -OL https://github.com/dvassallo/s3-benchmark/raw/master/build/linux-arm64/s3-benchmark
```
### Credentials
This tool needs AWS credentials with full S3 permissions. If you run this on EC2, it will automatically use your EC2 instance profile. Otherwise it will try to find the credentials [from the usual places](https://aws.amazon.com/blogs/security/a-new-and-standardized-way-to-manage-credentials-in-the-aws-sdks/).
### Run
Make the file executable:
```
chmod +x s3-benchmark
```
Run a quick test (takes a few minutes):
```
./s3-benchmark
```
Or run the full test (takes a few hours):
```
./s3-benchmark -full
```
See [this](https://github.com/dvassallo/s3-benchmark/blob/master/main.go#L123-L134) for all the other options.
### Build
1. Install [Go](https://golang.org/)
```
sudo apt-get install golang-go
```
or
```
sudo yum install go
```
may work too.
2. Setup Go environment variables (Usually GOPATH and GOBIN) and test Go installation
3. Clone the repo
4. Install [```dep```](https://golang.github.io/dep/)
```
go get -u github.com/golang/dep/cmd/dep
```
5. Go to source directory and run ```dep ensure```
6. Run ```go run main.go```
## S3 to EC2 Bandwidth
I ran this benchmark on all current generation EC2 instance types as of 2019-04-02. I put the data in an interactive spreadsheet that you can [download here](/Analysis%20of%20S3%20Performance%20from%20EC2.xlsx?raw=true). Feel free to share it with your friends and colleagues.
### Analysis
Here's the maximum download throughput I got from all 155 instance types:
| EC2 Instance Type | Max S3 Throughput MB/s |
| :--- | ---: |
| c5n.18xlarge | 8,003 |
| p3dn.24xlarge | 6,269 |
| c5n.9xlarge | 5,741 |
| c5n.2xlarge | 2,861 |
| c5n.4xlarge | 2,851 |
| r5.metal | 2,718 |
| z1d.12xlarge | 2,718 |
| r5d.24xlarge | 2,718 |
| i3.metal | 2,716 |
| r5.24xlarge | 2,714 |
| m5d.24xlarge | 2,714 |
| r4.16xlarge | 2,714 |
| h1.16xlarge | 2,713 |
| m5.metal | 2,713 |
| x1.32xlarge | 2,711 |
| c5d.18xlarge | 2,710 |
| c5.18xlarge | 2,710 |
| p3.16xlarge | 2,710 |
| x1e.32xlarge | 2,709 |
| m5.24xlarge | 2,709 |
| z1d.metal | 2,708 |
| i3.16xlarge | 2,707 |
| f1.16xlarge | 2,706 |
| m4.16xlarge | 2,705 |
| g3.16xlarge | 2,705 |
| p2.16xlarge | 2,702 |
| m5d.metal | 2,673 |
| m5a.24xlarge | 1,801 |
| m5ad.24xlarge | 1,706 |
| r5ad.24xlarge | 1,699 |
| r5a.24xlarge | 1,562 |
| c5n.xlarge | 1,543 |
| x1.16xlarge | 1,410 |
| p2.8xlarge | 1,409 |
| g3.8xlarge | 1,400 |
| x1e.16xlarge | 1,400 |
| r4.8xlarge | 1,400 |
| i3.8xlarge | 1,400 |
| p3.8xlarge | 1,400 |
| h1.8xlarge | 1,399 |
| c5.9xlarge | 1,387 |
| r5.12xlarge | 1,387 |
| z1d.6xlarge | 1,387 |
| m5.12xlarge | 1,387 |
| m5d.12xlarge | 1,387 |
| c5d.9xlarge | 1,387 |
| r5d.12xlarge | 1,386 |
| g3.4xlarge | 1,163 |
| r4.4xlarge | 1,163 |
| f1.4xlarge | 1,163 |
| i3.4xlarge | 1,162 |
| x1e.8xlarge | 1,162 |
| h1.4xlarge | 1,161 |
| h1.2xlarge | 1,161 |
| x1e.4xlarge | 1,160 |
| m5.4xlarge | 1,157 |
| r5a.4xlarge | 1,156 |
| r5.4xlarge | 1,156 |
| r5d.4xlarge | 1,156 |
| m5a.12xlarge | 1,156 |
| m5d.4xlarge | 1,156 |
| m5ad.4xlarge | 1,156 |
| c5d.4xlarge | 1,156 |
| r5ad.12xlarge | 1,156 |
| c5.4xlarge | 1,156 |
| m5ad.12xlarge | 1,156 |
| r5a.12xlarge | 1,156 |
| m5a.4xlarge | 1,155 |
| r5ad.4xlarge | 1,155 |
| z1d.3xlarge | 1,155 |
| a1.4xlarge | 1,153 |
| i3.2xlarge | 1,143 |
| p3.2xlarge | 1,142 |
| x1e.2xlarge | 1,138 |
| f1.2xlarge | 1,137 |
| r5ad.2xlarge | 1,136 |
| m5d.2xlarge | 1,136 |
| r4.2xlarge | 1,135 |
| r5d.2xlarge | 1,134 |
| m5.2xlarge | 1,133 |
| z1d.2xlarge | 1,132 |
| m5ad.xlarge | 1,132 |
| c5d.2xlarge | 1,132 |
| m5a.2xlarge | 1,132 |
| m5ad.2xlarge | 1,131 |
| r5a.2xlarge | 1,131 |
| c5.2xlarge | 1,131 |
| r5ad.xlarge | 1,130 |
| r5.2xlarge | 1,129 |
| r4.xlarge | 1,127 |
| m5a.xlarge | 1,125 |
| g3s.xlarge | 1,124 |
| r5a.xlarge | 1,123 |
| i3.xlarge | 1,119 |
| z1d.xlarge | 1,116 |
| m5.xlarge | 1,114 |
| c5.xlarge | 1,114 |
| m5d.xlarge | 1,114 |
| r5.xlarge | 1,114 |
| r5d.xlarge | 1,113 |
| c5d.xlarge | 1,113 |
| i2.8xlarge | 1,092 |
| d2.8xlarge | 1,066 |
| c4.8xlarge | 1,066 |
| m4.10xlarge | 1,066 |
| z1d.large | 1,002 |
| x1e.xlarge | 980 |
| c5.large | 949 |
| a1.2xlarge | 944 |
| c5d.large | 942 |
| r5d.large | 936 |
| m5d.large | 891 |
| m5.large | 873 |
| c5n.large | 851 |
| r5.large | 846 |
| r5ad.large | 783 |
| m5ad.large | 762 |
| r5a.large | 740 |
| a1.xlarge | 737 |
| m5a.large | 726 |
| i3.large | 624 |
| t3.2xlarge | 569 |
| t3.xlarge | 568 |
| t3.medium | 558 |
| t3.large | 553 |
| d2.4xlarge | 544 |
| c4.4xlarge | 544 |
| r4.large | 541 |
| a1.large | 514 |
| t3.small | 395 |
| t3.micro | 349 |
| t3.nano | 319 |
| c4.2xlarge | 272 |
| d2.2xlarge | 272 |
| m4.4xlarge | 246 |
| i2.4xlarge | 244 |
| g2.8xlarge | 237 |
| a1.medium | 169 |
| p2.xlarge | 154 |
| t2.nano | 118 |
| m4.2xlarge | 118 |
| i2.2xlarge | 118 |
| g2.2xlarge | 118 |
| t2.2xlarge | 116 |
| t2.xlarge | 113 |
| t2.large | 109 |
| t2.medium | 108 |
| c4.xlarge | 102 |
| d2.xlarge | 102 |
| m4.xlarge | 87 |
| i2.xlarge | 80 |
| c4.large | 71 |
| m4.large | 53 |
| t2.micro | 46 |
| t2.small | 39 |
Here's the performance of all instances with 32 MB objects (the legend is truncated, but all instances are plotted):
And here's the same chart with just the 3 outlier instances that have 50 Gigabit or more network bandwidth:
Here's a typical throughput profile showing how object size affects performance:
S3's 90th percentile time to first byte is typically around 20 ms regardless of the object size. However, small instances start to see elevated latencies with increased parallelization due to their limited resources. Here's the p90 first byte latency on a small instance:
And here's the p90 first byte latency on a larger instance:
Unlike the first byte latency, the time to last byte obviously follows the object size. S3 seems to deliver downloads at a rate of about 93 MB/s per thread, so this latency is a function of that and the first byte latency — at least until the network bandwidth gets saturated. Here's one example:
If you want to analyze the data further, you can [download the spreadsheet](/Analysis%20of%20S3%20Performance%20from%20EC2.xlsx?raw=true) with all the raw data and the interactive features shown in the screenshots. Here's a [45 second demo](https://www.youtube.com/watch?v=f5CacEcWC9s) of what you can do with it.
## License
This project is released under the [MIT License](LICENSE).