{"id":14110056,"url":"https://github.com/penso/test-s3-direct-upload","last_synced_at":"2026-04-09T02:32:37.531Z","repository":{"id":43888347,"uuid":"450477016","full_name":"penso/test-s3-direct-upload","owner":"penso","description":"Benchmark for S3 direct uploads for AWS, GCP and Scaleway","archived":false,"fork":false,"pushed_at":"2022-02-14T13:51:45.000Z","size":52,"stargazers_count":7,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-08-14T10:15:42.946Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/penso.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-01-21T12:08:12.000Z","updated_at":"2023-11-30T02:06:09.000Z","dependencies_parsed_at":"2022-08-25T11:31:30.287Z","dependency_job_id":null,"html_url":"https://github.com/penso/test-s3-direct-upload","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/penso%2Ftest-s3-direct-upload","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/penso%2Ftest-s3-direct-upload/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/penso%2Ftest-s3-direct-upload/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/penso%2Ftest-s3-direct-upload/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/penso","download_url":"https://codeload.github.com/penso/test-s3-direct-upload/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228360571,"owners_count":17907939,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-14T10:02:38.375Z","updated_at":"2026-04-09T02:32:37.223Z","avatar_url":"https://github.com/penso.png","language":"Ruby","funding_links":[],"categories":["Ruby"],"sub_categories":[],"readme":"\t\t                 \nThis is a small script to benchmark different S3 providers\n\n1. Run `make create_files`\n2. Copy `storage-example.yml` to `storage.yml` and fill it up\n4. Run `make run`\n\nIf you run those on a specific ISP, send a Pull Requests with results\n\n* AWS: fastest and very consistent\n* DigitalOcean: impressive, almost as fast as AWS\n* Google: slightly slower than S3\n* Scaleway: much slower, and massive standard deviation\n\n\t\t+------------------------------------------------------------------------------+\n\t\t|                       S3 benchmarks on multi provider                        |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| Provider             |      Median |         Min |         Max |   Deviation |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| Testing with file size: 0B                                                   |\n\t\t+------------------------------------------------------------------------------+\n\t\t| Direct Upload                                                                |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| aws eu-west-3        |          50 |          38 |          68 |           8 |\n\t\t| backblaze            |         450 |         351 |         690 |          96 |\n\t\t| digitalocean ams3    |         269 |          73 |         289 |          88 |\n\t\t| digitalocean fra1    |          75 |          68 |          86 |           6 |\n\t\t| google europe-west-1 |          83 |          75 |         109 |          10 |\n\t\t| scaleway ams         |         410 |         166 |         443 |          80 |\n\t\t| scaleway par         |         355 |         161 |         508 |         111 |\n\t\t| scaleway war         |         448 |         378 |         505 |          42 |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| Direct Download                                                              |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| aws eu-west-3        |          45 |          35 |          54 |           6 |\n\t\t| backblaze            |         317 |         300 |         389 |          29 |\n\t\t| digitalocean ams3    |         333 |          67 |        1627 |         529 |\n\t\t| digitalocean fra1    |          58 |          49 |          65 |           5 |\n\t\t| google europe-west-1 |          66 |          49 |          92 |          11 |\n\t\t| scaleway ams         |         317 |         144 |         418 |          92 |\n\t\t| scaleway par         |         184 |         115 |         211 |          39 |\n\t\t| scaleway war         |         330 |         301 |         390 |          33 |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t\n\t\t+------------------------------------------------------------------------------+\n\t\t| Testing with file size: 1KiB                                                 |\n\t\t+------------------------------------------------------------------------------+\n\t\t| Direct Upload                                                                |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| aws eu-west-3        |          65 |          52 |         314 |          80 |\n\t\t| backblaze            |         429 |         320 |         491 |          58 |\n\t\t| digitalocean ams3    |         124 |          78 |        1675 |         513 |\n\t\t| digitalocean fra1    |          72 |          66 |          87 |           7 |\n\t\t| google europe-west-1 |          98 |          72 |         287 |          63 |\n\t\t| scaleway ams         |         480 |         218 |         640 |         151 |\n\t\t| scaleway par         |         352 |         180 |         501 |          96 |\n\t\t| scaleway war         |         395 |         380 |         418 |          13 |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| Direct Download                                                              |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| aws eu-west-3        |          47 |          31 |          60 |           8 |\n\t\t| backblaze            |         318 |         311 |         387 |          23 |\n\t\t| digitalocean ams3    |         251 |          75 |         679 |         181 |\n\t\t| digitalocean fra1    |          56 |          51 |          69 |           5 |\n\t\t| google europe-west-1 |          62 |          45 |          88 |          13 |\n\t\t| scaleway ams         |         265 |         122 |         442 |          95 |\n\t\t| scaleway par         |         159 |         110 |         236 |          47 |\n\t\t| scaleway war         |         314 |         305 |         384 |          24 |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t\n\t\t+------------------------------------------------------------------------------+\n\t\t| Testing with file size: 10KiB                                                |\n\t\t+------------------------------------------------------------------------------+\n\t\t| Direct Upload                                                                |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| aws eu-west-3        |          77 |          57 |         111 |          18 |\n\t\t| backblaze            |         422 |         335 |         747 |         157 |\n\t\t| digitalocean ams3    |         597 |         255 |        1654 |         424 |\n\t\t| digitalocean fra1    |          75 |          63 |         102 |          13 |\n\t\t| google europe-west-1 |          89 |          65 |         137 |          23 |\n\t\t| scaleway ams         |         566 |         294 |         781 |         175 |\n\t\t| scaleway par         |         346 |         219 |         688 |         127 |\n\t\t| scaleway war         |         439 |         375 |         473 |          36 |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| Direct Download                                                              |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| aws eu-west-3        |          49 |          39 |          91 |          15 |\n\t\t| backblaze            |         328 |         319 |         364 |          15 |\n\t\t| digitalocean ams3    |         262 |          69 |         782 |         263 |\n\t\t| digitalocean fra1    |          55 |          48 |          66 |           5 |\n\t\t| google europe-west-1 |          65 |          48 |         107 |          17 |\n\t\t| scaleway ams         |         312 |         156 |         471 |         110 |\n\t\t| scaleway par         |         161 |         103 |         234 |          40 |\n\t\t| scaleway war         |         311 |         294 |         378 |          30 |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t\n\t\t+------------------------------------------------------------------------------+\n\t\t| Testing with file size: 100KiB                                               |\n\t\t+------------------------------------------------------------------------------+\n\t\t| Direct Upload                                                                |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| aws eu-west-3        |          79 |          66 |         143 |          29 |\n\t\t| backblaze            |         671 |         610 |        1330 |         212 |\n\t\t| digitalocean ams3    |         475 |         106 |        1526 |         477 |\n\t\t| digitalocean fra1    |         108 |         100 |         119 |           6 |\n\t\t| google europe-west-1 |         119 |         100 |         146 |          16 |\n\t\t| scaleway ams         |         568 |         324 |         856 |         152 |\n\t\t| scaleway par         |         309 |         257 |        1056 |         246 |\n\t\t| scaleway war         |         392 |         382 |         427 |          16 |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| Direct Download                                                              |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| aws eu-west-3        |          47 |          42 |          76 |          11 |\n\t\t| backblaze            |         576 |         569 |         654 |          26 |\n\t\t| digitalocean ams3    |         761 |         210 |        4983 |        1536 |\n\t\t| digitalocean fra1    |         101 |          89 |         220 |          49 |\n\t\t| google europe-west-1 |          97 |          68 |         172 |          42 |\n\t\t| scaleway ams         |         325 |         198 |         412 |          71 |\n\t\t| scaleway par         |         184 |         137 |         415 |          86 |\n\t\t| scaleway war         |         429 |         400 |         489 |          34 |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t\n\t\t+------------------------------------------------------------------------------+\n\t\t| Testing with file size: 1MiB                                                 |\n\t\t+------------------------------------------------------------------------------+\n\t\t| Direct Upload                                                                |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| aws eu-west-3        |         220 |         197 |         271 |          26 |\n\t\t| backblaze            |         941 |         890 |        1100 |          72 |\n\t\t| digitalocean ams3    |         806 |         260 |        2094 |         698 |\n\t\t| digitalocean fra1    |         265 |         186 |         322 |          52 |\n\t\t| google europe-west-1 |         175 |         168 |         200 |          14 |\n\t\t| scaleway ams         |         594 |         368 |         793 |         141 |\n\t\t| scaleway par         |         444 |         321 |         645 |          95 |\n\t\t| scaleway war         |         455 |         429 |         515 |          26 |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| Direct Download                                                              |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| aws eu-west-3        |         100 |          69 |         131 |          19 |\n\t\t| backblaze            |         892 |         873 |         938 |          21 |\n\t\t| digitalocean ams3    |         183 |         147 |       33182 |       10418 |\n\t\t| digitalocean fra1    |         122 |         117 |         237 |          42 |\n\t\t| google europe-west-1 |          86 |          73 |         112 |          14 |\n\t\t| scaleway ams         |         422 |         262 |         542 |          88 |\n\t\t| scaleway par         |         196 |         155 |         254 |          29 |\n\t\t| scaleway war         |         573 |         518 |         606 |          28 |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t\n\t\t+------------------------------------------------------------------------------+\n\t\t| Testing with file size: 10MiB                                                |\n\t\t+------------------------------------------------------------------------------+\n\t\t| Direct Upload                                                                |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| aws eu-west-3        |         506 |         458 |         539 |          25 |\n\t\t| backblaze            |        1778 |        1505 |        3061 |         467 |\n\t\t| digitalocean ams3    |         932 |         606 |        3554 |         922 |\n\t\t| digitalocean fra1    |         492 |         445 |        1506 |         330 |\n\t\t| google europe-west-1 |         821 |         416 |        1018 |         234 |\n\t\t| scaleway ams         |        1687 |         976 |        1811 |         282 |\n\t\t| scaleway par         |         734 |         502 |        1704 |         444 |\n\t\t| scaleway war         |         715 |         701 |         784 |          27 |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| Direct Download                                                              |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| aws eu-west-3        |         203 |         185 |         218 |          10 |\n\t\t| backblaze            |        1296 |        1275 |        1792 |         158 |\n\t\t| digitalocean ams3    |       12373 |         168 |      344914 |      137730 |\n\t\t| digitalocean fra1    |         216 |         205 |         265 |          19 |\n\t\t| google europe-west-1 |         173 |         156 |         203 |          14 |\n\t\t| scaleway ams         |         523 |         351 |         556 |          74 |\n\t\t| scaleway par         |         352 |         289 |         605 |          90 |\n\t\t| scaleway war         |         710 |         660 |         797 |          43 |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t\n\t\t+------------------------------------------------------------------------------+\n\t\t| Testing with file size: 100MiB                                               |\n\t\t+------------------------------------------------------------------------------+\n\t\t| Direct Upload                                                                |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| aws eu-west-3        |        1857 |        1644 |        2600 |         300 |\n\t\t| backblaze            |        4786 |        4461 |       13149 |        2691 |\n\t\t| digitalocean ams3    |        3985 |        1937 |       15225 |        5123 |\n\t\t| digitalocean fra1    |        1989 |        1773 |        3291 |         486 |\n\t\t| google europe-west-1 |        1777 |        1740 |        2723 |         386 |\n\t\t| scaleway ams         |        5338 |        1952 |        7262 |        2008 |\n\t\t| scaleway par         |        1944 |        1788 |        2683 |         348 |\n\t\t| scaleway war         |        2301 |        2095 |        2431 |         101 |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| Direct Download                                                              |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n\t\t| aws eu-west-3        |        1101 |        1066 |        1489 |         179 |\n\t\t| backblaze            |        3923 |        3773 |        4132 |         121 |\n\t\t| digitalocean ams3    |         331 |         171 |     2506343 |      792437 |\n\t\t| digitalocean fra1    |        1105 |        1035 |        1236 |          61 |\n\t\t| google europe-west-1 |        1023 |        1007 |        1058 |          15 |\n\t\t| scaleway ams         |        1368 |        1215 |        1603 |         101 |\n\t\t| scaleway par         |        1238 |        1139 |        2221 |         381 |\n\t\t| scaleway war         |        2004 |        1683 |        2477 |         241 |\n\t\t+----------------------+-------------+-------------+-------------+-------------+\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpenso%2Ftest-s3-direct-upload","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpenso%2Ftest-s3-direct-upload","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpenso%2Ftest-s3-direct-upload/lists"}