{"id":19289825,"url":"https://github.com/brefphp/benchmarks","last_synced_at":"2025-04-22T05:31:48.792Z","repository":{"id":56175983,"uuid":"134237151","full_name":"brefphp/benchmarks","owner":"brefphp","description":"Benchmarks for Bref running on AWS Lambda","archived":false,"fork":false,"pushed_at":"2024-05-02T13:39:36.000Z","size":433,"stargazers_count":19,"open_issues_count":0,"forks_count":3,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-01T20:00:16.962Z","etag":null,"topics":["aws-lambda","benchmark","bref","lambda","php","serverless"],"latest_commit_sha":null,"homepage":"https://github.com/brefphp/bref","language":"PHP","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/brefphp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-05-21T07:56:50.000Z","updated_at":"2024-05-02T13:39:36.000Z","dependencies_parsed_at":"2023-02-17T04:16:10.778Z","dependency_job_id":null,"html_url":"https://github.com/brefphp/benchmarks","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brefphp%2Fbenchmarks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brefphp%2Fbenchmarks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brefphp%2Fbenchmarks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brefphp%2Fbenchmarks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/brefphp","download_url":"https://codeload.github.com/brefphp/benchmarks/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250175068,"owners_count":21387132,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws-lambda","benchmark","bref","lambda","php","serverless"],"created_at":"2024-11-09T22:17:18.523Z","updated_at":"2025-04-22T05:31:48.517Z","avatar_url":"https://github.com/brefphp.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"Benchmarks for [Bref](https://github.com/brefphp/bref) running on AWS Lambda.\n\n## Scenarios\n\n- PHP function: a simple PHP function, see `function/index.php`\n- HTTP application: a simple PHP web page that returns `Hello world`, see `fpm/index.php`\n\n## Median (warm) execution time\n\nAverage execution time for a lambda that doesn't do anything.\n\nNumber of samples: 900\n\n### Bref 2.x (PHP 8.3)\n\n| Memory                       |   128 |  512 | 1024 | 1769 |\n|------------------------------|------:|-----:|-----:|-----:|\n| PHP function                 | 135ms | 27ms | 14ms | 14ms |\n| PHP function (BREF_LOOP_MAX) |       |      |  1ms |  1ms |\n| HTTP application             |   1ms |  1ms |  1ms |  1ms |\n| Laravel                      |       |      |  7ms |      |\n\n### Bref 2.x ARM (PHP 8.3)\n\n| Memory                       |   128 |  512 | 1024 | 1769 |\n|------------------------------|------:|-----:|-----:|-----:|\n| PHP function                 | 135ms | 30ms | 14ms | 14ms |\n| PHP function (BREF_LOOP_MAX) |       |      |  1ms |  1ms |\n| HTTP application             |   1ms |  1ms |  1ms |  1ms |\n| Laravel                      |       |      |  8ms |      |\n\nFor comparison on a 512M Digital Ocean droplet we get 1ms for \"HTTP application\" and 6ms for Symfony.\n\n## CPU performance\n\nThe more RAM, the more CPU power is allocated to the lambda. This is clearly visible when running [PHP's official `bench.php` script](https://github.com/php/php-src/blob/master/Zend/bench.php).\n\n| Memory      |  128 |  512 |  1024 |  1769 |\n|-------------|-----:|-----:|------:|------:|\n| `bench.php` | 5.7s | 1.4s | 0.65s | 0.33s |\n\nFor comparison  `bench.php` runs in 1.3s on a 512M Digital Ocean droplet, in 0.8s on a 2.8Ghz i7 and in 0.6s on a 3.2Ghz i5.\n\n## Cold starts\n\nNumber of samples: 20\n\nHigh level learnings:\n\n- Don't use less than 512M of memory\n- Cold starts are not faster on ARM, they are same or slightly slower\n- Containers have the fastest cold starts\n- When using containers, you need to warm up the first requests following a deployment\n\nComparison (1024M):\n\n|                    | x86 container | x86 layer | ARM layer |\n|--------------------|--------------:|----------:|----------:|\n| HTTP (duration)    |         180ms |     300ms |     300ms |\n| HTTP (latency)     |         350ms |     540ms |     530ms |\n| Laravel (duration) |         850ms |    1190ms |    1160ms |\n| Function           |         160ms |     230ms |     220ms |\n\n### Bref 2.x containers (PHP 8.3)\n\nNote: the first cold starts are much slower (see https://aaronstuyvenberg.com/posts/containers-on-lambda), e.g. 5s, because the container cache is warming up. In production warm up Lambda after deploying.\n\nFunction duration:\n\n| Memory           |  1024 |\n|------------------|------:|\n| PHP function     | 160ms |\n| HTTP application | 180ms |\n| Laravel          | 850ms |\n\nTotal latency (measured from API Gateway or X-Ray):\n\n| Memory           |  1024 |\n|------------------|------:|\n| PHP function     |       |\n| HTTP application | 350ms |\n| Laravel          |       |\n\n### Bref 2.x (PHP 8.3)\n\nFunction duration:\n\n| Memory           |   128 |   512 |   1024 |  1769 |\n|------------------|------:|------:|-------:|------:|\n| PHP function     | 500ms | 260ms |  230ms | 230ms |\n| HTTP application | 430ms | 330ms |  300ms | 310ms |\n| Laravel          |       |       | 1190ms |       |\n\nTotal latency (measured from API Gateway or X-Ray):\n\n| Memory           |   128 |   512 |   1024 |  1769 |\n|------------------|------:|------:|-------:|------:|\n| PHP function     | 700ms | 430ms |  440ms | 410ms |\n| HTTP application | 580ms | 450ms |  460ms | 450ms |\n| Laravel          |       |       | 1400ms |       |\n\n### Bref 2.x ARM (PHP 8.3)\n\nFunction duration:\n\n| Memory           |   128 |   512 |   1024 |  1769 |\n|------------------|------:|------:|-------:|------:|\n| PHP function     | 450ms | 240ms |  220ms | 210ms |\n| HTTP application | 410ms | 310ms |  300ms | 300ms |\n| Laravel          |       |       | 1160ms |       |\n\nTotal latency (measured from API Gateway or X-Ray):\n\n| Memory           |   128 |   512 |   1024 |  1769 |\n|------------------|------:|------:|-------:|------:|\n| PHP function     | 670ms | 470ms |  430ms | 430ms |\n| HTTP application | 620ms | 510ms |  540ms | 510ms |\n| Laravel          |       |       | 1460ms |       |\n\nMeasuring cold starts in CloudWatch Logs Insights:\n\n```\nfilter @type = “REPORT” and @initDuration\n| stats\n count(@type) as count,\n min(@billedDuration) as min,\n avg(@billedDuration) as avg,\n pct(@billedDuration, 50) as p50,\n max(@billedDuration) as max\nby @log\n| sort @log\n```\n\n## Reproducing\n\nYou will need [to install dependencies of Bref](https://bref.sh/docs/installation.html). Then:\n\n- clone the repository\n- `make setup`\n- `make deploy`\n\nThen run the `make bench-*` commands.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrefphp%2Fbenchmarks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbrefphp%2Fbenchmarks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrefphp%2Fbenchmarks/lists"}