{"id":28089330,"url":"https://github.com/vpalmisano/webrtcperf","last_synced_at":"2025-05-13T12:56:45.177Z","repository":{"id":44137039,"uuid":"344202200","full_name":"vpalmisano/webrtcperf","owner":"vpalmisano","description":"WebRTC performance and quality evaluation tool.","archived":false,"fork":false,"pushed_at":"2025-05-02T13:35:12.000Z","size":4477,"stargazers_count":182,"open_issues_count":5,"forks_count":39,"subscribers_count":9,"default_branch":"devel","last_synced_at":"2025-05-02T14:57:29.359Z","etag":null,"topics":["chromium","concurrent-webrtc-sessions","edumeet","janus","jitsi","load","mediasoup","nodejs","performance","statistics","test","test-automation","video","webrtc","webrtcperf"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vpalmisano.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-03-03T17:08:05.000Z","updated_at":"2025-05-02T13:35:16.000Z","dependencies_parsed_at":"2023-10-12T22:59:51.593Z","dependency_job_id":"3b9cc09c-dc37-46d8-a01a-68be18a713a8","html_url":"https://github.com/vpalmisano/webrtcperf","commit_stats":null,"previous_names":[],"tags_count":23,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vpalmisano%2Fwebrtcperf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vpalmisano%2Fwebrtcperf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vpalmisano%2Fwebrtcperf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vpalmisano%2Fwebrtcperf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vpalmisano","download_url":"https://codeload.github.com/vpalmisano/webrtcperf/tar.gz/refs/heads/devel","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253948346,"owners_count":21988953,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chromium","concurrent-webrtc-sessions","edumeet","janus","jitsi","load","mediasoup","nodejs","performance","statistics","test","test-automation","video","webrtc","webrtcperf"],"created_at":"2025-05-13T12:56:44.476Z","updated_at":"2025-05-13T12:56:45.163Z","avatar_url":"https://github.com/vpalmisano.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"![logo](media/logo.svg \"WebRTC Perf\")\n# WebRTC Perf\n[GitHub page](https://github.com/vpalmisano/webrtcperf) | [Documentation](https://vpalmisano.github.io/webrtcperf)\n\n[![Build Docker image](https://github.com/vpalmisano/webrtcperf/actions/workflows/build.yaml/badge.svg)](https://github.com/vpalmisano/webrtcperf/actions/workflows/build.yaml)\n\nWebRTC performance and quality evaluation tool.\nIt allows to validate the audio/video quality and the client CPU/memory usage\nwhen multiple connections join the same WebRTC service.\n\nMain features:\n- A NodeJS application/library using Puppeteer for controlling chromium instances.\n- It can be executed:\n  - using the pre built Docker image; this is the suggested way to run the tool\n    without installing any dependency;\n  - from sources (using git pull or npm install);\n  - using the pre built executables generated for each platform.\n- It allows to inject custom Javascript source files that will run into the\nbrowser page context for automating some tasks (e.g. pressing a button to join\na conference room).\n- It allows to throttle the networking configuration, limiting the ingress/egress\navailable bandwidth, the RTT or the packet loss %.\n- It uses a patched version of chromium (see `./chromium` directory) that allows\nto disable the video decoding, lowering the CPU requirements when running multiple\nbrowser sessions.\n- It contains an RTC stats logging module that allows to collect metrics and\nsend them to a Prometheus Pushgateway server for live visualization with Grafana.\n- It allows to override getUserMedia and getDisplayMedia calls.\n- It allows to define alert rules and generate reports.\n\n## Install\nThe tool can be executed from sources, using the pre built executables or using the Docker image.\n\nUsing Npm:\n\n```bash\nnpm install -g @vpalmisano/webrtcperf\n\n# Install FFMpeg:\nsudo apt install ffmpeg # Linux\n# or:\nbrew install ffmpeg # MacOS\n\n# Run a Jitsi test:\nwebrtcperf \\\n    --url=\"https://meet.jit.si/${JITSI_ROOM_URL}#config.prejoinPageEnabled=false\" \\\n    --sessions=1 \\\n    --display='' \\\n    --show-page-log=false\n# Press \u003cq\u003e to stop.\n```\n\nUsing Docker:\n\n```bash\ndocker pull ghcr.io/vpalmisano/webrtcperf\ndocker run -it --rm \\\n    -v /dev/shm:/dev/shm \\\n    ghcr.io/vpalmisano/webrtcperf \\\n    --url=\"https://meet.jit.si/$JITSI_ROOM_URL#config.prejoinPageEnabled=false\" \\\n    --show-page-log=false \\\n    --sessions=1 \\\n    --tabs-per-session=1\n```\n\nStop the tool pressing `q` (normal browser close) or `x` (it will close the\nprocess immediately).\n\n## Configuration options\n\nSee the [config documentation](https://vpalmisano.github.io/webrtcperf/documents/config.html).\n\n## Statistics\n\nExample output:\n\n```\n-- Mon, 06 Feb 2023 20:46:34 GMT -------------------------------------------------------------------\n                          name    count      sum     mean   stddev       5p      95p      min      max\n                    System CPU        1             15.89     0.00    15.89    15.89    15.89    15.89 %\n                    System GPU        1              0.00     0.00     0.00     0.00     0.00     0.00 %\n                 System Memory        1             72.18     0.00    72.18    72.18    72.18    72.18 %\n                      CPU/page        1    84.42    84.42     0.00    84.42    84.42    84.42    84.42 %\n                   Memory/page        1  1206.90  1206.90     0.00  1206.90  1206.90  1206.90  1206.90 MB\n                         Pages        1        1        1        0        1        1        1        1\n                        Errors        1        0        0        0        0        0        0        0\n                      Warnings        1        0        0        0        0        0        0        0\n              Peer Connections        1        2        2        0        2        2        2        2\n-- Inbound audio -----------------------------------------------------------------------------------\n                          rate        2    28.73    14.36    14.36     0.00    28.73     0.00    28.73 Kbps\n                          lost        1              0.00     0.00     0.00     0.00     0.00     0.00 %\n                        jitter        2              0.00     0.00     0.00     0.00     0.00     0.00 s\n          avgJitterBufferDelay        1             35.29     0.00    35.29    35.29    35.29    35.29 ms\n-- Inbound video -----------------------------------------------------------------------------------\n                      received        2     2.66     1.33     1.32     0.01     2.64     0.01     2.64 MB\n                          rate        2   967.41   483.71   483.71     0.00   967.41     0.00   967.41 Kbps\n                          lost        1              0.00     0.00     0.00     0.00     0.00     0.00 %\n                        jitter        2              0.01     0.01     0.01     0.02     0.01     0.02 s\n          avgJitterBufferDelay        1             50.48     0.00    50.48    50.48    50.48    50.48 ms\n                         width        2               960      320      640     1280      640     1280 px\n                        height        2               540      180      360      720      360      720 px\n                           fps        1                15        0       15       15       15       15 fps\n-- Outbound audio ----------------------------------------------------------------------------------\n                          rate        2    42.84    21.42    21.42     0.00    42.84     0.00    42.84 Kbps\n                          lost        1              0.00     0.00     0.00     0.00     0.00     0.00 %\n                 roundTripTime        1             0.001    0.000    0.001    0.001    0.001    0.001 s\n-- Outbound video ----------------------------------------------------------------------------------\n                          sent        2     3.25     1.62     1.58     0.04     3.21     0.04     3.21 MB\n                          rate        2  1131.25   565.63   565.63     0.00  1131.25     0.00  1131.25 Kbps\n                          lost        1              0.00     0.00     0.00     0.00     0.00     0.00 %\n                 roundTripTime        1             0.001    0.000    0.001    0.001    0.001    0.001 s\n qualityLimitResolutionChanges        2        2        1        1        0        2        0        2\n          qualityLimitationCpu        2        0        0        0        0        0        0        0 %\n    qualityLimitationBandwidth        2       20       10       10        0       20        0       20 %\n           sentActiveEncodings        2                 2        1        1        3        1        3 encodings\n                sentMaxBitrate        2  3700.00  1850.00   350.00  1500.00  2200.00  1500.00  2200.00 Kbps\n                         width        2               640      640        0     1280        0     1280 px\n                        height        2               360      360        0      720        0      720 px\n                           fps        2                12       12        0       25        0       25 fps\n              pliCountReceived        2                 1        0        1        2        1        2\n```\n\nStatistics values:\n\n| Name                      | Count        | Description |\n| :------------------------ | :----------- | :----------- |\n| cpu                       | Total sessions | The browser process cpu usage. |\n| memory                    | Total sessions | The browser process memory usage. |\n| tabs                      | Total sessions | The browser current opened tabs. |\n| received                  | Total inbound streams | The `bytesReceived` value for each stream. |\n| sent                      | Total outbound streams | The `bytesSent` value for each stream. |\n| retransmitted             | Total outbound streams | The `retransmittedBytesSent` value for each stream. |\n| rate                      | Total streams | The stream bitrate. |\n| lost                      | Total streams | The stream [lost packets](https://www.w3.org/TR/webrtc-stats/#dom-rtcreceivedrtpstreamstats-packetslost) %. |\n| jitter                    | Total streams | The stream [jitter](https://www.w3.org/TR/webrtc-stats/#dom-rtcreceivedrtpstreamstats-jitter) in seconds. |\n| avgJitterBufferDelay      | Total decoded tracks | The inbound average [jitter buffer delay](https://www.w3.org/TR/webrtc-stats/#dom-rtcinboundrtpstreamstats-jitterbufferdelay). |\n| qualityLimitResolutionChanges   | Total outbound video streams | The `qualityLimitationResolutionChanges` [value](https://w3c.github.io/webrtc-stats/#dom-rtcoutboundrtpstreamstats-qualitylimitationresolutionchanges) for each outbound video stream. |\n| width                     | Total sent or received videos | The sent or received video width. |\n| height                    | Total sent or received videos | The sent or received video height. |\n| fps                       | Total sent | The sent video frames per second. |\n\n## Prometheus / Grafana\nSee the [prometheus stack](prometheus-stack/README.md).\n\n## Examples\n\n### Mediasoup demo\n\nStarts one send-receive participant:\n\n```sh\ndocker run -it --rm --name=webrtcperf-publisher \\\n    -v /dev/shm:/dev/shm \\\n    ghcr.io/vpalmisano/webrtcperf \\\n    --url=$MEDIASOUP_DEMO_URL \\\n    --url-query='roomId=test\u0026displayName=Publisher($s-$t)' \\\n    --sessions=1 \\\n    --tabs-per-session=1\n```\n\nStarts 10 receive-only participants:\n\n```sh\ndocker run -it --rm --name=webrtcperf-viewer \\\n    -v /dev/shm:/dev/shm \\\n    ghcr.io/vpalmisano/webrtcperf \\\n    --url=$MEDIASOUP_DEMO_URL \\\n    --url-query='roomId=test\u0026displayName=Viewer($s-$t)\u0026produce=false' \\\n    --sessions=1 \\\n    --tabs-per-session=10\n```\n\n### Edumeet\n\nStarts one send-receive participant, with a random audio activation pattern:\n\n```sh\ndocker run -it --rm \\\n    -v /dev/shm:/dev/shm \\\n    -v $PWD/examples:/scripts:ro \\\n    ghcr.io/vpalmisano/webrtcperf \\\n    --url=$EDUMEET_URL \\\n    --url-query='displayName=Publisher($s-$t)' \\\n    --script-path=/scripts/edumeet-sendrecv.js \\\n    --sessions=1 \\\n    --tabs-per-session=1\n```\n\nStarts 10 receive-only participants:\n\n```sh\ndocker run -it --rm \\\n    -v /dev/shm:/dev/shm \\\n    -v $PWD/examples:/scripts:ro \\\n    ghcr.io/vpalmisano/webrtcperf \\\n    --url=$EDUMEET_URL \\\n    --url-query='displayName=Viewer($s-$t)' \\\n    --script-path=/scripts/edumeet-recv.js \\\n    --sessions=1 \\\n    --tabs-per-session=10\n```\n\n### Jitsi\n\nStarts one send-receive participant:\n\n```sh\ndocker run -it --rm \\\n    -v /dev/shm:/dev/shm \\\n    ghcr.io/vpalmisano/webrtcperf \\\n    --url=$JITSI_ROOM_URL \\\n    --url-query='#config.prejoinPageEnabled=false\u0026userInfo.displayName=Participant($s-$t)' \\\n    --sessions=1 \\\n    --tabs-per-session=1\n```\n\nStarts 10 receive-only participants:\n\n```sh\ndocker run -it --rm \\\n    -v /dev/shm:/dev/shm \\\n    ghcr.io/vpalmisano/webrtcperf \\\n    --url=$ROOM_URL \\\n    --url-query='#config.prejoinPageEnabled=false\u0026userInfo.displayName=Participant($s-$t)' \\\n    --sessions=1 \\\n    --tabs-per-session=10\n```\n\n## Running from source code\n\nThe `DEBUG_LEVEL` environment variable can be used to enable debug messages;\nsee [debug-level](https://github.com/commenthol/debug-level#readme) for syntax.\n\n```sh\ngit clone https://github.com/vpalmisano/webrtcperf.git\n\ncd webrtcperf\n\n# Optional: build the chromium customized version\n# cd chromium\n# ./build.sh setup\n# ./build.sh apply_patch\n# ./build.sh build\n# install the package (on Ubuntu/Debian)\n# dpkg -i ./chromium-browser-unstable_\u003cversion\u003e-1_amd64.deb\n# cd ..\n\nyarn build\n\n# sendrecv test\nDEBUG_LEVEL=DEBUG:* yarn start \\\n    --url=https://127.0.0.1:3443/test \\\n    --url-query='displayName=SendRecv($s/$S-$t/$T)' \\\n    --script-path=./examples/edumeet-sendrecv.js \\\n    --sessions=1 \\\n    --tabs-per-session=1\n\n# recv only\nDEBUG_LEVEL=DEBUG:* yarn start \\\n    --url=https://127.0.0.1:3443/test \\\n    --url-query='displayName=Recv($s/$S-$t/$T)' \\\n    --script-path=./examples/edumeet-recv.js \\\n    --sessions=1 \\\n    --tabs-per-session=10\n```\n\n## Using the VMAF calculator\nRun a test adding the following options:\n```sh\n--script-params=\"{timestampWatermarkVideo:true,saveSendVideoTrack:'0',saveRecvVideoTrack:'1'}\" \\\n--server-port=5000 \\\n--server-use-https=true \\\n--server-data=/data \\\n--vmaf-path /data\n```\nWith `saveSendVideoTrack` and `saveRecvVideoTrack` you can specify the sessions that will be saved at sender and receiver side (in this example it will save all the video streams sent in the session with index `0` and received in session `1`).\nThe sent/received videos will be saved in the `/data` directory.\nThe tool will generate a `.vmaf.json` and a `.vmaf.png` files in the `data/vmaf` directory. Adding the `--vmaf-preview` option, a `.mp4` file containing the side-by-side video comparison will be generated.\n\n## Using the VISQOL calculator\nRun a test adding the following options:\n```sh\n--script-params=\"{saveSendAudioTrack:'0',saveRecvAudioTrack:'1'}\" \\\n--server-port=5000 \\\n--server-use-https=true \\\n--server-data=/data \\\n--visqol-path /data\n```\nWith `saveSendAudioTrack` and `saveRecvAudioTrack` you can specify the sessions that will be saved at sender and receiver side (in this example it will save all the audio streams sent in the session with index `0` and received in session `1`).\nThe sent/received audio files will be saved in the `/data` directory.\nThe tool will generate a `visqol.csv` file in the `data` directory.\n\n## Authors\n- Vittorio Palmisano [[github](https://github.com/vpalmisano)]\n\n## License\n[AGPL](./LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvpalmisano%2Fwebrtcperf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvpalmisano%2Fwebrtcperf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvpalmisano%2Fwebrtcperf/lists"}