{"id":14965732,"url":"https://github.com/dumijay/pfreak","last_synced_at":"2025-10-25T12:30:33.269Z","repository":{"id":57322552,"uuid":"382818334","full_name":"dumijay/pFreak","owner":"dumijay","description":"pFreak is a unit-level 2-in-1 JavaScript benchmarking and testing framework.","archived":false,"fork":false,"pushed_at":"2021-07-15T17:11:22.000Z","size":642,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-31T07:11:11.659Z","etag":null,"topics":["automation","benchmark","developer-tools","devtools","framework","headless","javasscript","performance","puppeteer","test","ui","unit-testing","ux"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dumijay.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-07-04T10:03:09.000Z","updated_at":"2023-06-29T04:56:29.000Z","dependencies_parsed_at":"2022-08-26T01:11:28.195Z","dependency_job_id":null,"html_url":"https://github.com/dumijay/pFreak","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dumijay%2FpFreak","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dumijay%2FpFreak/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dumijay%2FpFreak/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dumijay%2FpFreak/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dumijay","download_url":"https://codeload.github.com/dumijay/pFreak/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238137862,"owners_count":19422715,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automation","benchmark","developer-tools","devtools","framework","headless","javasscript","performance","puppeteer","test","ui","unit-testing","ux"],"created_at":"2024-09-24T13:35:12.718Z","updated_at":"2025-10-25T12:30:32.823Z","avatar_url":"https://github.com/dumijay.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pFreak\n\n## What if,\n... we can automate DevTool's Performance tab, benchmark \u0026 assert multiple tasks, save raw trace data of DevTool's Graphical Timeline for post investigations \u0026 summarize all that into a nice web-based dashboard where we can compare key metrics to optimize JavaScript \u0026 UI/UX performance?\n\n## Intro\npFreak is a unit-level 2-in-1 JavaScript benchmarking and testing framework.\n\nThe framework implements a flexible way to test and benchmark multiple candidates. Enabling meaningful A/B testing and measuring performance changes between different implementations and versions.\n\nAlso, it's very suitable to benchmark asynchronous executions since pFreak is based on Chromium DevTool's raw trace data. It calculates the execution duration of each unit of work instead of marking start-end based timing approach.\n\nBasically, it provides a highly flexible and scalable framework to separate pre-execution setup, execution and assert function.\n\nThis was originally implemented as a part of [CalDOM](https://caldom.org) UI library development.\n\n## How it works?\nBehind the scene, pFreak is using [Puppeteer](https://github.com/puppeteer/puppeteer/) to automate the process and capture trace data through Chrome DevTools Protocol(CDP) session (same as Developer Tools Performance tab). The captured raw trace data is processed by [devtools-timeline-model](https://github.com/paulirish/devtools-timeline-model). Then the formatted results can be viewed on the browser.\n\n___\n\n### Sample benchmark result preview\n\u003cp\u003e\n\u003cimg src=\"./etc/benchmark_preview.png\" alt=\"pFreak Benchmark Results\" /\u003e\n\u003c/p\u003e\n\n### Sample test result preview\n\u003cp\u003e\n\u003cimg src=\"./etc/tests_preview.png\" alt=\"pFreak Test Results\" /\u003e\n\u003c/p\u003e\n\n### View raw trace data using [Timeline Viewer](https://github.com/ChromeDevTools/timeline-viewer)\n\u003cp\u003e\n\u003cimg src=\"./etc/timeline_preview.png\" alt=\"pFreak Timeline Viewer\" /\u003e\n\u003c/p\u003e\n\n## Benchmark Mechanism\n- Each task is iterated X times and mean execution duration is taken.\n- Each iteration Steps: \n    1. Open a new page (\"empty\\_page.html\")\n    2. Slowdown CPU speed by ?X\n    3. Load respective library through a script tag\n    4. Load respective task through a script tag\n    5. Start tracing (to capture Dev Tools' performance metrics)\n    6. Run Task (Puppeteer's code injection execution time is excluded by scheduling the task using setTimeout before tracing is started.)\n    7. Wait X seconds for all task operations to be completed. (This can be configured per task/test)\n    8. Stop tracing\n    9. Assert whether the task is completed.\n    10. Parse raw trace data using devtools-timeline-model to extract javascript and layout/paint execution times.\n    11. Close Page\n\n- The mean execution duration is in microseconds. (1000 Microseconds = 1 Millisecond).\n- Coefficient of variation is shown below the execution duration.\n    - This is how much iterations deviated from its mean execution duration. Lower deviation means the test is stable.\n    - Variation for smaller operations can be high. If that's the case, repeat the task equally for all candidates to increase the execution time.\n    - Refer _task_template.js for details.\n\n- Factor of slowness is compared against the base candidate. Eg: \n    - Vanilla JS execution duration = 400ms\n    - Candidate 1 execution duration= 600ms\n    - Candidate 1 is 1.5x slower than Vanilla JS\n\n___\n\n## How to use?\n\n```shell\nnpm install pfreak\n\ncd path/to/tests\n```\n\n### 1. Initiate pFreak. This creates \u0026 link all necessary file structure\n\n```shell\npfreak init\n```\nTip: Have a look at config.json \u0026 ./tasks/_task_template.js. Configure config.json if you want.\n_task_template.js is the base template to create new tests/tasks. You can modify this to suit your default template.\n\n### 2. Create a new task/test\n\n```shell\npfreak new-task --candidate candidate_name --category category_name --task task_name\n```\nThis creates a new JS file in the ./tasks/ folder. Define your test/benchmark in the file using the given structure.\n\n### 3. Run the benchmark or test-only mode\n```shell\npfreak benchmark\n\n#or\n\npfreak test\n```\n\n### 4. View Results\n```shell\npfreak show\n```\nThis starts an http-server at localhost:8080 and opens it.\n\n### Refer help for details\n```shell\npfreak --help\n```\n\n___\n\n## Future Development\n* Hope to expand this to Node based benchmark/tests as well (outside of the browser)\n* Need a detailed documentation\n* Same candidate, multiple library versions support (config.json)\n\n___\n\n## How to contribute?\nYour contributions are very welcome. I just created this as a side project to benchmark and test the [CalDOM](https://caldom.org) UI library I created. Figured that this could be useful for others as well. I don't have a grand plan for this yet, please feel free to jump in :)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdumijay%2Fpfreak","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdumijay%2Fpfreak","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdumijay%2Fpfreak/lists"}