{"id":22699837,"url":"https://github.com/enactjs/performance-tests","last_synced_at":"2025-03-29T19:09:59.516Z","repository":{"id":39749605,"uuid":"165144857","full_name":"enactjs/performance-tests","owner":"enactjs","description":"Tooling and test suite to validate @enactjs/enact performance","archived":false,"fork":false,"pushed_at":"2024-04-29T13:57:34.000Z","size":520,"stargazers_count":0,"open_issues_count":2,"forks_count":1,"subscribers_count":21,"default_branch":"master","last_synced_at":"2024-04-29T20:06:36.813Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/enactjs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-01-10T23:09:24.000Z","updated_at":"2024-05-08T09:44:49.520Z","dependencies_parsed_at":"2023-12-20T19:46:42.226Z","dependency_job_id":"53d4c46e-4b0b-4e8f-8dcc-d57561f5163e","html_url":"https://github.com/enactjs/performance-tests","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/enactjs%2Fperformance-tests","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/enactjs%2Fperformance-tests/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/enactjs%2Fperformance-tests/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/enactjs%2Fperformance-tests/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/enactjs","download_url":"https://codeload.github.com/enactjs/performance-tests/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246230542,"owners_count":20744349,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-10T06:08:51.366Z","updated_at":"2025-03-29T19:09:59.492Z","avatar_url":"https://github.com/enactjs.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Enact Performance Testing\r\n\r\nApplication to perform automated performance testing on Enact components.\r\n\r\nWe utilize puppeteer to get chrome performance traces.\r\n\r\n## Testing on PC\r\n\r\n### Testing Sandstone components\r\n\r\nYou can start the server and run the test suite on it separately. Sandstone is the default theme, so you can simply use:\r\n\r\n```\r\nnpm run serve\r\nnpm run test\r\n```\r\n\r\nAlternatively, you can use the test-all command to start the server and run the test suite together:\r\n\r\n```\r\nnpm run test-all\r\n```\r\n\r\n### Testing Agate components\r\n\r\nStart the server with Agate components and run the test suite on it. You can specify the theme by adding --theme=agate at the end of the command:\r\n\r\n```\r\nnpm run serve-agate\r\nnpm run test -- --theme=agate\r\n```\r\n\r\n```\r\nnpm run test-all -- --theme=agate\r\n```\r\n\r\nOn Windows OS you might need to install `cross-env` globally with `npm install -g cross-env`.\r\n\r\n## Testing on TV\r\n\r\nPass the IP address of the TV as an environment variable and use the `npm run test` task:\r\n\r\n```bash\r\nTV_IP=10.0.1.1 npm run test-all -- --target=TV --theme=sandstone\r\n```\r\n\r\n```bash\r\nnpm run serve-sandstone\r\nTV_IP=10.0.1.1 npm run test -- --target=TV --theme=sandstone\r\n```\r\n\r\n## CPU Throttling\r\n\r\nYou can simulate a low-end device with a CPU throttling option. 1 is no throttle, 2 is 2x slowdown.\r\nSee the [Puppeteer API docs](https://pptr.dev/api/puppeteer.page.emulatecputhrottling) for the detailed information.\r\n\r\nAvailable commands are:\r\n\r\n### Testing on PC\r\nExample:\r\nIf you want to run tests on the PC with CPU throttling with 3x slowdown, you can run this command:\r\n```\r\nnpm run test -- --target=PC --theme=sandstone --throttling=3\r\nnpm run test -- --target=PC --theme=agate --throttling=3\r\n```\r\n\r\n### Testing on TV board\r\nExample: \r\nIf you want to run tests on the TV with a CPU throttling with 2x slowdown, you can run this command:\r\n```\r\nnpm run test -- --target=TV --theme=sandstone --throttling=2\r\nnpm run test -- --target=TV --theme=agate --throttling=2\r\n```\r\n\r\n## Adding Tests\r\n\r\nThis project works a bit differently than a regular test suite for now. We have Jest installed more as a test runner, but we don't really use assertions for now. We use it more to gather and report numbers.\r\n\r\n### FCP\r\n\r\nFirst Contentful Paint (FCP) \"measures the total time taken from the beginning of a page load to the point any content is rendered on the screen\" (see https://web.dev/fcp/).\r\nTo get FCP we use the `PageLoadingMetrics` function from `TraceModel`.\r\n\r\n### DCL\r\n\r\nDOM Content Loaded (DCL) \"marks the point when both the DOM is ready and there are no stylesheets that are blocking JavaScript execution\" (see https://web.dev/critical-rendering-path-measure-crp/). In other words, it measures the time when the HTML document has been completely loaded and read.\r\nTo get DCL we use the `PageLoadingMetrics` function from `TraceModel`.\r\n\r\n### LCP\r\n\r\nThe Largest Contentful Paint (LCP) metric \"reports the render time of the largest image or text block visible within the viewport, relative to when the page first started loading\" (see https://web.dev/lcp/).\r\nTo get LCP we use the `PageLoadingMetrics` function from `TraceModel`.\r\n\r\n### FPS\r\n\r\nFrames per second (FPS) measures video quality (how many images are displayed consecutively each second). We use it to measure component animation performance.\r\nFPS is read using the performance.now() method and Window.requestAnimationFrame() for the entire life span of the page. Just before the page is closed the average FPS is calculated.\r\n\r\n### INP\r\n\r\nInteraction to Next Paint (INP) is a web performance metric that measures how quickly a website updates or shows changes after a user interacts with it. It specifically captures the delay between when a user interacts with your site—like clicking a link, pressing a key on the keyboard, or tapping a button—and when they see a visual response (see https://web.dev/articles/inp).\r\nINP is calculated from the onINP() webVitals function. It logs the INP value in the console and we are monitoring the console in order to read the actual value.\r\n\r\n### CLS\r\n\r\nCumulative Layout Shift (CLS) is \"a measure of the largest burst of layout shift scores for every unexpected layout shift that occurs during the entire lifespan of a page. A layout shift occurs any time a visible element changes its position from one rendered frame to the next\" (see https://web.dev/cls/).\r\nCLS is calculated using the PerformanceObserver interface. Its observer() method specifies the set of entry types to observe (in this case layout-shift). The performance observer's callback function will be invoked when a performance entry is recorded for one of the specified entryTypes.\r\n\r\n### Example\r\n\r\nEach component is tested repeatedly for both `click` and `keypress` events to measure average FPS. \r\nCLS is tested separately because requires interactions. We can check the React Devtools to see which component is at the top of a specific component.\r\nCLS is tested separately because it is read from using the `web-vitals` library.\r\nDCL, FCP and LCP are also tested together as they are measured at page load time.\r\n\r\nResults can be found in `testResults` folder.\r\n\r\nTest results are compared to the optimum values which are stored in global variables declared in `puppeteer.setup` file.\r\n\r\n**Metrics threshholds:**\r\n- global.maxCLS = 0.1; \r\n- global.maxDCL = 2000; \r\n- global.maxFCP = 1800;\r\n- global.maxINP = 200;\r\n- global.minFPS = 20; \r\n- global.maxLCP = 2500;\r\n\r\n```javascript\r\nconst TestResults = require('../TestResults');\r\nconst {CLS, FPS, getAverageFPS, PageLoadingMetrics} = require('../TraceModel');\r\nconst {clsValue, getFileName, newPageMultiple} = require('../utils');\r\n\r\ndescribe('Dropdown', () =\u003e {\r\n\tconst component = 'Dropdown';\r\n\tTestResults.newFile(component);\r\n\r\n\tdescribe('click', () =\u003e {\r\n\t\tit('animates', async () =\u003e {\r\n\t\t\tawait FPS();\r\n\t\t\tawait page.goto(`http://${serverAddr}/dropdown`);\r\n\t\t\tawait page.waitForSelector('#dropdown');\r\n\t\t\tawait page.click('#dropdown'); // to move mouse on dropdown\r\n\t\t\tawait page.mouse.down();\r\n\t\t\tawait new Promise(r =\u003e setTimeout(r, 200));\r\n\t\t\tawait page.mouse.up();\r\n\t\t\tawait page.mouse.down();\r\n\t\t\tawait new Promise(r =\u003e setTimeout(r, 200));\r\n\t\t\tawait page.mouse.up();\r\n\t\t\tawait page.mouse.down();\r\n\t\t\tawait new Promise(r =\u003e setTimeout(r, 200));\r\n\t\t\tawait page.mouse.up();\r\n\t\t\tawait page.mouse.down();\r\n\t\t\tawait new Promise(r =\u003e setTimeout(r, 200));\r\n\t\t\tawait page.mouse.up();\r\n\r\n\t\t\tconst averageFPS = await getAverageFPS();\r\n\t\t\tTestResults.addResult({component: component, type: 'FPS Click', actualValue: Math.round((averageFPS + Number.EPSILON) * 1000) / 1000});\r\n\r\n\t\t\texpect(averageFPS).toBeGreaterThan(minFPS);\r\n\t\t});\r\n\t});\r\n\r\n\tdescribe('keypress', () =\u003e {\r\n\t\tit('animates', async () =\u003e {\r\n\t\t\tawait FPS();\r\n\t\t\tawait page.goto(`http://${serverAddr}/dropdown`);\r\n\t\t\tawait page.waitForSelector('#dropdown');\r\n\t\t\tawait page.focus('#dropdown');\r\n\t\t\tawait new Promise(r =\u003e setTimeout(r, 200));\r\n\t\t\tawait page.keyboard.down('Enter');\r\n\t\t\tawait new Promise(r =\u003e setTimeout(r, 200));\r\n\t\t\tawait page.keyboard.up('Enter');\r\n\t\t\tawait page.keyboard.down('Enter');\r\n\t\t\tawait new Promise(r =\u003e setTimeout(r, 200));\r\n\t\t\tawait page.keyboard.up('Enter');\r\n\t\t\tawait page.keyboard.down('Enter');\r\n\t\t\tawait new Promise(r =\u003e setTimeout(r, 200));\r\n\t\t\tawait page.keyboard.up('Enter');\r\n\t\t\tawait page.keyboard.down('Enter');\r\n\t\t\tawait new Promise(r =\u003e setTimeout(r, 200));\r\n\t\t\tawait page.keyboard.up('Enter');\r\n\r\n\t\t\tconst averageFPS = await getAverageFPS();\r\n\t\t\tTestResults.addResult({component: component, type: 'FPS Keypress', actualValue: Math.round((averageFPS + Number.EPSILON) * 1000) / 1000});\r\n\r\n\t\t\texpect(averageFPS).toBeGreaterThan(minFPS);\r\n\t\t});\r\n\t});\r\n\r\n\tit('should have a good CLS', async () =\u003e {\r\n\t\tawait page.evaluateOnNewDocument(CLS);\r\n\t\tawait page.goto(`http://${serverAddr}/dropdown`);\r\n\t\tawait page.waitForSelector('#dropdown');\r\n\t\tawait page.focus('#dropdown');\r\n\t\tawait page.keyboard.down('Enter');\r\n\t\t\r\n\t\tlet actualCLS = await clsValue();\r\n\r\n\t\tTestResults.addResult({component: component, type: 'CLS', actualValue: Math.round((actualCLS + Number.EPSILON) * 1000) / 1000});\r\n\r\n\t\texpect(actualCLS).toBeLessThan(maxCLS);\r\n\t});\r\n\r\n\tit('should have a good INP', async () =\u003e {\r\n\t\tawait page.goto(`http://${serverAddr}/dropdown`);\r\n\t\tawait page.addScriptTag({url: webVitalsURL});\r\n\t\tawait page.waitForSelector('#dropdown');\r\n\t\tawait page.focus('#dropdown');\r\n\t\tawait new Promise(r =\u003e setTimeout(r, 100));\r\n\t\tawait page.keyboard.down('Enter');\r\n\t\tawait new Promise(r =\u003e setTimeout(r, 100));\r\n\r\n\t\tlet inpValue;\r\n\r\n\t\tpage.on(\"console\", (msg) =\u003e {\r\n\t\t\tinpValue = Number(msg.text());\r\n\t\t\tTestResults.addResult({component: component, type: 'INP', actualValue: Math.round((inpValue + Number.EPSILON) * 1000) / 1000});\r\n\t\t\texpect(inpValue).toBeLessThan(maxINP);\r\n\t\t});\r\n\r\n\t\tawait page.evaluateHandle(() =\u003e {\r\n\t\t\twebVitals.onINP(function (inp) {\r\n\t\t\t\t\tconsole.log(inp.value); // eslint-disable-line no-console\r\n\t\t\t\t},\r\n\t\t\t\t{\r\n\t\t\t\t\treportAllChanges: true\r\n\t\t\t\t}\r\n\t\t\t);\r\n\t\t});\r\n\t\tawait new Promise(r =\u003e setTimeout(r, 1000));\r\n\t});\r\n\r\n\tit('should have a good DCL, FCP and LCP', async () =\u003e {\r\n\t\tconst filename = getFileName(component);\r\n\r\n\t\tlet passContDCL = 0;\r\n\t\tlet passContFCP = 0;\r\n\t\tlet passContLCP = 0;\r\n\t\tlet avgDCL = 0;\r\n\t\tlet avgFCP = 0;\r\n\t\tlet avgLCP = 0;\r\n\t\tfor (let step = 0; step \u003c stepNumber; step++) {\r\n\t\t\tconst dropdownPage = targetEnv === 'TV' ? page : await newPageMultiple();\r\n\t\t\tawait dropdownPage.emulateCPUThrottling(CPUThrottling);\r\n\r\n\t\t\tawait dropdownPage.tracing.start({path: filename, screenshots: false});\r\n\t\t\tawait dropdownPage.goto(`http://${serverAddr}/dropdown`);\r\n\t\t\tawait dropdownPage.waitForSelector('#dropdown');\r\n\t\t\tawait new Promise(r =\u003e setTimeout(r, 200));\r\n\r\n\t\t\tawait dropdownPage.tracing.stop();\r\n\r\n\t\t\tconst {actualDCL, actualFCP, actualLCP} = PageLoadingMetrics(filename);\r\n\t\t\tavgDCL = avgDCL + actualDCL;\r\n\t\t\tif (actualDCL \u003c maxDCL) {\r\n\t\t\t\tpassContDCL += 1;\r\n\t\t\t}\r\n\r\n\t\t\tavgFCP = avgFCP + actualFCP;\r\n\t\t\tif (actualFCP \u003c maxFCP) {\r\n\t\t\t\tpassContFCP += 1;\r\n\t\t\t}\r\n\r\n\t\t\tavgLCP = avgLCP + actualLCP;\r\n\t\t\tif (actualLCP \u003c maxLCP) {\r\n\t\t\t\tpassContLCP += 1;\r\n\t\t\t}\r\n\r\n\t\t\tif (targetEnv === 'PC') await dropdownPage.close();\r\n\t\t}\r\n\t\tavgDCL = avgDCL / stepNumber;\r\n\t\tavgFCP = avgFCP / stepNumber;\r\n\t\tavgLCP = avgLCP / stepNumber;\r\n\r\n\t\tTestResults.addResult({component: component, type: 'DCL', actualValue: Math.round((avgDCL + Number.EPSILON) * 1000) / 1000});\r\n\t\tTestResults.addResult({component: component, type: 'FCP', actualValue: Math.round((avgFCP + Number.EPSILON) * 1000) / 1000});\r\n\t\tTestResults.addResult({component: component, type: 'LCP', actualValue: Math.round((avgLCP + Number.EPSILON) * 1000) / 1000});\r\n\r\n\t\texpect(passContDCL).toBeGreaterThan(passRatio * stepNumber);\r\n\t\texpect(avgDCL).toBeLessThan(maxDCL);\r\n\r\n\t\texpect(passContFCP).toBeGreaterThan(passRatio * stepNumber);\r\n\t\texpect(avgFCP).toBeLessThan(maxFCP);\r\n\r\n\t\texpect(passContLCP).toBeGreaterThan(passRatio * stepNumber);\r\n\t\texpect(avgLCP).toBeLessThan(maxLCP);\r\n\t});\r\n});\r\n```\r\n\r\n### Google Sheets\r\n\r\nWe have the ability to send data to a Google Spreadsheet. If you wish to use this, include an environment variable. \r\n\r\n```\r\n// .env\r\nAPI_URL=https://script.google.com/macros/s/SCRIPT_ID/exec\r\n```\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fenactjs%2Fperformance-tests","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fenactjs%2Fperformance-tests","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fenactjs%2Fperformance-tests/lists"}