{"id":20156243,"url":"https://github.com/web-infra-dev/rspack-ecosystem-benchmark","last_synced_at":"2025-04-09T22:23:35.228Z","repository":{"id":166404639,"uuid":"638792582","full_name":"web-infra-dev/rspack-ecosystem-benchmark","owner":"web-infra-dev","description":"Monitor Rspack performance.","archived":false,"fork":false,"pushed_at":"2025-03-18T00:40:04.000Z","size":15184,"stargazers_count":10,"open_issues_count":7,"forks_count":5,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-03-23T13:02:22.938Z","etag":null,"topics":["rspack"],"latest_commit_sha":null,"homepage":"https://rspack-ecosystem-benchmark.rspack.dev/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/web-infra-dev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-10T05:50:01.000Z","updated_at":"2025-02-06T06:46:08.000Z","dependencies_parsed_at":"2025-03-31T01:33:21.817Z","dependency_job_id":null,"html_url":"https://github.com/web-infra-dev/rspack-ecosystem-benchmark","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/web-infra-dev%2Frspack-ecosystem-benchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/web-infra-dev%2Frspack-ecosystem-benchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/web-infra-dev%2Frspack-ecosystem-benchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/web-infra-dev%2Frspack-ecosystem-benchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/web-infra-dev","download_url":"https://codeload.github.com/web-infra-dev/rspack-ecosystem-benchmark/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248120917,"owners_count":21051046,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["rspack"],"created_at":"2024-11-13T23:38:09.227Z","updated_at":"2025-04-09T22:23:30.206Z","avatar_url":"https://github.com/web-infra-dev.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Rspack Benchmark\n\nThis repository is used to monitor Rspack performance.\n\n## Usage\n\nYou can use the scripts in the `bin` directory to prepare and run benchmark. All interactions are of the form\n\n```\nnode bin/cli.js [command] [args]\n```\n\nIf no command is specified, then `build`, `bench` and `compare` commands will all be ran by default.\n\n### Commands\n\n#### build\n\n```\nnode bin/cli.js build [--ref \u003cref\u003e]\n```\n\nClone and build [Rspack](https://github.com/web-infra-dev/rspack) in the folder which defined by the environment variable of `RSPACK_DIR` or `\u003cproject_root\u003e/.rspack`. You can set Git ref with the parameter, e.g.:\n\n```bash\nnode bin/cli.js build # use the main branch to build\nnode bin/cli.js build --ref main # use the main branch to build\nnode bin/cli.js build --ref pull/1000/head # use the pull request with index 1000 to build\n```\n\n#### bench\n\n```\nnode bin/cli.js bench [--job \u003cbenchmark-names\u003e ...]\n```\n\nRun benchmark with Rspack and write the output to `output` folder. You can configure the environment variable of `RSPACK_DIR` to set the Rspack project path, and it will use the Rspack from the `.rspack` folder by default. eg.\n\n```bash\nnode bin/cli.js bench # run all benchmarks\nnode bin/cli.js bench --job 10000_development-mode --job 10000_production-mode # run benchmarks named 10000_development-mode and 10000_production-mode\nRSPACK_DIR=\u003cyour-rspack-path\u003e node bin/cli.js bench --job 10000_development-mode_hmr # set the rspack command path, and run 10000_development-mode_hmr\n```\n\n#### compare\n\n```\nnode bin/cli.js compare [--base \u003cbaseDate\u003e] [--current \u003ccurrentDate\u003e]\n```\n\nCompare and print the difference between `\u003cbaseDate\u003e` and `\u003ccurrentDate\u003e`. The parameter has three types, `\"current\"` will use the data from `output` folder. `\"latest\"` will use the latest data from `data` branch. A date string like `YYYY-MM-DD` will use the data of that day from `data` branch. eg.\n\n```bash\nnode bin/cli.js compare --base current --current latest # use output data as base, and latest data as current\nnode bin/cli.js compare --base latest --current 2023-08-17 # use latest data as base, and the data of 2023-08-17 as current\n```\n\n## Glossary\n\n#### Benchmark Name\n\nBenchmark name is a string containing the case name and the addon names. A benchmark name is separated by \"\\_\", the first part is the case name, amd the other parts are the addon names.\n\n#### Case\n\nThe benchmark case is the Rspack project in `cases` folder. It must contain `rspack.config.js` and `hmr.js`, the first one is the default config for Rspack, the next one is used to tell benchmark how to change file when hmr.\n\n- `10000` is a project with 10000 modules\n- `threejs` is a copy of three js\n\n#### Addon\n\nThe addon is used to change Rspack configuration and benchmark parameters. All addons are registered in `lib/addons`.\n\n- `development-mode` is used to set the development mode\n- `production-mode` is used to set the production mode\n- `10x` is used to make the project module 10 times larger\n- `hmr` is used to change the phase of collected metrics to hmr instead of build\n\n#### Metric\n\nThe metrics collected through benchmarking includes `stats`, `exec`, `dist size`, etc.\n\n#### Tag\n\nThe tag is `benchmarkName` + `metric`, it is used to display information on website and comparison results.\n\n## How benchmark works\n\nRspack benchmark consists of the following steps.\n\n```mermaid\nflowchart LR\n    A(\"Setup\") --\u003e B(\"Generate\")\n    B --\u003e C(\"Warmup\")\n    C --\u003e D(\"Run\")\n    D --\u003e E(\"Statistic\")\n    E --\u003e F(\"Teardown\")\n```\n\n- `Setup` is used to prepare the benchmark case environment and global context.\n- `Generate` will generate the benchmark case.\n- `Warmup` will attempt to run the benchmark case.\n- `Run` will run the benchmark case multiple times and collect metric data.\n- `Statistic` will calculate the final result through multiple benchmarks.\n- `Teardown` is used to clean up.\n\nThe main process is controlled by `scenario`, and its structure is:\n\n```typescript\ninterface Context {\n    caseDir: string\n    originalFiles: Record\u003cstring, string\u003e\n    // final rspack.config.js string\n    config: string\n    // hmr config\n    hmrConfig: Array\u003c{\n        rebuildChangeFile: string\n        generateContent(originalContent, runTimes): string\n    }\u003e\n    rspackArgs: string[]\n    // how many times should run\n    runTimes: number\n    // timeout milliseconds\n    timeout: number\n    runData: Array\u003c{\n        stats: number\n        exec: number\n        ...\n    }\u003e\n    result: Record\u003c\"stats\" | \"heap memory\" | \"rss memory\"\n     | \"external memory\" | \"array buffers memory\" | \"exec\"\n     | \"dist size\", {\n        min: number,\n        max: number,\n        mean: number,\n        median: number,\n        variance: number,\n        stdDev: number,\n        confidence: number,\n        low: number,\n        high: number,\n        count: number,\n        base: number\n     }\u003e\n}\n\ninterface Scenario {\n    async setup(): Context\n    async generate(ctx: Context): void\n    async warmup(ctx: Context): void\n    async run(ctx: Context): void\n    async statistic(ctx: Context): void\n    async teardown(ctx: Context): void\n}\n```\n\nThe addons can change context in most steps, and its structure is:\n\n```typescript\ninterface Addon {\n\tbeforeSetup(): void;\n\tafterSetup(ctx: Context): void;\n\n\tbeforeGenerate(ctx: Context): void;\n\tafterGenerate(ctx: Context): void;\n\n\tbeforeStatistic(ctx: Context): void;\n\tafterStatistic(ctx: Context): void;\n\n\tbeforeTeardown(ctx: Context): void;\n\tafterTeardown(ctx: Context): void;\n}\n```\n\n## How to add a benchmark case\n\n1. move your project into the `cases` folder\n2. move the project dependencies to global package.json\n3. add `rspack.config.js` to make the project runnable by Rspack\n4. add `hmr.js` to make the project support hmr changes\n5. try run `RSPACK_DIR=\u003cyour-rspack-path\u003e node bin/cli.js bench \u003cyour case\u003e_\u003cyour addons\u003e` to test\n\n## How to create a addon\n\n1. create your addons in `lib/addons` with kebab case like \"a-b-c.js\"\n2. export default a class that extends `lib/addons/common` and implement the hooks you want to listen\n3. try run `RSPACK_DIR=\u003cyour-rspack-path\u003e node bin/cli.js bench \u003ccase\u003e_\u003cyour addons\u003e` to test\n4. if you want to run some benchmark name by default, you can add it to `jobs` field in `./bench.config.js`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fweb-infra-dev%2Frspack-ecosystem-benchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fweb-infra-dev%2Frspack-ecosystem-benchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fweb-infra-dev%2Frspack-ecosystem-benchmark/lists"}