{"id":18071964,"url":"https://github.com/stereobooster/test-coverage-calculation","last_synced_at":"2025-04-05T17:41:56.974Z","repository":{"id":252639590,"uuid":"840682168","full_name":"stereobooster/test-coverage-calculation","owner":"stereobooster","description":"purely theoretical speculations about how code coverage may be calculated","archived":false,"fork":false,"pushed_at":"2024-08-16T20:29:15.000Z","size":80,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-11T14:47:03.164Z","etag":null,"topics":["c8","coverage","istanbuljs","javascript","js","monocart","monocart-reporter","vitest"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stereobooster.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-10T11:09:39.000Z","updated_at":"2024-08-16T20:29:18.000Z","dependencies_parsed_at":"2024-08-16T21:39:35.082Z","dependency_job_id":null,"html_url":"https://github.com/stereobooster/test-coverage-calculation","commit_stats":null,"previous_names":["stereobooster/test-coverage-calculation"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stereobooster%2Ftest-coverage-calculation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stereobooster%2Ftest-coverage-calculation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stereobooster%2Ftest-coverage-calculation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stereobooster%2Ftest-coverage-calculation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stereobooster","download_url":"https://codeload.github.com/stereobooster/test-coverage-calculation/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247378087,"owners_count":20929291,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["c8","coverage","istanbuljs","javascript","js","monocart","monocart-reporter","vitest"],"created_at":"2024-10-31T09:18:16.334Z","updated_at":"2025-04-05T17:41:56.956Z","avatar_url":"https://github.com/stereobooster.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# test-coverage-calculation\n\n**This is purely theoretical speculations about how code coverage may be calculated.**\n\nIt's all started with this code:\n\n```js\nexport function comp(a, b) {\n  if (a \u003e b) return 1;\n  if (a \u003c b) return -1;\n  return 0;\n}\n```\n\nMost of code coverage tools would say this code has 4 branches. Which seemed strange to me. So I wondered why...\n\n## Comparison of branch coverage\n\n|               | `@vitest/coverage-istanbul` | `vitest-monocart-coverage` | `@vitest/coverage-v8` | what I expect |\n| ------------- | --------------------------- | -------------------------- | --------------------- | ------------- |\n| example0.1.js | 0/0                         | 0/0                        | 1/1                   | 1/1           |\n| example0.2.js | 2/2                         | 2/2                        | 2/2                   | 2/2           |\n| example1.1.js | 4/4                         | 4/4                        | 4/4                   | 3/3 or 4/4    |\n| example1.2.js | 4/4                         | 4/4                        | [3/3][v6300]          | 4/4           |\n| example1.3.js | 4/4                         | 4/4                        | [3/3][v6300]          | 4/4           |\n| example1.4.js | 6/6                         | 6/6                        | [4/4][v6300]          | 6/6           |\n| example1.5.js | 4/4                         | 4/4                        | [5/5][v6300]          | 3/3 or 4/4    |\n| example2.1.js | 4/4                         | 4/4                        | 4/4                   | 3/3 or 4/4    |\n| example3.1.js | 2/2                         | 2/2                        | 2/2                   | 2/2 or 1/1    |\n| example3.2.js | 2/2                         | 2/2                        | 2/2                   | 2/2 or 1/1    |\n| example3.3.js | 2/2                         | 2/2                        | **3/3**               | 2/2 or 1/1    |\n| example3.4.js | 4/4                         | 4/4                        | **3/3**               | 2/2 or 4/4    |\n| example4.1.js | 2/2                         | 2/2                        | **3/3**               | 2/2           |\n| example4.2.js | [1/1][i795]                 | **1/1**                    | 2/2                   | 2/2           |\n| example4.3.js | 2/2                         | 2/2                        | 2/2                   | 2/2           |\n| example4.4.js | 3/3                         | 3/3                        | 3/3                   | 3/3           |\n| example5.1.js | 0/0                         | 0/0                        | 2/2                   | 2/2           |\n| example6.1.js | 0/0                         | 0/0                        | 2/2                   | 2/2 or 1/1    |\n| example6.2.js | 0/0                         | 0/0                        | 2/2                   | 2/2           |\n| example7.1.js | [0/0][i516]                 | 0/0                        | 2/2                   | 2/2           |\n| example7.2.js | [0/0][i516]                 | 0/0                        | 1/2                   | 1/2           |\n| example8.1.js | 0/0                         | 0/0                        | **1/2**               | 1/3           |\n\n**Note**: if you have 100% coverage you probably don't care if it is 3/3 or 5/5. This would make a difference it you have less than 100%, than numbers can be skewed.\n\n[mcr68]: https://github.com/cenfun/monocart-coverage-reports/issues/68\n[i795]: https://github.com/istanbuljs/istanbuljs/issues/795\n[v6300]: https://github.com/vitest-dev/vitest/issues/6300\n[i516]: https://github.com/istanbuljs/istanbuljs/issues/516\n\n## Example 1\n\n1 branch (or no branching):\n\n```js\nconsole.log(1);\n```\n\n```mermaid\nflowchart LR\n  s --\u003e e\n```\n\n2 branches:\n\n```js\nconsole.log(1);\n\nif (a) {\n  console.log(2);\n}\n```\n\n```mermaid\nflowchart LR\n  s(s) --- if[\"if(a)\"] -- true  --\u003e e\n  if -- false --\u003e e(e)\n```\n\n**Important** even so for `a = true` it would visit all lines of code, we as well need to execute code with `a = false` to claim that all cases have been covered.\n\nAnd this is basically explains why it counts 4 branches here:\n\n```js\nexport function comp(a, b) {\n  if (a \u003e b) return 1;\n  if (a \u003c b) return -1;\n  return 0;\n}\n```\n\n|     | `a \u003e b` | `a \u003c b` |\n| --- | ------- | ------- |\n| 1   | true    | true    |\n| 2   | true    | false   |\n| 3   | false   | true    |\n| 4   | false   | false   |\n\n**But** there is no difference between cases 1 and 2. If the first condition is true we will never reach code in the second condition, because of early return.\n\n```mermaid\nflowchart LR\n  s(s) --- if1[\"if(a \u003e b)\"] -- true --\u003e e\n  if1 -- false --- if2[\" if(a \u003c b)\"] -- true --\u003e e\n  if2 -- false --\u003e e(e)\n```\n\nOn the other hand, code like this:\n\n```js\nexport function comp(a, b) {\n  let result = 0;\n  if (a \u003e b) result = 1;\n  if (a \u003c b) result = -1;\n  return result;\n}\n```\n\nIndeed has 4 branches:\n\n```mermaid\nflowchart LR\n  s(s) --- if1[\"if(a \u003e b)\"]\n  if1 -- true  --\u003e if2\n  if1 -- false --\u003e if2\n  if2[\"if(a \u003c b)\"] -- true --\u003e e\n  if2 -- false --\u003e e(e)\n```\n\n### Branches != paths\n\nIn example above we have 4 branches and coincidentally 4 paths. All branches can be reached, but not all paths, because there are no such values that `a \u003e b` and `a \u003c b`.\n\nLet's take a different example:\n\n```js\nexport function experiment(a, b) {\n  let result = 0;\n  if (a) result += 1;\n  if (b) result += 2;\n  return result;\n}\n```\n\nAll branches can be covered with two tests:\n\n```js\nexpect(experiment(false, false)).toBe(0);\nexpect(experiment(true, true)).toBe(3);\n```\n\nBut to cover all paths you need 2 more tests:\n\n```js\nexpect(experiment(true, false)).toBe(1);\nexpect(experiment(false, true)).toBe(2);\n```\n\nOne more example:\n\n```js\nexport function experiment(a, b, c) {\n  let result = 0;\n  if (a) result += 1;\n  if (b) result += 2;\n  if (c) result += 4;\n  return result;\n}\n```\n\nIt has 6 branches, but 8 paths.\n\n### 100% branch or path coverage may be not enough\n\nLet's take the same example, we started with:\n\n```js\nexport function comp(a, b) {\n  if (a \u003e b) return 1;\n  if (a \u003c b) return -1;\n  return 0;\n}\n```\n\nAnd write 100% test coverage:\n\n```js\nexpect(comp(1, 1)).toBe(0);\nexpect(comp(1, 0)).toBe(1);\nexpect(comp(0, 1)).toBe(-1);\n```\n\nWe still miss edge cases for `NaN`:\n\n```js\n(comp(1, 1) === comp(1, NaN)) === comp(NaN, 1);\n```\n\nWhich may be not a desired behaviour.\n\n## Example 2\n\n```js\nexport function comp(a, b) {\n  let result;\n  if (a === b) result = 0;\n  else if (a \u003e b) result = 1;\n  else result = -1;\n  return result;\n}\n```\n\nThis code has 3 or 4 branches (depending on how you define \"branches\"):\n\n```mermaid\nflowchart LR\n  s(s) --- if1[\"if(a === b)\"]\n  if1 -- true --\u003e e\n  if1 -- false --- if2\n  if2[\"if(a \u003e b)\"] -- true --\u003e e\n  if2 -- false --\u003e e(e)\n```\n\n## Example 3\n\nSo far we talked only about `if/else`. Let's talk about other \"branching\" constructs\n\n```js\na \u0026\u0026 b();\n// is the same as\nif (a) b();\n\na || b();\n// is the same as\nif (!a) b();\n\na ? b() : c():\n// is the same as\nif (!a) b(); else c();\n```\n\nWhich makes sense. But what about this example:\n\n```js\nif (a || b) {\n  console.log(1);\n}\n```\n\nUsing logic above this code can be estimated to have 4 branches. But it seems more natural to count it as 2 branches (4 paths?). WDYT?\n\nWith exceptions if second operand (`b`) is a function call (`b()`) or property accessor (`b.something`), which may be [a getter](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/get).\n\nShall we count code like this:\n\n```js\nlet x = a ? 1 : 2;\n```\n\nas 2 branches or as 1 branch (but 2 paths)?\n\n## Example 4\n\nThis should count as 2 branches:\n\n```js\nswitch (a) {\n  case 1:\n    //...\n    break;\n  default:\n  //...\n}\n```\n\nThis is 2 branches as well:\n\n```js\nswitch (a) {\n  case 1:\n    //...\n    break;\n}\n```\n\nThis is 2 branches as well:\n\n```js\nswitch (a) {\n  case 1:\n  //...\n  default:\n  //...\n}\n```\n\nThis is 3 branches:\n\n```js\nswitch (a) {\n  case 1:\n  //...\n  case 3:\n  //...\n  default:\n  //...\n}\n```\n\nIs this 2 or 3 branches:\n\n```js\nswitch (a) {\n  case 1:\n  case 3:\n  //...\n  default:\n  //...\n}\n```\n\n## Example 5\n\nThis should count as 2 branches (?):\n\n```js\ntry {\n  a(x);\n  b(y);\n  //...\n} catch (e) {\n  //...\n}\n```\n\nBut what if each function (`a`, `b`) can throw an exception. Shall we count it as 3 (or 4) branches? On the other hand there is no way to know this from statical analysis unless we have type system with effects, like in [koka](https://koka-lang.github.io/koka/doc/book.html#why-effects).\n\n## Example 6\n\nThis should count as 2 branches\n\n```js\nfor (let i = 0; i \u003c j; i++) {\n  //\n}\n```\n\nBecause depending on the value of `j` we may or may not \"get inside\" `for` statement. On the other hand - this is 1 branch:\n\n```js\nfor (let i = 0; i \u003c 10; i++) {\n  //\n}\n```\n\nSame argument applies to `while`:\n\n```js\nwhile (j \u003c 3) {\n  //...\n}\n```\n\n`do` always counts as 1 branch\n\n```js\ndo {\n  //...\n} while (j \u003c 3);\n```\n\n## Example 7\n\nDo we count [optional chaining](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Optional_chaining) as branching?\n\n```js\nlet x = a?.something;\n```\n\nIt should be counted the same way as:\n\n```js\nlet x = a == null ? undefined : a.something;\n```\n\nDo we count whole chain as 2 branches or do we add branch for each link:\n\n```js\nlet x = a?.something?.else;\n```\n\nSame goes to [nullish coalescing](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Nullish_coalescing) and [nullish coalescing assignment](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Nullish_coalescing_assignment)\n\n## Example 8\n\nDo we count each `yield` in [generator](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Generator) as branch?\n\n```js\nconst x = function* () {\n  yield \"a\";\n  yield \"b\";\n  yield \"c\";\n};\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstereobooster%2Ftest-coverage-calculation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstereobooster%2Ftest-coverage-calculation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstereobooster%2Ftest-coverage-calculation/lists"}