{"id":21312036,"url":"https://github.com/0xade1a1de/measuresuite","last_synced_at":"2025-07-11T23:33:07.991Z","repository":{"id":50457108,"uuid":"515940529","full_name":"0xADE1A1DE/MeasureSuite","owner":"0xADE1A1DE","description":"This library measures the execution time for code. Can measure asm (with Assemblyline), o, so, bin files. Can check correctness (equality of all functions on output data) and the output is a JSON with robust cycle counts.","archived":false,"fork":false,"pushed_at":"2023-12-18T17:49:55.000Z","size":1531,"stargazers_count":8,"open_issues_count":5,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-11-20T19:41:33.064Z","etag":null,"topics":["cycles","measurement","pmc","time","typescript"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/0xADE1A1DE.png","metadata":{"files":{"readme":"Readme.md","changelog":"Changelog","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-07-20T10:43:36.000Z","updated_at":"2024-04-22T16:15:51.000Z","dependencies_parsed_at":"2024-11-10T07:31:30.941Z","dependency_job_id":"aa59ce39-be48-4c02-bdf6-2db627513e3d","html_url":"https://github.com/0xADE1A1DE/MeasureSuite","commit_stats":null,"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0xADE1A1DE%2FMeasureSuite","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0xADE1A1DE%2FMeasureSuite/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0xADE1A1DE%2FMeasureSuite/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0xADE1A1DE%2FMeasureSuite/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/0xADE1A1DE","download_url":"https://codeload.github.com/0xADE1A1DE/MeasureSuite/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225770694,"owners_count":17521517,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cycles","measurement","pmc","time","typescript"],"created_at":"2024-11-21T17:23:32.880Z","updated_at":"2024-11-21T17:23:33.333Z","avatar_url":"https://github.com/0xADE1A1DE.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MeasureSuite\n![Bin Tests](https://github.com/0xADE1A1DE/MeasureSuite/actions/workflows/bin-check.yml/badge.svg)\n![Lib Tests (no AssemblyLine)](https://github.com/0xADE1A1DE/MeasureSuite/actions/workflows/lib-check-noal.yml/badge.svg)\n![Lib Tests](https://github.com/0xADE1A1DE/MeasureSuite/actions/workflows/lib-check.yml/badge.svg)\n![TS Tests](https://github.com/0xADE1A1DE/MeasureSuite/actions/workflows/ts-check.yml/badge.svg)\n![C Code Style](https://github.com/0xADE1A1DE/MeasureSuite/actions/workflows/lib-formatting.yml/badge.svg)\n![TS Code Style](https://github.com/0xADE1A1DE/MeasureSuite/actions/workflows/ts-formatting.yml/badge.svg)\n![Version](https://img.shields.io/npm/v/measuresuite?logo=github\u0026style=flat)\n\nThis library measures the execution time of code.\n\n## Features\n- You can load `.o`, `.so`, `.bin`, `.asm` files and out comes a `JSON` with cycle counts. \n- Runs all loaded programs in a random order with random inputs.\n- Can check if the results matches the results of a the other loaded methods.\n- C-interface\n- TypeScript-interface\n- CLI tool `ms`, takes files in and outputs `JSON` measurements\n- CLI script `msc`, takes files in, reports relative performance toward first file.\n- Supports functions of the C-like type `void A(uint64_t *out, const uint64_t *in)` (up to six parameters)\n- It assembles assembly code using [AssemblyLine](https://github.com/0xADE1A1DE/AssemblyLine)\n- Reports chunk size counting. (i.e. How many instructions of a function beaks a chunk boundary, when assembling assemblyfiles with AssemblyLine)\n- Returns a JSON string with the measurement metrics.\n- Uses Performance Counters (PMC), falls back to `RDTSC` if PMC are unavailable.\n- Installable with `npm i measuresuite`\n\n## Repo Contents\n\nThis repository contains\n- the C-library `libmeasuresuite` in [lib](./lib)\n- a cli tool `ms` in [bin](./bin); use like `./ms ./fileA.asm ./fileB.o`, out comes `JSON`.\n- a cli script `msc` in [bin](./bin); use `./msc --check base.asm change_a.asm change_b.asm`\n- a TypeScript-Wrapper in [ts](./ts), around `libmeasuresuite` built with `node-gyp`.\n\n*Build*-instructions in [Build.md](./Build.md).\n\n## Examples\n\nFind full `C` and `TS` examples in the [examples](./examples) directory. Some need AssemblyLine installed.\n\n\n### Sneak Peak C-lib\n\n```c\n#include \u003cmeasuresuite.h\u003e\n\n/*\n * void add_two_numbers(uint64_t *out0, const uint64_t *in0, const uint64_t *in1) {\n *    *out0 = *in0 + *in1;\n * }\n */\nchar add_two_asm[] = {\"mov rax, [rsi]\\n\"\n                      \"add rax, [rdx]\\n\"\n                      \"mov [rdi], rax\\n\"\n                      \"ret\\n\"};\n\n\n  const int arg_width = 1;\n  const int arg_num_out = 1;\n  const int arg_num_in = 2;\n\n  // our measuresuite handle\n  measuresuite_t ms = NULL;\n\n  // initializing the measuresuite\n  ms_initialize(\u0026ms, arg_width, arg_num_in, arg_num_out);\n\n  int id = -1;\n  ms_load_data(ms,                     // handle\n               ASM,                    // type of input data is assembly\n               (uint8_t *)add_two_asm, // pointer to input data\n               strlen(add_two_asm),    // length of input data\n               NULL, // symbol (ignored for BIN/ASM, optional for ELF, required\n                     // for SHARED_OBJECT)\n               \u0026id); // ID (in/out)\n\n  const int number_of_batches = 10; // 10 batches of\n  const int batch_size = 100;       // 100 iterations of the function-unter-test (add_two_asm), each\n\n  ms_measure(ms, batch_size, number_of_batches);\n\n  const char *json = NULL;\n  size_t len = 0;\n  ms_get_json(ms, \u0026json, \u0026len);\n\n  assert(json != NULL);\n  assert(len != 0);\n\n  printf(\"%s\\n\", json);\n  // prints:\n  //\n  // {\n  // \"stats\": {\n  //   \"numFunctions\": 1,\n  //   \"runtime\": 0,\n  //   \"incorrect\": 0\n  // },\n  // \"functions\": [ { \"type\": \"ASM\", \"chunks\": 0 } ],\n  // \"cycles\": [ [ 1352, 890, 895, 884, 888, 886, 886, 886, 884, 888 ] ]\n  // }\n\n\n  ms_terminate(ms);\n  return 0;\n}\n```\n\n### Sneak Peak CLI\n\n\n```bash\n$ make ms\n$ make -B -C ./examples/elf add_two_numbers.o\n$ ./ms -n 3 ./examples/elf/add_two_numbers.o ./examples/elf/add_two_numbers.asm | jq\n``` \n\n```json\n{\n  \"stats\": {\n    \"numFunctions\": 2,\n    \"runtime\": 1,\n    \"incorrect\": 0\n  },\n  \"functions\": [\n    {\n      \"type\": \"ELF\"\n    },\n    {\n      \"type\": \"ASM\",\n      \"chunks\": 0\n    }\n  ],\n  \"cycles\": [\n    [ 7074, 7705, 7358 ],\n    [ 7640, 7259, 7858 ]\n  ]\n}\n\n```\n\n### Sneak Peak CLI MSC\n\n\n```bash\n$ make ms\n$ make -B -C ./examples/elf add_two_numbers.o \n$ ./bin/msc ./examples/elf/add_two_numbers.asm ./examples/elf/add_two_numbers_very_slow.asm  ./examples/elf/add_two_numbers.o\n``` \n```\nCycles (#3): 2941 3027 2941\n    2941 / 2941 =   1.0000 (./examples/elf/add_two_numbers.asm)\n    3027 / 2941 =   1.0292 (./examples/elf/add_two_numbers_very_slow.asm)\n    2941 / 2941 =   1.0000 (./examples/elf/add_two_numbers.o)\n```\n\n### Sneak Peak TypeScript\n\n```ts\nimport { type MeasureResult, Measuresuite } from \"measuresuite\";\n\n/**\n * our assembly strings to measure, same as in ./add_two_numbers.c\n */\n\nconst functionA = [\n  \"mov rax, [rsi]\",\n  \"add rax, [rdx]\",\n  \"mov [rdi], rax\",\n  \"ret\",\n].join(\"\\n\");\n\nconst functionB = [\n  \"mov rax, [rsi]\",\n  \"push rbx\",\n  \"mov rbx, [rdx]\",\n  \"lea rax, [rbx + rax]\",\n  \"mov [rdi], rax\",\n  \"pop rbx\",\n  \"ret\",\n].join(\"\\n\");\n\nconst arg_width = 1;\nconst arg_num_out = 1;\nconst arg_num_in = 2;\n\n// create a Measuresuite object\nconst ms = new Measuresuite(arg_width, arg_num_in, arg_num_out);\nms.enableChecking();\n\nconst nob = 50;\nconst batchSize = 10000;\n\n// measure\nconst measurementResult: MeasureResult | null = ms.measure(batchSize, nob, [ functionA, functionB ]);\n\n\n\nconsole.log( `The Measurement took ${measurementResult.stats.runtime}ms to complete`);\n\nconst [functionARes, functionBRes] = measurementResult.cycles;\nconsole.log(`Function A's cycles: ${functionARes.join(\",\")}`);\nconsole.log(`Function B's cycles: ${functionBRes.join(\",\")}`);\n\nconst medianA = functionARes.sort().at(Math.floor(nob / 2));\nconst medianB = functionBRes.sort().at(Math.floor(nob / 2));\n\nconsole.log(`Function A's median: ${medianA}`);\nconsole.log(`Function B's median: ${medianB}`);\n\nconsole.log(\n  `Function A is ${medianA \u003c medianB ? \"\" : \"not\"} faster than Function B`\n);\n```\n\n## Acknowledgements\n#### This project was supported by:  \n* The Air Force Office of Scientific Research (AFOSR) under award number FA9550-20-1-0425\n* An ARC Discovery Early Career Researcher Award (project number DE200101577) \n* An ARC Discovery Project (project number DP210102670)  \n* The Blavatnik ICRC at Tel-Aviv University  \n* the Defense Advanced Research Projects Agency (DARPA) and Air Force Research Laboratory (AFRL) under contracts FA8750-19-C-0531 and HR001120C0087\n* the National Science Foundation under grant CNS-1954712\n* Gifts from AMD, Google, and Intel  \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F0xade1a1de%2Fmeasuresuite","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F0xade1a1de%2Fmeasuresuite","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F0xade1a1de%2Fmeasuresuite/lists"}