{"id":23187342,"url":"https://github.com/svenssonjoel/emacs-bencher","last_synced_at":"2025-04-05T05:12:10.423Z","repository":{"id":140042158,"uuid":"138438067","full_name":"svenssonjoel/Emacs-Bencher","owner":"svenssonjoel","description":"Use Emacs to run benchmarks and to collect data","archived":false,"fork":false,"pushed_at":"2018-08-20T20:12:34.000Z","size":194,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-10T13:12:01.950Z","etag":null,"topics":["benchmarking","benchmarking-framework","csv","emacs"],"latest_commit_sha":null,"homepage":"","language":"Emacs Lisp","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/svenssonjoel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-06-23T22:39:00.000Z","updated_at":"2018-08-21T05:24:14.000Z","dependencies_parsed_at":null,"dependency_job_id":"8241105a-1691-4c54-b1e5-3d8670ceebc0","html_url":"https://github.com/svenssonjoel/Emacs-Bencher","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/svenssonjoel%2FEmacs-Bencher","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/svenssonjoel%2FEmacs-Bencher/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/svenssonjoel%2FEmacs-Bencher/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/svenssonjoel%2FEmacs-Bencher/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/svenssonjoel","download_url":"https://codeload.github.com/svenssonjoel/Emacs-Bencher/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247289429,"owners_count":20914464,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmarking","benchmarking-framework","csv","emacs"],"created_at":"2024-12-18T10:19:11.985Z","updated_at":"2025-04-05T05:12:10.414Z","avatar_url":"https://github.com/svenssonjoel.png","language":"Emacs Lisp","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Emacs-Bencher\nEmacs-Bencher automates the running of a program over a space of inputs while\ngathering statistics about for example running time into a csv buffer/file. \n\nLike the Haskell-based [HSBencher](https://hackage.haskell.org/package/hsbencher)\ntool, Emacs-Bencher looks for the occurence of user defined \"tags\" in the\noutput stream of the running program. Tags are of the form \"key: value\" and are\nparsed by Emacs-Bencher and added to the csv result. \n\n# Installation example\n\n1. Clone the repository for example in the .emacs.d directory.\n```\ncd ~/.emacs.d \ngit clone https://github.com/svenssonjoel/Emacs-Bencher.git'\n```\n\n2. Add the following to your .emacs file.\n```\n(add-to-list 'load-path \"~/.emacs.d/Emacs-Bencher/\")\n(require 'bencher)\n``` \n# .bench file syntax example\nIn Emacs-Bencher, one or more benchmarks are specified in a file (or Emacs buffer) as a set of colon separated key value pairs.\nBenchmarks are separated using %%. For some keys the associated value can be expressed using an embedded elisp expression. \n\n```\n%%\nname: test2\ncsv: test2.csv\nruns: 5\nvarying: a '(1 2 3 4 5 6)\nvarying: b '(1 2)\nexecutable: ./bench1 (* a 5) (+ b 10)\ntags: '(\"TAG0\" \"TAG1\") \n%%\n```\n* name: specifies a name for this set of benchmark runs.\n* csv: specifies a name for a csv output buffer (and file, todo).\n* runs: specifies how many times the benchmark should run at each varying setting. \n* varying: specifies a variable and a space for it to range over (an elisp expression evaluating to a list of values) for it to range over.\n* executable: specifies that an executable called \"bench1\" should be executed with two arguments\nthat range over the lists specified as varying. Each combination of one value from the a list and one from the b constitutes a run. Lisp expressions in this string are evaluated and the result of evaluation passed to the executable.\n* tags: specifies a list of names of tags. If the program under benchmarking outputs a string 'TAG: value', value will be parsed and stored in output csv. The separating colon in the benchmark output tag line is required. \n\n# Running a set of benchmarks\nThere are two Emacs-Bencher functions for starting a set of benchmarks.\n\n1. bencher-run-benchmarks-file \"\\path\\to\\file.bench\"\n2. bencher-run-benchmarks-buffer buffer\n\nThese can be run using 'M-x' or 'M-:'\n\n# Extensible through data processing \"plugins\"\n\nAdditional methods of processing the benchmark result data can be added to the bencher-data-processing-plugin-list.\nA \"plugin\" is function taking a list of key-value pairs as input. The example below just outputs the results\nof each benchmark to the Messages buffer. \n\n```\n;; Example of a data processing plugin\n(defun bencher-dummy-echo-to-messages (values)\n   \"dummy\"\n   (message (format \"DUMMY: %s\" values)))\n\n(setq bencher-data-processing-plugin-list\n       (cons #'bencher-dummy-echo-to-messages\n \t     bencher-data-processing-plugin-list))\n```\n\n# Extensible using pre- and post-benchmark \"information harvesters\"\n\nAn information harvester is a function that returns a key-value pair when given\nthe argument nil. If the argument to the harvester is t it should return its key.\nAll harvesters are run for each benchmark run. The key-value pairs returned by harvesters\nare added to the CSV output (and passed to whatever data processing plugins there are).\n\npre- and post-harvesters are added to the following lists:\n```\nbencher-pre-information-harvester-list\nbencher-post-information-harvester-list\n```\n\nExample harvester that grabs machine name:\n```\n(defun bencher-uname-n-information-harvester (just-header)\n  (let ((key \"Node\"))\n    (if just-header\n\tkey\n      (with-temp-buffer\n\t(call-process \"uname\" 'nil (current-buffer) 'nil \"-n\")\n\t(cons key\n\t      (bencher-newline-to-space (buffer-string)))))))\n``` \n\n\n# Example benchmark and csv output\n\n```\n%%\nname: sleep\nruns: 2\nvarying: a  '(1 2 3 4 5 6)\nexecutable: sleep a\n%%\n```\n\n```\nName, Run-id, Arg0, TimeReal, TimeUser, TimeSys, TimeNCS, Date, Time\nsleep, 1, 6, 6.00, 0.00, 0.00, 0, 2018-08-12, 08-12-12\nsleep, 0, 6, 6.00, 0.00, 0.00, 0, 2018-08-12, 08-12-19\nsleep, 1, 5, 5.00, 0.00, 0.00, 0, 2018-08-12, 08-12-26\nsleep, 0, 5, 5.00, 0.00, 0.00, 0, 2018-08-12, 08-12-32\nsleep, 1, 4, 4.00, 0.00, 0.00, 0, 2018-08-12, 08-12-38\nsleep, 0, 4, 4.00, 0.00, 0.00, 0, 2018-08-12, 08-12-43\nsleep, 1, 3, 3.00, 0.00, 0.00, 0, 2018-08-12, 08-12-48\nsleep, 0, 3, 3.00, 0.00, 0.00, 0, 2018-08-12, 08-12-52\nsleep, 1, 2, 2.00, 0.00, 0.00, 0, 2018-08-12, 08-12-56\nsleep, 0, 2, 2.00, 0.00, 0.00, 0, 2018-08-12, 08-12-59\nsleep, 1, 1, 1.00, 0.00, 0.00, 0, 2018-08-12, 08-13-02\nsleep, 0, 1, 1.00, 0.00, 0.00, 0, 2018-08-12, 08-13-04\n```\n\n# TODO\n  * Learn to code LISP.\n  * BUG: Reset does not seem to \"reset\" enough after aborting an ongoing benching session (by reload of file)\n  * Add error checking to many places. \n  * Log output buffers.\n  * Makefile projects and compile-time argument support.\n  * Editing mode for .bench files (with run-benchmark-closest-to-cursor function and run-all-benchmarks function).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsvenssonjoel%2Femacs-bencher","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsvenssonjoel%2Femacs-bencher","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsvenssonjoel%2Femacs-bencher/lists"}