{"id":28446599,"url":"https://github.com/parttimenerd/temci","last_synced_at":"2025-06-30T01:31:42.820Z","repository":{"id":4037661,"uuid":"50309641","full_name":"parttimenerd/temci","owner":"parttimenerd","description":"An advanced benchmarking tool","archived":false,"fork":false,"pushed_at":"2022-07-07T11:04:19.000Z","size":1817,"stargazers_count":177,"open_issues_count":8,"forks_count":17,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-06-06T11:07:33.112Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://temci.readthedocs.org/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/parttimenerd.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-01-24T21:41:29.000Z","updated_at":"2025-06-02T23:51:02.000Z","dependencies_parsed_at":"2022-09-26T17:40:59.461Z","dependency_job_id":null,"html_url":"https://github.com/parttimenerd/temci","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/parttimenerd/temci","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/parttimenerd%2Ftemci","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/parttimenerd%2Ftemci/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/parttimenerd%2Ftemci/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/parttimenerd%2Ftemci/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/parttimenerd","download_url":"https://codeload.github.com/parttimenerd/temci/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/parttimenerd%2Ftemci/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262693203,"owners_count":23349691,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-06T11:07:33.032Z","updated_at":"2025-06-30T01:31:42.811Z","avatar_url":"https://github.com/parttimenerd.png","language":"Python","readme":".. title:: temci\n\ntemci\n=====\n\n.. image:: https://github.com/parttimenerd/temci/actions/workflows/test.yml/badge.svg\n    :target: https://github.com/parttimenerd/temci/actions/workflows/test.yml\n\n.. image:: https://readthedocs.org/projects/temci/badge/?version=latest\u0026style=plain\n    :target: https://temci.readthedocs.org\n\nAn advanced benchmarking tool written in Python 3 that supports\n`setting up an environment for benchmarking \u003chttps://temci.readthedocs.io/en/latest/temci_exec.html#plugins\u003e`_\nand the generation of `visually appealing reports \u003chttp://mostlynerdless.de/files/report_readme/report.html\u003e`_.\n\nIt runs on Linux systems and (rudimentarily) on macOS.\n\nWhy should you use temci?\n-------------------------\n\ntemci allows you to easily measure the execution time (and other things)\nof programs and compare them against each other resulting in a pretty\nHTML5 based report. Furthermore it can set up the environment to ensure\nbenchmarking results with a low variance. The latter feature can be used\nwithout using temci for benchmarking\nby using `temci short shell \u003chttps://temci.readthedocs.io/en/latest/temci_shell.html\u003e`_.\n\nUsage\n-----\n\nThe main commands of temci are `temci exec \u003chttps://temci.readthedocs.io/en/latest/temci_exec.html\u003e`_ and\n`temci report \u003chttps://temci.readthedocs.io/en/latest/temci_report.html\u003e`_.\n\nSuppose you want to see whether grepping for the strings that consist of ``a`` and ``b`` in the current\nfolder is slower than for strings that consist only of ``a``.\n\nFirst we have to install temci (using `Nix \u003chttps://nixos.org/nix/\u003e`_, see below for more instructions):\n\n.. code:: sh\n\n    nix-env -f https://github.com/parttimenerd/temci/archive/master.tar.gz -i\n\nAfter this, we can benchmark both commands with temci:\n\n.. code:: sh\n\n    # benchmark both commands 20 times\n    temci short exec \"grep '[ab]*' -R .\" \"grep 'a*' -R .\" --runs 10\n\n    # append --watch to get report (in which you can move with the arrow keys and scroll)\n    # after every benchmark completed (use --watch_every to decrease interval)\n    temci short exec \"grep '[ab]*' -R .\" \"grep 'a*' -R .\" --runs 10 --watch\n\n    # if you want to improve the stability your benchmarks, run them with root privileges\n    # the benchmarked programs are run with your current privileges\n    temci short exec \"grep '[ab]*' -R .\" \"grep 'a*' -R .\" --runs 10 --sudo --preset usable\n\nThis results in a ``run_output.yaml`` file that should look like:\n\n.. code:: yaml\n\n    - attributes: {description: 'grep ''[ab]*'' -R .'}\n      data:\n        etime: [0.03, 0.02, 0.02, 0.03, 0.03, 0.03, 0.02, 0.03, 0.03, 0.02]\n        … # other properties\n    - attributes: {description: grep 'a*' -R .}\n      data:\n        etime: [0.02, 0.03, 0.02, 0.03, 0.03, 0.02, 0.03, 0.03, 0.02, 0.02]\n        … # other properties\n    - property_descriptions: {etime: elapsed real (wall clock) time, … }\n\nFor more information on the support measurement tools (like\n`perf stat \u003chttps://temci.readthedocs.io/en/latest/temci_exec.html#perf-stat-runner\u003e`_ and\n`rusage \u003chttps://temci.readthedocs.io/en/latest/temci_exec.html#rusage-runner\u003e`_),\nthe supported `plugins for setting up the environment \u003chttps://temci.readthedocs.io/en/latest/temci_exec.html#plugins\u003e`_\nand more, see `temci exec \u003chttps://temci.readthedocs.io/en/latest/temci_exec.html\u003e`_.\n\nWe can now create a report from these benchmarking results using\n`temci report \u003chttps://temci.readthedocs.io/en/latest/temci_report.html\u003e`_.\nWe use the option ``--properties`` to include only the elapsed time in the\nreport to keep the report simple:\n\n\n.. code:: sh\n\n    \u003e temci report run_output.yaml --properties etime\n    Report for single runs\n    grep '[ab]*' -R .    (   10 single benchmarks)\n         etime mean =     2(6).(000)m, deviation = 18.84223%\n\n    grep 'a*' -R .       (   10 single benchmarks)\n         etime mean =     2(5).(000)m, deviation = 20.00000%\n\n    Equal program blocks\n         grep '[ab]*' -R .  ⟷  grep 'a*' -R .\n             etime confidence =        67%, speed up =      3.85%\n\nWe see that there is no significant difference between the two commands.\n\nThere are multiple reporters besides the default\n`console reporter \u003chttps://temci.readthedocs.io/en/latest/temci_report.html#console\u003e`_.\nAnother reporter is the `html2 reporter \u003chttps://temci.readthedocs.io/en/latest/temci_report.html#html2\u003e`_\nthat produces an HTML report, use it by adding the ``--reporter html2`` option:\n\n.. image:: http://mostlynerdless.de/files/report_readme/html_report.png\n    :target: http://mostlynerdless.de/files/report_readme/report.html\n\nInstallation\n------------\n\nThe simplest way is to use the `Nix package manager \u003chttps://nixos.org/nix/\u003e`_, after installing Nix, run:\n\n.. code:: sh\n\n          nix-env -f https://github.com/parttimenerd/temci/archive/master.tar.gz -i\n\nUsing pip requiring at least Python 3.6:\n\n.. code:: sh\n\n        sudo pip3 install temci\n\nFor more information see the Installation_ page.\n\n\nAuto completion\n~~~~~~~~~~~~~~~\n\nTemci can generate auto completion files for bash and zsh. Add the following line to your `.bashrc` or `.zshrc`:\n\n.. code:: sh\n\n    . `temci_completion $0`\n\n\nUsing temci to set up a benchmarking environment\n------------------------------------------------\nUse the ``temci short shell COMMAND`` to run a command (``sh`` by default) in a shell that is inside\nthe benchmarking environment. Most options of ``temci short exec`` are supported.\nFor more information, see `temci shell \u003chttps://temci.readthedocs.io/en/latest/temci_shell.html\u003e`_.\n\n\nWhy is temci called temci?\n--------------------------\n\nThe problem in naming programs is that most good program names are\nalready taken. A good program or project name has (in my opinion) the\nfollowing properties:\n\n* it shouldn't be used on the relevant platforms (in this case: github and pypi)\n* it should be short (no one wants to type long program names)\n* it should be pronounceable\n* it should have at least something to do with the program\n\ntemci is such a name. It's lojban for time (i.e. the time duration between two moments or events).\n\n\nContributing\n------------\n\n`Bug reports \u003chttps://github.com/parttimenerd/temci/issues\u003e`_ and\n`code contributions \u003chttps://github.com/parttimenerd/temci\u003e`_ are highly appreciated.\n\nFor more information, see the `Contributing \u003chttps://temci.readthedocs.io/en/latest/contributing.html\u003e`_ page.\n\n\n.. _Installation: https://temci.readthedocs.io/en/latest/installation.html\n\n.. _Resources: https://temci.readthedocs.io/en/latest/resources.html\n","funding_links":[],"categories":["\u003ca name=\"programming\"\u003e\u003c/a\u003eProgramming","Other"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparttimenerd%2Ftemci","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fparttimenerd%2Ftemci","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparttimenerd%2Ftemci/lists"}