{"id":14972577,"url":"https://github.com/django/djangobench","last_synced_at":"2025-04-07T15:10:24.258Z","repository":{"id":949634,"uuid":"729602","full_name":"django/djangobench","owner":"django","description":"Harness and benchmarks for evaluating Django's performance over time","archived":false,"fork":false,"pushed_at":"2022-11-09T21:03:39.000Z","size":246,"stargazers_count":210,"open_issues_count":15,"forks_count":57,"subscribers_count":28,"default_branch":"master","last_synced_at":"2025-03-31T13:18:42.948Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/django.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2010-06-19T19:01:45.000Z","updated_at":"2025-03-21T05:01:39.000Z","dependencies_parsed_at":"2023-01-11T15:48:40.784Z","dependency_job_id":null,"html_url":"https://github.com/django/djangobench","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/django%2Fdjangobench","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/django%2Fdjangobench/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/django%2Fdjangobench/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/django%2Fdjangobench/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/django","download_url":"https://codeload.github.com/django/djangobench/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247675607,"owners_count":20977378,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-24T13:47:09.289Z","updated_at":"2025-04-07T15:10:24.229Z","avatar_url":"https://github.com/django.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Djangobench\n===========\n\nA harness and a set of benchmarks for measuring Django's performance over\ntime.\n\nRunning the benchmarks\n----------------------\n\nHere's the short version::\n\n    mkvirtualenv djangobench\n    pip install -e git://github.com/django/djangobench.git#egg=djangobench\n    git clone git://github.com/django/django.git\n    cd django\n    djangobench --control=1.2 --experiment=master\n\nOkay, so what the heck's going on here?\n\nFirst, ``djangobench`` doesn't test a single Django version in isolation --\nthat wouldn't be very useful. Instead, it benchmarks an \"experiment\" Django\nagainst a \"control\", reporting on the difference between the two and\nmeasuring for statistical significance.\n\nBecause a Git clone can contain all the project development history, you can\ntest against a single repository specifying individual commit IDs, tag (as we've\ndone above) and even possibly branches names with the ``--control`` and\n``--experiment`` options.\n\nBefore ``djangobench`` 0.10 you had to use ``--vcs=git`` to get this behavior.\nNow it's the default. There is also support for Mercurial (``--vcs=hg``).\n\nAnother way to use ``djangobench``, is to run it against two complete Django\nsource trees, you can specify this mode by using ``--vcs=none``. By default it\nlooks for directories named ``django-control`` and ``django-experiment`` in the\ncurrent working directory::\n\n    djangobench --vcs=none\n\nbut you can change that by using the ``--control`` or ``--experiment`` options::\n\n    djangobench --vcs=none --control pristine --experiment work\n\nNow, it's impractical to install the Django source code trees under test (this\nis particularly true in the two-trees scenario): ``djangobench`` works its magic\nby mucking with ``PYTHONPATH``.\n\nHowever, the benchmarks themselves need access to the ``djangobench`` module, so\nyou'll need to install it.\n\nYou can specify the benchmarks to run by passing their names on the command\nline.\n\nThis is an example of not-statistically-significant results::\n\n    Running 'startup' benchmark ...\n    Min: 0.138701 -\u003e 0.138900: 1.0014x slower\n    Avg: 0.139009 -\u003e 0.139378: 1.0027x slower\n    Not significant\n    Stddev: 0.00044 -\u003e 0.00046: 1.0382x larger\n\nPython 3\n~~~~~~~~\n\nNot only is ``djangobench`` Python 3 compatible, but can also be used to\ncompare Python 2 vs Python 3 code paths. To do this, you need to provide the\nfull paths to the corresponding Python executables in ``--control-python`` and\n``--experiment-python``. The short version (assuming you have also the\n``djangobench`` environment setup like above)::\n\n    mkvirtualenv djangobench-py3 -p python3\n    pip install -e git://github.com/django/djangobench.git#egg=djangobench\n    cd django\n    djangobench --vcs=none --control=. --experiment=. \\\n        --control-python=~/.virtualenvs/djangobench/bin/python \\\n        --experiment-python=~/.virtualenvs/djangobench-py3/bin/python \\\n\nWriting new benchmarks\n----------------------\n\nBenchmarks are very simple: they're a Django app, along with a settings\nfile, and an executable ``benchmarks.py`` that gets run by the harness. The\nbenchmark script needs to honor a simple contract:\n\n* It's an executable Python script, run as ``__main__`` (e.g. ``python\n  path/to/benchmark.py``). The subshell environment will have\n  ``PYTHONPATH`` set up to point to the correct Django; it'll also have\n  ``DJANGO_SETTINGS_MODULE`` set to ``\u003cbenchmark_dir\u003e.settings``.\n\n* The benchmark script needs to accept a ``--trials`` argument giving\n  the number of trials to run.\n\n* The output should be simple RFC 822-ish text -- a set of headers,\n  followed by data points::\n\n        Title: some benchmark\n        Description: whatever the benchmark does\n\n        1.002\n        1.003\n        ...\n\n  The list of headers is TBD.\n\nThere's a couple of utility functions in ``djangobench.utils`` that assist\nwith honoring this contract; see those functions' docstrings for details.\n\nThe existing benchmarks should be pretty easy to read for inspiration. The\n``query_delete`` benchmark is probably a good place to start.\n\n**Please write new benchmarks and send us pull requests on Github!**\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdjango%2Fdjangobench","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdjango%2Fdjangobench","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdjango%2Fdjangobench/lists"}