{"id":13708319,"url":"https://github.com/sophilabs/django-test-query-counter","last_synced_at":"2025-08-14T21:31:34.688Z","repository":{"id":145428392,"uuid":"97185362","full_name":"sophilabs/django-test-query-counter","owner":"sophilabs","description":"A Django toolkit for controlling query count when testing.","archived":false,"fork":false,"pushed_at":"2017-12-14T12:27:10.000Z","size":123,"stargazers_count":20,"open_issues_count":0,"forks_count":1,"subscribers_count":17,"default_branch":"master","last_synced_at":"2024-12-08T05:03:09.497Z","etag":null,"topics":["django","django-testing"],"latest_commit_sha":null,"homepage":"https://sophilabs.co/open-source","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sophilabs.png","metadata":{"files":{"readme":"README.rst","changelog":"HISTORY.rst","contributing":"CONTRIBUTING.rst","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS.rst","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-07-14T02:41:51.000Z","updated_at":"2024-10-30T02:46:58.000Z","dependencies_parsed_at":"2023-09-18T11:06:16.277Z","dependency_job_id":null,"html_url":"https://github.com/sophilabs/django-test-query-counter","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sophilabs%2Fdjango-test-query-counter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sophilabs%2Fdjango-test-query-counter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sophilabs%2Fdjango-test-query-counter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sophilabs%2Fdjango-test-query-counter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sophilabs","download_url":"https://codeload.github.com/sophilabs/django-test-query-counter/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":229678800,"owners_count":18106299,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["django","django-testing"],"created_at":"2024-08-02T23:00:20.370Z","updated_at":"2024-12-15T19:32:13.345Z","avatar_url":"https://github.com/sophilabs.png","language":"Python","funding_links":[],"categories":["Database"],"sub_categories":["Tools"],"readme":"=============================\nDjango Test Query Counter\n=============================\n\n.. image:: https://badge.fury.io/py/django-test-query-counter.svg\n    :target: https://badge.fury.io/py/django-test-query-counter\n\n.. image:: https://travis-ci.org/sophilabs/django-test-query-counter.svg?branch=master\n    :target: https://travis-ci.org/sophilabs/django-test-query-counter\n\n.. image:: https://codecov.io/gh/sophilabs/django-test-query-counter/branch/master/graph/badge.svg\n    :target: https://codecov.io/gh/sophilabs/django-test-query-counter\n\n\n.. image:: logo.png\n   :width: 200\n   :alt: Logo\n   :align: center\n\nA Django Toolkit for controlling Query count when testing. It controls the\nnumber of queries done in the tests stays below a particular threshold between\nTest Runs.Specially useful when paired with a CI like Jenkins or Travis to\ncontrol each commit doesn't slow down the Database considerably.\n\nRequirements\n------------\n\n* Python 3\n* Django\n\nDocumentation\n-------------\n\nThe full documentation is at https://django-test-query-counter.readthedocs.io.\n\nInstallation\n-------------\n\nThere are ways to install it into your project\n\nClone this repository into your project:\n\n.. code-block:: bash\n\n    git clone https://github.com/sophilabs/django-test-query-counter.git\n\nDownload the zip file and unpack it:\n\n.. code-block:: bash\n\n    wget https://github.com/sophilabs/django-test-query-counter/archive/master.zip\n    unzip master.zip\n\nInstall with pip:\n\n.. code-block:: bash\n\n    pip install django-test-query-counter\n\nAdd it to your `INSTALLED_APPS`:\n\n.. code-block:: python\n\n    INSTALLED_APPS = (\n        ...\n        'test_query_counter',\n        ...\n    )\n\nUsage\n-----\n\nRun your django tests as you do. After the run the\n``reports`` directory with two files ``query_count.json`` and\n``query_count_detail.json``.\n\nTo check your tests Query Counts run:\n\n``$ python manage.py check_query_count``\n\nThen you would see an output like the following one (if you at least ran the\ntests twice):\n\n.. code-block::\n\n    All Tests API Queries are below the allowed threshold.\n\nIf the current test run had more queries than the last one, it will tell you:\n\n.. code-block::\n\n    There are test cases with API calls that exceeded threshold:\n\n        In test case app.organizations.tests.api.test_division_api.DivisionViewTest.test_bulk_create_division, POST /api/divisions. Expected at most 11 queries but got 15 queries\n        In test case app.shipments.tests.api.test_leg_api.LegViewTest.test_create_bulk_leg, POST /api/legs. Expected at most 59 queries but got 62 queries\n        In test case app.plans.tests.functional.test_plan_api.PlannedDatesTest.test_unassign_and_assign_driver_to_leg, POST /api/assignments/assign-driver. Expected at most 261 queries but got 402 queries\n    CommandError: There was at least one test with an API call excedding the allowed threshold.\n\nConfiguration\n-------------\n\nYou can customize how the Test Query Count works by defining ``TEST_QUERY_COUNTER``\nin your settings file as a dictionary. The following code shows an example\n\n.. code-block:: python\n\n    TEST_QUERY_COUNTER = {\n        # Global switch\n        'ENABLE': True,\n\n        # Paths where the count files are generated (relative to the current\n        # process dir)\n        'DETAIL_PATH': 'reports/query_count_detail.json',\n        'SUMMARY_PATH': 'reports/query_count.json',\n\n        # Tolerated percentage of count increase on successive\n        # test runs.A value of 0 prevents increasing queries altoghether.\n        'INCREASE_THRESHOLD': 10\n    }\n\n\nExcluding Tests\n---------------\n\nIndividual tests or classes can be excluded for the count using the\n``@exclude_query_count`` decorator. For example\n\n.. code-block:: python\n\n        # To exclude all methods in the class\n        @exclude_query_count()\n        class AllTestExcluded(TestCase):\n            def test_foo(self):\n                self.client.get('path-1')\n\n            def test_foo(self):\n                self.client.get('path-2')\n\n        # To exclude one test only\n        class OneMethodExcluded():\n            def test_foo(self):\n                self.client.get('path-1')\n\n            @exclude_query_count()\n            def test_foo(self):\n                self.client.get('path-2')\n\n\nMore specifically, ``exclude_query_count`` accept parameters to conditionally\nexclude a query count by path, method or count. Where ``path`` the or regex of\nthe excluded path(s). The ``method`` specifies the regex of the method(s) to\nexclude, and ``count`` is minimum number of queries tolerated. Requests with\nless or same amount as \"count\" will be excluded.\n\nFor example:\n\n.. code-block:: python\n\n    class Test(TestCase):\n        @exclude_query_count(path='url-2')\n        def test_exclude_path(self):\n            self.client.get('/url-1')\n            self.client.post('/url-2')\n\n        @exclude_query_count(method='post')\n        def test_exclude_method(self):\n            self.client.get('/url-1')\n            self.client.post('/url-2')\n\n        @exclude_query_count(count=2)\n        def test_exclude_count(self):\n            #  succesive urls requests are additive\n            for i in range(3):\n                self.client.get('/url-1')\n\nImplementing into your CI\n-------------------------\n\nCurrently the query count works locally. However, it shines when it is\nintegrated with `Jenkins \u003chttps://jenkins.io/\u003e`_, or other CIs. You have to do\nthis manually:\n\nRequirements\n\n* Jenkins with a Job that build you project.\n\nFrom now on let's suppose your job is available at ``http://127.0.0.1:8080/job/ZAP_EXAMPLE_JOB/``.\n\n\n1. `Activate Build Archive`: Go to the job `configuration page \u003chttps://wiki.jenkins.io/display/JENKINS/Configure+the+Job\u003e`_ and add the `archive artifacts \u003chttps://jenkins.io/doc/pipeline/tour/tests-and-artifacts/\u003e`_ build\n   Post-build actions.\n\n    .. image:: archive-artifact.png\n       :alt: Jenkins Post Build Action showing an archive artifact example\n       :align: center\n\n2. Set ``reports/query_count.json`` in the files to archive path\n\n3. Create a new Django custom Command to run the validation against the\n   archived Jenkins file. For example:\n\n    .. code-block:: python\n\n        from urllib.request import urlretrieve\n        from django.core.management import BaseCommand, CommandError\n        from django.conf import settings\n        from test_query_counter.apps import RequestQueryCountConfig\n        from test_query_counter.query_count import QueryCountEvaluator\n\n\n        class Command(BaseCommand):\n            JENKINS_QUERY_COUNT = 'https://yourci/job/' \\\n                                  'yourproject/lastSuccessfulBuild/' \\\n                                  'artifact/app/reports/query_count.json'\n\n            def handle(self, *args, **options):\n                current_file_path = RequestQueryCountConfig.get_setting('SUMMARY_FILE')\n\n                with open(current_file_path) as current_file, \\\n                        open(urlretrieve(self.JENKINS_QUERY_COUNT)[0]) as last_file:\n                    violations = QueryCountEvaluator(10, current_file,last_file).run()\n\n                    if violations:\n                        raise CommandError('There was at least one test with an API '\n                                           'call excedding the allowed threshold.')\n\n4. Add a build step to run this command:\n\n    .. image:: build-action.png\n       :alt: Jenkins Build Action showing the script action\n       :align: center\n\n   After that, it will run fine, and the build would fail if any Query count is over the limit.\n\nTODO\n----\n\n* Include support for stacktraces in ``query_count_detail.json``.\n* Generate an HTML report of executed Queries.\n* Make query count configurable\n* Include Jenkins support out of the box (using `django_jenkins`)\n\nRunning Tests\n-------------\n\nDoes the code actually work?\n\n.. code-block:: bash\n\n    source \u003cYOURVIRTUALENV\u003e/bin/activate\n    (myenv) $ pip install tox\n    (myenv) $ tox\n\nLicense\n-------\n\nDjango Test Query Counter is MIT Licensed. Copyright (c) 2017 Sophilabs, Inc.\n\n\nCredits\n-------\n\n.. image:: https://s3.amazonaws.com/sophilabs-assets/logo/logo_300x66.gif\n    :target: https://sophilabs.co\n\nThis tool is maintained and funded by Sophilabs, Inc. The names and logos for\nsophilabs are trademarks of sophilabs, inc.\n\nTools used in rendering this package:\n\n*  Cookiecutter_\n*  `cookiecutter-djangopackage`_\n\n.. _Cookiecutter: https://github.com/audreyr/cookiecutter\n.. _`cookiecutter-djangopackage`: https://github.com/pydanny/cookiecutter-djangopackage\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsophilabs%2Fdjango-test-query-counter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsophilabs%2Fdjango-test-query-counter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsophilabs%2Fdjango-test-query-counter/lists"}