{"id":19963239,"url":"https://github.com/bilbottom/db-query-profiler","last_synced_at":"2025-05-03T22:31:53.221Z","repository":{"id":164868023,"uuid":"636437165","full_name":"Bilbottom/db-query-profiler","owner":"Bilbottom","description":"Lightweight database query profiler.","archived":false,"fork":false,"pushed_at":"2024-07-23T08:07:45.000Z","size":227,"stargazers_count":2,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-09-23T10:47:50.431Z","etag":null,"topics":["python","sql"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Bilbottom.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-04T21:06:28.000Z","updated_at":"2024-07-23T08:07:47.000Z","dependencies_parsed_at":"2023-10-03T12:07:33.004Z","dependency_job_id":"ca5be6a0-5728-4565-b558-04fa389afe13","html_url":"https://github.com/Bilbottom/db-query-profiler","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bilbottom%2Fdb-query-profiler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bilbottom%2Fdb-query-profiler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bilbottom%2Fdb-query-profiler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bilbottom%2Fdb-query-profiler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Bilbottom","download_url":"https://codeload.github.com/Bilbottom/db-query-profiler/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224374655,"owners_count":17300691,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["python","sql"],"created_at":"2024-11-13T02:15:21.399Z","updated_at":"2025-05-03T22:31:53.208Z","avatar_url":"https://github.com/Bilbottom.png","language":"Python","readme":"\u003cdiv align=\"center\"\u003e\n\n[![Python](https://img.shields.io/badge/Python-3.8+-blue.svg)](https://www.python.org/downloads/)\n[![uv](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json)](https://github.com/astral-sh/uv)\n[![tests](https://github.com/Bilbottom/db-query-profiler/actions/workflows/tests.yaml/badge.svg)](https://github.com/Bilbottom/db-query-profiler/actions/workflows/tests.yaml)\n[![coverage](https://raw.githubusercontent.com/Bilbottom/db-query-profiler/main/coverage.svg)](https://github.com/dbrgn/coverage-badge)\n![GitHub last commit](https://img.shields.io/github/last-commit/Bilbottom/db-query-profiler)\n\n[![code style: prettier](https://img.shields.io/badge/code_style-prettier-ff69b4.svg?style=flat-square)](https://github.com/prettier/prettier)\n[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/Bilbottom/db-query-profiler/main.svg)](https://results.pre-commit.ci/latest/github/Bilbottom/db-query-profiler/main)\n\n\u003c/div\u003e\n\n---\n\n# Database Query Profiler 🗃️⏱️\n\nLightweight database query profiler.\n\nThis tool is database-agnostic -- just provide a class that connects to your database with an `execute` method, and the queries that you want to profile.\n\n\u003e [!WARNING]\n\u003e\n\u003e **_This is NOT a replacement for analysing the [query plan](https://en.wikipedia.org/wiki/Query_plan). This should just support the analysis done with it._**\n\n## Installation ⬇️\n\nGrab a copy from PyPI like usual:\n\n```\npip install db-query-profiler\n```\n\nIf you'd prefer, you can install from source:\n\n```\npip install git+https://github.com/Bilbottom/db-query-profiler.git@main\n```\n\n## Sample Output 📝\n\nGiven a set of queries (details below), this package prints the average time in seconds taken to run each query, as well as the percentage of the total time taken by each query.\n\nThe [`tqdm`](https://github.com/tqdm/tqdm) package is used to show progress of the queries being run.\n\nA typical output will look something like this:\n\n```\nStart time: 2023-05-07 12:38:06.879738\n----------------------------------------\n100%|██████████| 5/5 [00:01\u003c00:00,  3.29it/s]\nquery-1.sql: 0.10063192s (33.4%)\nquery-2.sql: 0.20044784s (66.6%)\n----------------------------------------\nEnd time: 2023-05-07 12:38:08.757555\n```\n\n## Usage 📖\n\nThe package exposes a single function, `time_queries`, which currently requires:\n\n1. A database connection/cursor class that implements an `execute` method.\n2. The number of times to re-run each query.\n3. A directory containing the SQL files with the queries to run.\n\nThere should only be a single query in each file, and the file name will be used as the query name in the output.\n\nFor the following examples, assume that there are SQL files in the `queries` directory.\n\n### SQLite Example\n\n\u003e Official documentation: https://docs.python.org/3/library/sqlite3.html\n\n```python\nimport sqlite3\n\nimport db_query_profiler\n\n\ndef main() -\u003e None:\n    db_conn = sqlite3.connect(\":memory:\")  # Or a path to a database file\n    db_query_profiler.time_queries(\n        conn=db_conn,\n        repeat=5,\n        directory=\"queries\"\n    )\n\n\nif __name__ == \"__main__\":\n    main()\n```\n\n### Snowflake Example\n\n\u003e Official documentation: https://docs.snowflake.com/en/developer-guide/python-connector/python-connector-example\n\nSome databases, like Snowflake, have [extra layers of caching](https://docs.snowflake.com/en/user-guide/querying-persisted-results) that can affect the results of the profiling. To avoid this and make the runtime comparisons more genuine, it's recommended to turn off these extra caching options (where this is supported).\n\n```python\nimport db_query_profiler\nimport snowflake.connector  # snowflake-connector-python\n\n\n# This dictionary is just for illustration purposes and\n# you should use whatever connection method you prefer\nCREDENTIALS = {\n    \"user\": \"XXX\",\n    \"password\": \"XXX\",\n    \"account\": \"XXX\",\n    \"warehouse\": \"XXX\",\n    \"role\": \"XXX\",\n    \"database\": \"XXX\",\n}\n\n\ndef main() -\u003e None:\n    db_conn = snowflake.connector.SnowflakeConnection(**CREDENTIALS)\n    with db_conn.cursor() as cursor:\n        cursor.execute(\"\"\"ALTER SESSION SET USE_CACHED_RESULT = FALSE;\"\"\")\n        db_query_profiler.time_queries(\n            conn=cursor,\n            repeat=5,\n            directory=\"queries\",\n        )\n        cursor.execute(\"\"\"ALTER SESSION SET USE_CACHED_RESULT = TRUE;\"\"\")\n    db_conn.close()\n\n\nif __name__ == \"__main__\":\n    main()\n```\n\n## Warnings ⚠️\n\nThis package will open and run all the files in the specified directory, so be careful about what you put in there -- potentially unsafe SQL commands could be run.\n\nThis package only reads from the database, so it's encouraged to configure your database connection in a read-only way.\n\n### SQLite\n\n\u003e Official documentation:\n\u003e\n\u003e - https://docs.python.org/3/library/sqlite3.html#sqlite3.connect\n\u003e - https://docs.python.org/3/library/sqlite3.html#how-to-work-with-sqlite-uris\n\nTo connect to a SQLite database in a read-only way, use the `uri=True` parameter with `file:` and `?mode=ro` surrounding the database path when connecting:\n\n```python\ndb_conn = sqlite3.connect(\"file:path/to/database.db?mode=ro\", uri=True)\n```\n\n## Contributing 🤝\n\nThe Python packaging is managed with [uv](https://github.com/astral-sh/uv), but that should be the only dependency.\n\nTo get started, just clone the repo, install the dependencies, and enable [pre-commit](https://pre-commit.com/):\n\n```bash\nuv sync --all-groups\npre-commit install --install-hooks\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbilbottom%2Fdb-query-profiler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbilbottom%2Fdb-query-profiler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbilbottom%2Fdb-query-profiler/lists"}