{"id":20238492,"url":"https://github.com/marcinn/django-sqltemplate","last_synced_at":"2025-06-28T00:04:43.783Z","repository":{"id":57422129,"uuid":"55814924","full_name":"marcinn/django-sqltemplate","owner":"marcinn","description":"Database query tool for Django, based on SQL templates","archived":false,"fork":false,"pushed_at":"2019-12-22T00:11:38.000Z","size":41,"stargazers_count":5,"open_issues_count":3,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-23T06:44:48.774Z","etag":null,"topics":["django","python","python3","query-database","sql-template"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/marcinn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-04-08T23:07:42.000Z","updated_at":"2020-05-26T15:19:05.000Z","dependencies_parsed_at":"2022-09-26T16:31:21.244Z","dependency_job_id":null,"html_url":"https://github.com/marcinn/django-sqltemplate","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/marcinn/django-sqltemplate","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcinn%2Fdjango-sqltemplate","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcinn%2Fdjango-sqltemplate/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcinn%2Fdjango-sqltemplate/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcinn%2Fdjango-sqltemplate/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/marcinn","download_url":"https://codeload.github.com/marcinn/django-sqltemplate/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcinn%2Fdjango-sqltemplate/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262352619,"owners_count":23297689,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["django","python","python3","query-database","sql-template"],"created_at":"2024-11-14T08:34:28.222Z","updated_at":"2025-06-28T00:04:43.756Z","avatar_url":"https://github.com/marcinn.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# django-sqltemplate\nDatabase query tool for Django, based on SQL templates\n\n## Development status\n\n* Alpha\n* API may be radically changed until first Beta release\n* Not tested automatically yet\n \n### Roadmap\n\n* 0.6 - stable API\n* 0.7 - automated tests, first Beta release\n* 0.8,\u003c1.0 - minor improvements without API changes, bugfixes\n* 1.0 - first Stable release\n\n## Introduction\n\nSQL is a great and poweruful DSL, which is easeier in maintenance \nwhen you're working on complext queries (i.e. reporting queries).\nBut the main problem of raw SQL is a commonly used \"spaghetti\" anti-pattern, \nwhen you're embedding/building SQLs directly in your code.\n\nThe solution comes from templating SQLs idea and `django-sqltemplate` \nis a simple implementation of it.\n\n## Quickstart\n\n### Install the package\n\n```pip install django-sqltemplate```\n\n### Add application to the INSTALLED_APPS\n\n```\n    INSTALLED_APPS = [\n       ...\n       'djsqltemplate',\n       ...\n       ]\n```\n\n### Add SQL template(s)\n\nMake `sqltemplates` directory in your Django app (your app must be added to\n`INSTALLED_APPS`):\n\n```mkdir \u003cdjango-project\u003e/\u003cmy-app\u003e/sqltemplates```\n\nPut `hello.sql` template in `sqltemplates` directory:\n\nhello.sql (assuming sqlite syntax)\n```sql\nselect 'Hello ' || :name as message\n```\n\n### Query the database\n\n```python\n\u003e\u003e\u003e import djsqltemplate\n\u003e\u003e\u003e hello = djsqltemplate.get('hello.sql')\n\u003e\u003e\u003e print hello.values(name='Marcin')\n[{'message': u'Hello Marcin'}]\n```\n\nIf query returns just one row (as in example above) you may read result\ndirectly using `.scalar()` method:\n\n```python\n\u003e\u003e\u003e print hello.scalar(name='Marcin')\nHello Marcin\n```\n\nTo fetch results as a list of dictionaries use `.values()` method:\n\n```python\n\u003e\u003e\u003e print hello.values(name='Marcin')\n[{'message': u'Hello Marcin'}]\n```\n\nTo fetch results as a list of tuples use `.values_list()` method:\n\n```python\n\u003e\u003e\u003e print hello.values_list(name='Marcin')\n[(u'Hello Marcin',)]\n```\n\nTo fetch results as iterator over tuples use `.iterator()` method:\n\n```python\n\u003e\u003e\u003e print hello.iterator(name='Marcin')\n\u003cgenerator object _fetch at 0x7f8abd202870\u003e\n```\n\nTo fetch results as iterator over dictionaries use `.dictiterator()` method:\n\n```python\n\u003e\u003e\u003e print hello.dictiterator(name='Marcin')\n\u003cgenerator object _fetch at 0x7f8abd202820\u003e\n```\n\n\n## Advanced examples\n\n### The Counter\n\nLet's assume that we want to count rows returning from `hello.sql` query.\nTo do that we should create a sql for the counter. But instead of making\na new file, we'll create it from string, to show how `.from_string()`\nworks:\n\n```python\n\u003e\u003e\u003e count = djsqltemplate.from_string(\n    'select count(*) from ({{ sql|safe }}) x')\n```\n\nThen join the queries together:\n\n```python\n\u003e\u003e\u003e import djsqltemplate\n\u003e\u003e\u003e hello = djsqltemplate.get('hello.sql').bind(name='Marcin')\n\u003e\u003e\u003e count = djsqltemplate.from_string(\n    'select count(*) from ({{ sql|safe }}) x')\n\u003e\u003e\u003e print count.scalar(sql=hello)\n1\n```\n\nAs you can see the `:name` variable was replaced with `Marcin` string,\nand the `sql` template variable (from The Counter query) was replaced\nby `hello.sql` subquery.\n\n#### How it looks?\n\n```python\n\u003e\u003e\u003e print count.bind(sql=hello).pretty()\n```\n```sql\nSELECT count(*)\nFROM\n  (SELECT 'Hello ' || :name AS message) x\n```\n\n#### How it works?\n\n`count` and `hello` objects are `TemplateQuery` instances:\n\n```python\n\u003e\u003e\u003e count, hello\n(\u003csqltemplate.query.TemplateQuery at 0x7f8abd1ee610\u003e,\n \u003csqltemplate.query.TemplateQuery at 0x7f8abd1ee210\u003e)\n```\n\nThe `TemplateQuery` wraps Django `Template` instance together with specified\ncontext.  Calling `TemplateQuery` produces new instance with extended context\n(internally using `.bind()` method), and the outermost context is extended by\ncontext of embedded templates.\n\nContext may be set at the factory time setting `context` argument or by\nimplicit call of `.bind()` method. Also you can pass extra context arguments\ndirectly to `.values()`, `.values_list()`, `.iterator()`, `.dictiterator()` and\n`.scalar()`.\n\n```python\n\u003e\u003e\u003e hello = djsqltemplate.get('hello.sql', context={'name': 'Marcin'})\n\u003e\u003e\u003e print hello.context\n{'name': 'Marcin'}\n\n\u003e\u003e\u003e hello = djsqltemplate.get('hello.sql')\n\u003e\u003e\u003e print hello.context\n{}\n\n\u003e\u003e\u003e hello_marcin = hello.bind(name='Marcin')\n\u003e\u003e\u003e print hello_marcin.context\n{'name': 'Marcin'}\n\n\u003e\u003e\u003e print hello.scalar(name='Marcin')\nHello Marcin\n```\n\nSo in the Counter example we're setting `hello` instance as a `sql` variable\nfor the `counter.sql` template, which is resolved and rendered by\n`{{ sql|safe }}` expression, and then (at the execution time) the `name`\nvariable is passed to `cursor.execute()` (which is safe and the preferred way\nof passing query parameters). \n\nRemeber that preparing templates with additional context makes a new instance\n(a copy) of the original object. This will allow you for easy query\ncustomization dependend of your requirements.\n\n### Countries searcher\n\nLet's prepare a test table (still assuming sqlite as a db engine):\n\n```bash\necho \"create table countries (id int, name varchar(64));\" | sqlite3 db.sqlite3\n```\n\nFill the example data:\n\n```sql\necho \"insert into countries (id, name) values (1, 'Poland'), (2, 'Slovakia'), (3, 'Czech Republic');\" | sqlite3 db.sqlite3\n```\n\nAdd `countries.sql` query template:\n\n```sql\nselect id, name from countries\n{% if search_for %}where name like '%'||:search_for||'%'{% endif %}\n{% if limit %}limit :limit{% endif %} \n```\n\nInstantiate `count` and `countries` templates:\n\n```python\n\u003e\u003e\u003e count = djsqltemplate.get('counter.sql')\n\u003e\u003e\u003e countries = djsqltemplate.get('countries.sql')\n```\n\nAsk for countries containg letter \"a\" in their names:\n\n```python\n\u003e\u003e\u003e print countries.values(search_for='a')\n[{'id': 1, 'name': u'Poland'}, {'id': 2, 'name': u'Slovakia'}]\n```\n\nthen count the results:\n\n```python\n\u003e\u003e\u003e print count.scalar(sql=countries.bind(search_for='a'))\n2\n```\n\nand limit results if you want:\n\n```python\n\u003e\u003e\u003e print countries.values(search_for='a', limit=1)\n[{'id': 1, 'name': u'Poland'}]\n```\n\nSimple?\n\n### Multiple database connections\n\n`TemplateQuery` class provides `.using()` method which allow you to\nchange connection used to querying database. Just provide connection\nname (alias) same as for Django's `QuerySet`.\n\n```python\n\u003e\u003e\u003e print countries.using('default').values()\n```\n\nYou can set connection name at factory time:\n\n```python\n\u003e\u003e\u003e countries = djsqltemplate.get('countries.sql', using='default')\n```\n\nAnd you can use `djsqltemplate.using()` as a context manager:\n\n```python\nwith djsqltemplate.using('default') as tpl:\n    countries = tpl.get('countries.sql')\n    print countries.values()\n```\n\nPlease note that `tpl` variable is a new factory instance, which will\nautomatically set proper connection to all created `TemplateQuery`\nobjects. Direct call to `djsqltemplate.get()` will create objects same\nas before, without connection set, because it is a shortcut for default\nfactory method.\n\n### Default context\n\nSometimes you may need to set some defaults. To do that you can set\ndefault context at a factory time:\n\n```python\n\u003e\u003e\u003e countries = djsqltemplate.get('countries.sql', context={'limit': 2})\n```\n\nAnd by using `djsqltemplate.context()` context manager:\n\n```python\nwith djsqltemplate.context(limit=1) as tpl:\n    countries = tpl.get('countries.sql')\n    print countries.values()\n```\n\n### Setting default context and connection together\n\nIf you want to set default context together with specific connection,\nuse `djsqltemplate.scope()` context manager:\n\n```python\nwith djsqltemplate.scope(context={'limit': 2}, using='default') as tpl:\n    countries = tpl.get('countries.sql')\n    print countries.values()\n```\n\n## Motivation\n\n* `django-sqltemplate` is designed for managing queries of mid/large complexity\n  (like queries above 100 SLOCs, incl. window functions, non-generic syntax,\n  etc)\n* Maintenance of a complex queries is way faster using raw SQL instead of ORM\n  objects (`Q()`,`F()`,`.aggregate()`, etc)\n* The querying should be simplest as possible, incl. joining / embedding\n  templates (we don't want to handle cursors and connections instances manually)\n* It is not a replacement for Django ORM nor SQLAlchemy, and may be used\n  together with (i.e.`sqlalchemy.text(str(countries(search_for='a')))`\n  or Django's `Manager.raw()`)\n* There are many good template engines (Django Templates, Jinja2), so we just\n  need to use them and not reinvent the wheel\n* Django 1.8+ has support for multiple templating engines\n* Django is a most popoular RAD framework for Python, ~~but with limited ORM~~ (it's quite expressive nowdays, but still there are cases where plain SQL is better, ie. multiline and complex reporting queries)\n\n## Requirements\n\n* Django 1.11+, Django 2.x, Django 3.x\n* Python 2.7 (deprecated), Python 3.5+\n\nDependencies:\n\n* sqltemplate \u003e= 0.5.0\n\n## License\n\nBSD\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcinn%2Fdjango-sqltemplate","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarcinn%2Fdjango-sqltemplate","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcinn%2Fdjango-sqltemplate/lists"}