{"id":16417515,"url":"https://github.com/guettli/django-tips","last_synced_at":"2025-08-19T14:33:20.262Z","repository":{"id":54970759,"uuid":"321145901","full_name":"guettli/django-tips","owner":"guettli","description":"Güttli's opinionated Django Tips","archived":false,"fork":false,"pushed_at":"2023-04-27T23:11:38.000Z","size":420,"stargazers_count":78,"open_issues_count":1,"forks_count":9,"subscribers_count":8,"default_branch":"main","last_synced_at":"2024-12-16T23:07:42.430Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/guettli.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-12-13T19:43:02.000Z","updated_at":"2024-07-04T04:36:17.000Z","dependencies_parsed_at":"2024-10-11T07:12:04.583Z","dependency_job_id":null,"html_url":"https://github.com/guettli/django-tips","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guettli%2Fdjango-tips","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guettli%2Fdjango-tips/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guettli%2Fdjango-tips/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guettli%2Fdjango-tips/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/guettli","download_url":"https://codeload.github.com/guettli/django-tips/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230359943,"owners_count":18214158,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-11T07:11:33.249Z","updated_at":"2024-12-19T01:12:52.135Z","avatar_url":"https://github.com/guettli.png","language":"Python","readme":"# Güttli's opinionated Django Tips\n\n# If you are new to Software Development\n\nIf you are new to software development, then there is a long road before you. But Django is a good choice, since it is a well established and very good documented framework.\n\nFirst learn Python, then some HTML and CSS. Then Django and SQL.\n\nAfter you learned the basics (Python, some HTML, some CSS), then use the [Django tutorial](https://docs.djangoproject.com/en/dev/intro/tutorial01/).\n\nAvoid ReactJS or Vue, since you might not need them. First start with the traditional approach: Create HTML on\nthe server and send it to the web browser.\n\nIf you want to use a CSS library, then I recommend [Bootstrap5](https://getbootstrap.com/).\n\nYou can start with SQLite, but sooner or later you should switch to PostgreSQL.\n\nMy hint: Don't dive too deep into JavaScript. It is less important than most people think.\n\n# How to extend the user model in Django?\n\nAccording to Vitor Freitas: \"always replace the default User model\" \n[What You Should Know About The Django User Model](https://simpleisbetterthancomplex.com/article/2021/07/08/what-you-should-know-about-the-django-user-model.html)\n\n# Project vs App\n\nIt is important to understand the difference between a project and an application in the context of Django.\nPlease read [Projects and Applications](https://docs.djangoproject.com/en/3.1/ref/applications/#projects-and-applications)\n\nI always try to keep the project small. The project is a small container. One projects contains several apps.\nThe project contains settings, but almost no code.\n\n# Project `mysite`\n\nI always call the project `mysite`, like in the Django tutorial.\n \nThis has the benefit that the env var `DJANGO_SETTINGS_MODULE` is always \"mysite.settings\" in all my personal projects.\n\n# Templates\n\n## I don't trust the django template language\n\nErrors get silently ignored by the django template language. I avoid it. \n\nI like to create small html snippets with [format_html()](https://docs.djangoproject.com/en/dev/ref/utils/#django.utils.html.format_html).\n\nI love Python methods. They are simple to write, easy to understand and exceptions show a nice stacktrace.\n\nBut if you use the template language, you can not easily mix custom template tags with custom template filters. Example: https://stackoverflow.com/a/12249207/633961\n\nNext thing why I don't prefer to create HTML with Python:\n\n```\n    def my_special_method(self, obj):\n        return ...\n    my_special_method.short_descript = 'super name'\n```\nAbove is a common pattern. Now try to access 'short_description' via the template language:\n\n```\n{{ obj.my_special_method.short_description }}\n```\n\nThis does not work, since the template language does this `obj.my_special_method().short_description`. Grrr\n\nThat's why I prefer [format_html()](https://docs.djangoproject.com/en/4.0/ref/utils/#django.utils.html.format_html) in Python code.\n\n## Use CSS, not \"cycle\"\n\nDon't use (or try to understand) the [Django cycle](https://docs.djangoproject.com/en/3.1/ref/templates/builtins/#cycle) templatetag. Today you don't need to alter the css class to create cebra-tables. Vanilla CSS is enough.\n\n## How to debug Django's url resolving?\n\nIf you are new to a big project, you might not easily see which view function does handle an URL.\n\nYou can use [resolve()](https://docs.djangoproject.com/en/dev/ref/urlresolvers/#resolve) to get the answer:\n\n```\nimport django\ndjango.setup()\nfrom django.urls import resolve\nprint(resolve('/foo/bar/'))\n\n--\u003e ResolverMatch(func=foocms.views.bar_handler, args=('/foo/bar/',), kwargs={}, url_name=foocms, ...)\n\n```\n\nNow you know that `foocms.views.bar_handler` will handle requests to the URL `/foo/bar/`.\n\n\n## Connect to production DB (read-only)\n\nSometimes you want to check some code against the production DB just for testing.\n\nGetting changes through CI would take too long.\n\nYou can connect you local Django to the production DB, **and** make the connection\nread-only.\n\n```\nDATABASES = {\n    'default': {\n        'ENGINE': 'django.db.backends.postgresql_psycopg2',\n        'NAME': ...\n        'USER': ...\n        'PASSWORD': ...\n        'OPTIONS': {\n            'options': '-c default_transaction_read_only=on'\n        }\n    }\n}\n```\n\nNow you can run your local code connected to the production DB, and be sure that you don't break things.\n\nSee: https://stackoverflow.com/a/66986980/633961\n## Keep opening and closing tag together\n\nfoo/start-overview.html\n```\n\u003ctable\u003e\n \u003ctr\u003e\u003cth\u003eCol1\u003c/th\u003e...\u003c/tr\u003e\n ...\n```\n\nfoo/end-overview.html\n```\n\u003c/table\u003e\n```\n\n===\u003e NO!\n\nKeep the opening and closing tag together!\n\nfoo/overview.html\n```\n\u003ctable\u003e\n {% include 'foo/overview-heading.html' %}\n ...\n\u003c/table\u003e\n```\n\n# dot-env\n\nsettings.py\n\n```\nfrom dotenv import load_dotenv\nfrom distutils.util import strtobool\n\nload_dotenv()\nDEBUG = strtobool(os.getenv('DEBUG'))\n...\n\n```\n\n\n# Django Debug Toolbar\n\nThe [Django Debug Toolbar](https://django-debug-toolbar.readthedocs.io/en/latest/) is really useful.\n\nFor small projects it is even fine to enable it in production via [SHOW_TOOLBAR_CALLBACK](https://django-debug-toolbar.readthedocs.io/en/latest/configuration.html#show-toolbar-callback). Example: The toolbar gets shown, only\nif the URL contains `_debug=SomeMagicString`.\n\nThird party panel for DDT:\n\n* https://github.com/mikekeda/django-debug-toolbar-line-profiler/\n\n\n# Testing\n\n## pytest-django for Unittests\n\nI use the library [pytest-django](https://github.com/pytest-dev/pytest-django).\n\nIf you want to get an exception (as opposed to an empty string) if a template variable is unknown, then you can use this config:\n\npytest.ini:\n```\n[pytest]\nDJANGO_SETTINGS_MODULE = mysite.settings\nFAIL_INVALID_TEMPLATE_VARS = True\n```\n\n`FAIL_INVALID_TEMPLATE_VARS` causes the rendering of a template to fail, if a template variable does not exist. I like this. See Zen-of-Python \"Errors should never pass silently.\"\n\nSee [pytest-django docs \"fail for invalid variables in templates](https://pytest-django.readthedocs.io/en/latest/usage.html#fail-on-template-vars-fail-for-invalid-variables-in-templates)\n\n## django_assert_num_queries\n\n[django_assert_num_queries](https://pytest-django.readthedocs.io/en/latest/helpers.html#django-assert-num-queries) is handy, if you want to ensure that the number of SQL queries does not increase over time.\n\n## Avoid Selenium/Playwright Tests\n\nSelenium automates browsers. It can automated modern browsers and IE. It is flaky. It will randomly fail, and you will waste a lot of time.\nAvoid to support IE, and prefer to focus on development.\n\n[Google Trend \"Selenium\"](https://trends.google.com/trends/explore?date=all\u0026q=%2Fm%2F0c828v) is going down.\n\nI heared that PlayWright is modern solution to automate Chromium, Firefox and WebKit with a single API.\n\n## html_form_to_dict()\n\nIf possible, I test methods without a http request in a small unittest.\n\nOn the other hand I would like to know if the html forms are usable by a web-browser.\n\nI like to test my django forms like this:\n\n1. I use `reverse()` to get an URL\n1. I use the [pytest client](https://pytest-django.readthedocs.io/en/latest/helpers.html#client-django-test-client) to get a http response\n1. I use [html_form_to_dict()](https://github.com/guettli/html_form_to_dict) to get a dictionary of the form which is in `response.content`\n1. I set some values on `data`\n1. I use the `client.post(url, data)` to submit the form\n\nFor an example, please see the README of [html_form_to_dict()](https://github.com/guettli/html_form_to_dict)\n\n## Fixtures\n\nSoftware test fixtures initialize test functions. They provide a fixed baseline so that tests execute reliably and produce consistent, repeatable, results. \n\nI don't need [Django Fixtures](https://docs.djangoproject.com/en/3.1/howto/initial-data/). If I need data in a production systems, I can use a script or database migration.\n\nIf I need data for a test, then I use [pytest fixtures](https://docs.pytest.org/en/stable/fixture.html)\n\nIt is unfortunate that the term \"fixture\" has two completely different meanings in the django terminology. I never use django-fixtures, but use pytest-fixtures daily.\n\nAnd I don't see a reason to use a library like [factory-boy](https://factoryboy.readthedocs.io/en/stable/). I prefer [pytest fixtures](https://docs.pytest.org/en/stable/fixture.html). Pytest fixtures and Django-ORM gives me all I need.\n\nI avoid creating random data. I don't use libraries like [faker](https://github.com/joke2k/faker). I want things to be repeatable and predictable.\n\n## Edit-Test Cycle via runserver and ctrl-r?\n\nI know coding and testing your changes manually via runserver and ctrl-r in browser is like a magnet. Even if you know that test-driven-development is\nbetter you will get trapped in the \"edit, ctrl-r\" loop.\n\nWhat do you mean with \"edit ctrl-r\" loop? It works like this: You edit your source code, and then your use the browser\nto see if your changes have the desired effect. And looking at your changes happens by using your browser.\n\nThis works, but has one draw-back: After you got your code change done, you have no automated test. You did manually testing, and it is likely\nthat someone will break the things you implemented in your change.\n\nThis loop makes you faster in the long run: \"Edit, run automated test\".\n\n\n# Keep models.py small\n\nThe file `models.py` is a mixture. A mixture of schema definition and logic. That's handy and makes you fast at the beginning.\n\nBut don't put too much logic into this file. Your subclasses of `Model` should only contain basic methods.\n\nBusiness Logic, creating HTML and other stuff should live somewhere else.\n\nI usualy create a file per model: If the model is called `Foo`, then I create a file called `fooutils.py` which contains\nmethods which get instance of the class `Foo` as first argument.\n\nExample for a model called `Customer`:\n\n```\n# customerutils.py\n\ndef create_weekly_report(customer):\n    ...\n```\n# Django Typed Models\n\n[Django Typed Models](https://github.com/craigds/django-typed-models) brings [Single Table Inheritance](https://en.wikipedia.org/wiki/Single_Table_Inheritance) to Django.\n\nI like it for use-cases like this: Imagine you want to track the changes which get done by a user. The user creates an account,\nthen the user adds his address. Later he creates an order, ....\n\nYou could create a model like this:\n\n```Python\nclass Log(models.Model):\n    user = models.ForeignKey(User)\n    time = models.DateTimeField(auto_add_now=True)\n    action = ....\n    data = models.JSONField()    \n```\n\n`action` could be a CharField with choices, or a ForeignKey to a Model which contains all the possible choices as rows.\n\n`data` stores the changes which fit to the specific action.\n\nExample: the action \"Order Created\" needs a link to the relevant offer.\n\nEvery action has its own data schema.... Things get fuzzy if you use JSON.\n\nOR you could use Single Table Inheritance:\n\n```Python\nclass Log(TypedModel):\n    user = models.ForeignKey(User)\n    time = models.DateTimeField(auto_add_now=True)\n\nclass OrderCreatedLog(Log):\n    offer = models.ForeignKey(Offer)\n```    \n\nThis way you don't need JSON, you can use database column to store the values.\n\n# Check HTML Middleware\n\nI wrote a small Check HTML Middleware, so that typos in HTML get detected soon:\n\n[django-check-html-middleware](https://github.com/guettli/django-check-html-middleware)\n\n\n# Signal on changed fields\n\n[django-fieldsignals](https://github.com/craigds/django-fieldsignals) is a handy library, so\nthat you can easily receive a signal if a field of a model has changed.\n\n\n\n# Django Packages Overview\n\n[djangopackages.org](https://djangopackages.org/)\n\n# htmx\n\nIf you follow the current hype, you get the impression that web applications must be build like this: \n\nThere is a backend (for example Django) which provides an http-API. This API gets used by a\nJavaScript application written in Angular, React, or Vue.\n\nWait, slow down.\n\nI think there is a simpler and more efficient way to develop an web application: Just create\nHTML on the server side with Django.\n\nTo make your application load/submit HTML snippets (to avoid the full screen refresh) you can use [htmx](https://htmx.org).\n\nThis way you have a simple stack which gives you a solid foundation for your application.\n\n\n# Creating timezone aware datetimes.\n\n```\nfrom django.utils.timezone import get_current_timezone\n\nget_current_timezone().localize(datetime.datetime(1999, 1, 1, 0, 0, 0))\n\n--\u003e datetime.datetime(1999, 1, 1, 0, 0, tzinfo=\u003cDstTzInfo 'Europe/Berlin' CET+1:00:00 STD\u003e)\n```\n\nFrom: [Create timezone aware datetime objects](https://stackoverflow.com/questions/71768804)\n\n\n# One Page, three forms\n\nYou want to create one HTML page which contains three forms. The Django forms library is great, but it does not solve this problem for you.\n\nYou could use [Prefixes for forms](https://docs.djangoproject.com/en/3.1/ref/forms/api/#prefixes-for-forms) and put your three django-forms\nonto one big page. Depending on the context, this often feels too heavy. \n\nThat's where [htmx](#htmx) can help you: With htmx you can load/submit html fragments easily. No need to write a SPA (Single Page Application),\nbut if you want to, htmx gives you all you need.\n\n\n# Responsive Web Design with Bootstrap\n\nTo make your HTML look good on mobile and desktop I use [Bootstrap5](https://getbootstrap.com/docs/5.0/getting-started/introduction/).\n\n# Learn to distinguish between Django and Python\n\nThe Python ecosystem is much bigger than the Django ecosystem. \n\nFor example you want to visualize data.\nDon't search for a django solution, search for a Python solution. For example: [Holoviews](https://github.com/holoviz/holoviews)\n\nOr use a JS based solution. For example [d3](https://github.com/d3/d3)\n\n# Avoid request.user\n\nImagine you write a page so that the user is able to edit his/her address. You use request.user and everything works fine. \n\nNow you want to make the same form available\nto a superuser, so that the superuser can edit the address of a user.\n\nNow things get mixed up. Because `request.user` is not longer the user of the address ....\n\nYou can avoid the confusion if you avoid `request.user` and instead require that the caller explicitly gives you\nan `user` object.\n\nIt is perfectly fine to store the current user in a threadlocal variable. But just use this global user for permission checking.\n\n# Global request middleware\n\nUp to now I use a global request middleware. But I am not happy with it. I will use a global user middleware in the future.\n\nThis way I can check for permission without passing the request to every method which will be called during handling the http-request.\n\nBut making the request available via a global variable is too much implicit input to methods. \n\n# Don't give a customer `is_superuser`\n\nDon't give a customer `is_superuser`. Give him `is_staff` and Permissions. \n\nSooner or later you want to add functionality which should be only available to you (the SaaS maintainer).\n\n# Development Environment\n\nIn the past I had the whole stack installed in my local development environment (Apache or Nginx/Gunicorn), but\nI don't do this any more. The `runserver` of Django is enough for development. You usualy don't need https during development.\n\nThis contradicts the guideline that the development environment and the production environment should be as similar as possible.\n\nThe runserver reloads code fast, which is great for a fluent \"edit test\" cycle.\n\nI develop on Ubuntu Linux with PyCharm.\n\nBut I use PostgreSQL on production and for development. If you use the same username for your PostgreSQL-user like for your\nLinux-user, then you don't need to configure a password for the database.\n\n# Production Environment\n\nI use a cheap VPS from Hetzner. Every system I run on the VPS has its own Linux user. Every Linux-User has a virtualenv in $HOME,\nwhich gets activated in .bashrc. This is handy if you want to check something with SSH.\n\nI run `gunicorn` webserver via an Systems like explained in the [Gunicorn Deploying Docs](https://docs.gunicorn.org/en/stable/deploy.html#systemd)\n\nThis way I can run several systems on one VPS. This means there are N gunicorn processes.\n\nAs reverse proxy and https endpoint I use Nginx.\n\nSooner or alter I will switch to containers, but at the moment my current setup works fine.\n\n# Django's Jobs vs Webserver's Jobs: GZipMiddleware\n\n\nYou should understand that's the job of the webserver to provide `https` (Certificates ...) or to compress the responses.\n\nIt makes no sense to use the Django [GZipMiddleware](https://docs.djangoproject.com/en/dev/ref/middleware/#module-django.middleware.gzip).\n\n# Django's Jobs vs Webserver's Jobs: SECURE_SSL_REDIRECT\n\nThe setting [SECURE_SSL_REDIRECT](https://docs.djangoproject.com/en/dev/ref/settings/#std:setting-SECURE_SSL_REDIRECT): I think\nredirecting from http to https should\nbe done by the web-server, not by Django.\n\n\n# Full text search\n\nPostgreSQL can do full text search, and Django supports it: [PG full text search](https://docs.djangoproject.com/en/dev/ref/contrib/postgres/search/)\n\nYou make your life easier, if you avoid a second system (for example ElasticSearch).\n\n# Backup\n\nFirst I run pg_dump, then [timegaps](https://pypi.org/project/timegaps/) to remove outdated dumps, then rsync to a second VPS.\n\n# Login via Google, Amazon, ...?\n\nUse [django-allauth](https://django-allauth.readthedocs.io/en/latest/)\n\n# Static files\n\nUse the library WhiteNoise, [even during development](http://whitenoise.evans.io/en/latest/django.html#using-whitenoise-in-development)\n\n# Ask Questions\n\nAsking questions is important. It pushes you and others forward.\n\nBut you need to dinstinguish between vague and specific questions.\n\n\n## Vague Question?\n\nAsk in the [Django Forum](https://forum.djangoproject.com/) or in a Django Facebook Group. For example [Django Python Web Framework](https://www.facebook.com/groups/python.django)\n\nFor example: \"Which library should I use for ...\". \n\n## Specific Question?\n\nAsk on [Stackoverflow](https://stackoverflow.com/questions/tagged/django)\n\nThe best questions provide some example code which can be executed easily. This increases the likelihood that\nsomeone will provide an answer soon.\n\nIf your code creates a stacktrace, then please add the whole stacktrace to the question.\n\n# Survey\n\n[Community Survey 2020](https://www.djangoproject.com/weblog/2020/jul/28/community-survey-2020/)\n\nUnfortunately not that cool like [state of js](https://stateofjs.com/) or [state of css](https://stateofcss.com/),\nbut the Django Survey gives you a nice overview, too.\n\n# Forms\n\nI like the Django Forms Library. I don' use third party packages like crispy forms.\n\nRule of thumb: Don't access request.GET or request.POST. Always use a form to retrieve the values from the request.\n\n# ORM: No Manager methods\n\nI know [Adding extra manager methods](https://docs.djangoproject.com/en/dev/topics/db/managers/#custom-managers). But I don't like it.\n\nI prefer to write a `@classmethod` if I want to return a custom querysets.\n\nSame for subclassing `QuerySet`. I don't think this is useful. A classmethod is easier to understand (my POV).\n\nWhy do I prefer classmehods? This is a gut feeling. Maybe because Python is bigger than Django. There are more\npeople who know Python, than people who know Django. If someone knows Python, but not Django, then he would\nunderstand a classmethod, but not a custom django manager class.\n\n# ORM: No GenericForeignKey\n\nMy POV: Use real ForeignKeys, not [GenericForeignKeys](https://docs.djangoproject.com/en/4.0/ref/contrib/contenttypes/#generic-relations)\n\n# FBV vs CBV\n\n\"Function based Views\" vs \"Class based Views\"?\n\nSince I switched to [htmx.org](//htmx.org) I prefer FBV.\n\n\u003e You don’t need to learn any of the CBV APIs - TemplateView, ListView, DetailView, FormView, \n\u003e MultipleObjectMixin etc. etc. and all their inheritance trees or method flowcharts. \n\u003e They will only make your life harder.You don’t need to learn any of the CBV APIs -\n\u003e TemplateView, ListView, DetailView, FormView, MultipleObjectMixin etc. etc. \n\u003e and all their inheritance trees or method flowcharts. They will only make your life harder.\n\nSource: [Django Views - The Right Way](https://spookylukey.github.io/django-views-the-right-way/index.html) by Luke Plant.\n\n# Misc\n\n* [get_object_or_404()](https://docs.djangoproject.com/en/dev/topics/http/shortcuts/#get-object-or-404) is handy.\n\n# CSRF token is not needed.\n\nIf you use the default of [SESSION_COOKIE_SAMESITE](https://docs.djangoproject.com/en/dev/ref/settings/#session-cookie-samesite), then\nyou don't need a CSRF token. \n\n# Thumbnails\n\n[sorl-thumbnail](https://github.com/jazzband/sorl-thumbnail) is a handy library to create thumbnails.\n\n# Uncaught Exception Handling: Sentry\n\nDuring development on your local machine you get a nice debug-view if there is an uncaught exception.\n\nOn the production system you get a white page \"Server Error (500)\".\n\nAnd of course, you want to know if users of your site get server error.\n\nPersonally I prefer simple open source solutions to free and great commercial solutions. But in this case I settled with [Sentry](//sentry.io/).\n\nIt is simple to set up, is free and shows all uncaught exceptions in an easy to use web GUI.\n\nAnd Sentry is a [Gold Corporate Member](https://www.djangoproject.com/foundation/corporate-members/) of the Django Software Foundation.\n\n# Uptime Monitoring\n\nSentry won't help you, if there is something broken and your http server does not start. To be sure that your site is running you can use a free service.\nThere are several, I use https://uptimerobot.com/\n\nThey check your site every N minutes from outside via https.\n\n# Cache for ever.\n\nNew content, new URL.\n\n[django-storages](https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html) `AWS_S3_FILE_OVERWRITE = False` forces you to use create+delete instead of updating content.\n\n# Page Speed\n\nYou can use [Lighthouse](https://developers.google.com/web/tools/lighthouse) (via Chrome) or [PageSpeed Insights (Google)](https://developers.google.com/speed/pagespeed/insights/) to check your page.\n\n# Software built with Django\n\n* [OpenSlides](https://openslides.com/) OpenSlides is the all-in-one solution for running your plenary meetings and conferences. \n* [Taiga](https://github.com/taigaio/taiga-back) Agile project management platform. Built on top of Django and AngularJS\n* [Saleor](https://saleor.io/) e-commerce plattform\n\n# Things which could get improved\n\n\"There are only two hard things in Computer Science: cache invalidation and naming things.\"\n\nIn ModelForm it is called \"instance\". In a class-based-view it is called \"object\". In Admin templates it is called \"original\". It would be nice to have **one** term for the thing.\n\nDon't ask me why the [reverse()](https://docs.djangoproject.com/en/3.1/ref/urlresolvers/#reverse) method is called \"url\" in Django templates.\n\nThe Django template language hides errors. I prefer **[format_html()](https://docs.djangoproject.com/en/3.1/ref/utils/#django.utils.html.format_html)**\n\nAdam has a great article about [How to Make Django Raise an Error for Missing Template Variables](https://adamj.eu/tech/2022/03/30/how-to-make-django-error-for-undefined-template-variables/).\n\nThe term \"Field\" has two meanings, and this creates confusion. There are ModelFields and FormFields.\n\nTemplate language: allow for `{% if foo in [\"one\", \"two\", \"three\"] %}`. Related: [check for presence in a list django template](https://stackoverflow.com/questions/7481750/check-for-presence-in-a-list-django-template)\n\nParentheses in expression in the template language would be nice. Related docs [Boolean operators](https://docs.djangoproject.com/en/4.1/ref/templates/builtins/#boolean-operators)\n\n# Migrations\n\n## Don't change old migrations\n\nDon't change old migrations which you already pushed to a central repository. It is likely that someone already pulled your changes into his\ndevelopment environment. This developer will have trouble if you change the old migration. Create a new migration which fixes the error of the\nold migration.\n\n## Linear Migrations\n\nIf you develop in a team you will sooner or later get trouble with your migrations, because two developers create a new migration\nin their branch. The CI for each branch is fine, but after merging both to the main branch you have two migrations with the same number. Grrr\n\n[django-linear-migrations](https://pypi.org/project/django-linear-migrations/) helps you. At least you know during merging that there is a conflict.\n\nThe solution is simple:\n\n\u003e It does this by creating new files in your apps’ migration directories, called max_migration.txt. These files contain the latest migration name for that app, and are automatically updated by makemigrations.\n\nBig thank you to Adam Chainz for this package.\n\n# SQL is great\n\nSince the ORM solves most use-cases many developers don't use raw SQL queries in Django. This does not mean SQL is \"evil\". SQL is great!\n\nWith the help of [Subquery](https://docs.djangoproject.com/en/dev/ref/models/expressions/#subquery-expressions) and [OuterRef](https://docs.djangoproject.com/en/dev/ref/models/expressions/#django.db.models.OuterRef) complex queries are possible. But\nnevertheless, I think SQL is more readable than ORM queries containing `OuterRef(OuterRef(...))`.\n\nIf a raw SQL query is easier to understand than the ORM counterpart (and there is a test for it), then go for SQL.\n\nBut of course you should be aware of SQL-injection and use parameters like documented:\n\n```\nPerson.objects.raw('SELECT * FROM myapp_person WHERE last_name = %s', [lname])\n```\n\nEspescialy in data migrations I often prefer SQL to ORM.\n\nAs a Django developer you might feel more comfortable with the ORM than with SQL. I know. But look at the whole plant earth, then\nthere are far more people who know SQL than people who now the Django ORM.\n\n# Serving private files\n\nImagine you have private files which you don't want to serve via a CDN. You could use a hard to guess random\nstring in the URL, but that's not a real solution.\n\nYou can use the \"x-sendfile\" approach. This way you can do the authentication and permission checking in your\nPython code, and then let the webserver (for example Nginx) handle the slow work of sending the file over the wire.\n\nSetting the appropriate http headers is not hard. But you can use [django-sendfile2](https://github.com/moggers87/django-sendfile2), too.\n\n# Fast Inner Feedback Loop\n\nThe inner feedback loop:\n\n1. Edit\n2. Save\n3. Compile\n4. Run\n5. Check result\n\nWith Python and a modern IDE \"Save\" and \"Compile\" are not needed.\n\nNevertheless it takes some time to see the result.\n\nIf you are fiddling with HTML+CSS you might be faster if you edit the HTML+CSS directly in the browser. For example in devtools (Chrome).\n\nIn most cases step \"5. Check result\" means to see if a test was successful or not. Test-Driven Development makes you faster in the long run.\n\n# ORM: All Users which are at least in one group:\n\n```\nUser.objects.filter(groups__isnull=False)\n```\n\nRelated: https://stackoverflow.com/questions/54367178/django-orm-all-users-which-have-at-least-one-group\n\n# ORM: is annotate and aggregate usefull?\n\nI am unsure wheter `annotate` and `aggregate` are cool features or not. Every time I use\nthem, I know the SQL which I would write. Then I need to translate the SQL in my head\nto the Django syntax. Is this useful? I would be faster if I could write SQL directely.\nOf course I could, but I don't, since writing raw SQL is frowned upon in the Django context....\n\n\n\n# Show SQL\n\n```\nUser.objects.filter(...).query\n```\n\n# Disk-Cache instead of Redis\n\n[python-diskcache](https://github.com/grantjenks/python-diskcache) is an replacement for the [FileBasedCache](https://docs.djangoproject.com/en/dev/topics/cache/#filesystem-caching). For small projects you might not need\na Redis server.\n\n# Signals as Hooks: Use the return-value\n\nDjango has an excelent hook-system, but many don't know it: Signals.\n\nYou can use the return values of signals handlers.\n\nThis is handy if you have a core-app with several plugins.\n\n1. Create a custom signal in the core-app.\n2. Implement a signal handler in the optional plugin-app.\n3. Call the signal handler from the core-app.\n4. Use the [return values of `Signal.send()](https://docs.djangoproject.com/en/4.1/topics/signals/#django.dispatch.Signal.send)\n\nDocs:\n\n\u003e Both send() and send_robust() return a list of tuple pairs [(receiver, response), ... ],\n\u003e representing the list of called receiver functions and their response values.\n\n\n# PostgreSQL Extensions\n\n[django.contrib.postgres](https://docs.djangoproject.com/en/dev/ref/contrib/postgres/) has some nice features.\n\nFor example:\n\n* [CICharField](https://docs.djangoproject.com/en/dev/ref/contrib/postgres/fields/#django.contrib.postgres.fields.CICharField) Case-insensitive CharField\n\n[django-pghistory](https://github.com/Opus10/django-pghistory) record changes to models via PostgreSQL triggers.\n\n\n# Template Language\n\nI like to create HTML with [format_html()](https://docs.djangoproject.com/en/dev/ref/utils/#django.utils.html.format_html)\n\n# Task Queue\n\nI like to have all valuable data in PostgreSQL. That's why I like to tasks in the DB (and not in Redis or RabbitMQ).\n\n[djanog-db-queue](https://github.com/dabapps/django-db-queue)\n\n\n\n## Comma separated list of HTML\nYou want to list some hyperlinks separated by comma?\n```\n{% for obj in mylist %}\n \u003ca href=\"{{obj.get_absolute_url}}\"\u003e{{obj}}\u003c/a\u003e{% if not forloop.last %}, {% endif %}\n{% endfor %}\n```\n\nSee [\"for\" in Built-in Template Language](https://docs.djangoproject.com/en/dev/ref/templates/builtins/#for)\n\n# Not solved yet: Creating a Demo-System\n\nYou can easily create data for unittest with pytest-fixtures.\n\nBut these DB rows are temporary and only for unit-testing.\n\nYou want a demo system populated with some dummy data if you want\nto use your django system with the web-browser.\n\nFor small projects, this is a minor issue. You can create some dummy data by hand.\n\nOr you can dump the production database and restore it in your development DB. But wait!\nThis is unprofessional. This might work for small projects, but for bigger projects\nthis is against data protection regulations. Developer should not see\nthe data of the customers.\n\nOf course you can help yourself with a script.\n\nBut it would be really cool to have a plugin-system so that\nevery app can contribute to setting up a demo data.\n\nInvesting time into this makes sense.\n\nRelated: https://code.djangoproject.com/ticket/33467\n\nI am not looking for libraries like factory-boy, faker or django-seed. I think\nevery app should create the data the way the app wants to create the data.\n\nThe interesting part is the dependency management. If app \"cms-plugin\" needs the data\nof \"cms-core\", then you need a way specify the dependency. Where \"dependency\" means, that\nthe cms-plugin uses the primary-key of a row created by cms-core as foreign-key.\n\nFor example every page needs a category. Imagine there are two plugins. cms-plugin1 creates\na category, and cms-plugin2 depends on the existance of this category.\n\nEvery app should provide N demo data creators. For example the cms-core could provide demo-data-users,\ndemo-data-countries. An other app should be able to depend on demo-data-users without\ndepending on demo-data-countries.\n\n# Not solved yet: Row based permissions\n\nRow based (AKA \"per object permissions\") are not easy. There are several solutions:\n\n[django-gardian](https://django-guardian.readthedocs.io/en/stable/) Drawback: Slow. See [Performance](https://django-guardian.readthedocs.io/en/stable/userguide/performance.html)\n\nUse [get_queryset() in the admin](https://docs.djangoproject.com/en/dev/ref/contrib/admin/#django.contrib.admin.ModelAdmin.get_queryset). This is\nfast and flexible. But does not distinguish between \"read permissions\" and \"write permissions\".\n\n[Video of Madelaine Boyd about row based permissions](https://www.youtube.com/watch?v=1NGGmHZJvyU)\n\n# Not solved yet: Set operations on admin results.\n\nThe django admin has filters so you can filter your models.\n\nI would like to have set operations:\n\nCreate a set with the admin filters. Store this list.\n\nThen do a send filter. Then do set operations on both lists:\n\n* A minus B\n* A plus B\n* B minus A\n* Intersection of A and B.\n* ....\n\n# Not solved yet: Generic way to check if a user is allowed to access a view\n\nImagine you want to render a hyperlink, if a user is allowed to access this view.\nBut if the user is not allowed to access the hyperlink, then you want to display\nsome text.\n\nOf course you can handle this on your own. This is easy if the view and the code\nwhich creates the hyperlink are under your control. Django gives you all the tools to solve\nthis.\n\nBut there is no generic way of doing this. If the view is third party code, then\nit is likely that permission checking gets done inside the view. Then you can't check\nif the user is allowed or not access the page without calling the view.\n\n# Not solved yet: Building URL including protocol outside a request handler\n\nIf you have a request, then it is easy: [request.build_absolute_uri()](https://docs.djangoproject.com/en/dev/ref/request-response/#django.http.HttpRequest.build_absolute_uri).\n\nBut in a command line script, you don't have a request.\n\nAFAIK there is no standard way. See [How can I get the URL (with protocol and domain) in Django (without request)?](https://stackoverflow.com/questions/34989031/how-can-i-get-the-url-with-protocol-and-domain-in-django-without-request)\n\n# Not solved yet: File Upload, create file with PK\n\n\u003e In most cases, this object will not have been saved to the database yet, so if it uses the default AutoField, it might not yet have a value for its primary key field.\n\n[Django FileField.upload_to docs](https://docs.djangoproject.com/en/dev/ref/models/fields/#django.db.models.FileField.upload_to)\n\nIt would be great if I could use the PK for storing the file.\n\n# Not solved yet: Invalid Template --\u003e Exception (and default)\n\nYou can use [string_if_invalid](https://docs.djangoproject.com/en/dev/ref/templates/api/#how-invalid-variables-are-handled),\nbut this means you can use a \"default\" for the string. See: [Django string_if_invalid and default](https://stackoverflow.com/questions/40505917/django-string-if-invalid-and-default)\n\nFor pytest you can use [FAIL_INVALID_TEMPLATE_VARS = True](https://pytest-django.readthedocs.io/en/latest/usage.html#fail-on-template-vars-fail-for-invalid-variables-in-templates) but again, \"defaults\" does not work.\n\nIf the project uses [htmx](//htmx.org), this is not an issue. Because with htmx I often write small methods returning small snippets. This means I use `format_html()` instead of the template language.\n\n# Not solved yet: Call method with arguments via template language\n\nYou can't call a method with arguments via the template language. I wish you could. Related [How to call function that takes an argument in a Django template?](https://stackoverflow.com/questions/2468804/how-to-call-function-that-takes-an-argument-in-a-django-template/2468830#2468830)\n\n# FileField\n\nIn most cases you don't your files to override each other. This means instance-1 should be allowed to\nhave a file called image.png and instance-2 should have a file with the same name (but different content).\n\n```\nfoo_file = FileField(_('Foo File'),\n    upload_to=lambda instance, filename: datetime.datetime.now().strftime(\n              # use random to avoid clashing filenames\n              '%Y/%m/%d-{}/{}'.format(random.randint(100000, 999999),\n              filename)),\n              unique=True,\n              max_length=1024, blank=True, null=True)\n```\n\nUnfortunately it is not possible up to now to use instance.pk in \"upload_to\".\n          \n# The admin interface is for you and your crew\n\nDon't sell the Admin-Interface to your customer. \n\nThis might make you faster on day-1, but in the long run it will slow you down.\n\n\n# Testing: Ensure your number of SQL queries does not increase\n\n[django-perf-rec](https://pypi.org/project/django-perf-rec/)\n\n\u003e django-perf-rec is like Django’s assertNumQueries on steroids. It lets you track the individual queries and cache operations that occur in your code. It stores a YAML file alongside the test file that tracks the queries and operations.\n\n# Maybe you don't need Celery\n\nCelery is very common task-queue.\n\nNevertheless it is big and a second data-store. I prefer to have all data in one DB.\n\nWith PostgreSQL you can write simple task-queues yourself. Just create a table and\nand add new rows for each task.\n\nThen you can use `SELECT ... FOR UPDATE SKIP LOCKED` in your worker processes.\n\nThis should work for most cases. This means there is no need for a second data-store and the huge celery library.\n\nYou can use [LISTEN](https://www.postgresql.org/docs/current/sql-listen.html), to process new data immediately.\n\n\n# 500 Apps instead of Microservices\n\n[Dan Palmer - Scaling Django to 500 apps](https://youtu.be/NsHo-kThlqI)\n\n# Avoid django.contrib.sites\n\nIn my experience [django.contrib.sites](https://docs.djangoproject.com/en/4.0/ref/contrib/sites/) creates more confusion, then it brings value.\n\nAs soon as you use one database for several sites, you need to make sure for every request that you apply the matching filters, so that only\nthe corresponding database rows get used.\n\nWhich site to choose if there not http request, like in CronJobs?\n\nThis makes software development complicated every day.\n\nRunning two systems makes more sense for me. Running two system is more work at day-one, but once it is set up and CI/CD is automated,\nyou are done.\n\nI have seen (non-django) projects which use a single database for all of their customers, although each customer must only see his data.\nThere was no data shared between the customers, so that there was no reason for this architecture. The single-DB architecture \nslowed down the development daily at several layers (development, deployment, fear of bringing the single DB down, ...)\n\n# select_related\n\nDon't use `select_related()` except you check the performance with a benchmark and write a unit-tests which\nchecks that the number of SQL queries does not increase.\n\nI have seen code where developers seem to have used select_related() randomly.\n\nMaybe an annotated query does provide even better performance.\n\nNobody knows, except you benchmark it, and leave a link to the benchmark in the source code.\n\n# How to debug \"ManagementForm data is missing or has been tampered with\"\n\nIf you get the above error message, it is sometimes hard to understand what is missing.\n\nStep 1: Create a test to reproduce the error.\n\nStep 2: Alter the django source code, to give you a better message:\n\n\nFile django/forms/formsets.py\n\n```\n if not form.is_valid():\n     raise ValidationError(\n-        _('ManagementForm data is missing or has been tampered with'),\n+        _('ManagementForm data is missing or has been tampered with xxxxxx prefix={} errors: {}'.format(\n+             form.prefix, form.errors)),\n         code='missing_management_form',\n     )\n```\n\nNow you see what is missing.\n\n# Upgrading 100+ Repositories\n\n[Jeremy Bowman - Herding Ponies: Coordinating and Automating Django Upgrades Across 100+ Repositories]([https://www.youtube.com/watch?v=ny-3AaNHbbs)\n\n# Blogs\n\n* [Adam Johnson](https://adamj.eu/tech/)\n\n# manage command to check state?\n\nNot all invalid states can be avoided by database contstraints. Nevertheless you want to check that\neverything is in a valid state.\n\nYou could write a custom manage command for this. See [custom management commands](https://docs.djangoproject.com/en/dev/howto/custom-management-commands/).\n\nBut I think it makes more sense to write a view which returns a HTML page and a corresponding http status code.\n\nHTML gives you much more freedom. For example you can create tables, which you can't if you use text output.\n\n# get_or_create() --\u003e unique constraint\n\nIf you use `get_or_create()` (or `update_or_create()`), then you most likely want your\nmodel to have a corresponding unique constraint.\n\nIf you don't have a unique constraint, and two http-requests get handled by your application, which\nexecute the same `get_or_create()`, then both http-requests will create a new row.\n\nNow your state in the DB is broken, because the next call to `get_or_create()` with the same values,\nwill fail:\n\n\n`MultipleObjectsReturned: get() returned more than one FooModel -- it returned 2!`\n\nExamples: \n\nYou use `FooModel.objects.get_or_create(foo=...)`, then the attribute `foo` on you\nmodel needs to be unique.\n\n\nIf you use `OtherModel.objects.get_or_create(my_col=..., my_other_col=...)`, then you need `unique_together`.\n\n# How to see which templates got loaded in a http request?\n\nIn the development environment open `template/base.py` and add a print statement:\n\n```\nclass Template(object):\n    def __init__(self, template_string, origin=None,\n                 name='\u003cUnknown Template\u003e'):\n        print('.............loading', name)\n```   \n\n# Related\n\n* [Güttli django-htmx-fun](https://github.com/guettli/django-htmx-fun) Small intro to htmx.\n* [Güttli's opinionated Python Tips](https://github.com/guettli/python-tips)\n* [Güttli's Programming Guidelines](https://github.com/guettli/programming-guidelines)\n* [Güttli, why I like PyCharm](https://github.com/guettli/why-i-like-pycharm/)\n* [Güttli working-out-loud](https://github.com/guettli/wol)\n\n\n","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fguettli%2Fdjango-tips","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fguettli%2Fdjango-tips","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fguettli%2Fdjango-tips/lists"}