{"id":13468723,"url":"https://github.com/occipital/django-consistency-model","last_synced_at":"2025-03-26T05:31:17.187Z","repository":{"id":57419493,"uuid":"446851676","full_name":"occipital/django-consistency-model","owner":"occipital","description":"DCM is a set of tools that helps you to keep your data in your Django Models consistent.","archived":false,"fork":false,"pushed_at":"2024-02-25T08:57:19.000Z","size":180,"stargazers_count":64,"open_issues_count":1,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-08T00:45:50.085Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/occipital.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-01-11T14:21:22.000Z","updated_at":"2024-04-30T22:40:40.000Z","dependencies_parsed_at":"2024-10-29T21:55:20.410Z","dependency_job_id":null,"html_url":"https://github.com/occipital/django-consistency-model","commit_stats":{"total_commits":7,"total_committers":2,"mean_commits":3.5,"dds":0.2857142857142857,"last_synced_commit":"b0521dda2323c8158254a774aee0d4aa3e88d659"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/occipital%2Fdjango-consistency-model","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/occipital%2Fdjango-consistency-model/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/occipital%2Fdjango-consistency-model/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/occipital%2Fdjango-consistency-model/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/occipital","download_url":"https://codeload.github.com/occipital/django-consistency-model/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245597283,"owners_count":20641864,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T15:01:17.484Z","updated_at":"2025-03-26T05:31:15.948Z","avatar_url":"https://github.com/occipital.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"[![Stand With Ukraine](https://raw.githubusercontent.com/vshymanskyy/StandWithUkraine/main/banner-direct-single.svg)](https://stand-with-ukraine.pp.ua)\n\n[![PyPI version fury.io](https://badge.fury.io/py/django-consistency-model.svg)](https://pypi.python.org/pypi/django-consistency-model/) \n[![PyPI pyversions](https://img.shields.io/pypi/pyversions/django-consistency-model.svg)](https://pypi.python.org/pypi/django-consistency-model/)\n[![PyPI - Django Version](https://img.shields.io/pypi/djversions/django-consistency-model)](https://pypi.python.org/pypi/django-consistency-model/)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n\n# Django Consistency Model\n\nDCM is a set of tools that helps you to keep your data in your Django Models consistent.\n\n![Django Consistency Model](https://github.com/occipital/django-consistency-model/blob/master/title-800.png)\n\n## Motivation\n\n* You have a lot of legacy and inconsistent data in your project and you need to clean it out\n* You want to monitor the broken data\n* You are looking for a very simple solution.\n\n## Quick Start\n\nInstall the package:\n\n```bash\npip install django-consistency-model\n```\n\nAdd new app into `INSTALLED_APPS`:\n\n```python\nINSTALLED_APPS = (\n    # ...\n    \"consistency_model\",\n)\n```\n\nAdd your first validator using decorator consistency_validator:\n\n```python\nfrom decimal import Decimal\nfrom django.db import models\nfrom consistency_model import consistency_validator\n\nclass Order(models.Model):\n    total = models.DecimalField(\n        default=Decimal(\"0.00\"), decimal_places=2, max_digits=10\n    )\n    refund = models.DecimalField(\n        default=Decimal(\"0.00\"), decimal_places=2, max_digits=10\n    )\n    revenue = models.DecimalField(\n        default=Decimal(\"0.00\"), decimal_places=2, max_digits=10\n    )\n\n    @consistency_validator\n    def validate_revenue(self):\n        assert self.revenue == self.total - self.refund, \"revenue = total - refund\"\n```\n\nRun command to check validators:\n\n```bash\n./manage.py consistency_model_check\n```\n\n## What if I need to check more than one condition in one validator\n\nThe first thing you may think of is using more than one validator, and it is common to have more than one validator (for example, one validator per field).\n\nSometimes, you want to check more than one aspect in one validator or have a complex calculation you don't want to do for every validator.\n\nFor those cases, you may want to use function `consistency_error`. It shows the system an error without raising an exception, so one validator can generate more than one error.\n\n```python\nfrom decimal import Decimal\n\nfrom django.db import models\n\nfrom consistency_model import consistency_validator, consistency_error\n\n\nclass Order(models.Model):\n    total = models.DecimalField(\n        default=Decimal(\"0.00\"), decimal_places=2, max_digits=10\n    )\n    refund = models.DecimalField(\n        default=Decimal(\"0.00\"), decimal_places=2, max_digits=10\n    )\n    revenue = models.DecimalField(\n        default=Decimal(\"0.00\"), decimal_places=2, max_digits=10\n    )\n\n    @consistency_validator\n    def validate_total(self):\n        assert self.total \u003e= 0, \"can't be negative\"\n\n    @consistency_validator\n    def validate_revenue(self):\n        if self.revenue \u003c 0:\n            consistency_error(\"can't be negative\", \"negative\")\n\n        if self.revenue != self.total - self.refund:\n            consistency_error(\"revenue = total - refund\", \"formula\")\n```\n\nAs you can see, one validator (`validate_revenue`) checks two factors of the field revenue.\n\nThe function `consistency_error` has two arguments - message and name(optional). The name is a unique value for the validator and will be used in monitoring.\n\n## I don't want to check all of the data, but only one model instead.\n\nWhen you add a new validator, you don't want to check all the data. You want to test only one validator instead.\n\nArgument `--filter` can help you with that\n\n```bash\n./manage.py consistency_model_check --filter storeapp.Order.validate_revenue\n```\n\nCheck only one model\n\n```bash\n./manage.py consistency_model_check --filter storeapp.Order\n```\n\nCheck the model but excluding one validator. Argument `--exclude` excludes validator from validation circle.\n\n```bash\n./manage.py consistency_model_check --filter storeapp.Order --exclude storeapp.Order.validate_revenue\n```\n\nCheck only one object. Using `--object` you can check a specific object in db.\n\n```bash\n./manage.py consistency_model_check --object storeapp.Order.56\n```\n\nYou can combine `--object` with `--filter` and `--exclude` as well.\n\n## I want to monitor my DB on consistency constantly.\n\nThe idea of consistency monitoring is very simple. You add the command `consistency_model_monitoring` to your cron. The command checks DB and saves all of the errors in `ConsistencyFail`. Nothing is too complicated.\n\nAs the result, you can see all of the inconsistency errors in admin panel. Or you can connect `pre_save` signal to `consistency_model.ConsistencyFail` and send an email notification in case of any new inconsistency.\n\n## Monitoring configuration.\n\nA typical situation is when you don't want to monitor all the data but only recently added/updated data. By default, the system checks only 10k recent IDs, but you have a lot of flexibility to change that with function `register_consistency`.\n\nLet's take a look of how one can be used.\n\nFor model `Order` you want to check only 10 last ids.\n\n```python\nfrom consistency_model import register_consistency\nregister_consistency(Order, limit=10)\n```\n\n`register_consistency` can be used as class decorator\n\n```python\nfrom consistency_model import register_consistency\n\n@register_consistency(limit=10)\nclass Order(models.Model):\n    # ...\n```\n\nyou can order not by id, but `modified_on` field\n\n```python\nfrom consistency_model import register_consistency\nregister_consistency(Order, order_by='modified_on')\n```\n\nyou can use a consistency checker class to overwrite the whole query for consistency check\n\n```python\nfrom django.db import models\n\nfrom consistency_model import register_consistency, ConsistencyChecker\n\n\nclass Order(models.Model):\n    is_legacy = models.BooleanField(dafult=False)\n    # ...\n\n\nclass OrderConsistencyChecker(ConsistencyChecker):\n    limit = None # I don't want to have any limitation\n    order_by = 'modified_on'\n\n    def get_queryset(self):\n        return self.cls.objects.filter(is_legacy=False)\n\nregister_consistency(Order, OrderConsistencyChecker)\n```\n\nAgain, it is possible to be used as class decorator for any  on both classes.\n\nFor Model:\n\n```python\nfrom django.db import models\n\nfrom consistency_model import register_consistency, ConsistencyChecker\n\n\nclass OrderConsistencyChecker(ConsistencyChecker):\n    # ...\n\n@register_consistency(OrderConsistencyChecker)\nclass Order(models.Model):\n    is_legacy = models.BooleanField(dafult=False)\n    # ...\n\n```\n\nFor Checker:\n\n```python\nfrom django.db import models\n\nfrom consistency_model import register_consistency, ConsistencyChecker\n\n\nclass Order(models.Model):\n    is_legacy = models.BooleanField(dafult=False)\n    # ...\n\n\n@register_consistency(Order)\nclass OrderConsistencyChecker(ConsistencyChecker):\n    # ...\n\n```\n\n## Settings\n\n`CONSISTENCY_DEFAULT_MONITORING_LIMIT` (default: `10_000`) - default limit rows per model\n\n`CONSISTENCY_DEFAULT_ORDER_BY` (default: `\"-id\"`) - defaul model ordering for monitoring\n\n`CONSISTENCY_DEFAULT_CHECKER` (default: `\"consistency_model.tools.ConsistencyChecker\"`) - default class for consistency monitoring\n\nIf you have `pid` package installed, one will be used for monitoring command to prevent running multiple monitpring process. The following settings will be used for monitoring\n\n`CONSISTENCY_PID_MONITORING_FILENAME` (default: `\"consistency_monitoring\"`) \n\n`CONSISTENCY_PID_MONITORING_FOLDER` (default: `None`) - folder the pid file is stored. `tempfile.gettempdir()` is using if it is `None`\n\n## Contributing\n\nWe’re looking to grow the project and get more contributors. Feel free to submit bug reports, pull requests, and feature requests.\n\nTools:\n\n* [tox](https://tox.wiki/en/latest/)\n* [pre-commit](https://pre-commit.com/)\n* [black](https://github.com/psf/black)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foccipital%2Fdjango-consistency-model","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foccipital%2Fdjango-consistency-model","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foccipital%2Fdjango-consistency-model/lists"}