{"id":44164040,"url":"https://github.com/mathandpencil/django-bulk-sync","last_synced_at":"2026-02-09T08:44:23.311Z","repository":{"id":34888326,"uuid":"187276056","full_name":"mathandpencil/django-bulk-sync","owner":"mathandpencil","description":"Combine bulk add, update, and delete into a single call.","archived":false,"fork":false,"pushed_at":"2022-03-22T14:35:22.000Z","size":61,"stargazers_count":52,"open_issues_count":3,"forks_count":16,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-09-27T17:21:41.719Z","etag":null,"topics":["django","django-bulk-sync","django-orm","python"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mathandpencil.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-05-17T20:13:31.000Z","updated_at":"2025-07-16T01:03:40.000Z","dependencies_parsed_at":"2022-08-08T02:15:35.694Z","dependency_job_id":null,"html_url":"https://github.com/mathandpencil/django-bulk-sync","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/mathandpencil/django-bulk-sync","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathandpencil%2Fdjango-bulk-sync","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathandpencil%2Fdjango-bulk-sync/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathandpencil%2Fdjango-bulk-sync/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathandpencil%2Fdjango-bulk-sync/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mathandpencil","download_url":"https://codeload.github.com/mathandpencil/django-bulk-sync/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathandpencil%2Fdjango-bulk-sync/sbom","scorecard":{"id":624052,"data":{"date":"2025-08-11","repo":{"name":"github.com/mathandpencil/django-bulk-sync","commit":"50da4c02ae5b08fc049edb211fcfaf5d954dadd5"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":2,"checks":[{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Code-Review","score":2,"reason":"Found 5/21 approved changesets -- score normalized to 2","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 14 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":0,"reason":"32 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: PYSEC-2022-190 / GHSA-2gwj-7jmv-h26r","Warn: Project is vulnerable to: PYSEC-2020-32 / GHSA-2m34-jcjv-45xf","Warn: Project is vulnerable to: PYSEC-2020-36 / GHSA-3gh2-xw74-jmcw","Warn: Project is vulnerable to: PYSEC-2022-1 / GHSA-53qw-q765-4fww","Warn: Project is vulnerable to: PYSEC-2021-98 / GHSA-68w8-qjq3-2gfm","Warn: Project is vulnerable to: GHSA-6c3j-c64m-qhgq","Warn: Project is vulnerable to: PYSEC-2019-10 / GHSA-6c7v-2f49-8h26","Warn: Project is vulnerable to: PYSEC-2022-20 / GHSA-6cw3-g6wv-c2xv","Warn: Project is vulnerable to: PYSEC-2019-13 / GHSA-6r97-cj55-9hrq","Warn: Project is vulnerable to: PYSEC-2019-79 / GHSA-7rp2-fm2h-wchj","Warn: Project is vulnerable to: GHSA-7xr5-9hcq-chf9","Warn: Project is vulnerable to: PYSEC-2022-2 / GHSA-8c5j-9r9f-c6w8","Warn: Project is vulnerable to: GHSA-8x94-hmjh-97hq","Warn: Project is vulnerable to: PYSEC-2022-19 / GHSA-95rw-fx8r-36v6","Warn: Project is vulnerable to: PYSEC-2019-11 / GHSA-c4qh-4vgv-qc6g","Warn: Project is vulnerable to: PYSEC-2020-34 / GHSA-fr28-569j-53c4","Warn: Project is vulnerable to: PYSEC-2021-9 / GHSA-fvgf-6h6h-3322","Warn: Project is vulnerable to: PYSEC-2019-12 / GHSA-h5jv-4p7w-64jg","Warn: Project is vulnerable to: PYSEC-2020-35 / GHSA-hmr4-m2h5-33qx","Warn: Project is vulnerable to: PYSEC-2019-15 / GHSA-hvmf-r92r-27hr","Warn: Project is vulnerable to: PYSEC-2022-3 / GHSA-jrh2-hc4r-7jwx","Warn: Project is vulnerable to: PYSEC-2020-33 / GHSA-m6gj-h9gm-gw44","Warn: Project is vulnerable to: PYSEC-2021-99 / GHSA-p99v-5w3c-jqq9","Warn: Project is vulnerable to: PYSEC-2021-8 / GHSA-qm57-vhq3-3fwf","Warn: Project is vulnerable to: GHSA-rrqc-c2jx-6jgv","Warn: Project is vulnerable to: PYSEC-2021-7 / GHSA-rxjp-mfm9-w4wr","Warn: Project is vulnerable to: PYSEC-2021-439 / GHSA-v6rh-hp5x-86rv","Warn: Project is vulnerable to: PYSEC-2019-14 / GHSA-v9qg-3j8p-r63v","Warn: Project is vulnerable to: PYSEC-2019-16 / GHSA-vfq6-hq5r-27r6","Warn: Project is vulnerable to: PYSEC-2022-191 / GHSA-w24h-v9qh-8gxj","Warn: Project is vulnerable to: PYSEC-2020-31 / GHSA-wpjr-j57x-wxfw","Warn: Project is vulnerable to: PYSEC-2021-6 / GHSA-xgxc-v2qg-chmh"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-21T05:56:13.649Z","repository_id":34888326,"created_at":"2025-08-21T05:56:13.649Z","updated_at":"2025-08-21T05:56:13.649Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29260322,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-09T04:11:57.159Z","status":"ssl_error","status_checked_at":"2026-02-09T04:11:56.117Z","response_time":56,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["django","django-bulk-sync","django-orm","python"],"created_at":"2026-02-09T08:44:22.081Z","updated_at":"2026-02-09T08:44:23.301Z","avatar_url":"https://github.com/mathandpencil.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# django-bulk-sync\n\nCombine bulk create, update, and delete into a single call.\n\n`django-bulk-sync` is a package for the Django ORM that combines bulk_create, bulk_update, and delete into a single method call to `bulk_sync`.\n\nIt manages all necessary creates, updates, and deletes with as few database calls as possible to maximize performance.\n\nIt can use either database PKs or `key_fields` to match up objects with existing records.\n\n## Installation\n\nThe package is available on pip as [django-bulk-sync][django-bulk-sync]. Run:\n\n`pip install django-bulk-sync`\n\nthen import via:\n\n`from bulk_sync import bulk_sync`\n\n## A Usage Scenario\n\nCompanies have zero or more Employees. You want to efficiently sync the names of all employees for a single `Company` from an import from that company, but some are added, updated, or removed. The simple approach is inefficient -- read the import line by line, and:\n\nFor each of N records:\n\n-   SELECT to check for the employee's existence\n-   UPDATE if it exists, INSERT if it doesn't\n\nThen figure out some way to identify what was missing and delete it. As is so often the case, the speed of this process is controlled mostly by the number of queries run, and here it is about two queries for every record, and so O(N).\n\nInstead, with `bulk_sync`, we can avoid the O(N) number of queries, and simplify the logic we have to write as well.\n\n## Example Usage\n\n```python\nfrom django.db.models import Q\nfrom bulk_sync import bulk_sync\n\nnew_models = []\nfor line in company_import_file:\n\t# The `.id` (or `.pk`) field should not be set. Instead, `key_fields`\n\t# tells it how to match.\n\te = Employee(name=line['name'], phone_number=line['phone_number'], ...)\n\tnew_models.append(e)\n\n# `filters` controls the subset of objects considered when deciding to\n# update or delete.  Here we sync only company 501 employees.\nfilters = Q(company_id=501)\n\n# `key_fields` matches an existing object if all `key_fields` are equal.\nkey_fields = ('name', )\n\nret = bulk_sync(\n        new_models=new_models,\n        filters=filters,\n        fields=['name', 'phone_number', ...],\n        key_fields=key_fields)\n\nprint(\"Results of bulk_sync: \"\n      \"{created} created, {updated} updated, {deleted} deleted.\"\n      \t\t.format(**ret['stats']))\n```\n\nUnder the hood, it will atomically call `bulk_create`, `bulk_update`, and a single queryset `delete()` call, to correctly and efficiently update all fields of all employees for the filtered Company, using `name` to match properly.\n\n## Argument Reference\n\n`def bulk_sync(new_models, key_fields, filters, batch_size=None, fields=None, skip_creates=False, skip_updates=False, skip_deletes=False):`\nCombine bulk create, update, and delete. Make the DB match a set of in-memory objects.\n\n-   `new_models`: An iterable of Django ORM `Model` objects that you want stored in the database. They may or may not have `id` set, but you should not have already called `save()` on them.\n-   `key_fields`: Identifying attribute name(s) to match up `new_models` items with database rows. If a foreign key is being used as a key field, be sure to pass the `fieldname_id` rather than the `fieldname`. Use `['pk']` if you know the PKs already and want to use them to identify and match up `new_models` with existing database rows.\n-   `filters`: Q() filters specifying the subset of the database to work in. Use `None` or `[]` if you want to sync against the entire table.\n-   `batch_size`: (optional) passes through to Django `bulk_create.batch_size` and `bulk_update.batch_size`, and controls how many objects are created/updated per SQL query.\n-   `fields`: (optional) List of fields to update. If not set, will sync all fields that are editable and not auto-created.\n-   `exclude_fields`: (optional) list of fields to exclude from updates. Subtracts from the passed-in `fields` or default-calculated `fields` (see `fields` documentation above).\n-   `exclude_fields`: (optional) List of fields to exclude from updates.\n-   `skip_creates`: (optional) If truthy, will not perform any object creations needed to fully sync. Defaults to not skip.\n-   `skip_updates`: (optional) If truthy, will not perform any object updates needed to fully sync. Defaults to not skip.\n-   `skip_deletes`: (optional) If truthy, will not perform any object deletions needed to fully sync. Defaults to not skip.\n-   `db_class`: (optional) Model class to operate on. If new_models always contains at least one object, this can be set automatically so is optional.\n-   `select_for_update_of`: (optional) Iterable passed directly to select_for_update `of` clause to control locking of related models.\n            See https://docs.djangoproject.com/en/dev/ref/models/querysets/#select-for-update for more information.\n\n-   Returns a dict:\n    ```\n    {\n    'stats': {\n        \"created\": number of `new_models` not found in database and so created,\n        \"updated\": number of `new_models` that were found in database as matched by `key_fields`,\n        \"deleted\": number of deleted objects - rows in database that matched `filters` but were not present in `new_models`.\n        }\n    }\n    ```\n\n`def bulk_compare(old_models, new_models, key_fields, ignore_fields=None):`\nCompare two sets of models by `key_fields`.\n\n-   `old_models`: Iterable of Django ORM objects to compare.\n-   `new_models`: Iterable of Django ORM objects to compare.\n-   `key_fields`: Identifying attribute name(s) to match up `new_models` items with database rows. If a foreign key\n    is being used as a key field, be sure to pass the `fieldname_id` rather than the `fieldname`.\n-   `ignore_fields`: (optional) If set, provide field names that should not be considered when comparing objects.\n-   Returns dict:\n    ```\n        {\n            'added': list of all added objects.\n            'unchanged': list of all unchanged objects.\n            'updated': list of all updated objects.\n            'updated_details': dict of {obj: {field_name: (old_value, new_value)}} for all changed fields in each updated object.\n            'removed': list of all removed objects.\n        }\n    ```\n\n## Frameworks Supported\n\nThis library is tested using Python 3 against Django 2.2+. If you are looking for versions that work with Django \u003c 2.2,\nplease use the 1.x releases.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmathandpencil%2Fdjango-bulk-sync","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmathandpencil%2Fdjango-bulk-sync","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmathandpencil%2Fdjango-bulk-sync/lists"}