{"id":44570001,"url":"https://github.com/dual/daplug-core","last_synced_at":"2026-02-14T02:14:15.354Z","repository":{"id":323243638,"uuid":"1092613293","full_name":"dual/daplug-core","owner":"dual","description":"Core services for other daplug-* projects","archived":false,"fork":false,"pushed_at":"2025-12-07T04:33:07.000Z","size":115,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-12-27T07:56:39.594Z","etag":null,"topics":["adapter-pattern","database","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dual.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-09T00:27:59.000Z","updated_at":"2025-12-07T04:31:57.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/dual/daplug-core","commit_stats":null,"previous_names":["dual/daplug-core","dual/daplug-base"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/dual/daplug-core","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dual%2Fdaplug-core","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dual%2Fdaplug-core/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dual%2Fdaplug-core/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dual%2Fdaplug-core/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dual","download_url":"https://codeload.github.com/dual/daplug-core/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dual%2Fdaplug-core/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29431643,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-13T22:20:51.549Z","status":"online","status_checked_at":"2026-02-14T02:00:07.626Z","response_time":53,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adapter-pattern","database","python"],"created_at":"2026-02-14T02:14:12.235Z","updated_at":"2026-02-14T02:14:15.343Z","avatar_url":"https://github.com/dual.png","language":"Python","readme":"# 🧩 daplug-core (da•plug)\n\n\u003e **Shared schema + event plumbing for daplug-* adapters**\n\n[![CircleCI](https://circleci.com/gh/dual/daplug-core.svg?style=shield)](https://circleci.com/gh/dual/daplug-core)\n[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=dual_daplug-core\u0026metric=alert_status)](https://sonarcloud.io/summary/new_code?id=dual_daplug-core)\n[![Bugs](https://sonarcloud.io/api/project_badges/measure?project=dual_daplug-core\u0026metric=bugs)](https://sonarcloud.io/summary/new_code?id=dual_daplug-core)\n[![Coverage](https://sonarcloud.io/api/project_badges/measure?project=dual_daplug-core\u0026metric=coverage)](https://sonarcloud.io/summary/new_code?id=dual_daplug-core)\n[![Python](https://img.shields.io/badge/python-3.10%2B-blue)](https://www.python.org/downloads/)\n[![PyPI package](https://img.shields.io/pypi/v/daplug-core?color=blue\u0026label=pypi%20package)](https://pypi.org/project/daplug-core/)\n[![License](https://img.shields.io/badge/license-apache%202.0-blue)](LICENSE)\n[![Contributions](https://img.shields.io/badge/contributions-welcome-blue)](https://github.com/paulcruse3/daplug-core/issues)\n\n`daplug-core` is the tiny layer of glue that both `daplug-ddb`, `daplug-cypher`, and future `daplug-*` projects, relied on in their old `common/` directories. It bundles a publisher, logging shim, schema utilities, and merge helpers so the higher-level adapters can stay laser-focused on their respective datastores. This repository is not meant to be a fully fledged adapter on its own—it simply centralizes the primitives the adapters share.\n\n---\n\n## 🌈 Why this exists\n\n- **Single source of truth** – The DynamoDB and Cypher adapters used to carry duplicate copies of the same helpers. `daplug-core` keeps those modules in one place.\n- **Batteries-included SNS publishing** – The base `publisher` encapsulates SNS fan-out, FIFO metadata, and logging so consuming packages can just hand it messages.\n- **Schema-first tooling** – `schema_loader` and `schema_mapper` read OpenAPI/JSON schemas and project payloads to the shapes your adapters expect.\n- **Deterministic merging** – `dict_merger` upgrades nested payloads with configurable list/dict strategies (add, replace, remove) so you can keep optimistic writes tight.\n\nIf you are migrating `daplug-ddb` or `daplug-cypher`, remove their legacy `common/` folder and import from `daplug_core` instead. Nothing else changes.\n\n---\n\n## 📦 Installation\n\n```bash\npip install daplug-core\n# or\npipenv install daplug-core\n```\n\n\u003e Not on PyPI yet? Until release, install straight from the repo:\n\u003e\n\u003e ```bash\n\u003e pip install git+https://github.com/paulcruse3/daplug-core.git\n\u003e ```\n\n---\n\n## 🔁 How consuming packages use the base\n\n1. **Declare the dependency** in the adapter package (e.g. `daplug-ddb`) via Pipfile/pyproject.\n2. **Drop the duplicated modules** (`common/logger.py`, `common/publisher.py`, etc.).\n3. **Import from `daplug_core`** wherever those utilities were previously referenced.\n\n```python\n# inside daplug-ddb\nfrom daplug_core import dict_merger, json_helper, publisher, schema_mapper\n\nmerged = dict_merger.merge(original, incoming, update_list_operation=\"replace\")\npublisher.publish(arn=sns_arn, data=merged, attributes={\"event\": \"updated\"})\n```\n\nBecause the API surface stayed the same, adapter code typically only needs import-path updates.\n\n---\n\n## 🧱 Building blocks\n\n| Module | Purpose |\n| ------ | ------- |\n| `base_adapter.BaseAdapter` | Minimal SNS-aware adapter scaffold (used as a mixin by higher-level adapters). |\n| `publisher.publish` | Thin wrapper over `boto3` SNS clients with FIFO group/dedupe support and structured logging. |\n| `logger.log` | Consistent JSON stdout logging that honors `RUN_MODE=unittest`. |\n| `json_helper` | Best-effort `try_encode_json` / `try_decode_json` helpers used by loggers and publishers. |\n| `schema_loader.load_schema` | Loads an OpenAPI/JSON schema and resolves `$ref`s using `jsonref`. |\n| `schema_mapper.map_to_schema` | Recursively projects payloads into schema-shaped dictionaries (supports `allOf` inheritance). |\n| `dict_merger.merge` | Deep merge with per-call list/dict strategies (`add`, `remove`, `replace`, `upsert`). |\n\nMix and match these pieces inside datastore-specific adapters.\n\n---\n\n## 🧭 Example: refactoring `daplug-ddb`\n\n```python\n# before (inside daplug_ddb/common/publisher.py)\nfrom . import logger\nimport boto3\n\n# after\nfrom daplug_core import publisher\n\npublisher.publish(\n    arn=self.sns_arn,\n    data=payload,\n    fifo_group_id=fifo_group,\n    fifo_duplication_id=fifo_dedupe,\n    attributes={\"source\": \"daplug-ddb\"},\n)\n```\n\n```python\n# before\nfrom .common.dict_merger import merge\n\n# after\nfrom daplug_core import dict_merger\n\nupdated_item = dict_merger.merge(original, patch, update_list_operation=\"replace\")\n```\n\nThe same pattern applies inside `daplug-cypher` when merging node payloads or formatting SNS events.\n\n---\n\n## ⚙️ Local development\n\n```bash\ngit clone https://github.com/paulcruse3/daplug-core.git\ncd daplug-core\npipenv install --dev\n```\n\n### Run tests \u0026 coverage\n\n```bash\npipenv run test       # pytest tests/\npipenv run test-cov   # pytest --cov=daplug_core --cov-report=term-missing\npipenv run lint       # pylint --fail-under 10 daplug_core\n```\n\n### Ship updates downstream\n\n1. Bump the version in `setup.py` (and `setup.cfg` if needed).\n2. Publish to PyPI or deliver a git tag.\n3. Update `daplug-ddb` and `daplug-cypher` to depend on the new version.\n4. Remove any residual `common/` references in those repos and re-run their suites.\n\n---\n\n## 🤝 Contributing\n\nPull requests are welcome—especially improvements that make life easier for the DynamoDB and Cypher adapters. If you add a helper here, remember to wire it up in the consuming packages as well.\n\n```bash\ngit checkout -b feat/better-schema-mapper\npipenv run test-cov\npipenv run lint\ngit commit -am \"feat: better schema mapper\"\n```\n\n---\n\n## 📄 License\n\nApache 2.0 – see [LICENSE](LICENSE).\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdual%2Fdaplug-core","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdual%2Fdaplug-core","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdual%2Fdaplug-core/lists"}