{"id":29097396,"url":"https://github.com/lambdaisland/plenish","last_synced_at":"2025-10-27T22:35:52.487Z","repository":{"id":154421243,"uuid":"506290021","full_name":"lambdaisland/plenish","owner":"lambdaisland","description":null,"archived":false,"fork":false,"pushed_at":"2025-03-28T09:18:50.000Z","size":97,"stargazers_count":50,"open_issues_count":2,"forks_count":4,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-09-19T12:41:08.697Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lambdaisland.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-06-22T14:49:19.000Z","updated_at":"2025-08-21T18:37:19.000Z","dependencies_parsed_at":null,"dependency_job_id":"8ea139c4-d196-41dd-b280-5a59cfbbafec","html_url":"https://github.com/lambdaisland/plenish","commit_stats":{"total_commits":49,"total_committers":5,"mean_commits":9.8,"dds":0.3877551020408163,"last_synced_commit":"d9f6f9491708200f2b0e18e76218e63ee3edb03a"},"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/lambdaisland/plenish","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lambdaisland%2Fplenish","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lambdaisland%2Fplenish/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lambdaisland%2Fplenish/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lambdaisland%2Fplenish/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lambdaisland","download_url":"https://codeload.github.com/lambdaisland/plenish/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lambdaisland%2Fplenish/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279015352,"owners_count":26085684,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-13T02:00:06.723Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-28T13:42:08.575Z","updated_at":"2025-10-13T13:41:30.928Z","avatar_url":"https://github.com/lambdaisland.png","language":"Clojure","funding_links":["https://opencollective.com/lambda-island","http://opencollective.com/lambda-island"],"categories":[],"sub_categories":[],"readme":"# Plenish\n\nRestock your warehouse.\n\nSync Datomic to a RDBMS.\n\n\u003c!-- badges --\u003e\n[![cljdoc badge](https://cljdoc.org/badge/com.lambdaisland/plenish)](https://cljdoc.org/d/com.lambdaisland/plenish) [![Clojars Project](https://img.shields.io/clojars/v/com.lambdaisland/plenish.svg)](https://clojars.org/com.lambdaisland/plenish)\n\u003c!-- /badges --\u003e\n\n## Requirements\n\nPlenish requires Clojure version \u003e= 1.11.2.\n\n## Usage\n\nFor the most common use case, copying all transactions that haven't been copied\nyet, this is what you need:\n\n```clj\n(def datomic-conn (d/connect \"datomic:...\"))\n(def pg-conn (jdbc/get-datasource \"jdbc:pgsql://...\"))\n\n(def metaschema\n  {:tables {:user/name {}}})\n\n(def db-adapter (postgres/db-adapter))\n\n(plenish/sync-to-latest datomic-conn pg-conn metaschema db-adapter)\n```\n\nThere are more fine-grained functions if you want to have greater control over\nthe process.\n\n```clj\n(let [;; find the most recent transaction that has been copied, or `nil` if this\n      ;; is the first run\n      max-t (plenish/find-max-t pg-conn)\n\n      ;; query the current datomic schema. plenish will track schema changes as\n      ;; it processes transcations, but it needs to know what the schema looks\n      ;; like so far.\n      ctx   (plenish/initial-ctx datomic-conn metaschema db-adapter max-t)\n\n      ;; grab the datomic transactions you want plenish to process. this grabs\n      ;; all transactions that haven't been processed yet.\n      txs   (d/tx-range (d/log datomic-conn) (when max-t (inc max-t)) nil)]\n\n  ;; get to work\n  (plenish/import-tx-range ctx datomic-conn pg-conn txs))\n```\n\nNote that Plenish will ensure that a transaction is never processed twice\n(through a PostgreSQL uniqueness constraint on the tranactions table), but it\nwon't check if you are skipping transactions. This is not a problem if you are\nusing `find-max-t` as shown above, but if you are piping the tx-report-queue\ninto Plenish then you will have to build in your own guarantees to make sure you\ndon't lose any transactions.\n\n## Configuration\n\nPlenish takes a Metaschema, a map with (currently) a single key, `:tables`, it's\nvalue being a map. Each map entry creates a table, where the map entry is the\nmembership attribute that determines whether an entity becomes a row in that\ntable. The value is a map of configuration keys for that table.\n\n- `:name` Name of the table, optional, defaults to the namespace name of the membership attribute\n- `:rename` Alternative names for specific columns\n- `:rename-many-table` Alternative names for join tables created for has-many attributes\n\n```clj\n{:tables\n {:user/name {:name \"users\"\n              :rename {:user/profile \"profile_url\"}\n  :user-group/name {:rename-many-table {:user-group/users \"group_members\"}}}}\n```\n\nThis above configuration will result in three tables, `users`, `user-group`, and\n`group_members`. Had the `:rename-many-table` been omitted, the last would be\ncalled `user_group_x_user`.\n\nThe columns in each table are determined by which attributes coincide with the\nmembership attributes. The column names are the attribute names without\nnamespace. You can use `:rename` to set them explicitly.\n\n## Running tests\n\nRequires PostgreSQL to be running. To not have to mess around with permissions we run it like so:\n\n```\ndocker run -e POSTGRES_HOST_AUTH_METHOD=trust -p 5432:5432 postgres\n```\n\nNow you can \n\n```\nbin/kaocha\n```\n\nas usual.\n\n## Adding new SQL database adapter\n\nTo add a new adapter, one should consider doing the following steps:\n\n1. Create a adapter inside `src/lambdaisland/plenish/adapters` directory, reify the `IDatomicEncoder` protocol.\n2. Add necessary test in file `test/lambdaisland/$ADAPTER/plenish_test.clj`\n\nWhen trying to make the test pass, consider temporily to remove the `jdbc/with-transaction` in `plenish.clj` so as to make the writing-to-db becomes writing in granuality of command instead of a series of commands.\n\n\u003c!-- opencollective --\u003e\n## Lambda Island Open Source\n\nThank you! plenish is made possible thanks to our generous backers. [Become a\nbacker on OpenCollective](https://opencollective.com/lambda-island) so that we\ncan continue to make plenish better.\n\n\u003ca href=\"https://opencollective.com/lambda-island\"\u003e\n\u003cimg src=\"https://opencollective.com/lambda-island/organizations.svg?avatarHeight=46\u0026width=800\u0026button=false\"\u003e\n\u003cimg src=\"https://opencollective.com/lambda-island/individuals.svg?avatarHeight=46\u0026width=800\u0026button=false\"\u003e\n\u003c/a\u003e\n\u003cimg align=\"left\" src=\"https://github.com/lambdaisland/open-source/raw/master/artwork/lighthouse_readme.png\"\u003e\n\n\u0026nbsp;\n\nplenish is part of a growing collection of quality Clojure libraries created and maintained\nby the fine folks at [Gaiwan](https://gaiwan.co).\n\nPay it forward by [becoming a backer on our OpenCollective](http://opencollective.com/lambda-island),\nso that we continue to enjoy a thriving Clojure ecosystem.\n\nYou can find an overview of all our different projects at [lambdaisland/open-source](https://github.com/lambdaisland/open-source).\n\n\u0026nbsp;\n\n\u0026nbsp;\n\n\u0026nbsp;\n\u003c!-- /opencollective --\u003e\n\n\u003c!-- license --\u003e\n## License\n\nCopyright \u0026copy; 2023 Arne Brasseur and Contributors\n\nLicensed under the term of the Mozilla Public License 2.0, see LICENSE.\n\u003c!-- /license --\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flambdaisland%2Fplenish","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flambdaisland%2Fplenish","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flambdaisland%2Fplenish/lists"}