{"id":13801011,"url":"https://github.com/metasoarous/datsys","last_synced_at":"2025-09-01T01:16:59.760Z","repository":{"id":66280474,"uuid":"54090202","full_name":"metasoarous/datsys","owner":"metasoarous","description":"(+ clj cljs datomic datascript re-frame-esque-frp)","archived":false,"fork":false,"pushed_at":"2017-12-18T18:33:28.000Z","size":5047,"stargazers_count":232,"open_issues_count":21,"forks_count":25,"subscribers_count":22,"default_branch":"dev","last_synced_at":"2025-06-15T06:48:36.830Z","etag":null,"topics":["clojure","clojurescript","datascript","datomic","react","reagent"],"latest_commit_sha":null,"homepage":"","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"epl-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/metasoarous.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2016-03-17T05:08:03.000Z","updated_at":"2025-05-14T09:08:54.000Z","dependencies_parsed_at":null,"dependency_job_id":"82f9a611-2b22-4cf4-b10e-f85bfc7d0ca6","html_url":"https://github.com/metasoarous/datsys","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/metasoarous/datsys","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metasoarous%2Fdatsys","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metasoarous%2Fdatsys/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metasoarous%2Fdatsys/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metasoarous%2Fdatsys/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/metasoarous","download_url":"https://codeload.github.com/metasoarous/datsys/tar.gz/refs/heads/dev","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metasoarous%2Fdatsys/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273061089,"owners_count":25038596,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-31T02:00:09.071Z","response_time":79,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clojure","clojurescript","datascript","datomic","react","reagent"],"created_at":"2024-08-04T00:01:18.565Z","updated_at":"2025-09-01T01:16:59.740Z","avatar_url":"https://github.com/metasoarous.png","language":"Clojure","funding_links":[],"categories":["Web Framework"],"sub_categories":[],"readme":"\n![Datsys](datsys.png)\n\n(Formerly Catalysis)\n\n\u003cbr/\u003e\n\n## A web application development framework embracing:\n\n* Unidirectional flow of derived data \u0026 generally the power of the stream processing model for concurrency and distribution on the server and client (a la [Re-frame](https://github.com/Day8/re-frame), Elm, Redux, Samza)\n* The vision described in the Web After Tomorrow: Scoped Datomic/DataScript database sync as a model for state management and communication\n* The power of the Datom model, datalog and pull-expressions, and the power\n* Stuart Sierra's system component model + protocols \u0026 specs for system modularity (Arachne module ready eventually hopefully as well)\n\nSee the talk from Clojure/West 2016: [Datalog all the way down](https://www.youtube.com/watch?v=aI0zVzzoK_E)\n\n[![Join the chat at https://gitter.im/metasoarous/datsys](https://badges.gitter.im/metasoarous/datsys.svg)](https://gitter.im/metasoarous/datsys?utm_source=badge\u0026utm_medium=badge\u0026utm_campaign=pr-badge\u0026utm_content=badge)\n\n\u003cbr/\u003e\n\n## System (Component) architecture\n\nDatsys is built out of the following libraries:\n\n* [Datsync](https://github.com/metasoarous/datsync): Humble basis upon which we may approach the Web After Tomorrow; `Datomic \u003c-\u003e DataScript` sync utilities\n* Datview: Reusable Reagent components for dynamically and flexibly translating data in a Datomic or DataScript database into UI representations and forms, input, and controls.\n* Datreactor: Re-frame style event handling transaction coordination for DataScript connections.\n* Datspec: Specifications, protocols and lib version coordination for Dat`*` systems.\n\nIt is all of these things and none of these things.\nIt is whichever of them you decide to hook up.\n\nEach of these libraries has been designed around a set of system abstractions, declared as protocols.\nThese protocols define an abstraction over the shape of the various system components in the overall architecture.\nThe goal is that it be possible to swap implementations in and out depending on what pieces are needed for a particular application.\n(So far this system modularization only extends as far as the client, since that's where most of the code is.\n However, the server will end up following this pattern as well.)\n\nLet's take a look at `dat.sys.client.app`:\n\n```clj\n(defn new-system []\n  (-\u003e (component/system-map\n        ;; Remote is our main server connection\n        :remote     (sente/new-sente-remote)\n        ;; For dispatching event streams to the reactor\n        :dispatcher (dispatcher/new-strictly-ordered-dispatcher)\n        ;; Remote is our main server connection\n        :app        (component/using\n                      (dat.view/new-datview)\n                      [:remote :dispatcher])\n        :reactor    (component/using\n                      (reactor/new-simple-reactor)\n                      [:remote :dispatcher :app])\n        :datsync    (component/using\n                      (dat.sync/new-datsync)\n                      [:remote :dispatcher]))))\n```\n\nThis describes a system that is syncing a DataScript database on a client with (most likely) a Datomic database on a server, with messages transmitted by a default Sente-based implementation of the Remote abstraction's protocol(s).\nThis DataScript database then feeds a view using the datview component `:app`.\nThe one core piece in orchestrating all of this is the reactor and the dispatcher.\nTogether these manage the flow of events and their updates on the app state, as well as orchestrate side effects.\n\nSince these pieces are modular, you could just use the reactor and dispatcher, but nothing else.\nOr you could use that plus Datsync, but not use Datview.\nOr use Datview, but not Datsync.\nOr use DataScript on the server instead of Datomic.\nOr whatever.\n\nThe goal is for this to be flexible enough to describe a lot of different systems within the general span of this set of pieces.\nSo if it isn't working for you let us know.\n\n(Aside: Why haven't people been building system components on top of protocols abstracting subsystem boundaries more?)\n\nOf course, as the name might suggest, we'll also be building specs into this project.\nSo that'll be another nice layer of expressiveness open to us for describing subsystem boundaries.\n\n\u003cbr/\u003e\n\n\n## Usage\n\nTo get running, clone, and cd into the project directory (`datsys`).\n\n```\nlein repl\n```\n\nWait for the prompt, then type\n\n```clj\n(run)\n```\n\nThis will initialize a `system` var in the `user` ns, bind the new system instance to it, and then start that system (see Stuart Sierra's Component for more information about systems and components in this sense).\n\nIf you need to reset the system, call `reset`.\nYou can also call something like `(run {:server {:port 8882}})` to specify config overrides.\nThe schema for this is in `datsys.config`.\nUnfortunately, not sure yet how to get the `reset` function to also accept the `config-overrides` option.\n\nAnd maybe eventually we'll also helpers for running multiple instances at once to test different things...\nBut one step at a time :-)\n\n### Browser Repl\n\nYou can get the browser-connected figwheel repl with the command:\n\n```clj\n(browser-repl)\n```\n\nAnd return to the server repl with:\n\n```clj\n:cljs/quit\n```\n\n\n### Open Browser\n\nNext, point browser to:\n\u003chttp://localhost:2358\u003e (or whatever you set $PORT to)\nYou should see a page that says \"Congrats! You've got a datsys app running :-)\".\n\nAfter a few seconds or so, once connections have established and data transacted, you should see a todo list render.\nIf not, check your console.\n(Actually, right now there's a bug and it's possible nothing shows up; You can save a file to get figwheel to trigger an update, but we should have a real fix soon).\n\n\u003cbr/\u003e\n\n\n\n## Customizing your app\n\nYay!\nIf you've gotten through the Usage section, you have a running system with some data loaded and ready to tinker with!\nAt this point, you might want to look at the Todo app a little bit to feel out how things work.\n\n\n### Schema \u0026 Seed data\n\nBecause of the seamless shuttling of data back and forth between server and client, much of the customization work to do on the server is in the schema.\n\nThe schema file is located in `resources/schema.edn`.\nThe data in this file is a [conformity](https://github.com/rkneufeld/conformity) spec, so you can continue to use this same file for all your migrations.\nIf you want to see how these migrations are hooked up, take a look at the `dat.sys.datomic/Datomic` component.\n\nThere's also some seed data in `resources/test-data.edn` for you to play with.\n\n\n### Front end\n\nViews are in `dat.sys.client.views`, and you'll note that the `main` function there is hooked up at the end of the `dat.sys.client.app` namespace (where we created our system).\n\nThe view code is written in a combination of [Reagent](https://github.com/reagent-project/reagent) and [Posh](https://github.com/mpdairy/posh) within the context of the [Datview](https://github.com/metasoarous/datview) UI toolkit.\nMore or less, Datview is a set of helper functions for translating queries and/or query data into hiccup (and thus DOM).\nThese functions vary in granularity (`pull-view \u003e attribute-view \u003e value-view`), letting you compose things as you see fit.\nAdditionally, there is a notion of rendering data within an abstract _context_ with instructs a particular Datview component function how to do it's job.\nAs such, the semantics of a particular context map depend on the function in which it's being used.\nAnd actually, under the hood, these will all just be mapping to some general purpose function: `(transform datview-app context data)`.\nThis itself, in turn, may call `(transform datview-app sub-context sub-data)`, recursively.\nThe context should optimally be DataScript data, so that different views can just be different base context values created from within the database.\nThe `transform` function then can just be a multimethod on the structure of this data.\nOr if you wanted to, you could use pattern matching.\n(For more information please see the Datview README.)\n\nSo aside from constructing context specifications (and wiring/composing them together), you'll also have to handle events and trigger side effects.\nTo read more about this, see [Datreactor](https://github.com/metasoarous/datreactor).\nIf you are using [Datsync](https://github.com/metasoarous/datsync), there's a Datremote abstraction you can use for coordinating messages with the synchronization channels.\nMore information about hooking those thing up can be found in their respective repositories, and see whatever code samples are provided with this project template.\n\n### Other stuff\n\nReactions... etc\n\n\u003cbr/\u003e\n\n\n\n## Contributing to Datsys (and other Dat\\* projects)\n\nWe would love your help!\nAnything from filing bug reports, to documentation, to hacking on code is infinitely appreciated.\nFor docs, feel free to contribute directly to the wiki.\nFor code or code-comment/README docs, feel free to submit a pull-request.\nBefore you start working on anything big though please get in touch with us about it using either a github issue, or by reaching out to us on the Gitter channel.\n\n### Dealing with multiple repos\n\nIf you're contributing code to one of the other Dat projects, you'll likely want to use the [Checkout dependencies](https://github.com/technomancy/leiningen/blob/master/doc/TUTORIAL.md#checkout-dependencies) feature of Leiningen.\nIn short, clone whatever projects you're working on into a `checkouts` dir in `datsys` (note: linking will not work if you want client reloading via Figwheel).\nBe aware of the caveats listed in the link above.\n\n### Roadmap\n\nThere's obviously also a ton smaller things that need to be done and fixed, and you can take a look at project issues for those.\nBut the following is a very big picture overview of some of the more major things we'd like to accomplish with the system.\n\n* Datsync:\n    * Security and scoping filters (+ posh query exports for automated scoping)\n    * Offline availability\n    * Entity validations\n    * Onyx distribution\n    * Long term:\n        * P2P (non-centralized) distribution/sharing (perhaps via CRDTs or CVDCS)\n        * Log history plugins\n* Datview:\n    * Rewrite in terms more general `(transform app context data)` context-based translation function, with multimethod dispatching for extensible rendering.\n    * Finish setting up default context hooks/implementations.\n    * Build out more default control widgets and clean way of grouping them.\n    * Documentation\n    * Package together some example widgets (as context multimethod implementations).\n* Datspec\n    * Add Clojure spec hotness to abstract architectural descriptions :-) (and generally spec rest of project(s) as well)\n* Datsys:\n    * Documentation\n    * More example apps\n    * System Componentization of server, similar to what we've done for client:\n      Going to be a little more challenging here since the flow of data is a bit more complicated, and scalability more of a concern.\n      But we should still be able to come up with some nice abstractions so folks can switch out different implementations.\n    * Generative testing (based on specs)\n\n\u003cbr/\u003e\n\n\n\n\n## Environment and Deployment\n\n### Config\n\nThis application uses a system configuration component found in `dat.sys.config`.\nThere's a default settings map there you can edit, and all settings are customizable via environment variables or run time arguments.\nFeel free to extend this for your needs to keep all your config logic in one place.\nOr replace with whatever you like here.\nWe may move to a proper library for doing this.\n\n### Datomic Pro\n\nAssuming you want to use the free version of Datomic to test things out, things _should_ run out of the box here.\nThere's a partial description in the wiki of how to get Datomic Pro set up with this application, should you need it (as well as some commented code in the project.clj).\n\n### Deploying to Heroku\n\n(Disclaimer: I haven't tried this; copied from Rente.)\n\nTo make Datsys run on Heroku, you need to let Leiningen on Heroku use the \"package\" build task.\n\nTo do this, and point Leiningen on Heroku to the \"package\" target, add the following config variable to Heroku by running this command:\n\n```\nheroku config:add LEIN_BUILD_TASK=package\n```\n\nEverything is nicely wrapped in shiny purple foil if you simply click this button:\n\n[![Deploy](https://www.herokucdn.com/deploy/button.png)](https://heroku.com/deploy)\n\nEnjoy!\n\n### Mobile App?\n\nThere's been talk of folks starting to use this in mobile apps.\nI've you've been using it for mobile apps, please write about it (blog post, tweet, GH wiki page, whatevs) and PR a link here or message me.\n\n## More coming soon...\n\n* Datsys as an Arachne plugin?\n\n\n\n## Contributions\n\nThis code was initially developed as a fork of Rente, but has diverged.\nWe thank the authors of Rente for their contribution.\n\nThis library is authored by Christopher T. Small, with the contributions from the following individuals:\n\nKyle Langford\n\nSee LICENSE for license.\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmetasoarous%2Fdatsys","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmetasoarous%2Fdatsys","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmetasoarous%2Fdatsys/lists"}