{"id":32191560,"url":"https://github.com/vermilionsands/ashtree","last_synced_at":"2025-10-22T01:43:17.510Z","repository":{"id":62435110,"uuid":"112784636","full_name":"vermilionsands/ashtree","owner":"vermilionsands","description":"Clojure + Apache Ignite","archived":false,"fork":false,"pushed_at":"2018-03-12T22:09:39.000Z","size":108,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-10-22T01:43:13.561Z","etag":null,"topics":["apache-ignite","clojure","ignite"],"latest_commit_sha":null,"homepage":"","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"epl-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vermilionsands.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-12-01T20:39:39.000Z","updated_at":"2022-09-22T16:12:10.000Z","dependencies_parsed_at":"2022-11-01T21:02:36.231Z","dependency_job_id":null,"html_url":"https://github.com/vermilionsands/ashtree","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/vermilionsands/ashtree","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vermilionsands%2Fashtree","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vermilionsands%2Fashtree/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vermilionsands%2Fashtree/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vermilionsands%2Fashtree/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vermilionsands","download_url":"https://codeload.github.com/vermilionsands/ashtree/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vermilionsands%2Fashtree/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":280365580,"owners_count":26318385,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-21T02:00:06.614Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-ignite","clojure","ignite"],"created_at":"2025-10-22T01:43:11.303Z","updated_at":"2025-10-22T01:43:17.500Z","avatar_url":"https://github.com/vermilionsands.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ashtree\n\nPlaying with Apache Ignite in Clojure. \n\n## Features\n\nCaveat emptor! This is work in progress.\n\n### Distributed tasks\n\n`vermilionsands.ashtree.compute` contains some helpers to work with Ignite's \n[compute grid \u0026 distributed closures](https://apacheignite.readme.io/docs/distributed-closures). \n\nRight now only a part of the API is implemented. \n\n#### Sample usage\n\nStart Ignite in REPL:\n```clj\n;; Create ignite instance\n(import '[org.apache.ignite Ignition Ignite IgniteCompute])\n(import '[org.apache.ignite.cache CacheMode])\n(import '[org.apache.ignite.configuration IgniteConfiguration AtomicConfiguration])\n\n(def ignite \n  (Ignition/start\n    (doto (IgniteConfiguration.)\n      (.setPeerClassLoadingEnabled true)\n      (.setAtomicConfiguration\n      (doto (AtomicConfiguration.)\n        (.setCacheMode CacheMode/REPLICATED))))))\n            \n;; define a function, normally this should be AOT-compiled but we can use a call with symbol as a workaround\n(defn hello! [x] (println \"Hello\" x \"from Ignite!\") :ok)        \n```\nRepeat the above in a different REPL (different JVM instance). \n\nIn one of the REPLs:\n```clj         \n(require '[vermilionsands.ashtree.ignite :as ignite])  \n(require '[vermilionsands.ashtree.compute :as compute])\n\n;; broadcast a function to all nodes\n(compute/with-compute (ignite/compute ignite)\n  (compute/invoke 'user/hello! :args [\"Stranger\"] :broadcast true)) \n```\n\nYou should see a printed string on both REPLs and a vector `[:ok :ok]` as a function\nresult on the calling one. \n\n#### Details\n\n##### Sending a distributed task\n\nTo send a task to a single node use `invoke` function:\n\n```clj\n(compute/with-compute compute-instance\n  (compute/invoke some-fn))\n  \n;; would return a result from some-fn  \n```\n\nAs a result you would get a return value from `some-fn`.\n\n`ignite/with-compute` binds a compute API instance to `compute/*compute*` which is later used by `invoke`. \nAlternatively you can use `:compute` option to submit a compute instance to invoke. \n\n```clj\n(compute/invoke some-fn :compute compute-instance)\n```\n\nTo pass arguments to a function use `:args args-vector` option. For no-arg functions you can\nskip this, or pass `nil` or `[]`.\n\n```clj \n(compute/with-compute compute-instance \n  (compute/invoke some-fn-with-args :args [arg1 arg2 arg3]))\n```\n\n##### Broadcast\n\nTo send a task to all nodes use `:broadcast` option. Tasks would be executed on each node in cluster group associated\nwith compute instance and it would return a vector of results from all nodes.\n\n```clj\n(compute/with-compute compute-instance\n  (compute/invoke some-fn :broadcast true))\n\n;; would return a vector of results    \n```\n\n##### Call many tasks in one call\n\nThere's also an option to to send multiple tasks at once. You can use it to mix multiple different functions \nand call them together.\n\n```clj \n(compute/with-compute compute-instance \n  (compute/invoke some-fn-with-args1 some-fn-with-arg2 :args [first-fn-args-vector] [second-fn-args-vector])\n  \n;; would return a vector of results   \n``` \n\n##### Async\n\nTo enable async mode add `:async true` option like this:\n\n```clj\n(compute/with-compute compute-instance\n  (compute/invoke compute-instance some-fn :async true)\n\n;; would return an IgniteFuture instance\n```\n\nReturned future is an instance of `IgniteFuture` which implements `IDeref`, `IPending` and `IBlockingDeref` so it \ncan be used in `deref` or `@` like standard `future`.\n\nThere is a `finvoke` function which is a variant of `invoke` that adds `:async true` option.\n\n##### Tasks\n\nTask function have to be available on all nodes that would be executing given task. Function can be either a no-arg\nfunction (you can use `partial` to turn any function into a no-arg version) or can accept args which can be passed as \none of the task options. Right now it has to be an instance of `AFn` so some classes that implement `IFn` interface \nwould not qualify (this may change in future). \n\nApart from standard functions there are two alternative ways to submit functions\n\n* using a fully qualified symbol - internally this would be wrapped in a function that would try to resolve the symbol\n  on executing node and call the function. \n\n  ```clj\n  (compute/with-compute compute-instance\n    (compute/invoke 'some-ns/some-fn)) \n  ```  \n  \n* using serializable function from `vermilionsands.ashtree.function`. This is right now _experimental_. Serializable \n  functions would be passed in source form (along with local bindings) and evaled on execution to a concrete function.\n  When working in REPL you can use this to submit arbitrary function to the cluster.\n  \n  ```clj\n  (require '[vermilionsands.ashtree.function :as function])\n  \n  (compute/with-compute compute-instance\n    (compute/invoke (function/sfn [] (println \"Hello from arbitrary function!\")))) \n  ```\n  \n  There's a `*callable-eval*` var in `function` ns which defines an `eval` function used to eval these functions.\n  By default it would memoize 100 functions using LRU caching.      \n\n##### Selecting nodes for execution\nWhich nodes would be eligible to process the task depends on cluster group used to get a compute instance. \n\n##### Task options\n\n`invoke` supports the following options, which can be supplied as additional arguments:\n\n```clj\n(compute/invoke some-fn :args [1 2] :async true :name \"distributed-function\" :timeout 1000)\n```\n\n* `:args` - seqs of arguments for tasks\n* `:async` - enable async execution if true\n* `:broadcast` - would be executed on all nodes if true\n* `:compute` - compute API instance, would override `*compute*`\n* `:name` - name for this task\n* `:no-failover` - execute with no failover mode if true\n* `:timeout` - timeout, after which ComputeTaskTimeoutException would be returned, in milliseconds\n\nFor `invoke` with single task:\n* `:affinity-cache` - cache name(s) for [affinity call](https://apacheignite.readme.io/docs/collocate-compute-and-data)\n* `:affinity-key` - affinity key or partition id\n \nFor `invoke` with multiple tasks:\n* `:reduce` - if provided it would be called on the results reducing them into a single value.\nIt should accept 2 arguments (state, x) and should follow the same rules as task.\n* `:reduce-init` - initial state of reducer state\n\nSee `invoke` doc for more info. If instead of additional args you would like to pass options as map check `invoke*`.\n\n##### Distributed function \n\n`distribute` function accepts a task and additional options and returns a new function which would\ncall `invoke` with supplied task, options and args. If it is called with bounded `*compute*` or with `:compute` \noption, new function would keep compute API instance.\n\n```clj\n;; using hello! from earlier example\n;; create a distributed version\n(def distributed-hello! \n  (compute/invoke 'user/hello! :compute (ignite/compute ignite)))\n  \n;; call it later, it would be executed on cluster\n(distributed-hello! \"John\")  \n```\n\nThere is a `fdistribute` function which is a variant of `distribute` that adds `:async true` option.\n\n##### misc\n* in case of Binary Marshaller exceptions double check that you have your namespace with offending code AOT compiled. \n  Usually it solves the problem. \n \n### Distributed atom\n\n`vermilionsands.ashtree.data` contains an atom-like reference type, that uses Ignite to store it's state in a \ndistributed fashion.\n\n#### Sample usage\n\nIn REPL:\n\n```clj\n;; Create Ignite instance first, see example for distributed task for example.\n;; After that:\n\n;; Create atom\n(require '[vermilionsands.ashtree.data :as data])\n(def ignite-atom (data/distributed-atom ignite :test-atom 0))\n```\n\nRepeat the above steps in a different REPL (different JVM instance). \nAfter that you can use distributed atom like a regular one.\n\nAtom value after calls to `swap!` and `reset!` in one JVM would be available in another.\n\nIn one REPL:\n```clj\n;; set value in one REPL\n(reset! ignite-atom \"my shared value\")\n```\n\nAfter that in another one:\n```clj\n;; check that value is visible\n@ignite-atom ;; =\u003e would return \"my shared value\"\n```\n\n#### Details\n\n`vermilionsands.ashtree.data` defines an `IgniteAtom` type which implements the same interfaces as `clojure.lang.Atom`, \nas well as a constructor function `distributed-atom`.\n \n##### swap! and reset!\n\n`swap!` is implemented using `compareAndSet` and keeps data in \n[IgniteAtomicReference](https://apacheignite.readme.io/docs/atomic-types). Values need to be serializable.\nFunctions (for `swap!`) are executed locally and do not need to be serializable. To use custom deftypes as values make \ntheir namespaces AOTed.  \n\n##### validators\n\nValidator added using `set-validator!` is not shared between instances. Shared validator can be added using \n`set-shared-validator` from `vermilionsands.ashtree.data/DistributedAtom` protocol. This functions would be \nshared between instances.\n  \n**Note:** validator function should be AOTed on all JVM instances and has to be serializable.\n    \nValidation is called only on the instance which is changing the data.\n \n##### watches\n \nWatches added using `add-watch` are local (like validator). Shared watches can be added using `add-shared-watch` \nfrom `DistributedAtom` protocol are shared between instances. Functions need to meet the same requirements as shared \nvalidators.\n  \nBy default notification that triggers watches is executed only on the instance which is changing the data (like with \nvalidation). \nThere is an option to enable global notifications, which would trigger watches on all instances.\n \nIn order to turn on global notifications add `:global-notifications true` in options map to `distributed-atom` function when first \natom instance is created.\nThis would enable notifications that would be used to trigger watches when a change occurs.\nNotifications can have a timeout, set using `:notification-timeout milliseconds` option.\n\nNotifications are implemented using `IgniteMessaging`. Each notification is a vector od `[old-value new-value]`.  \n  \n##### metadata\n\nMetadata is stored on instance and is not shared.\n\n##### skipping identity updates\n`:skip-identity true` option would make the atom skip updating it's state if `(= new-val old-val)`. No notifications \nwould be called as well. This can improve performance if you have lot's of such updates, but use this option at your own \ndiscretion. \n\n##### misc\n  \n* partitioning and replication are controlled by Ignite instance\n* to free resources when no longer needed call `(destroy! distributed-atom)`. This would destroy all underlying \n  distributed objects\n* when notifications are enabled atom needs to be closed using `.close()` from `java.io.Closeable`. \n  Otherwise notification listener would not be removed. This would remove the listener, but would not destroy \n  state objects. `destroy!` would do both.\n  \n \n## License\n\nCopyright © 2018 vermilionsands\n\nDistributed under the Eclipse Public License either version 1.0 or (at\nyour option) any later version.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvermilionsands%2Fashtree","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvermilionsands%2Fashtree","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvermilionsands%2Fashtree/lists"}