{"id":13760405,"url":"https://github.com/lilactown/pyramid","last_synced_at":"2025-12-12T01:27:52.195Z","repository":{"id":38018258,"uuid":"308228837","full_name":"lilactown/pyramid","owner":"lilactown","description":"A library for storing and querying graph data in Clojure","archived":false,"fork":false,"pushed_at":"2024-11-15T23:10:33.000Z","size":682,"stargazers_count":234,"open_issues_count":3,"forks_count":11,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-05-14T11:07:54.256Z","etag":null,"topics":["clojure","clojurescript","eql","graph-database","memory-database","pathom"],"latest_commit_sha":null,"homepage":"","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"epl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lilactown.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-10-29T05:39:43.000Z","updated_at":"2025-04-30T14:01:49.000Z","dependencies_parsed_at":"2023-11-28T18:28:27.180Z","dependency_job_id":"4d349310-faba-423f-893a-4f8c846a42d7","html_url":"https://github.com/lilactown/pyramid","commit_stats":{"total_commits":250,"total_committers":8,"mean_commits":31.25,"dds":"0.10799999999999998","last_synced_commit":"c7e48514ba12bb8512235f3a666d494820170eb4"},"previous_names":["lilactown/autonormal"],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lilactown%2Fpyramid","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lilactown%2Fpyramid/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lilactown%2Fpyramid/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lilactown%2Fpyramid/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lilactown","download_url":"https://codeload.github.com/lilactown/pyramid/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254337612,"owners_count":22054253,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clojure","clojurescript","eql","graph-database","memory-database","pathom"],"created_at":"2024-08-03T13:01:09.679Z","updated_at":"2025-12-12T01:27:52.157Z","avatar_url":"https://github.com/lilactown.png","language":"Clojure","funding_links":[],"categories":["Clojure"],"sub_categories":[],"readme":"# pyramid\n\nA library for storing and querying graph data in Clojure.\n\nFeatures:\n* Graph query engine that works on any in-memory data store\n* Algorithm for taking trees of data and storing them in normal form in Clojure\n  data.\n\n## Install \u0026 docs\n\n[![Clojars Project](https://img.shields.io/clojars/v/town.lilac/pyramid.svg)](https://clojars.org/town.lilac/pyramid) [![cljdoc badge](https://cljdoc.org/badge/town.lilac/pyramid)](https://cljdoc.org/d/town.lilac/pyramid/CURRENT)\n\n\n## Why\n\nClojure is well known for its graph databases like datomic and datascript which\nimplement a datalog-like query language. There are contexts where the power vs\nperformance tradeoffs of these query languages don't make sense, which is where\npyramid can shine.\n\nPyramid focuses on doing essential things like selecting data from and\ntraversing relationships between entities, while eschewing arbitrary logic like\nwhat SQL and datalog provide. What it lacks in features it makes up for in read\nperformance when combined with a data store that has fast in-memory look ups of\nentities by key, such as Clojure maps. It can also be extended to databases like\nDatomic, DataScript and Asami.\n\n\n## What\n\nPyramid can be useful at each evolutionary stage of a program where one needs\nto traverse and make selections out of graphs of data.\n\n### Selection\n\nPyramid starts by working with Clojure data structures. A simple program that\nuses pyramid can use a query to select specific data out of a large, deeply\nnested tree, like a supercharged `select-keys`.\n\n```clojure\n(def data\n  {:people [{:given-name \"Bob\" :surname \"Smith\" :age 29}\n            {:given-name \"Alice\" :surname \"Meyer\" :age 43}]\n   :items {}})\n\n(def query [{:people [:given-name]}])\n\n(pyramid.core/pull data query)\n;; =\u003e {:people [{:given-name \"Bob\"} {:given-name \"Alice\"}]}\n```\n\n### Transformation\n\nPyramid combines querying with the [Visitor pattern](https://en.wikipedia.org/wiki/Visitor_pattern)\nin a powerful way, allowing one to easily perform transformations of selections\nof data. Simply annotate parts of your query with metadata\n`{:visitor (fn visit [data selection] ,,,)}` and the `visit` function will be\nused to transform the data in a depth-first, post-order traversal (just like\n`clojure.walk/postwalk`).\n\n```clojure\n(def data\n  {:people [{:given-name \"Bob\" :surname \"Smith\" :age 29}\n            {:given-name \"Alice\" :surname \"Meyer\" :age 43}]\n   :items {}})\n\n(defn fullname\n  [_db {:keys [given-name surname] :as person}]\n  (str given-name \" \" surname))\n\n(def query [{:people ^{:visitor fullname} [:given-name :surname]}])\n\n(pyramid.core/pull data query)\n;; =\u003e {:people [\"Bob Smith\" \"Alice Meyer\"]}\n ```\n\n### Accretion\n\nA more complex program may need to keep track of that data over time, or query\ndata that contains cycles, which can be done by creating a `pyramid.core/db`.\n\"Pyramid dbs\" are maps that have a particular structure:\n\n```clojure\n;; for any entity identified by `[key id]`, it follows the shape:\n{key {id {,,,}}\n```\n\nAdding data to a db will [normalize](https://en.wikipedia.org/wiki/Database_normalization)\nthe data into a flat structure allowing for easy updating of entities as new\ndata is obtained and allow relationships that are hard to represent in trees.\nQueries can traverse the references inside this data.\n\nSee [docs/GUIDE.md](docs/GUIDE.md).\n\n### Durability\n\nA program may grow to need durable storage and other features that more full\nfeatured in-memory databases provide. Pyramid provides a protocol, `IPullable`,\nwhich can be extended to allow queries to run over any store that data can be\nlooked up by a tuple, `[primary-key value]`. This is generalizable to most\ndatabases like Datomic, DataScript, Asami and SQLite.\n\n### Full stack\n\nThe above shows the evolution of a single program, but many programs never grow\nbeyond the accretion stage. Pyramid has been used primarily in user interfaces\nwhere data is stored in a data structure and queried over time to show different\nviews on a large graph. Through its protocols, it can now be extended to be used\nwith durable storage on the server as well.\n\n## Concepts\n\n**Query**: A query is written using [EQL](https://edn-query-language.org/eql/1.0.0/what-is-eql.html),\na query language implemented inside Clojure. It provides the ability to select\ndata in a nested, recursive way, making it ideal for traversing graphs of data.\nIt does not provide arbitrary logic like SQL or Datalog.\n\n**Entity map**: a Clojure map which contains information that uniquely identifies\nthe domain entity it is about. E.g. `{:person/id 1234 :person/name \"Bill\"\n:person/age 67}` could be uniquely identified by it's `:person/id` key. By\ndefault, any map which contains a key which `(= \"id\" (name key))` is true, is an\nentity map and can be normalized using `pyramid.core/db`.\n\n**Ident function**: a function that takes a map and returns a tuple `[:key val]`\nthat uniquely identifies the entity the map describes.\n\n**Lookup ref**: a 2-element Clojure vector that has a keyword and a value which\ntogether act as a pointer to a domain entity. E.g. `[:person/id 1234]`.\n`pyramid.core/pull` will attempt to look them up in the db value if they appear\nin the result at a location where the query expects to do a join.\n\n## Usage\n\nSee [docs/GUIDE.md](docs/GUIDE.md)\n\n## Prior art\n\n- [Fulcro](https://fulcro.fulcrologic.com/)\n- @souenzzo's POC [eql-refdb](https://github.com/souenzzo/eql-refdb)\n- [MapGraph](https://github.com/stuartsierra/mapgraph/blob/master/test/com/stuartsierra/mapgraph/compare.clj)\n- [EntityDB](https://keechma.com/guides/entitydb/)\n- [DataScript](https://github.com/tonsky/datascript/) and derivatives\n- [Pathom](https://github.com/wilkerlucio/pathom)\n- [juxt/pull](https://github.com/juxt/pull)\n- [ribelo/doxa](https://github.com/ribelo/doxa)\n\n## Copyright\n\nCopyright © 2023 Will Acton. Distributed under the EPL 2.0.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flilactown%2Fpyramid","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flilactown%2Fpyramid","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flilactown%2Fpyramid/lists"}