{"id":32183896,"url":"https://github.com/timokramer/datahike","last_synced_at":"2025-10-21T23:10:03.106Z","repository":{"id":45539985,"uuid":"262585441","full_name":"TimoKramer/datahike","owner":"TimoKramer","description":"A durable datalog implementation adaptable for distribution. ","archived":false,"fork":true,"pushed_at":"2025-09-22T14:37:14.000Z","size":4170,"stargazers_count":0,"open_issues_count":2,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-20T05:39:42.445Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Clojure","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"replikativ/datahike","license":"epl-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TimoKramer.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-05-09T14:12:40.000Z","updated_at":"2025-09-22T14:22:13.000Z","dependencies_parsed_at":"2023-10-16T03:26:09.786Z","dependency_job_id":"a1364dd6-11ef-4607-a424-fdcc71a05432","html_url":"https://github.com/TimoKramer/datahike","commit_stats":null,"previous_names":[],"tags_count":166,"template":false,"template_full_name":null,"purl":"pkg:github/TimoKramer/datahike","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TimoKramer%2Fdatahike","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TimoKramer%2Fdatahike/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TimoKramer%2Fdatahike/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TimoKramer%2Fdatahike/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TimoKramer","download_url":"https://codeload.github.com/TimoKramer/datahike/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TimoKramer%2Fdatahike/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":280348105,"owners_count":26315376,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-21T02:00:06.614Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-21T23:10:00.657Z","updated_at":"2025-10-21T23:10:03.096Z","avatar_url":"https://github.com/TimoKramer.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003ca align=\"center\" href=\"https://datahike.io\" target=\"_blank\"\u003e\n    \u003cimg alt=\"Datahike\" src=\"./doc/assets/datahike-logo.svg\" height=\"128em\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n\u003ca href=\"https://clojurians.slack.com/archives/CB7GJAN0L\"\u003e\u003cimg src=\"https://badgen.net/badge/-/slack?icon=slack\u0026label\"/\u003e\u003c/a\u003e\n\u003ca href=\"https://clojars.org/io.replikativ/datahike\"\u003e \u003cimg src=\"https://img.shields.io/clojars/v/io.replikativ/datahike.svg\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://circleci.com/gh/replikativ/datahike\"\u003e\u003cimg src=\"https://circleci.com/gh/replikativ/datahike.svg?style=shield\"/\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/replikativ/datahike/tree/main\"\u003e\u003cimg src=\"https://img.shields.io/github/last-commit/replikativ/datahike/main\"/\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n[Datahike](https://datahike.io) is a durable [Datalog](https://en.wikipedia.org/wiki/Datalog) database\npowered by an efficient Datalog query engine. This project started as a port of\n[DataScript](https://github.com/tonsky/DataScript) to the\n[hitchhiker-tree](https://github.com/datacrypt-project/hitchhiker-tree). All\nDataScript tests are passing, but we are still working on the internals. Having\nsaid this we consider Datahike usable for medium sized projects, since DataScript is\nvery mature and deployed in many applications and the hitchhiker-tree\nimplementation is heavily tested through generative testing. We are\nbuilding on the two projects and the storage backends for the hitchhiker-tree\nthrough [konserve](https://github.com/replikativ/konserve). We would like to\nhear experience reports and are happy if you join us.\n\nYou can find [API documentation on cljdoc](https://cljdoc.org/d/io.replikativ/datahike) and articles on Datahike on our company's [blog page](https://lambdaforge.io/articles).\n\n[![cljdoc](https://badgen.net/badge/cljdoc/datahike/blue)](https://cljdoc.org/d/io.replikativ/datahike)\n\nWe presented Datahike also at meetups,for example at:\n\n- [2021 Bay Area Clojure meetup](https://www.youtube.com/watch?v=GG-S-xrDS5M)\n- [2019 scicloj online meetup](https://www.youtube.com/watch?v=Hjo4TEV81sQ).\n- [2019 Vancouver Meetup](https://www.youtube.com/watch?v=A2CZwOHOb6U).\n- [2018 Dutch clojure meetup](https://www.youtube.com/watch?v=W6Z1mkvqp3g).\n\n## Usage\n\nAdd to your dependencies:\n\n[![Clojars Project](http://clojars.org/io.replikativ/datahike/latest-version.svg)](http://clojars.org/io.replikativ/datahike)\n\nWe provide a small stable API for the JVM at the moment, but the on-disk schema\nis not fixed yet. We will provide a migration guide until we have reached a\nstable on-disk schema. _Take a look at the ChangeLog before upgrading_.\n\n```clojure\n(require '[datahike.api :as d])\n\n\n;; use the filesystem as storage medium\n(def cfg {:store {:backend :file :path \"/tmp/example\"}})\n\n;; create a database at this place, per default configuration we enforce a strict\n;; schema and keep all historical data\n(d/create-database cfg)\n\n(def conn (d/connect cfg))\n\n;; the first transaction will be the schema we are using\n;; you may also add this within database creation by adding :initial-tx\n;; to the configuration\n(d/transact conn [{:db/ident :name\n                   :db/valueType :db.type/string\n                   :db/cardinality :db.cardinality/one }\n                  {:db/ident :age\n                   :db/valueType :db.type/long\n                   :db/cardinality :db.cardinality/one }])\n\n;; lets add some data and wait for the transaction\n(d/transact conn [{:name  \"Alice\", :age   20 }\n                  {:name  \"Bob\", :age   30 }\n                  {:name  \"Charlie\", :age   40 }\n                  {:age 15 }])\n\n;; search the data\n(d/q '[:find ?e ?n ?a\n       :where\n       [?e :name ?n]\n       [?e :age ?a]]\n  @conn)\n;; =\u003e #{[3 \"Alice\" 20] [4 \"Bob\" 30] [5 \"Charlie\" 40]}\n\n;; add new entity data using a hash map\n(d/transact conn {:tx-data [{:db/id 3 :age 25}]})\n\n;; if you want to work with queries like in\n;; https://grishaev.me/en/datomic-query/,\n;; you may use a hashmap\n(d/q {:query '{:find [?e ?n ?a ]\n               :where [[?e :name ?n]\n                       [?e :age ?a]]}\n      :args [@conn]})\n;; =\u003e #{[5 \"Charlie\" 40] [4 \"Bob\" 30] [3 \"Alice\" 25]}\n\n;; query the history of the data\n(d/q '[:find ?a\n       :where\n       [?e :name \"Alice\"]\n       [?e :age ?a]]\n  (d/history @conn))\n;; =\u003e #{[20] [25]}\n\n;; you might need to release the connection for specific stores\n(d/release conn)\n\n;; clean up the database if it is not need any more\n(d/delete-database cfg)\n```\n\nThe API namespace provides compatibility to a subset of Datomic functionality\nand should work as a drop-in replacement on the JVM. The rest of Datahike will\nbe ported to core.async to coordinate IO in a platform-neutral manner.\n\nRefer to the docs for more information:\n\n- [backend development](./doc/backend-development.md)\n- [benchmarking](./doc/benchmarking.md)\n- [garbage collection](./doc/gc.md)\n- [contributing to Datahike](./doc/contributing.md)\n- [configuration](./doc/config.md)\n- [differences to Datomic](./doc/datomic_differences.md)\n- [entity spec](./doc/entity_spec.md)\n- [logging and error handling](./doc/logging_and_error_handling.md)\n- [schema flexibility](./doc/schema.md)\n- [time variance](./doc/time_variance.md)\n- [unstructured input support](./doc/unstructured.md) (experimental)\n- [versioning](./doc/versioning.md)\n\n\nFor simple examples have a look at the projects in the `examples` folder.\n\n## Example Projects\n\n- [Invoice creation](https://gitlab.com/replikativ/datahike-invoice)\n  demonstrated at the [Dutch Clojure\n  Meetup](https://www.meetup.com/de-DE/The-Dutch-Clojure-Meetup/events/trmqnpyxjbrb/).\n\n## Relationship to Datomic and DataScript\n\nDatahike provides similar functionality to [Datomic](http://Datomic.com) and can\nbe used as a drop-in replacement for a subset of it. The goal of Datahike is not\nto provide an open-source reimplementation of Datomic, but it is part of the\n[replikativ](https://github.com/replikativ) toolbox aimed to build distributed\ndata management solutions. We have spoken to many backend engineers and Clojure\ndevelopers, who tried to stay away from Datomic just because of its proprietary\nnature and we think in this regard Datahike should make an approach to Datomic\neasier and vice-versa people who only want to use the goodness of Datalog in\nsmall scale applications should not worry about setting up and depending on\nDatomic.\n\nSome differences are:\n\n- Datahike runs locally on one peer. A transactor might be provided in the\n  future and can also be realized through any linearizing write mechanism, e.g.\n  Apache Kafka. If you are interested, please contact us.\n- Datahike provides the database as a transparent value, i.e. you can directly\n  access the index datastructures (hitchhiker-tree) and leverage their\n  persistent nature for replication. These internals are not guaranteed to stay\n  stable, but provide useful insight into what is going on and can be optimized.\n- Datahike supports [GDPR](https://gdpr.eu/) compliance by allowing to [completely remove database entries](./doc/time_variance.md#data-purging).\n- Datomic has a REST interface and a Java API\n- Datomic provides timeouts\n\nDatomic is a full-fledged scalable database (as a service) built from the\nauthors of Clojure and people with a lot of experience. If you need this kind\nof professional support, you should definitely stick to Datomic.\n\nDatahike's query engine and most of its codebase come from\n[DataScript](https://github.com/tonsky/DataScript). Without the work on\nDataScript, Datahike would not have been possible. Differences to Datomic with\nrespect to the query engine are documented there.\n\n## When to Choose Datahike vs. Datomic vs. DataScript\n\n### Datahike\n\nPick Datahike if your app has modest requirements towards a typical durable\ndatabase, e.g. a single machine and a few millions of entities at maximum.\nSimilarly, if you want to have an open-source solution and be able to study and\ntinker with the codebase of your database, Datahike provides a comparatively\nsmall and well composed codebase to tweak it to your needs. You should also\nalways be able to migrate to Datomic later easily.\n\n### Datomic\n\nPick Datomic if you already know that you will need scalability later or if you\nneed a network API for your database. There is also plenty of material about\nDatomic online already. Most of it applies in some form or another to Datahike,\nbut it might be easier to use Datomic directly when you first learn Datalog.\n\n### DataScript\n\nPick DataScript if you want the fastest possible query performance and do not\nhave a huge amount of data. You can easily persist the write operations\nseparately and use the fast in-memory index data structure of DataScript then.\nDatahike also at the moment does not support ClojureScript anymore, although we\nplan to recover this functionality.\n\n## ClojureScript Support\n\nClojureScript support is planned and work in progress. Please see [Discussions](https://github.com/replikativ/datahike/discussions/categories/ideas).\n\n## Migration \u0026 Backup\n\nThe database can be exported to a flat file with:\n\n```clojure\n(require '[datahike.migrate :refer [export-db import-db]])\n(export-db conn \"/tmp/eavt-dump\")\n```\n\nYou must do so before upgrading to a Datahike version that has changed the\non-disk format. This can happen as long as we are arriving at version `1.0.0`\nand will always be communicated through the Changelog. After you have bumped the\nDatahike version you can use\n\n```clojure\n;; ... setup new-conn (recreate with correct schema)\n\n(import-db new-conn \"/tmp/eavt-dump\")\n```\n\nto reimport your data into the new format.\n\nThe datoms are stored in the CBOR format, enabling migration of binary data, such as the byte array data type now supported by Datahike. You can also use the export as a backup.\n\nIf you are upgrading from pre `0.1.2` where we have not had the migration code\nyet, then just evaluate the `datahike.migrate` namespace manually in your\nproject before exporting.\n\nHave a look at the [change log](./CHANGELOG.md) for recent updates.\n\n## Roadmap and Participation\n\nInstead of providing a static roadmap, we have moved to working closely with the community to decide what will be worked on next in a dynamic and interactive way.\n\nHow it works?\n\nGo to [Discussions](https://github.com/replikativ/datahike/discussions/categories/ideas) and upvote all the _ideas_ of features you would like to be added to Datahike. As soon as we have someone free to work on a new feature, we will address one with the most upvotes. \n\nOf course, you can also propose ideas yourself - either by adding them to the Discussions or even by creating a pull request yourself. Please note thought that due to considerations about incompatibilities to earlier Datahike versions it might sometimes take a bit more time until your PR is integrated.\n\n\n## Commercial Support\n\nWe are happy to provide commercial support with\n[lambdaforge](https://lambdaforge.io). If you are interested in a particular\nfeature, please let us know.\n\n## License\n\nCopyright © 2014–2023 Konrad Kühne, Christian Weilbach, Chrislain Razafimahefa, Timo Kramer, Judith Massa, Nikita Prokopov, Ryan Sundberg\n\nLicensed under Eclipse Public License (see [LICENSE](LICENSE)).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimokramer%2Fdatahike","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftimokramer%2Fdatahike","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimokramer%2Fdatahike/lists"}