{"id":13623873,"url":"https://github.com/vmware/differential-datalog","last_synced_at":"2025-03-10T06:49:46.938Z","repository":{"id":33039109,"uuid":"126076620","full_name":"vmware/differential-datalog","owner":"vmware","description":"DDlog is a programming language for incremental computation. It is well suited for writing programs that continuously update their output in response to input changes. A DDlog programmer does not write incremental algorithms; instead they specify the desired input-output mapping in a declarative manner.","archived":false,"fork":false,"pushed_at":"2023-07-07T21:56:01.000Z","size":309115,"stargazers_count":1399,"open_issues_count":140,"forks_count":122,"subscribers_count":31,"default_branch":"master","last_synced_at":"2025-03-01T06:11:12.472Z","etag":null,"topics":["datalog","ddlog","incremental","programming-language","rust"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vmware.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2018-03-20T20:14:11.000Z","updated_at":"2025-02-28T12:00:43.000Z","dependencies_parsed_at":"2023-01-14T23:15:18.241Z","dependency_job_id":"958ed22b-564a-4d41-9c31-7e604fe6a67b","html_url":"https://github.com/vmware/differential-datalog","commit_stats":{"total_commits":1993,"total_committers":42,"mean_commits":47.45238095238095,"dds":0.5067737079779226,"last_synced_commit":"ca5ded823dc2c01bc33aac44e526c12d2fed03a6"},"previous_names":[],"tags_count":115,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmware%2Fdifferential-datalog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmware%2Fdifferential-datalog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmware%2Fdifferential-datalog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmware%2Fdifferential-datalog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vmware","download_url":"https://codeload.github.com/vmware/differential-datalog/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242805888,"owners_count":20187999,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["datalog","ddlog","incremental","programming-language","rust"],"created_at":"2024-08-01T21:01:36.559Z","updated_at":"2025-03-10T06:49:46.911Z","avatar_url":"https://github.com/vmware.png","language":"Java","readme":"[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)\n[![CI workflow](https://github.com/vmware/differential-datalog/actions/workflows/main.yml/badge.svg)](https://github.com/vmware/differential-datalog/actions)\n[![pipeline status](https://gitlab.com/ddlog/differential-datalog/badges/master/pipeline.svg)](https://gitlab.com/ddlog/differential-datalog/commits/master)\n[![rustc](https://img.shields.io/badge/rustc-1.52.1+-blue.svg)](https://blog.rust-lang.org/2021/05/10/Rust-1.52.1.html)\n[![Gitter chat](https://badges.gitter.im/vmware/differential-datalog.png)](https://gitter.im/vmware/differential-datalog)\n\n# Differential Datalog (DDlog)\n\nDDlog is a programming language for *incremental computation*. It is well suited for\nwriting programs that continuously update their output in response to input changes. With DDlog,\nthe programmer does not need to worry about writing incremental algorithms.\nInstead they specify the desired input-output mapping in a declarative manner, using a dialect of Datalog.\nThe DDlog compiler then synthesizes an efficient incremental implementation.\nDDlog is based on [Frank McSherry's](https://github.com/frankmcsherry/)\nexcellent [differential dataflow](https://github.com/TimelyDataflow/differential-dataflow) library.\n\nDDlog has the following key properties:\n\n1. **Relational**: A DDlog program transforms a set of input relations (or tables) into a set of output relations.\nIt is thus well suited for applications that operate on relational data, ranging from real-time analytics to\ncloud management systems and static program analysis tools.\n\n2. **Dataflow-oriented**: At runtime, a DDlog program accepts a *stream of updates* to input relations.\nEach update inserts, deletes, or modifies a subset of input records. DDlog responds to an input update\nby outputting an update to its output relations.\n\n3. **Incremental**: DDlog processes input updates by performing the minimum amount of work\nnecessary to compute changes to output relations.  This has significant performance benefits for many queries.\n\n4. **Bottom-up**: DDlog starts from a set of input facts and\ncomputes *all* possible derived facts by following user-defined rules, in a bottom-up fashion.  In\ncontrast, top-down engines are optimized to answer individual user queries without computing all\npossible facts ahead of time.  For example, given a Datalog program that computes pairs of connected\nvertices in a graph, a bottom-up engine maintains the set of all such pairs.  A top-down engine, on\nthe other hand, is triggered by a user query to determine whether a pair of vertices is connected\nand handles the query by searching for a derivation chain back to ground facts.  The bottom-up\napproach is preferable in applications where all derived facts must be computed ahead of time and in\napplications where the cost of initial computation is amortized across a large number of queries.\n\n5. **In-memory**: DDlog stores and processes data in memory.  In a typical use case, a DDlog program\nis used in conjunction with a persistent database, with database records being fed to DDlog as\nground facts and the derived facts computed by DDlog being written back to the database.\n\n    At the moment, DDlog can only operate on databases that completely fit the memory of a single\n    machine. We are working on a distributed version of DDlog that will be able to\n    partition its state and computation across multiple machines.\n\n6. **Typed**: In its classical textbook form Datalog is more of a mathematical formalism than a\npractical tool for programmers.  In particular, pure Datalog does not have concepts like types,\narithmetics, strings or functions.  To facilitate writing of safe, clear, and concise code, DDlog\nextends pure Datalog with:\n\n    1. A powerful type system, including Booleans, unlimited precision integers, bitvectors, floating point numbers, strings,\n    tuples, tagged unions, vectors, sets, and maps. All of these types can be\n    stored in DDlog relations and manipulated by DDlog rules.  Thus, with DDlog\n    one can perform relational operations, such as joins, directly over structured data,\n    without having to flatten it first (as is often done in SQL databases).\n\n    2. Standard integer, bitvector, and floating point arithmetic.\n\n    3. A simple procedural language that allows expressing many computations natively in DDlog without resorting to external functions.\n\n    4. String operations, including string concatenation and interpolation.\n\n    5. Syntactic sugar for writing imperative-style code using for/let/assignments.\n\n7. **Integrated**: while DDlog programs can be run interactively via a command line interface, its\nprimary use case is to integrate with other applications that require deductive database\nfunctionality.  A DDlog program is compiled into a Rust library that can be linked against a Rust,\nC/C++, Java, or Go program (bindings for other languages can be easily added).  This enables good performance,\nbut somewhat limits the flexibility, as changes to the relational schema or rules require re-compilation.\n\n## Documentation\n\n- Follow the [tutorial](doc/tutorial/tutorial.md) for a step-by-step introduction to DDlog.\n- DDlog [language reference](doc/language_reference/language_reference.md).\n- DDlog [command reference](doc/command_reference/command_reference.md) for writing and testing your own Datalog programs.\n- [How to](doc/java_api.md) use DDlog from Java.\n- [How to](doc/c_tutorial/c_tutorial.rst) use DDlog from C.\n- [How to](go/README.md) use DDlog from Go and [Go API documentation](https://pkg.go.dev/github.com/vmware/differential-datalog/go/pkg/ddlog).\n- [How to](test/datalog_tests/rust_api_test) use DDlog from Rust (by example)\n- [Tutorial](doc/profiling/profiling.md) on profiling DDlog programs\n- [DDlog overview paper](doc/datalog2.0-workshop/paper.pdf), Datalog 2.0 workshop, 2019.\n\n## Installation\n\n### Installing DDlog from a binary release\n\nTo install a precompiled version of DDlog, download the [latest binary release](https://github.com/vmware/differential-datalog/releases), extract it from archive, add `ddlog/bin` to your `$PATH`, and set `$DDLOG_HOME` to point to the `ddlog` directory. You will also need to install the Rust toolchain (see instructions below).\n\nIf you're using OS X, you will need to override the binary's security settings through [these instructions](https://support.apple.com/guide/mac-help/open-a-mac-app-from-an-unidentified-developer-mh40616/mac). Else, when first running the DDlog compiler (through calling `ddlog`), you will get the following warning dialog:\n```\n\"ddlog\" cannot be opened because the developer cannot be verified.\nmacOS cannot verify that this app is free from malware.\n```\n\nYou are now ready to [start coding in DDlog](doc/tutorial/tutorial.md).\n\n### Compiling DDlog from sources\n\n#### Installing dependencies manually\n\n- Haskell [stack](https://github.com/commercialhaskell/stack):\n  ```\n  wget -qO- https://get.haskellstack.org/ | sh\n  ```\n- Rust toolchain v1.52.1 or later:\n  ```\n  curl https://sh.rustup.rs -sSf | sh\n  . $HOME/.cargo/env\n  rustup component add rustfmt\n  rustup component add clippy\n  ```\n  **Note:** The `rustup` script adds path to Rust toolchain binaries (typically, `$HOME/.cargo/bin`)\n  to `~/.profile`, so that it becomes effective at the next login attempt.  To configure your current\n  shell run `source $HOME/.cargo/env`.\n- JDK, e.g.:\n  ```\n  apt install default-jdk\n  ```\n- Google FlatBuffers library.  Download and build FlatBuffers release 1.11.0 from\n  [github](https://github.com/google/flatbuffers/releases/tag/v1.11.0).  Make sure\n  that the `flatc` tool is in your `$PATH`.  Additionally, make sure that FlatBuffers\n  Java classes are in your `$CLASSPATH`:\n  ```\n  ./tools/install-flatbuf.sh\n  cd flatbuffers\n  export CLASSPATH=`pwd`\"/java\":$CLASSPATH\n  export PATH=`pwd`:$PATH\n  cd ..\n  ```\n- Static versions of the following libraries: `libpthread.a`, `libc.a`, `libm.a`, `librt.a`, `libutil.a`,\n  `libdl.a`, `libgmp.a`, and `libstdc++.a` can be installed from distro-specific packages.  On Ubuntu:\n  ```\n  apt install libc6-dev libgmp-dev\n  ```\n  On Fedora:\n  ```\n  dnf install glibc-static gmp-static libstdc++-static\n  ```\n\n#### Building\n\nTo build the software once you've installed the dependencies using one of the\nabove methods, clone this repository and set `$DDLOG_HOME` variable to point\nto the root of the repository.  Run\n\n```\nstack build\n```\n\nanywhere inside the repository to build the DDlog compiler.\nTo install DDlog binaries in Haskell stack's default binary directory:\n\n```\nstack install\n```\n\nTo install to a different location:\n\n```\nstack install --local-bin-path \u003ccustom_path\u003e\n```\n\nTo test basic DDlog functionality:\n\n```\nstack test --ta '-p path'\n```\n\n**Note:** this takes a few minutes\n\nYou are now ready to [start coding in DDlog](doc/tutorial/tutorial.md).\n\n### vim syntax highlighting\n\nThe easiest way to enable differential datalog syntax highlighting for `.dl` files in Vim is by\ncreating a symlink from `\u003cddlog-folder\u003e/tools/vim/syntax/dl.vim` into `~/.vim/syntax/`.\n\nIf you are using a plugin manager you may be able to directly consume the file from the upstream\nrepository as well. In the case of [`Vundle`](https://github.com/VundleVim/Vundle.vim), for example,\nconfiguration could look as follows:\n\n```vim\ncall vundle#begin('~/.config/nvim/bundle')\n...\nPlugin 'vmware/differential-datalog', {'rtp': 'tools/vim'} \u003c---- relevant line\n...\ncall vundle#end()\n```\n\n## Debugging with GHCi\n\nTo run the test suite with the GHCi debugger:\n\n```\nstack ghci --ghci-options -isrc --ghci-options -itest differential-datalog:differential-datalog-test\n```\n\nand type `do main` in the command prompt.\n\n## Building with profiling info enabled\n\n```\nstack clean\n```\n\nfollowed by\n\n```\nstack build --profile\n```\n\nor\n\n```\nstack test --profile\n```\n","funding_links":[],"categories":["Java","JVM语言","Technologies"],"sub_categories":["SpreadSheets"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvmware%2Fdifferential-datalog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvmware%2Fdifferential-datalog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvmware%2Fdifferential-datalog/lists"}