https://github.com/vmware/differential-datalog Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code + Explore + All features + Documentation + GitHub Skills + Blog * Solutions + By Plan + Enterprise + Teams + Compare all + By Solution + CI/CD & Automation + DevOps + DevSecOps + Case Studies + Customer Stories + Resources * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles + Repositories + Topics + Trending + Collections * Pricing [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this organization All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} vmware / differential-datalog Public * Notifications * Fork 103 * Star 1.1k DDlog is a programming language for incremental computation. It is well suited for writing programs that continuously update their output in response to input changes. A DDlog programmer does not write incremental algorithms; instead they specify the desired input-output mapping in a declarative manner. License MIT license 1.1k stars 103 forks Star Notifications * Code * Issues 126 * Pull requests 5 * Discussions * Actions * Projects 0 * Wiki * Security * Insights More * Code * Issues * Pull requests * Discussions * Actions * Projects * Wiki * Security * Insights vmware/differential-datalog This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. master Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags Name already in use A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch? Cancel Create 15 branches 115 tags Code * Clone HTTPS GitHub CLI [https://github.com/v] Use Git or checkout with SVN using the web URL. [gh repo clone vmware] Work fast with our official CLI. Learn more. * Open with GitHub Desktop * Download ZIP Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching Xcode If nothing happens, download Xcode and try again. Launching Visual Studio Code Your codespace will open once ready. There was a problem preparing your codespace, please try again. Latest commit @dependabot dependabot[bot] Bump calcite-core from 1.30.0 to 1.32.0 in /sql (# 1164) ... b3a3e14 Oct 3, 2022 Bump calcite-core from 1.30.0 to 1.32.0 in /sql (#1164) Bumps [calcite-core](https://github.com/apache/calcite) from 1.30.0 to 1.32.0. - [Release notes](https://github.com/apache/calcite/releases) - [Commits](apache/calcite@calcite-1.30.0...calcite-1.32.0) Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> b3a3e14 Git stats * 2,011 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time .github Override issue templates from the github org. Feb 11, 2022 adapters/ovsdb ovsdb2ddlog: --intern-strings option. Aug 19, 2021 app [Self-profiler revamp 2/2]: Profile generator. Nov 9, 2021 debugger Parse record from debugger dumped file Jul 10, 2020 doc Document and add a test for bit-shift operator behavior. Jul 19, 2022 gitlab-ci Use private docker repo in CI. Dec 22, 2020 go Implement original attribute for relations. Jun 17, 2021 java Fix recording of DDlogDynamic commands. Dec 6, 2021 lib Satisfy new Rust linter rules (#1149) Feb 12, 2022 rust Add missing Record::as_string() method. Aug 11, 2022 sql Bump calcite-core from 1.30.0 to 1.32.0 in /sql (#1164) Oct 3, 2022 src/Language/DifferentialDatalog Use LTS 18.28 for Apple M1 Jul 24, 2022 test Document and add a test for bit-shift operator behavior. Jul 19, 2022 tools improve souffle_converter.py (#1116) Nov 16, 2021 .gitignore Feature to panic on weight overflow Sep 13, 2021 .gitlab-ci.yml [Self-profiler revamp 2/2]: Profile generator. Nov 9, 2021 CHANGELOG.md Bump version to 1.2.3. Dec 13, 2021 CONTRIBUTING.md Contributor guide (#1150) Feb 12, 2022 Cargo.toml [Self-profiler revamp 2/2]: Profile generator. Nov 9, 2021 LICENSE Update LICENSE Mar 20, 2018 README.md [Self-profiler revamp 2/2]: Profile generator. Nov 9, 2021 Setup.hs stack-based project (#2) Apr 3, 2018 build_distro.sh build_distro.sh: Fix Windows check. Nov 10, 2021 default.nix Update default.nix Jun 24, 2021 install-dependencies.sh Avoid interactive prompt for apt install in install-dependencies.sh ( #... Nov 12, 2021 package.yaml Bump version to 1.2.3. Dec 13, 2021 stack.yaml Use LTS 18.28 for Apple M1 Jul 24, 2022 test.sh [Self-profiler revamp 2/2]: Profile generator. Nov 9, 2021 View code [ ] Differential Datalog (DDlog) Documentation Installation Installing DDlog from a binary release Compiling DDlog from sources Installing dependencies manually Building vim syntax highlighting Debugging with GHCi Building with profiling info enabled README.md License: MIT CI workflow pipeline status rustc Gitter chat Differential Datalog (DDlog) DDlog is a programming language for incremental computation. It is well suited for writing programs that continuously update their output in response to input changes. With DDlog, the programmer does not need to worry about writing incremental algorithms. Instead they specify the desired input-output mapping in a declarative manner, using a dialect of Datalog. The DDlog compiler then synthesizes an efficient incremental implementation. DDlog is based on Frank McSherry's excellent differential dataflow library. DDlog has the following key properties: 1. Relational: A DDlog program transforms a set of input relations (or tables) into a set of output relations. It is thus well suited for applications that operate on relational data, ranging from real-time analytics to cloud management systems and static program analysis tools. 2. Dataflow-oriented: At runtime, a DDlog program accepts a stream of updates to input relations. Each update inserts, deletes, or modifies a subset of input records. DDlog responds to an input update by outputting an update to its output relations. 3. Incremental: DDlog processes input updates by performing the minimum amount of work necessary to compute changes to output relations. This has significant performance benefits for many queries. 4. Bottom-up: DDlog starts from a set of input facts and computes all possible derived facts by following user-defined rules, in a bottom-up fashion. In contrast, top-down engines are optimized to answer individual user queries without computing all possible facts ahead of time. For example, given a Datalog program that computes pairs of connected vertices in a graph, a bottom-up engine maintains the set of all such pairs. A top-down engine, on the other hand, is triggered by a user query to determine whether a pair of vertices is connected and handles the query by searching for a derivation chain back to ground facts. The bottom-up approach is preferable in applications where all derived facts must be computed ahead of time and in applications where the cost of initial computation is amortized across a large number of queries. 5. In-memory: DDlog stores and processes data in memory. In a typical use case, a DDlog program is used in conjunction with a persistent database, with database records being fed to DDlog as ground facts and the derived facts computed by DDlog being written back to the database. At the moment, DDlog can only operate on databases that completely fit the memory of a single machine. We are working on a distributed version of DDlog that will be able to partition its state and computation across multiple machines. 6. Typed: In its classical textbook form Datalog is more of a mathematical formalism than a practical tool for programmers. In particular, pure Datalog does not have concepts like types, arithmetics, strings or functions. To facilitate writing of safe, clear, and concise code, DDlog extends pure Datalog with: 1. A powerful type system, including Booleans, unlimited precision integers, bitvectors, floating point numbers, strings, tuples, tagged unions, vectors, sets, and maps. All of these types can be stored in DDlog relations and manipulated by DDlog rules. Thus, with DDlog one can perform relational operations, such as joins, directly over structured data, without having to flatten it first (as is often done in SQL databases). 2. Standard integer, bitvector, and floating point arithmetic. 3. A simple procedural language that allows expressing many computations natively in DDlog without resorting to external functions. 4. String operations, including string concatenation and interpolation. 5. Syntactic sugar for writing imperative-style code using for/ let/assignments. 7. Integrated: while DDlog programs can be run interactively via a command line interface, its primary use case is to integrate with other applications that require deductive database functionality. A DDlog program is compiled into a Rust library that can be linked against a Rust, C/C++, Java, or Go program (bindings for other languages can be easily added). This enables good performance, but somewhat limits the flexibility, as changes to the relational schema or rules require re-compilation. Documentation * Follow the tutorial for a step-by-step introduction to DDlog. * DDlog language reference. * DDlog command reference for writing and testing your own Datalog programs. * How to use DDlog from Java. * How to use DDlog from C. * How to use DDlog from Go and Go API documentation. * How to use DDlog from Rust (by example) * Tutorial on profiling DDlog programs * DDlog overview paper, Datalog 2.0 workshop, 2019. Installation Installing DDlog from a binary release To install a precompiled version of DDlog, download the latest binary release, extract it from archive, add ddlog/bin to your $PATH, and set $DDLOG_HOME to point to the ddlog directory. You will also need to install the Rust toolchain (see instructions below). If you're using OS X, you will need to override the binary's security settings through these instructions. Else, when first running the DDlog compiler (through calling ddlog), you will get the following warning dialog: "ddlog" cannot be opened because the developer cannot be verified. macOS cannot verify that this app is free from malware. You are now ready to start coding in DDlog. Compiling DDlog from sources Installing dependencies manually * Haskell stack: wget -qO- https://get.haskellstack.org/ | sh * Rust toolchain v1.52.1 or later: curl https://sh.rustup.rs -sSf | sh . $HOME/.cargo/env rustup component add rustfmt rustup component add clippy Note: The rustup script adds path to Rust toolchain binaries (typically, $HOME/.cargo/bin) to ~/.profile, so that it becomes effective at the next login attempt. To configure your current shell run source $HOME/.cargo/env. * JDK, e.g.: apt install default-jdk * Google FlatBuffers library. Download and build FlatBuffers release 1.11.0 from github. Make sure that the flatc tool is in your $PATH. Additionally, make sure that FlatBuffers Java classes are in your $CLASSPATH: ./tools/install-flatbuf.sh cd flatbuffers export CLASSPATH=`pwd`"/java":$CLASSPATH export PATH=`pwd`:$PATH cd .. * Static versions of the following libraries: libpthread.a, libc.a, libm.a, librt.a, libutil.a, libdl.a, libgmp.a, and libstdc++.a can be installed from distro-specific packages. On Ubuntu: apt install libc6-dev libgmp-dev On Fedora: dnf install glibc-static gmp-static libstdc++-static Building To build the software once you've installed the dependencies using one of the above methods, clone this repository and set $DDLOG_HOME variable to point to the root of the repository. Run stack build anywhere inside the repository to build the DDlog compiler. To install DDlog binaries in Haskell stack's default binary directory: stack install To install to a different location: stack install --local-bin-path To test basic DDlog functionality: stack test --ta '-p path' Note: this takes a few minutes You are now ready to start coding in DDlog. vim syntax highlighting The easiest way to enable differential datalog syntax highlighting for .dl files in Vim is by creating a symlink from / tools/vim/syntax/dl.vim into ~/.vim/syntax/. If you are using a plugin manager you may be able to directly consume the file from the upstream repository as well. In the case of Vundle, for example, configuration could look as follows: call vundle#begin('~/.config/nvim/bundle') ... Plugin 'vmware/differential-datalog', {'rtp': 'tools/vim'} <---- relevant line ... call vundle#end() Debugging with GHCi To run the test suite with the GHCi debugger: stack ghci --ghci-options -isrc --ghci-options -itest differential-datalog:differential-datalog-test and type do main in the command prompt. Building with profiling info enabled stack clean followed by stack build --profile or stack test --profile About DDlog is a programming language for incremental computation. It is well suited for writing programs that continuously update their output in response to input changes. A DDlog programmer does not write incremental algorithms; instead they specify the desired input-output mapping in a declarative manner. Topics programming-language rust incremental datalog ddlog Resources Readme License MIT license Code of conduct Code of conduct Security policy Security policy Stars 1.1k stars Watchers 22 watching Forks 103 forks Releases 110 DDlog v1.2.3 Latest Dec 13, 2021 + 109 releases Packages 0 No packages published Contributors 33 * @ryzhyk * @mbudiu-vmw * @d-e-s-o * @Kixiron * @lalithsuresh * @smadaminov * @amytai * @falzberger * @blp * @krs85 * @arkivm + 22 contributors Languages * Java 37.7% * Rust 27.0% * Haskell 24.7% * C 2.9% * Python 2.5% * Shell 1.4% * Other 3.8% Footer (c) 2022 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.