DAML Driven Ledgers - ACID vs eventual consistency?

If the underlying ledger is ACID vs using eventual consistency, what aspect/properties are lost that developers need to be aware when working with DAML driven ledgers?

1 Like

I’m not sure I understand your question, but I guess you are asking what tradeoffs DAML is making between Consistency, Availability, and Partition Tolerance since we can’t have all three by the CAP Theorem. I’m not sure why you say that competes with ACID, though. ACID talks about the properties of a transaction, CAP talks about the behaviour of a system in case of network partition.

All DAML ledgers have full Atomicity, Consistency, and Isolation - ACI of ACID. Durability depends on the underlying infrastructure. Of course an in-memory Ledger doesn’t have durability. A Ledger with a SQL backing has roughtly the durability properties of that SQL backend.

On the CAP side, DAML itself doesn’t really make any tradeoffs. Rather, DAML is designed to have an execution model that can be run on a range of platforms that make very different tradeoffs in that regard.

DAML can be run on plain old Blockchains in which case you get plain old eventual consistency, meaning 49% of the network may diverge temporarily.

DAML can be run on a centralised database in which case you get full consistency, but no availability in case of network failure.

Really, the consistency vs availability tradeoff in case of network failure is determined by the infrastructure DAML runs on since that infrastructure synchronises the data between nodes and manages how data is committed.

7 Likes

As @bernhard said, ACID is a property of a transaction, and that transaction can be committed via a variety of protocols, some of which can be described as “eventually consistent”. To call the DAML Ledger Model “Eventually Consistent” can be misleading if you aren’t careful. The original, and to this day most?, eventually consistent transaction models relied on operational transforms, crdt’s, lattices, or similar algebraic structures. The result was the result of a committed transaction would eventually be consistent with some non-deterministic ordering of the transaction with other events/transactions in the system. This is nothing like what the DAML ledger model provides.

The DAML Ledger Model is completely deterministic from the perspective of the ledger participant. The transaction is fully evaluated before it is submitted, and is only committed if that evaulation remains consistent at the point it is sequenced on the ledger. All of the input parameters, all of the output events, and all relevant ledger state is captured as a part of evaluation. DAML itself is 100% deterministic with respect to input parameters and ledger state. Therefore before a participant node submits a transaction it knows with 100% certainty the updated ledger state should the transaction commit.

The only uncertainty is will the transaction remain valid by the time it is to be sequenced?

This means that with enough parallelisation, we can theoretically support extremely high throughput for contention free workflows. On the flip side, it means that the latency penalty for contention between workflows is also high.

This architecture has two important consequences for engineering DAML applications:

  1. The transaction is evaluated by the participant and then checked for consistency by the committer/sequencer. Therefore, all antecedents when reasoning within a DAML workflow must be guarded by predicates that will be checked by the committer/sequencer. Generally this means strictly monotonic reasoning based on existentials (ie. fetch). The one non-monotonic exception to this is the key uniqueness check when creating a contract. So if you absolutely must use non-monotonic reasoning, you must find a way to express it in terms of key uniqueness.

  2. Interactions with the ledger that rely on seeing a commit before proceeding are a) very slow; and b) extremely vulnerable to any contention. Almost all DAML performance engineering consists of a) finding and eliminating contract contention; b) finding ways to decompose operations into large numbers of independent, non-contending, ledger operations (high fan-out fork); while, minimising the number of resynchronisations (ie. joins). The real challenge is then doing this without violating the monotonic reasoning constraint above.

It takes a little getting used to, but I can speak from experience when I say that once you do, it is an extremely elegant and pleasant model to work with — and the performance you can get, once you understand how to lean into its strengths, is incredible.

6 Likes

ACID is a fuzzy term as it can mean a lot of things; e.g., there are many different isolation levels that behave quite differently (yet are all “ACID”), and “atomicity” is usually not meaningfully separable from “isolation”. As @bernhard hints, these notions come from the DB tradition, and are usually understood to talk about a single DB server. The eventual consistency term, on the other hand, comes from the distributed systems tradition, where terms usually refer to operations on a single replicated object (rather than multiple objects as in the DB tradition). People have really started looking at the “distributed ACID” notions relatively recently, e.g., this 2014 VLDB paper by Bailis. The point being, the language for the question you’re asking (or I think you’re asking) is kind of still in the process of construction.

In practice, I’d expect all “enterprise DLT”-based DAML Ledgers that are based on global-state topologies to be implemented by either a single central node, or a linearizable (in distributed system speak) replicated version of such a node, replicated using some form of classical consensus (aka total order broadcast) algorithm. This would give you a linearizable (or strictly serializable, if you prefer to think of a DAML transaction as operating on multiple objects) ledger. Aphyr’s write-up if a fun read on what these terms mean. In a nutshell, this would mean that you can think of the global DAML ledger as a linear sequence of events, and the system’s functional behavior (in terms of commands you send and transactions you receive) should be mostly indistinguishable from a DAML sandbox. Availability might improve over a sandbox, thanks to replication, but it can still be compromised if sufficiently many replicas are down, or if there is a network partition.

One exception would be a global state topology over an “open-membership” consensus algorithm such as proof of work or proof of stake. Here, I think we’re also in the process of building the language to describe the guarantees. I’d describe the transactions as still running atomically (i.e., in full isolation; in Bailis’s terms, I believe this would be a “monotonic atomic view” guarantee), but there is always a chance that a transaction’s effects will simply get ignored afterwards the transaction stream that you receive might have to be “rewound”, in case of forks to the underlying ledger. So the ledger is more like a tree order of events, where the transaction stream can decide to “jump” from one branch of the partial order to another unrelated branch at any point in time. You can lower the risk of the jump at the cost of adding (rather significant) latency. OTOH, availability-wise, such a ledger should be superior to a ledger replicated via “classic” consensus.

Ledger interoperability protocols such as Canton are also an interesting case. I’m also not sure if the behavior can be fully described with existing terms. You’d definitely get “monotonic atomic views” again. Moreover, you’ll never have to rewind your transaction stream. However, the “virtual global DAML ledger” that you access through your participant would be partially ordered, as the participant can connect to multiple ledgers, and can pick how to order unrelated events from two different ledgers. So different participants may output things in a different order. @Andreas_Lochbihler is in the process of clearing the ordering guarantees up. Availability-wise, it’s also an interesting story, because the system can be left in a “partly-available” state, where if some of the underlying ledgers become unavailable, some transactions could not be processed any more, while processing others might still be possible.

5 Likes

Moving to #general as this is an in-depth discussion without a single answer :slight_smile: