How distributed ledgers scale❓

News
1

This is an explanation of @Shaul on how distributed ledgers scale.

He went through those topics:
:white_check_mark: #Data Flow
:white_check_mark: Horizontal #Scaling
:white_check_mark: Scaling writes to the #ledger

DAML Ledgers follow a Command Query Responsibility Segregation (CQRS), event-sourced architecture. This means:

  • Writing to the ledger (Command) is a first data flow, a sequence of transactions
  • The ledger is a long log of transactions and is the source of truth for all data
  • Reading from the ledger (Query) is a second, segregated data flow
  • In DAML, all interaction between the write and read path is through the immutable ledger, there are no out-of-bounds workflows.

Which parts are trivial to scale horizontally?

  • Serving the API - similar to any API endpoint
  • Read path - by duplicating the read stream and creating multiple caches (copies). The ledger is an immutable stream of events, so no synchronization between these is necessary
  • Write path - not so trivial because we need to ensure we’re not writing conflicting transactions to the ledger

The write path has to validate three things about the transaction being written to the ledger:

  1. Conforms to the rules of DAML ← Highest need for computing resources (CPU and memory)
  2. Is authorized by someone who is allowed to write it
  3. Doesn’t conflict with any previous transaction
    We can now split the write path into two stages, one is horizontally scalable and the other is sequential

For CQRS check this link and for even-sourced this one.

Any questions to @Shaul ?

6 Likes
2

Because of GDPR, besides reading and writing you sometimes need to delete data safely from the ledger. Canton supports this operation. How deleting data safely from the ledger fits into this picture?

1 Like
3

@gyorgybalazsi Canton-style GDPR deletion is like garbage collection; we throw away (sufficiently old) archived contracts, since they don’t influence the future behavior of the system. So writing to the ledger is in principle sufficient; as you archive contracts, they become candidates for garbage collection. The tricky part is to be able to garbage collect while maintaining the ability to prove that an active contract is still active (so-called non-repudiation). How you can do that depends on the ledger and the trust assumptions.

3 Likes
4

To build a bit on top of @oggy’s answer, in a GDPR request the first step is for the DAML workflow to archive the contract as part of servicing the individual’s request to delete their data. The second step is for the ledger itself to prune (delete) the archived data, in ledgers which support pruning.

1 Like