Selective data redaction in Daml applications

Questions
, ,
1

In the following I seek answers and advices on how to redact data selectively in Daml applications. I hope I described the problem space well enough, and in a generally applicable way.

Problem space

  • Assume to have a set of templates for which the corresponding contracts should be redacted.
  • Redaction is needed for regulatory reasons.
  • In this context redaction means to replace sensitive information with dummy values instead of complete removal of data.
  • Since contracts are immutable “replacing” would effectively mean archive and recreate.
  • Redacted contracts should be accessible in some form. This means these might be archived.

Questions

  1. Am I right to think the redaction feature in PQS is not suitable for such purposes?
  2. Am I right to think that stable, application specific identifiers are required that the original contracts and their redacted counterparts would share, allowing the redacted contracts to be retrieved?
  3. Am I right to think that after redaction the ledger, all participant and PQS instances must be pruned to be compliant with such regulations?
  4. Am I right to think that the complexity of such a redaction feature on a large scale and in a generic fashion is close to the complexity of the upgrade tool? (Fail tolerance, potential batching, etc.)
2

I’m afraid that the problem description is a bit too vague to give an authoritative answer. Let me nevertheless provide some context:

Contracts on ledger cannot be redacted. This means that data inside a contract disappears only once the contract is archived and the participant nodes that have witnessed the contract during transaction processing have been pruned sufficiently. Witnessing is important here: Not only must we prune the participants that host a stakeholder of the contract, but also all participants of parties that have seen the creation of the contract or an exercise or fetch node that used the contract.

Redaction in PQS merely removes the contract data from the database rows of the targeted PQS instance. It does not affect the data stored on the corresponding participant node. Depending on the requirements of the application, this may or may not suffice for your use case.

An application may store contracts in further places. If you need to redact them in there as well, you’ll have to find an application-specific approach to this.

So in summary: Which copies of a contract do you need to be redacted for regulatory reasons?

3

I’ve noticed that my post could be misinterpreted to change contracts “inplace”. I’m aware contracts are immutable, so I updated the original post to avoid this confusion.

@Andreas_Lochbihler I tried to be generic, not vague. I think your answer confirms my points.

4

Indeed. I understood your original post that you want to redact the data in place. If you want to rewrite contracts by archiving and recreating them, then I fully agree with your points.

1 Like