Obviously this might depend on the underlying ledgers, but I think two things that would help to understand this aspect for DAML driven ledgers:
is there an abstraction where it can be a user choice to keep data as binary, opentext, encrypted? May be this flexibility is a bad thing - thoughts?
Whats default for various DAML driven ledgers?
thanks
I’m not close enough to any of the ledger implementation to give you specific details, but I can tell you one thing they all have in common: we consider the on-ledger storage format to be an internal implementation detail, and users should only ever interact with DAML data through the DAML API.
As explained in this thread, for the case of Fabric, we’re not actively trying to make it impossible for people to get access to the data (at least for ledgers on which the data is not encrypted), but it is unsupported, subject to change, and all the other bad things that come with relying on implementation details.
What @Gary_Verhaegen said, but in practice I believe most ledgers internally store data (ie contract arguments) as binary encoded protobuf messages in the DAML-LF value schema.
DAML on SQL and the participant index both store transactions as individual nodes, so that they can be filtered down according to privacy rules at the storage level to minimize the data moved around.
Each event has a set of structured data associated with it (like the identifier of the transaction or of the contract on which the event occurs), plus a few Protobuf-encoded BLOBs which represent DAML-LF values: contract creation argument and key for create events, exercise argument and result for exercise events. One of the reasons that lead to this choice is that our libraries to handle DAML-LF values already provide a way to seamlessly read values persisted in an old version of the encoding, to ensure transparent backwards compatibility. I would say there isn’t much to gain from storing these values in different formats.
With regards to “various DAML ledgers”, the interface between the participant and the ledger storage is defined in terms of DAML-LF transactions. Ledgers can decide to store full transactions as Protobuf BLOBs, find a way to store each piece of information in a structured (or semi-structured) fashion, or somewhere in the middle of the road (as DAML on SQL does). There isn’t anything in the interface that forces any specific approach, but different approaches have different pros and cons (in our case, the pros for our approach were facilitated backwards compatibility and pushing down Ledger API predicates, where a con could be not being able to perform more granular queries on individual contract fields).
As mentioned in the very first answer, this is entirely an implementation detail which no user should rely upon, if not for internal maintenance of components. The Ledger API is the only reliable gateway to the data stored on DAML ledgers.