Where are private keys stored?

I am fairly new to blockchain, but it is my understanding that generally, the way something like Sawtooth operates is that any actor in the system has a pair of public and private keys. The private key is used to sign the transactions in the blockchain (the signatory part in the DAML temples).

It is my understanding that it is intended that operations on the ledger can be mediated through the DAML HTTP JSON API.

However, the documentation for the DAML HTTP JSON API seems to indicate the only a JWT is required to perform actions on the ledger.

So, my question is: where are the private keys stored? Storing them somewhere in the node seems super insecure and defeats the whole purpose of the blockchain.

5 Likes

Hi @iko! DAML was designed to be blockchain agnostic. That means that the same DAML code can be deployed to several different ledgers (aka blockchains). For this, DAML abstracts the concept of signatories and leaves it to the underlying ledger to implement cryptographic primitives for signatures. In particular, the location of private keys in the system depends on the ledger you choose to deploy your DAML code on. Also, this underlying ledger will define what exactly a node is and what its capabilities and trust assumptions are. This could be a fully centralized architecture, say a central Postgres database server, or something fully decentralized more like Sawtooth.
The HTTP JSON API provides a simple ledger API that tries to capture most needs of typical applications. The JWT authorizes to read/act as a user by means of a token, but the provider of that token is again to your choosing. You could for example use an external provider like auth0.com, but you can just as well run your own provider somewhere in the system. Again, you’re very free to choose the security architecture that you like and trust. For example, you could just skip the HTTP JSON API altogether and run directly against the underlying ledger API.
I hope this answers your question!

3 Likes

Thanks for the answer!

After reading the DAML integration kit architectural overview things are still really fuzzy, but it seems like the evaluation of the DAML smart contract code is in the general case performed external to the actual host ledger (backend). (It definitely is in the case of using PostgreSQL as a backend.)

What is unclear from all of the documentation is: if I decide to write the smart contract logic in DAML, will I still be able to interact with the underlying ledger technology without going through DAML services (the DAML Ledger API)?

For example, Hyperledger Sawtooth has a list of languages that it supports writing smart contracts in (DAML not being hone of them). After I write and deploy smart contract logic using daml-on-sawtooth, will I be able to forget about the fact that the underlying logic was written in DAML and interact directly with Sawtooth APIs? Or will writing the underlying logic in DAML force me to only use some sort of DAML-wrapped version of Sawtooth? The fact that the DAML code is executed outside of the Sawtooth nodes seems to indicate the necessity of a wrapper.

2 Likes

After a bit more reading, it seems like Sawtooth in particular has an architecture where transaction validators are separate processes. It seems like in the case of Sawtooth in particular I am able to interact with the underlying ledger directly.

1 Like

To understand the general architecture of DAML ledgers, I recommend you have a look at https://docs.daml.com/app-dev/app-arch.html. The DAML integration kit documents how you can write your own integration of DAML to a yet unsupported ledger and might be a bit overwhelming to start with and there are already plenty of integrations that you can use and don’t have to write from scratch.

Usually in these integrations validation of the DAML smart contract code is run on a node as part of the validation of received transactions.

To answer your question on whether you can still access the underlying ledger technology: The answer is yes, you can (if you have the necessary access).

However, the whole idea of DAML is that you should be able to forget about the underlying architecture, not the other way around. That means you should write your high-level code in DAML. It will describe your business logic: who needs to sign what, and what is the process to arrive at a signature.
The low-level processes like node to node communication, consensus algorithms, cryptographic signatures, etc are taken care of by the DAML integration and the underlying ledger. That means you should be able to completely forget that you are writing your application against say Sawtooth and only focus to get the business logic right in DAML. This will also allow you to develop applications a lot faster because you don’t have to care for all the complexities of the underlying ledger.

2 Likes

I understand that DAML aims to abstract the underlying backend and allow me to focus on the business logic.

However, there are situations when the project requires high security guarantees like storing the private key only on the client side, and the requirement of transparency of all operations – the ability to view all transactions of all parties anonymously.

As far as I understand, neither of the two requirements can be met by the high-level DAML API, which requires me to dive into the lower level of a specific backend.

That being said, I think DAML is still worth using because it makes writing smart contract logic more pleasant and more familiar to developers with FP experience.

Please correct me if I misunderstand something.

1 Like

Hi Iko, welcome to the forum.

My very rudimentary understanding of this is that if you write to the ledger using DAML, you can inspect it through the underlying ledger’s native APIs, but the data that stored on those contracts will be DAML ‘bytecode’, and depending on the underlying ledger security model may be encrypted.

So I suspect you could read those back through another language, but would then need to decrypt them and then decode the DAML-LF (bytecode) into something human-interpretable.

1 Like

In all DAML integrations you can in principle use the native APIs. In DAML for PostgreSQL you can query the database. In DAML for Sawtooth you can use Sawtooth’s APIs to extract data, etc.

However, what you’ll see on the native APIs is far from uniform, and writing via the native APIs may well get you into a bad state. In DAML for PostgreSQL, for example, it would be easy to write an invalid transaction to the database.

What is fairly uniform is that DAML transactions are written to the underlying persistence (Sawtooth in your case) as binary-encoded protobuf messages. You can easily extract those with a block explorer or via an API, ad decode them in a client application.

As for you question about private keys, the trust boundary DAML tries to solve for with decentralized ledgers like Sawtooth is between nodes, not between node clients and nodes. That’s why there is no signing between client and node, and only the nodes have cryptographic identity on ledger, not the individual parties.

2 Likes

…the ledger using DAML, you can inspect it through the underlying ledger’s native APIs, but the data that stored on those contracts will be DAML ‘bytecode’…

That seems to line up with what I have experienced.

It is really suboptimal. Basically using DAML forces me to only use DAML tools in the future.

How hard would it be to extract the DAML-LF parser? What is it currently written in? And is there a spec of the DAML-LF representation?

1 Like

Okay, it now seems to me that using DAML would only complicate things for me.

1 Like

So, DAML requires that if I don’t trust the network as a whole, then every party would basically need to run its own node.

1 Like

Parsing DAML-LF is quite easy as it’s supported by the whole protobuf ecosystem. Note that you don’t have to parse the whole of DAML-LF, either, which includes packages, but just DAML-LF values and transactions, which are the things that get persisted when transactions are committed.

Your assessment of the trust assumptions is not far off. DAML is not designed for public networks where individual clients hold their own private keys to act as parties. As a client, you need to trust a node, either your own one, or a hosted one. Nodes don’t need to trust each other, however, so you don’t need to trust the whole network as you suggest, but you do need to trust one node, at least.

1 Like

@iko I should also note that we are actively thinking about a feature to enable cryptographic signing between client application and DAML Ledger API. This would reduce the trust assumptions further, but our current thinking is to not link that “external” signing to the on-chain cryptographic identities, as it would be close to impossible to have a usable API across all ledger integrations for that.

1 Like

I am not intimately familiar with the matter, and you have probably already thought about what I am about to propose, but I would still like to propose it for completion’s sake.

It seems to me like it would be possible to add an opaque “Key” type to the DAML Ledger API. What the “Key” actually contains would depend on the underlying backend, but it wouldn’t change the API – with any backend the user would get the “Key” when allocating a Party, and would pass it to the API when performing any action.

2 Likes

It’s a bit more complex than that. Since you don’t want to trust any node, no node must ever see your private key. The other challenge is that DAML supports ledgers where there is no central infrastructure that has all the data. Eg in DAML for Corda, or on Canton enabled DAML Ledgers, there is no central place where transactions are stored and can be retrieved. Transactions for a party are only available on a node hosting that party - that’s the privacy promise of DAML Ledgers with full privacy. In order for you do be able to receive transactions, there has to be a node that’s entitled to at least “read” on your behalf.

So at a minimum, the the identity management system has to enable you to create an identity without sharing the secret with anyone, and to have your party hosted on another node (either read-only or read-write). The Identity Management in Canton enables this. There are extensive docs on that here: https://www.canton.io/docs/stable/user-manual/architecture/identity.html

That is to say that your use-case may well be supported by a Canton-enabled DAML Ledger. Canton is currently available as a pre-alpha in a binary distribution here.

1 Like

Has there been any progress on this? This is a DAML show stopper for me.

I agree they don’t need to be tied to the underlying ledger cryptographic identities, but they should form part of the DAML application layer.

1 Like

Hi @CatDanMo, welcome to the Daml Forums!
Yes, in response to a number of requests in this direction, we added a feature that validates and stores user-signatories to make sure the non-repudiation chain extends all the way to the end-user/agent that submitted a request.

1 Like

Thanks. This looks like great progress. Couple of follow up questions if you don’t mind.

  1. do you plan to remove the dependency on Postgres such that the commands are stored in the underlying ledger?
  2. are you planning to add full support for REST?
  3. any ideas when this will be available as a stable feature?

I really like what you guys are doing, but I cannot commit to DAML without knowing I’ll have access to this for my particular application.

Great work though!

  1. Yes, storing the commands and signatures on-ledger can be a good idea to reduce the number of components in play. That “mode” isn’t currently implemented, but wouldn’t be all that expensive to add if there is sufficient demand. But note that this is still not “transaction signing” where the user signature is validated by ledger. If you don’t want to trust anyone in Daml, you need to run a node.
  2. Not in the near term, but possibly at some point. Right now the deployment model is that the JSON API communicates with the ledger through the same Ledger API that the user does. Ie there is a trust boundary that needs non-repudiation between the JSON API and Ledger API. To really do this well, we either need to integrate the JSON API more tightly to remove that trust boundary, or move to a two-step signing process where the end-user signs the final transaction in the style of public chains.
  3. The feature is pretty much there, but does not yet have sufficient use in the wild for us to call it “Stable” just yet.

As I said before, it’s currently only available in the Enterprise Edition of Daml. If you have a use-case for which you are looking at commercial Daml Drivers and Daml Connect Enterprise, this feature should not present a make-or-break obstacle - it’s ready for use. I’d recommend getting in touch with Digital Asset sales to discuss your specific requirements and on what timelines they could be met.