What is the best way to manage packageIds in production?

Hi there, I’m looking for a little guidance in forward thinking and changing an application over time.

Say, for example, I have a project with templates X, Y, and Z. In my consuming applications, I search for contracts matching these templates by using the Identifier generated by daml-codegen. That’s fine.

Then, I change a little bit of the logic for one of the choices in template Y. This will create a new packageId, meaning my application can no longer use the daml-codegen for the latest version of the project, because if I do that then I won’t be able to find any of my old contracts, due to mismatching packageId.

Looking at the docs, the only solution is to import the older package as a data-dependency, create an “upgrade” contract simply for creating the exact same contract, and then process every single old contract in the ledger simply so it can be visible under the new packageId. That, or store a list of packageIds for my consuming applications to iterate through to find all contracts, but that has its own downsides.

Have I missed something?

I don’t think so. Please keep in mind that Daml contracts are meant to carry a fairly strong sense of agreement from the signing parties, so

is a bit off: if the new contract has the same data but different choices, it is a different contract and, in the Daml model where signing a contract has weight, you do need the relevant parties to agree to the new code for the choices.

What I would recommend in practice is to create new code for new packages, and keep using the contracts from the “old” package if they have not changed. Essentially, once a package has been deployed to a production ledger, you should not modify its source code anymore, you should start a new package with new source code that only contains the new things. Those new things may include new templates that render the existing ones obsolete, but I think it makes more sense to think of them as new templates than as updates to the old ones.

In particular, you should not take the whole code for v1, change one of the three templates, and recompile the whole thing as v2. That will indeed result in two duplicated templates that only differ in their package ID, so in that case (assuming those two packages have truly not changed, i.e. they don’t reference the third template or use any function that was changed), you should just leave them out of the second package entirely.

So, yeah, I think it’s easier to just start with a blank slate and possibly copy/paste some specific bits from the “previous version”, rather than start from the whole code and make changes.

1 Like

Yeah, that makes sense, thanks for that.

My only concern there is the original point whereby you are a consuming application wanting to read out these contracts from the ledger. Would you need to keep a mapping of version to packageId? If you did so it seems like you would also need to manage documentation stating which version should be used for the latest version of each template?

Have you developed a better process to manage this kind of thing internally?

I’m not sure I understand your question here, but I would really encourage you not to think in those terms. If it has a different package ID, it’s a different template. From a code perspective, it does not really help to think of them as different versions of “the same” template.

You’ll need to deploy a new version of your UI before users can use the new templates. There is, in general, no way for you to force users to migrate to the new templates. One approach, if you control the clients (say, in a web context, for example, where you control the browser code), you could decide that the new version of the UI does not expose any way to interact with the old templates except for converting them to the new ones, but even that is not foolproof: people could still rely on the old templates through automation and direct access to the API. Or, if the UI needs to be installed somehow, they could keep using an old version.

If you’re in a truly distributed context, where other people write code that interact with your templates, then yes, when you deploy new templates that are intended to replace existing ones, you need to somehow let people know about it. That could be through documentation, email newsletters, etc. Presumably if you have templates in common you have some form of relationship. You could also communicate that on the ledger, for example by including a “upgradeTo: Maybe Text” field in every contract and giving you a choice that lets you put in the updated packageID in there, and having all other choices abort if that attribute is not None. Still, that only works for contracts you are a party to.

As a final note, please take everything I write with a healthy dose of skepticism. I’m really just the guy maintaining our CI machines; I don’t do any Daml myself.