Hello,
I have a general question about DAR files and the way we use them.
I created a simple Daml project (Sample-0.0.1.DAR) that has the following template:
template TestA
with
owner: Party
num: Int
where
signatory owner
I created several contracts of TestA on the Daml hub, and then i fixed a bug in the project. In order to review the fix on the hub, i uploaded the fixed DAR file to the hub.
However, when I fetch a contract from the ledger, I am getting errors with the contract, saying that it is not the same type of template.
I can overcome this problem?
I understand that each DAR file has a unique package id. So can I check my fix?
Do I need each time to create a new Daml Hub and re-create the contracts on it in order to test my fix?
It doesnât seem right, and I believe there is a way to do it.
Because of Damlâs core tenet that you should know what youâre signing for when you sign a contract, templates have to be uniquely identified. Therefore, changing a package generates new templates.
This raises an interesting question that is absent from most other systems: what to do with existing data, which is still defined in terms of the previous template version?
At development time, in many cases, you can drop the data. For (dev-speed) efficiency, you may even decide not to drop the data and instead ignore it - if all of your queries are derived from the codegen and thus use the âcurrentâ template IDs, you will simply not meet old contracts at all. Youâll probably still want to do some cleanup at some point, but you may not need to do it after every single change.
But that still leaves the question open for the production case: at some point you will need to upgrade templates on a live ledger, where deleting the data is not an option. This is an important consideration that you need to keep in mind while developing your Daml models.
Iâm not best placed to talk about specific approaches here, as I donât have much experience with this problem myself. Hopefully other, more-informed people will chime in.
But as a very basic starting point, you can try these two approaches:
Separate, as much as possible, your data from your operations in different contracts (possibly ephemeral contracts for actions, designed for createAndExercise, or long-standing so-called âroleâ contracts). That way, when you want to update the operations, your new âroleâ templates can still refer to the existing template IDs for the data, and you donât need to update the âdataâ contracts. If you do need to change your data schema, then this doesnât work, but that should be rarer.
Package your previous deployment as dar_1, your new version of the code as dar_2 (essentially an updated copy of the entire code base), and then create a third, separate code base packaged as dar_convert_1_2 that depends on both dar_1 and dar_2 and has âroleâ/âephemeralâ template definitions that collect signatures to âmanuallyâ upgrade contracts from dar_1 to dar_2.
Thank you @Gary_Verhaegen for a detailed answer.
I honestly have to say that I am very surprised about the approach DAML takes here.
For me, I have several templates that are created by different DAR files. And each DAR file is dependant on the other.
Hence, the process of creating a specific contract which I need to work on takes several minutes. That makes simple debugging into a real headache.
Each line of code that I check/fix requires me to rebuild the entire ledger each time from scratch (including the identities, initialise the ledger with the basic contracts and etc⌠)
Is this approach is considered to change?
This approach makes our work (the developers) extremely hard and frustrating.
And just to emphasise how much it is difficult, Take a look at the following code:
template A
with
owner: Party
number: Int
where
signatory owner
template B
with
owner: Party
aContract: ContractId A
where
signatory owner
Letâs say template A is declared by DAR file named A, and template B is created by a DAR file named B.
According to what you said, if I fix some code in A.DAR, then I need to convert only the contracts of type A.
However, in the following example, I will also need to fix/convert all contracts of type B as well since they point to templates of type A since they wonât exist anymore on the ledger.
Another issue of making the application not to be online whenever conversion is being processed⌠but this is less critical for me now.
Yes, if you have two separate Daml projects A and B, and B depends on A, in the general case a change to A will require a recompilation & redeployment of B too. There is no way to avoid that and preserve the âyou should know what you signâ property.
I can see two ways to make this a bit easier:
Donât have separate packages. If your code modules are tightly coupled, putting them in separate packages is only going to create management overhead.
Donât push every single change to production (Hub). Think of your Daml model as a database schema: changing the database is costly, you may want to wait until youâre fairly confident about your model before pushing it to production.
In principle you should be able to develop locally against a sandbox, without having to push to Hub for every single line of change. If you think there is an issue with the sandbox that makes it not a good development environment and a good reflection of a production system, please report it as a bug.
With the sandbox, recreating your state could be done as an init script and should thus be fairly quick as part of daml start and perhaps another explicitly-invoked daml script command.
Hi @Gary_Verhaegen, thanks for the answerâŚ
In order to make things easier, I would like to ask something about this way of work.
Letâs say that I make a DAR file that has these 2 templates:
template A
with
owner: Party
biggerValue: Bool
where
signatory owner
template B
with
owner: Party
number1: Int
where
signatory owner
choice CreateTemplateA : ContractId A
with
number2: Int
controller owner
do
create A with biggerValue = number2 > number1, ..
In my example I did the data validation inside the choice itself. Which means that whenever I need to update my validation (for example to be number2 == number1) than I need to rebuild my DAR file that contains my templates.
How can I resolve this, so whenever I change the validations, I wonât need to re-compile the DAR file holding the templates?
For example, having an additional DAR file that has a function which receives these values (number1 & number2) as arguments and returns a boolean into the choice, will this resolve it?
so in my second DAR file I will have the following function:
validateNumbers : Int -> Int -> Bool
validateNumbers num1 num2 = num1 > num2
And in my choice I will change it like that:
template B
with
owner: Party
number1: Int
where
signatory owner
choice CreateTemplateA : ContractId A
with
number2: Int
controller owner
do
create A with biggerValue = (validateNumbers number2 number1), ..
If you want to change the logic of a template, you have to redeploy that template, and somehow upgrade existing contracts on the ledger. Thereâs no way around that.
People who have signed the previous version must explicitly agree to switch to the new version. Otherwise agreements and signatures are completely meaningless.
The link from the template to the function will go through the Package ID of the DALF that contains the function. Pushing another DALF with a function with the same name (but another Package ID) will have no effect on existing templates and contracts. If you want a new template that uses the new validation function, youâll have to create and deploy that new template.
You cannot change an existing template, you can only push new ones.
@Gary_Verhaegen I am not sure I explained the situation correctly (maybe you understood it - bit I will explain in a more detailed way to be sure I am on the same page as you).
Letâs say I create a daml project named Templates, this project contains the following code:
template A
with
owner: Party
biggerValue: Bool
where
signatory owner
template B
with
owner: Party
number1: Int
where
signatory owner
choice CreateTemplateA : ContractId A
with
number2: Int
controller owner
do
create A with biggerValue = (validateNumbers number2 number1), ..
Then, I create a new daml project, named BusinessMdel, that has the following function:
validateNumbers : Int -> Int -> Bool
validateNumbers num1 num2 = num1 > num2
After building both projects I am having the following DAR files:
Templates-0.0.1.DAR
BusinessModel-0.0.1.DAR
After some time, I am changing the validateNumbers function to do something else.
In order for this to take affect when exercising the CreateTemplateA choice, do I need to rebuild both projects, or will I need to rebuild only the BusinessModel project?
To allow otherwise would mean that, even though parties had signed and agreed to various B contracts on a production ledger, you could just change the rules that govern B out from under them, without their consent. The fact that the definition of validateNumbers is in a different file does not change the fact that it is part of the rules that govern B.
During development of Daml templates, itâs expected that you rebuild everything all the time, constantly restarting your development ledger from scratch and reinitializing from a script. Thatâs why development should take place on a local sandbox, rather than via continuous deployment to a production environment like Daml Hub.
Navigate to the terminal window where the daml start process is running and press ârâ. This will
Compile our Daml code into a DAR file containing the new feature
Update the JavaScript library under ui/daml.js to connect the UI with your Daml code
Upload the new DAR file to the sandbox
As mentioned previously, Daml Sandbox uses an in-memory store, which means it loses its state â which here includes all user data and follower relationships â when stopped or restarted.
So letâs say you need a bunch of contracts to be created in development in order to test out your changes. The way you deal with that is, you donât set up everything via Navigator or other UI, you define an init-script that sets up a fresh, empty ledger into a state where youâre ready to test your app. The getting started app also uses an init-script to set up users, and this just happens automatically when you press r as described above in your local sandbox development environment.
For development, resetting the ledger is usually your option and you donât need to worry about this. You are right that this is difficult on Hub, I recommend to develop locally and only deploy to hub once youâre happy to roll out your changes.
Once you rolled out to your actual production ledger, resetting is ofc no longer an option and you do need to worry about upgrades. I think youâre right that this is currently somewhat difficult to handle and changes often propagate and require changes in other templates as well. We are currently working on making upgrade workflows easier and in general supporting decoupling of templates to avoid those propagating changes.