How to extract workflow state from the ledger

Hi all,

I am currently working on a project where we are trying to extract the the state of workflow so that it can notify some downstream process. However, we are struggling to find a good solution. We have defined some contracts in a similar way to the Multile party agreement:

type SomeKey = (Party, Text)

template Agreement
  with
    id: Text
    initiator: Party
    signatories: [Party]
  where
    signatory signatories
    ensure
      initiator `elem` signatories &&
      unique signatories
    key (initiator, id) : SomeKey
    maintainer key._1

template Proposal
  with
    initiator : Party
    finalContract : Agreement
    alreadySigned: [Party]
  where
    signatory  alreadySigned
    observer finalContract.signatories
    key (initiator, finalContract.id) : SomeKey
    maintainer key._1
    ensure
      -- Can't have duplicate signatories
      unique alreadySigned
    -- The parties who need to sign is the finalContract.signatories with alreadySigned filtered out
    let toSign = filter (`notElem` alreadySigned) finalContract.signatories

    choice Agree : Either (ContractId Agreement) (ContractId Proposal) with
        signer : Party
      controller signer
        do
          assert (signer `elem` toSign)
          let
            newAlreadySigned = signer :: alreadySigned
        
          -- if allSigned automatically finalize
          if sort newAlreadySigned == sort finalContract.signatories then do
            agreementCid <- create finalContract
            return (Left agreementCid)
          else do
            proposalCid <- create this with alreadySigned = newAlreadySigned
            return (Right proposalCid)

    choice Cancel : () with
      controller initiator
        do
          return ()

From these templates, we want to extract the status of the workflow. For example, the statuses could be:
PROPOSED, AGREED, CANCELLED where PROPOSED = when the Proposal exists but not all parties have agreed yet, AGREED = the Proposal has been archived and an Agreement contract with the same key now exists on the ledger and CANCELLED = where the Cancel choice has been exercised on the ledger and there does not exists a Proposal or Agreement on the ledger.

Even though, we can determine the status by querying the active contract set at any given time, we would like to STREAM out the changes to the state. We came up with 3 approaches to the solution:

  1. Using the TransactionService.getTransactions Ledger API method, we can extract a stream of updates by filtering by Proposal template. However, when a FlatTransaction is returned, the ArchivedEvent does not contain the exercising choice. Therefore, there is no way to distinguishe between a Cancel or a “finalising” Agree choice.
    Is there any consideration in adding the choice name to the ArchivedEvent in the future, or would you recommend adding the Agreement choice to the filter as well. With the later, this introduces extra complexity in my custom extractor.

  2. We can use the using the TransactionService.getTransactionTrees Ledger API method to extract the whole TransactionTree. With this, I am able to distinguish between the two choices as I have access to the exercising choice name.
    However, this method has the drawback of not being able to filter by template. Will this result in a lower extraction time and throughput of the incoming stream? This will also require additional computational time in the extractor side to filter out and traverse the whole tree.

  3. The last method, would be to create a template that store the statuses on the ledger. The status template is then update through each choice of the Proposal.
    However, this means that we are storing a long lived state on the ledger, when does this template then get archived? How will this affect performance of the ledger if it is not archive?

It would be really helpful to hear your opinions on this? Which method is better or is there another approach that I am missing?

1 Like

I’d try to get away with the flat transaction stream. I think you do actually have the information you need already:

Let’s assume that on the client side you maintain a map from id (the text field in your contract) to the state proposed | agreed | cancelled. We want to update that via the flat transaction stream. For that we subscribe to both Proposal and Agreement. Now we need to consider a few cases:

  1. A create of a Proposal. In that case add a new entry to the map with value proposed.
  2. An archive of a Proposal and a create of an Agreement with the same id. In that case, modify the entry to agreed.
  3. An archive of a Proposal without a create of an Agreement. In that case, modify the entry to cancelled.

If I understood your requirements correctly, that should do the trick without any additional state on the ledger.

2 Likes

Thanks for the response! Yep that makes sense, and it does satisfy the requirements.

Just one question regarding the id to check that my implementation is correct. In your second case:
2. An archive of a Proposal and a create of an Agreement with the same id . In that case, modify the entry to agreed .

To correlate the Proposal and Agreement, I will need the id which is also part of the contract keys. When I recieve a Proposal archivedEvent, only the COntractId is provided and the event does not coontaine a contract key (im assuming this is not to leak infromation as this event could be viewable through divulgence to non-stakeholder). Therefore, i will need to store come key/value store of ContractIds to contract keys (or just the id field) in order to compute case 2.

Or is there another way to query the contents of the archived event from the ledger?

Right, you need to store that on the client side. Shouldn’t be too difficult, you can store it as part of the value for the proposed state. Then when you see the create where you do have the id, you can look up the old value in the map which includes the contract id and then check that you also have the corresponding archive for that id.