Does Daml Trigger State update before Daml Trigger is run?

Questions
,
1

Hello we are setting up a Trigger which registered to below 3 templates

  1. SettlementFinalized
  2. SettlementRejected
  3. Instruction

What we hope to achieved is when SettlementFinalized or SettlementRejected is created, we Settle or Cancel the corresponding Instructions which are linked using a field groupId.
There could be multiple instructions with the same groupId but there can be only one SettlementFinalized or one SettlementRejected contract with the same groupId.

Settle or Cancel Instruction is a consuming choice while SettlementFinalized and SettlementRejected will stay on the ledger. Thus, to prevent same SettlementFinalized/SettlementRejected getting processed each time the trigger got triggered (even tho the corresponding instructions should all already be archived), we created a State for unprocessedSettlements.
Upon creations of a SettlementFinalized/SettlementRejected contract we will add it onto the list of unprocessedSettlements by updating the state. On the other hand, the trigger simply get the current state, iterate over the list of unprocessedSettlements and send commands to Settle/Cancel Instructions with its groupId. After sending commands, we empty the list of unprocessedSettlements.

See below for code snippet

addToUnProcessedTransactions : Message -> TriggerUpdateA UnProcessedSettlements ()
addToUnProcessedTransactions (MTransaction Transaction{events}) =
    forA_ events markedAsProcessed
  where
    markedAsProcessed (CreatedEvent created) = do
      whenSome (fromCreated @SettlementFinalized created) $
        \(_, _, SettlementFinalized{groupId}) -> modify (Finalized groupId ::)
      whenSome (fromCreated @SettlementRejected created) $
        \(_, _, SettlementRejected{groupId}) -> modify (Rejected groupId ::)
    markedAsProcessed _else = pure ()
addToUnProcessedTransactions _any = pure ()

updateBalance: Party -> TriggerA UnProcessedSettlements ()
updateBalance provider = do 
  let 
    settleInstructionsForTransaction: Settlement -> TriggerA UnProcessedSettlements ()
    settleInstructionsForTransaction transaction = do 
      case transaction of 
       Finalized groupId -> mapA_ (`dedupExercise` Settle) =<< getAllInstructionsForGroupId groupId
       Rejected groupId -> mapA_ (`dedupExercise` Cancel) =<< getAllInstructionsForGroupId groupId
       where getAllInstructionsForGroupId: GroupId -> TriggerA UnProcessedSettlements [ContractId Instruction]
             getAllInstructionsForGroupId targetGroupId = do 
               map fst <$> queryFilter @Instruction ((==targetGroupId) . (.groupId))
  unprocessedTransactions <- get
  forA_ unprocessedTransactions settleInstructionsForTransaction
  put []
  pure ()

We are wondering is it guaranteed that when SettlementFinalized/SettlementRejected create event is published, updateState (our addToUnProcessedTransactions) is guaranteed to run before the rule (updateBalance).
If not, is there any recommended approach to implement above flow?

Thanks so much in advance!

2

updateState is guaranteed to run before the rule.

I’m not sure you actually need any custom state for this though. At a high-level I’d describe what you want to do as follows:

  1. Find all SettlementFinalized and SettlementRejected contracts
  2. For each of those find the corresponding instructions and finalize/reject them.

So to implement that, a rule like the following should do the trick:

finalized <- query @SettlementFinlized`
forA_ finalized $ \(_, SettlementFinalized{groupId}) -> do
  instructions <- queryFilter @Instruction (\i -> groupId == i.groupId)
  forA_ instructions $ \(cid, _) -> emitCommands [exerciseCmd cid Settle] [toAnyContractId cid]
forA_ rejected $ \(_, SettlementRejected{groupId}) -> do
  instructions <- queryFilter @Instruction (\i -> groupId == i.groupId)
  forA_ instructions $ \(cid, _) -> emitCommands [exerciseCmd cid Reject] [toAnyContractId cid]

The crucial part here is that you mark the contracts you’re exercising the consuming choice on as pending via emitCommands. Once all of them have been marked queryFilter is just going to give you an empty list so while the SettlementFinalized/SettlementRejected contracts are still there, your trigger is not going to do anything.

1 Like
3

@cocreature again, thanks so much for the swift answer and suggestion, we were originally thinking of this implementation. But was wondering if the number of SettlementFinalized/SettlementRejected contracts are huge (say ~10k per second and manual clean up only run on daily basis), will calling queryFilter on all of them be really expensive?

4

Yes, at some point that becomes an issue. There are a few options here to speed this up:

  1. Only query once:
    queryFilter does a query and then a linear search. So an easy and very effective optimization is to call query once and then based on that build a map indexed by group id. Instead of calling queryFilter each time, you then do a lookup in that map.
finalized <- query @SettlementFinlized`
instructions <- query @Instruction
let instructionMap = Map.fromListWith (++) [(instruction.groupId, [cid]) | (cid, instruction) <- instructions ]
forA_ finalized $ \(_, SettlementFinalized{groupId}) -> do
  let instructions = fromOptional [] $ Map.lookup groupId instructionMap
  forA_ instructions $ \cid -> emitCommands [exerciseCmd cid Settle] [toAnyContractId cid]
forA_ rejected $ \(_, SettlementRejected{groupId}) -> do
  let instructions = fromOptional [] $ Map.lookup groupId instructionMap
  forA_ instructions $ \cid -> emitCommands [exerciseCmd cid Reject] [toAnyContractId cid]

  1. Cache the map:
    The previous option means the individual lookups are fairly fast. But at the beginning of each rule execution we still build the map so that can be somewhat slow. You could cache that map based on the user defined state. Note that you need to be careful to take the pending set into account for this to work properly.

  2. Cache “finished” SettlementFinalized/SettlementRejected contracts
    If we cache the map as in 2 we’ve eliminated most of the big bottlenecks but we still iterate over all SettlementFinalized/SettlementRejected contracts and do a map lookup. You could cache Settlement contracts for which there are no more instructions (or probably the opposite, cache the ones that do still have instructions) in your state and then only iterate over that.

1 is easy enough to just implement. For 2 and 3 I’d probably test things to see if you really need those optimizations before implementing them.

3 Likes
5

@cocreature wowww, Thanks for the great and detailed suggestion!

1 Like