Negative Testing

Hey team,

I’m writing some scripts to test some new features I’ve recently added to a Daml project. One thing I noticed is when I do negative testing, using submitMustFail that I’d like to also provide the reason why the submission should fail. Currently, I have to manually check that the submission failed as I cannot specify the reason why the submission should fail.

For example, I have choices that requires multiple signatures with multiple assertions. I cannot distinguish a failure on a missing signature or a specific assertion without manually checking each test case.

Would this type of feature be possible in Daml ?

2 Likes

I’m thinking of such functions as -

  • submitFailsOnMissingAuth [missingParty1, missingParty2]
  • submitFailsOnAbort "my abort message"
1 Like

Currently there is no way of doing that. My plan for that is to throw proper exceptions that you can catch (and perhaps some utilities to make that nice) but the issue is that currently we cannot get structured information out of the ledger to see what actually failed so since everything in Daml Script has to work over the Ledger API and not just in Daml Studio, we’re somewhat limited here.

1 Like

Thanks @cocreature for your quick reply as usual.

1 Like

You can mock this using Exceptions and a helper contract as follows:

module Test.User where

import User

import Daml.Script
import DA.Exception

tryExercise : (Template t, Choice t c r) => ContractId t -> c -> Update (Either Text r)
tryExercise cid choice =
  try do
    ret <- exercise cid choice
    return (Right ret)
  catch
    AssertionFailed m -> return (Left m)
    GeneralError m -> return (Left m)
    PreconditionFailed m -> return (Left m)
    ArithmeticError m -> return (Left m) 

template UserTestHelper
  with
    p : Party
  where
    signatory p
  
    controller p can
      Test_Follow : Either Text (ContractId User)
        with
          cid : ContractId User
          choice : Follow
        do
          tryExercise cid choice

my_test : Script ()
my_test = script do
  alice <- allocateParty "Alice"
  bob <- allocateParty "Bob"

  aliceUserCid <- submit alice do
    createCmd User with username = alice; following = []
  bobUserCid <- submit bob do
    createCmd User with username = bob; following = []

  aliceUserCid <- submit alice do
    exerciseCmd aliceUserCid Follow with userToFollow = bob

  result <- submit alice do
    createAndExerciseCmd (UserTestHelper alice) Test_Follow with
      cid = aliceUserCid
      choice = Follow with userToFollow = bob
  
  assert (result == Left "You cannot follow the same user twice")
  return ()

You cannot write your completely generic submitFailsOnAbort functions, though, because templates and choices don’t support type parameters. You have to write an explicit test wrapper for each type of submission you want to test negatively.

1 Like

That doesn’t actually allow you to catch authorization errors though which was part of the original question.

1 Like