Is it possible for negative test cases to get back just the error message from `daml test`?

Questions
1

I’m writing unit tests, and I want to include tests for negative use cases. I can see two options:

  1. sutmitMustFail with no feedback about the error

I can use in the script the submitMustFail or submitMultiMustFail expression, which lets the script run without any error message. This is useful to know that the negative case would be rejected by the ledger, but it gives no feedback about the error message, so I cannot include easily the error message into an automatically generated test log.

  1. Let the script fail, with a possibly huge error message

If I run the failing script as part of daml test, I get a feedback, but it contains too much information. This is a minimal example, where I messed up the choice in the skeleton template setup script with the following line:

choice Give : AssetId
      with
        newOwner : Party
      controller owner
      do 
        assertMsg "This is a test error message" False
        create this with 
           owner = newOwner

When I run daml test, I get the following message, which contains way more than I need. The message for a big contract can be huge:

gyorgybalazsi@BGY fail-test % daml test
File:     daml/Main.daml
Hidden:   no
Range:    26:1-26:6
Source:   Script
Severity: DsError
Message: 
  Scenario execution failed on commit at Main:42:12:
  Unhandled exception:
  DA.Exception.AssertionFailed:AssertionFailed@3f4deaf145a15cdcfa762c058005e2edb9baa75bb7f95a4f8f6f937378e86415
  with
  message = "This is a test error message"

  Ledger time: 1970-01-01T00:00:00Z

  Partial transaction:
  Failed exercise (unknown source):
  exercises Give on #0:0 (Main:Asset)
  with
  newOwner = 'Bob'
  Sub-transactions:
  0
  └─> 'Alice' exercises Give on #0:0 (Main:Asset)
  with
  newOwner = 'Bob'

  Committed transactions:
  TX 0 1970-01-01T00:00:00Z (Main:36:14)
  #0:0
  │ known to (since): 'Alice' (0)
  └─> create Main:Asset
  with
  issuer = 'Alice'; owner = 'Alice'; name = "TV"

All I would need is just the error message.

Is there maybe a flag to set this? I couldn’t find any. If there isn’t, this is a feedback that it would be useful.

2

Hi Gyorgy,

As far as I can tell, there isn’t currently a way to disable the “pretty printing” on failed submit, which is what you see in daml test (or in daml studio), or to catch the error message when performing a submit.

In cases where the error you want to test is a daml exception, like in this case, there’s a way to check the error, which is to wrap the choice in an exercise and use exception handling.

For this example:

template Runner with
    p1, p2 : Party
    cid : ContractId Asset
  where
    signatory p1
    choice Run : Optional Text
      controller p1
      do
        try do
          exercise cid (Give p2)
          pure None
        catch
          AssertionFailed msg ->
            pure (Some msg)

Then you replace your submit with:

msg <- submit alice do
  createAndExerciseCmd (Runner alice bob aliceTV) Run
msg === Some "This is a test error message"

i.e. msg ends up with the error message.

You could also catch an AnyException instead of a specific exception type:

        catch
          (e: AnyException) ->
            pure (Some (message e))

The only thing you really can’t do with this approach is catch uncatcheable errors, like authorization failures or fetch failures (e.g. “contract does not exist”).

2 Likes
3

Thank you!