How to get the runtime of tests (scripts) one by one

Questions
, , ,
1

I would like to see the runtime for each test, one by one.
With daml test and --log-level=DEBUG, I think I am able to see the length of time it took to run all the tests.

How is it possible to get results for each test?

I tried adding debug calls to my test script (wrapping each test script, start/end) and thought it would result in a timestamped log output, but seemingly not.

1 Like
2

At the moment, there is no way to get timings per test out of daml test.

daml test runs via the same Script service in Daml studio which does not emit trace messages to the terminal so any attempts at wrapping it in getTime + debugRaw isn’t going to work.

You could try daml script-test which runs via a real ledger and does emit (client-side) trace statements to a terminal but keep in mind that then there is also a lot more overhead to the ledger operations which may or may not skew measurements.

3

I think that trace/debug statements are not visible in the logs printed by daml script-test because it uses INFO level logging and my understanding is that one cannot change that (there is no --log-level option, for example, if I am not wrong).

1 Like
4

You are right unfortunately, that’s a bug. Try creating a logback.xml file with the following contents

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%replace(, context: %marker){', context: $', ''}%n</pattern>
        </encoder>
    </appender>

    <logger name="io.netty" level="WARN" />
    <logger name="io.grpc.netty" level="WARN" />
    <logger name="akka.event.slf4j" level="WARN" />
    <logger name="daml.tracelog" level="DEBUG" />
    <logger name="daml.warnings" level="WARN" />
    <logger name="com.daml.platform.store.FlywayMigrations" level="WARN" />
    <logger name="com.zaxxer.hikari" level="ERROR" />
    <logger name="com.daml.platform" level="WARN" />
    <logger name="org.flywaydb" level="ERROR" />
    <logger name="com.daml.ledger.on.sql" level="WARN" />

    <root level="INFO">
        <appender-ref ref="STDOUT" />
    </root>
</configuration>

and then run _JAVA_OPTIONS=-Dlogback.configurationFile=$PWD/logback.xml daml test-script --dar .daml/dist/foobar-0.0.1.dar to use that logfile.

1 Like