Canton domain reaching size limit

Questions
,
1

Hello team,

We have setup a canton network and are trying to do some performance test with 1.) creating 1000+ users and 2.) The daml trigger service is exercising a choice with creating 700+ contracts but giving us the size limit error as below

image
image1554Ɨ255 33.7 KB

Can you please advise if there is any way to config the domain size or any other walkaround?

thanks,
Dorrit Du

1 Like
2

Hi Dorrit!

This is a GRPC rejection by the domain, when the inbound message size is too big. You can increase this by increasing maxInboundMessageSize: https://www.canton.io/docs/dev/scaladoc/com/digitalasset/canton/domain/config/DomainParametersConfig.html

So, the domain config should be changed using

canton.domains.mydomain.domain-parameters.max-inbound-message-size =

However, note that the participant needs to be configured accordingly:

canton.participants.myparticipant.ledger-api.max-inbound-message-size =

Note that the participant setting always must be larger than the one on the domain, as otherwise, the participant will refuse to connect to the domain.

I hope that helps.

Cheers,
Ratko

1 Like
3

Hi Ratko,

After resolving the inbound message size, we only another heap size issue now message as below.
Is there a way to increase the heap size canton (i.e. Java) uses via config?

WARN c.d.c.c.ExecutionContextMonitor - Execution context canton-env-execution-context is just slow. Future got executed in the meantime.
WARN c.d.c.c.ExecutionContextMonitor - Execution context canton-env-execution-context is stuck or slow. My scheduled future has not been processed for at least 3 seconds (queue-size=43).
ForkJoinIdlenessExecutorService-canton-env-execution-context: [java.util.concurrent.ForkJoinPool@5c8e45a0Running, parallelism = 2, size = 4, active = 2, running = 2, steals = 1404, tasks = 873, submissions = 43]
Aborting due to java.lang.OutOfMemoryError: Java heap space

A fatal error has been detected by the Java Runtime Environment:

Internal Error (debug.cpp:338), pid=1, tid=34

fatal error: OutOfMemory encountered: Java heap space

JRE version: OpenJDK Runtime Environment Zulu11.48+21-CA (11.0.11+9) (build 11.0.11+9-LTS)

Java VM: OpenJDK 64-Bit Server VM Zulu11.48+21-CA (11.0.11+9-LTS, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)

Core dump will be written. Default location: Core dumps may be processed with ā€œ/usr/share/apport/apport %p %s %c %d %P %Eā€ (or dumping to /canton/core.1)

An error report file with more information is saved as:

/tmp/hs_err_pid1.log

If you would like to submit a bug report, please visit:

http://www.azulsystems.com/support/

many thanks,
Dorrit

4

Hi @Dorrit_Du

Canton is using the JVM, so you just need to control this using the standard JVM configuration mechanisms:

https://www.oracle.com/java/technologies/javase/vmoptions-jsp.html

You can pass these argument directly to bin/canton, i.e. write

./bin/canton -Xmx4G

Or you can pass them via the JAVA_OPTS environment variable:

JAVA_OPTS="-Xmx4G

The canton startup script is a very tiny wrapper that invokes java, and we just pass through the arguments you define.

I hope this helps.

Cheers,
Ratko