Facing error in the canton participant node (UNAVAILABLE: SERVICE_NOT_RUNNING(1,0): Ledger API offset dispatcher has been shut down.)

Hi Team,

I have canton 2.7.5 network setup in the VM. The network has one domain and 2 participant nodes. mydomain and participant1 is running in the VM1 and participant2 is running in the VM2. I am able to communicate the networks (participant1.health.ping(participant2).

In the VM1, I have started the HTTP Json server (java -jar http-json-2.7.5.jar --ledger-host --ledger-port 5011 --address --http-port 7575 --allow-insecure-tokens). Here, participant1 ledger port is 5011 and running in address.

  1. In the VM1 canton log, I got the following error. Can someone tell what is error ?

WARN c.d.c.p.a.ResilientTransactionsSubscription:participant=participant1 - Ledger subscription admin-ping for PingService failed with an error
io.grpc.StatusRuntimeException: UNAVAILABLE: SERVICE_NOT_RUNNING(1,0): Ledger API offset dispatcher has been shut down.
at io.grpc.Status.asRuntimeException(Status.java:535)
at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:479)
at io.opentelemetry.instrumentation.grpc.v1_6.TracingClientInterceptor$TracingClientCall$TracingClientCallListener.onClose(TracingClientInterceptor.java:161)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:562)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:70)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:743)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:722)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at com.daml.executors.QueueAwareExecutorService$TrackingRunnable.run(QueueAwareExecutorService.scala:98)
at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1425)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1016)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1665)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1598)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
WARN a.s.s.RestartWithBackoffSource - Restarting stream due to failure [1]: com.daml.error.ErrorCode$LoggedApiException: UNAVAILABLE: SERVICE_NOT_RUNNING(1,0): Ledger API offset dispatcher has been shut down.

  1. In the VM2, I have started the HTTP Json server. I used create-daml-app. The following are the commands which i ran,
    daml new --template=create-daml-app demo-daml
    cd demo-daml
    daml build

I have uploaded the demo-daml-0.1.0.dar in participant2 node.

daml codegen js .daml/dist/demo-daml-0.1.0.dar -o daml.js
cd ui
npm install
npm start

It is started in localhost:3000. When i open it in browser and tried to login as alice, I got the following error,
4-01-25 15:42:55.064 [http-json-ledger-api-akka.actor.default-dispatcher-23] INFO com.daml.http.Endpoints - Incoming GET request on http://localhost:3000/v1/user from, context: {instance_uuid: “dda7e793-de51-4df5-8e17-e3245e2c8bf4”, request_id: “de36e697-6196-4746-904c-ccbc26ad1bbb”}

Can anyone suggest how to resolve this?

Are you sure this is an error? Where do you see it? The message says, INFO, not ERROR.

Yes It is an info. When I ran this in my local machine, i am able to login as alice, bob and add the followers. The local setup uses sandbox. The same when I tried this in the VM, by updating the ledger address and port of canton participant node without sandbox, I am not able to login. I got the above mentioned info. Can you please give your suggestion for the first question?