"INTERNAL: RST_STREAM closed stream. HTTP/2 error code: INTERNAL_ERROR" upon gRPC stream comuunication

Hi,

After pingPongReactive managed to run against a real Ledger (thanks to @cocreature help on this fork) i noticed an error appearing on the client side (Java gRPC client) triggered from an action on the Ledger.

Whenever i’m manually experimenting with the “RespondPing” (or “RespondPong”) from the Ledger itself i get an expected behaviour (see first lines in the below log) followed by a weird HTTP close-stream error:

11:18:54.006 [client-2] INFO  e.p.reactive.PingPongProcessor - transaction = Transaction{transactionId='00000000000000a0', commandId='', workflowId='', effectiveAt=2022-02-26T09:18:54.443204Z, events=[ArchivedEvent{witnessParties=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], eventId='#00000000000000a0:0', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}, contractId='00268313b3df53237761181c6575f09823b3fd2bfd93600094892ea7254246b4fe'}, CreatedEvent{witnessParties=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], eventId='#00000000000000a0:1', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}, contractId='0025996ad01351bb26469f6146bcee072f74066a906436105dc324603681a5df12', arguments=DamlRecord{recordId=Optional[Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}], fields=[Field{label=Optional[sender], value=Party{value='ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246'}}, Field{label=Optional[receiver], value=Party{value='ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb'}}, Field{label=Optional[count], value=Int64{value=4}}]}, agreementText='Optional[]', contractKey=Optional.empty, signatories=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], observers=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb]}], offset='00000000000000a0'}
11:18:54.009 [client-1] INFO  e.p.reactive.PingPongProcessor - processTransaction: Transaction{transactionId='00000000000000a0', commandId='4e1a99d8-620c-4517-ba02-ce216c906616', workflowId='', effectiveAt=2022-02-26T09:18:54.443204Z, events=[ArchivedEvent{witnessParties=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], eventId='#00000000000000a0:0', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}, contractId='00268313b3df53237761181c6575f09823b3fd2bfd93600094892ea7254246b4fe'}, CreatedEvent{witnessParties=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], eventId='#00000000000000a0:1', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}, contractId='0025996ad01351bb26469f6146bcee072f74066a906436105dc324603681a5df12', arguments=DamlRecord{recordId=Optional[Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}], fields=[Field{label=Optional[sender], value=Party{value='ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246'}}, Field{label=Optional[receiver], value=Party{value='ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb'}}, Field{label=Optional[count], value=Int64{value=4}}]}, agreementText='Optional[]', contractKey=Optional.empty, signatories=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], observers=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb]}], offset='00000000000000a0'}
11:18:54.011 [client-3] INFO  e.p.reactive.PingPongProcessor - processTransaction: Transaction{transactionId='00000000000000a0', commandId='', workflowId='', effectiveAt=2022-02-26T09:18:54.443204Z, events=[ArchivedEvent{witnessParties=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], eventId='#00000000000000a0:0', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}, contractId='00268313b3df53237761181c6575f09823b3fd2bfd93600094892ea7254246b4fe'}, CreatedEvent{witnessParties=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], eventId='#00000000000000a0:1', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}, contractId='0025996ad01351bb26469f6146bcee072f74066a906436105dc324603681a5df12', arguments=DamlRecord{recordId=Optional[Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}], fields=[Field{label=Optional[sender], value=Party{value='ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246'}}, Field{label=Optional[receiver], value=Party{value='ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb'}}, Field{label=Optional[count], value=Int64{value=4}}]}, agreementText='Optional[]', contractKey=Optional.empty, signatories=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], observers=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb]}], offset='00000000000000a0'}
11:18:54.011 [client-3] INFO  e.p.reactive.PingPongProcessor - ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb is exercising RespondPong on 0025996ad01351bb26469f6146bcee072f74066a906436105dc324603681a5df12 in workflow  at count 4
11:18:54.039 [client-0] INFO  e.p.reactive.PingPongProcessor - transaction = Transaction{transactionId='00000000000000a0', commandId='4e1a99d8-620c-4517-ba02-ce216c906616', workflowId='', effectiveAt=2022-02-26T09:18:54.443204Z, events=[ArchivedEvent{witnessParties=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], eventId='#00000000000000a0:0', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}, contractId='00268313b3df53237761181c6575f09823b3fd2bfd93600094892ea7254246b4fe'}, CreatedEvent{witnessParties=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], eventId='#00000000000000a0:1', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}, contractId='0025996ad01351bb26469f6146bcee072f74066a906436105dc324603681a5df12', arguments=DamlRecord{recordId=Optional[Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}], fields=[Field{label=Optional[sender], value=Party{value='ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246'}}, Field{label=Optional[receiver], value=Party{value='ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb'}}, Field{label=Optional[count], value=Int64{value=4}}]}, agreementText='Optional[]', contractKey=Optional.empty, signatories=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], observers=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb]}], offset='00000000000000a0'}
11:18:54.243 [client-0] INFO  e.p.reactive.PingPongProcessor - transaction = Transaction{transactionId='00000000000000a1', commandId='', workflowId='', effectiveAt=2022-02-26T09:18:54.683774Z, events=[ArchivedEvent{witnessParties=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], eventId='#00000000000000a1:0', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}, contractId='0025996ad01351bb26469f6146bcee072f74066a906436105dc324603681a5df12'}, CreatedEvent{witnessParties=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], eventId='#00000000000000a1:1', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}, contractId='00635c16dcb9caa8bca9a873b902c5e5990b74889a065e58e4656fd940394f4f22', arguments=DamlRecord{recordId=Optional[Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}], fields=[Field{label=Optional[sender], value=Party{value='ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb'}}, Field{label=Optional[receiver], value=Party{value='ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246'}}, Field{label=Optional[count], value=Int64{value=5}}]}, agreementText='Optional[]', contractKey=Optional.empty, signatories=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], observers=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246]}], offset='00000000000000a1'}
11:18:54.243 [client-2] INFO  e.p.reactive.PingPongProcessor - transaction = Transaction{transactionId='00000000000000a1', commandId='7fc67b9e-fe41-4164-94ea-4676b3c18e3e', workflowId='', effectiveAt=2022-02-26T09:18:54.683774Z, events=[ArchivedEvent{witnessParties=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], eventId='#00000000000000a1:0', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}, contractId='0025996ad01351bb26469f6146bcee072f74066a906436105dc324603681a5df12'}, CreatedEvent{witnessParties=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], eventId='#00000000000000a1:1', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}, contractId='00635c16dcb9caa8bca9a873b902c5e5990b74889a065e58e4656fd940394f4f22', arguments=DamlRecord{recordId=Optional[Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}], fields=[Field{label=Optional[sender], value=Party{value='ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb'}}, Field{label=Optional[receiver], value=Party{value='ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246'}}, Field{label=Optional[count], value=Int64{value=5}}]}, agreementText='Optional[]', contractKey=Optional.empty, signatories=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], observers=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246]}], offset='00000000000000a1'}
11:18:54.246 [client-1] INFO  e.p.reactive.PingPongProcessor - processTransaction: Transaction{transactionId='00000000000000a1', commandId='', workflowId='', effectiveAt=2022-02-26T09:18:54.683774Z, events=[ArchivedEvent{witnessParties=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], eventId='#00000000000000a1:0', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}, contractId='0025996ad01351bb26469f6146bcee072f74066a906436105dc324603681a5df12'}, CreatedEvent{witnessParties=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], eventId='#00000000000000a1:1', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}, contractId='00635c16dcb9caa8bca9a873b902c5e5990b74889a065e58e4656fd940394f4f22', arguments=DamlRecord{recordId=Optional[Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}], fields=[Field{label=Optional[sender], value=Party{value='ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb'}}, Field{label=Optional[receiver], value=Party{value='ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246'}}, Field{label=Optional[count], value=Int64{value=5}}]}, agreementText='Optional[]', contractKey=Optional.empty, signatories=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], observers=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246]}], offset='00000000000000a1'}
11:18:54.246 [client-1] INFO  e.p.reactive.PingPongProcessor - ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246 is exercising RespondPing on 00635c16dcb9caa8bca9a873b902c5e5990b74889a065e58e4656fd940394f4f22 in workflow  at count 5
11:18:54.251 [client-3] INFO  e.p.reactive.PingPongProcessor - processTransaction: Transaction{transactionId='00000000000000a1', commandId='7fc67b9e-fe41-4164-94ea-4676b3c18e3e', workflowId='', effectiveAt=2022-02-26T09:18:54.683774Z, events=[ArchivedEvent{witnessParties=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], eventId='#00000000000000a1:0', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}, contractId='0025996ad01351bb26469f6146bcee072f74066a906436105dc324603681a5df12'}, CreatedEvent{witnessParties=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], eventId='#00000000000000a1:1', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}, contractId='00635c16dcb9caa8bca9a873b902c5e5990b74889a065e58e4656fd940394f4f22', arguments=DamlRecord{recordId=Optional[Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}], fields=[Field{label=Optional[sender], value=Party{value='ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb'}}, Field{label=Optional[receiver], value=Party{value='ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246'}}, Field{label=Optional[count], value=Int64{value=5}}]}, agreementText='Optional[]', contractKey=Optional.empty, signatories=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], observers=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246]}], offset='00000000000000a1'}
11:18:54.494 [client-0] INFO  e.p.reactive.PingPongProcessor - transaction = Transaction{transactionId='00000000000000a2', commandId='1d3d7515-a685-404f-94eb-7acdf9a79112', workflowId='', effectiveAt=2022-02-26T09:18:54.921602Z, events=[ArchivedEvent{witnessParties=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], eventId='#00000000000000a2:0', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}, contractId='00635c16dcb9caa8bca9a873b902c5e5990b74889a065e58e4656fd940394f4f22'}, CreatedEvent{witnessParties=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], eventId='#00000000000000a2:1', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}, contractId='00f6ac0e590b1a67b01da88684951b114f730fce3e4de4061b22a922dee8f159a1', arguments=DamlRecord{recordId=Optional[Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}], fields=[Field{label=Optional[sender], value=Party{value='ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246'}}, Field{label=Optional[receiver], value=Party{value='ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb'}}, Field{label=Optional[count], value=Int64{value=6}}]}, agreementText='Optional[]', contractKey=Optional.empty, signatories=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], observers=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb]}], offset='00000000000000a2'}
11:18:54.494 [client-3] INFO  e.p.reactive.PingPongProcessor - processTransaction: Transaction{transactionId='00000000000000a2', commandId='', workflowId='', effectiveAt=2022-02-26T09:18:54.921602Z, events=[ArchivedEvent{witnessParties=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], eventId='#00000000000000a2:0', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}, contractId='00635c16dcb9caa8bca9a873b902c5e5990b74889a065e58e4656fd940394f4f22'}, CreatedEvent{witnessParties=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], eventId='#00000000000000a2:1', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}, contractId='00f6ac0e590b1a67b01da88684951b114f730fce3e4de4061b22a922dee8f159a1', arguments=DamlRecord{recordId=Optional[Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}], fields=[Field{label=Optional[sender], value=Party{value='ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246'}}, Field{label=Optional[receiver], value=Party{value='ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb'}}, Field{label=Optional[count], value=Int64{value=6}}]}, agreementText='Optional[]', contractKey=Optional.empty, signatories=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], observers=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb]}], offset='00000000000000a2'}
11:18:54.494 [client-2] INFO  e.p.reactive.PingPongProcessor - transaction = Transaction{transactionId='00000000000000a2', commandId='', workflowId='', effectiveAt=2022-02-26T09:18:54.921602Z, events=[ArchivedEvent{witnessParties=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], eventId='#00000000000000a2:0', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}, contractId='00635c16dcb9caa8bca9a873b902c5e5990b74889a065e58e4656fd940394f4f22'}, CreatedEvent{witnessParties=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], eventId='#00000000000000a2:1', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}, contractId='00f6ac0e590b1a67b01da88684951b114f730fce3e4de4061b22a922dee8f159a1', arguments=DamlRecord{recordId=Optional[Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}], fields=[Field{label=Optional[sender], value=Party{value='ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246'}}, Field{label=Optional[receiver], value=Party{value='ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb'}}, Field{label=Optional[count], value=Int64{value=6}}]}, agreementText='Optional[]', contractKey=Optional.empty, signatories=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], observers=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb]}], offset='00000000000000a2'}
11:18:54.494 [client-1] INFO  e.p.reactive.PingPongProcessor - processTransaction: Transaction{transactionId='00000000000000a2', commandId='1d3d7515-a685-404f-94eb-7acdf9a79112', workflowId='', effectiveAt=2022-02-26T09:18:54.921602Z, events=[ArchivedEvent{witnessParties=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], eventId='#00000000000000a2:0', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}, contractId='00635c16dcb9caa8bca9a873b902c5e5990b74889a065e58e4656fd940394f4f22'}, CreatedEvent{witnessParties=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], eventId='#00000000000000a2:1', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}, contractId='00f6ac0e590b1a67b01da88684951b114f730fce3e4de4061b22a922dee8f159a1', arguments=DamlRecord{recordId=Optional[Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}], fields=[Field{label=Optional[sender], value=Party{value='ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246'}}, Field{label=Optional[receiver], value=Party{value='ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb'}}, Field{label=Optional[count], value=Int64{value=6}}]}, agreementText='Optional[]', contractKey=Optional.empty, signatories=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], observers=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb]}], offset='00000000000000a2'}
11:18:54.494 [client-3] INFO  e.p.reactive.PingPongProcessor - ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb is exercising RespondPong on 00f6ac0e590b1a67b01da88684951b114f730fce3e4de4061b22a922dee8f159a1 in workflow  at count 6
11:18:54.724 [client-2] INFO  e.p.reactive.PingPongProcessor - transaction = Transaction{transactionId='00000000000000a3', commandId='b26075ce-55fd-4fc3-8c2f-056255896095', workflowId='', effectiveAt=2022-02-26T09:18:55.170859Z, events=[ArchivedEvent{witnessParties=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], eventId='#00000000000000a3:0', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}, contractId='00f6ac0e590b1a67b01da88684951b114f730fce3e4de4061b22a922dee8f159a1'}, CreatedEvent{witnessParties=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], eventId='#00000000000000a3:1', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}, contractId='00b2d1072a347f395c8af49c44d20308655f30f31a1c0009a4c655dcc56cfe07aa', arguments=DamlRecord{recordId=Optional[Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}], fields=[Field{label=Optional[sender], value=Party{value='ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb'}}, Field{label=Optional[receiver], value=Party{value='ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246'}}, Field{label=Optional[count], value=Int64{value=7}}]}, agreementText='Optional[]', contractKey=Optional.empty, signatories=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], observers=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246]}], offset='00000000000000a3'}
11:18:54.725 [client-1] INFO  e.p.reactive.PingPongProcessor - processTransaction: Transaction{transactionId='00000000000000a3', commandId='', workflowId='', effectiveAt=2022-02-26T09:18:55.170859Z, events=[ArchivedEvent{witnessParties=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], eventId='#00000000000000a3:0', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}, contractId='00f6ac0e590b1a67b01da88684951b114f730fce3e4de4061b22a922dee8f159a1'}, CreatedEvent{witnessParties=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], eventId='#00000000000000a3:1', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}, contractId='00b2d1072a347f395c8af49c44d20308655f30f31a1c0009a4c655dcc56cfe07aa', arguments=DamlRecord{recordId=Optional[Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}], fields=[Field{label=Optional[sender], value=Party{value='ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb'}}, Field{label=Optional[receiver], value=Party{value='ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246'}}, Field{label=Optional[count], value=Int64{value=7}}]}, agreementText='Optional[]', contractKey=Optional.empty, signatories=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], observers=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246]}], offset='00000000000000a3'}
11:18:54.725 [client-1] INFO  e.p.reactive.PingPongProcessor - ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246 is exercising RespondPing on 00b2d1072a347f395c8af49c44d20308655f30f31a1c0009a4c655dcc56cfe07aa in workflow  at count 7
11:18:54.726 [client-0] INFO  e.p.reactive.PingPongProcessor - transaction = Transaction{transactionId='00000000000000a3', commandId='', workflowId='', effectiveAt=2022-02-26T09:18:55.170859Z, events=[ArchivedEvent{witnessParties=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], eventId='#00000000000000a3:0', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}, contractId='00f6ac0e590b1a67b01da88684951b114f730fce3e4de4061b22a922dee8f159a1'}, CreatedEvent{witnessParties=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246], eventId='#00000000000000a3:1', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}, contractId='00b2d1072a347f395c8af49c44d20308655f30f31a1c0009a4c655dcc56cfe07aa', arguments=DamlRecord{recordId=Optional[Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}], fields=[Field{label=Optional[sender], value=Party{value='ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb'}}, Field{label=Optional[receiver], value=Party{value='ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246'}}, Field{label=Optional[count], value=Int64{value=7}}]}, agreementText='Optional[]', contractKey=Optional.empty, signatories=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], observers=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246]}], offset='00000000000000a3'}
11:18:54.740 [client-3] INFO  e.p.reactive.PingPongProcessor - processTransaction: Transaction{transactionId='00000000000000a3', commandId='b26075ce-55fd-4fc3-8c2f-056255896095', workflowId='', effectiveAt=2022-02-26T09:18:55.170859Z, events=[ArchivedEvent{witnessParties=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], eventId='#00000000000000a3:0', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Ping'}, contractId='00f6ac0e590b1a67b01da88684951b114f730fce3e4de4061b22a922dee8f159a1'}, CreatedEvent{witnessParties=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], eventId='#00000000000000a3:1', templateId=Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}, contractId='00b2d1072a347f395c8af49c44d20308655f30f31a1c0009a4c655dcc56cfe07aa', arguments=DamlRecord{recordId=Optional[Identifier{packageId='b8e864550d5d351c0d66cc8d6a899a119d06e900dfedc64396e284f69a8e27e6', moduleName='PingPong', entityName='Pong'}], fields=[Field{label=Optional[sender], value=Party{value='ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb'}}, Field{label=Optional[receiver], value=Party{value='ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246'}}, Field{label=Optional[count], value=Int64{value=7}}]}, agreementText='Optional[]', contractKey=Optional.empty, signatories=[ledger-party-1dc946a0-0398-4d81-8dc0-d5a976096cdb], observers=[ledger-party-38f28393-45b6-4740-806a-ffc5d01a8246]}], offset='00000000000000a3'}
11:18:54.742 [client-0] ERROR c.d.g.a.SingleThreadExecutionSequencer - Unhandled exception in SingleThreadExecutionSequencer.
io.reactivex.exceptions.OnErrorNotImplementedException: INTERNAL: RST_STREAM closed stream. HTTP/2 error code: INTERNAL_ERROR
	at io.reactivex.internal.functions.Functions$OnErrorMissingConsumer.accept(Functions.java:704)
	at io.reactivex.internal.functions.Functions$OnErrorMissingConsumer.accept(Functions.java:701)
	at io.reactivex.internal.subscribers.LambdaSubscriber.onError(LambdaSubscriber.java:79)
	at io.reactivex.internal.operators.flowable.FlowableFlattenIterable$FlattenIterableSubscriber.checkTerminated(FlowableFlattenIterable.java:396)
	at io.reactivex.internal.operators.flowable.FlowableFlattenIterable$FlattenIterableSubscriber.drain(FlowableFlattenIterable.java:256)
	at io.reactivex.internal.operators.flowable.FlowableFlattenIterable$FlattenIterableSubscriber.onError(FlowableFlattenIterable.java:182)
	at io.reactivex.internal.subscribers.BasicFuseableSubscriber.onError(BasicFuseableSubscriber.java:101)
	at com.daml.grpc.adapter.client.rs.BufferingResponseObserver.lambda$onError$3(BufferingResponseObserver.java:81)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: io.grpc.StatusRuntimeException: INTERNAL: RST_STREAM closed stream. HTTP/2 error code: INTERNAL_ERROR
	at io.grpc.Status.asRuntimeException(Status.java:535)
	at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:478)
	at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:557)
	at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:69)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:738)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:717)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
	... 3 common frames omitted

According to google and by looking at the stack-trace this issue need to be investigated on the server side (Ledger) and DA might want to implement a callback handler (rather than OnErrorNotImplementedException) to process such error.

I assume this issue wasn’t tested before (manually creating a contract) since the pingPongReactive app was alive for 5 seconds (i extended it to 2 minute).

private static void processActiveContracts(DamlLedgerClient client, XsdMetadata metadata) {
    ActiveContractsClient activeContractsClient = client.getActiveContractSetClient();
    TransactionFilter filter = filterFor(TEMPLATE_ID, ADMIN_PARTY);
    Flowable<GetActiveContractsResponse> activeContracts =
        activeContractsClient.getActiveContracts(filter, true);
    activeContracts.blockingForEach(
        response -> {
          response.getCreatedEvents().stream()
              .map(PolicyPagesNotification.Contract::fromCreatedEvent)
              .forEach(
                  contract -> {
                      // Do something with the contract
                  });
        });
  }


  private static TransactionFilter filterFor(Identifier templateId, String party) {
    InclusiveFilter inclusiveFilter = new InclusiveFilter(Collections.singleton(templateId));
    Map<String, Filter> filter = Collections.singletonMap(party, inclusiveFilter);
    return new FiltersByParty(filter);
  }