Pruning impact on ledger API connections

Questions
,
1

Can I confirm that every time pruning happens, the gRPC ledger API connections may drop and not hang?
From a client application perspective, we are planning to re-attempt to establish the connection to the participant node if it drops (without the need to restart the client application). For this purpose, we wanted to confirm that the connections to the gRPC ledger API will drop with an exception and not hang, every time pruning is executed. Do you know what specific exception will be thrown that we should be on the lookout for?

2

If you prune a ledger, it will drop all events <= pruning-offset and delete all the contracts that have been archived before the pruning-offset. This dropping operation happens in the back-ground. The node remains fully operational and you normally don’t notice it, except for some reduction in maximum throughput. The pruning happens “chunk-wise”, starting at the oldest event, slowly rolling up event batch after event batch.

So, no, the connections will not be killed and won’t hang and you don’t need to expect specific exceptions.

However, it’s very recommendable to write the application in a way that deals with connection outages (can be infrastructure related) and deal with errors.

Generally, Canton provides rich errors as described in here:
https://docs.daml.com/canton/reference/error_codes.html#machine-readable-information

These map to GRPC errors but come with additional information that can support you with automated error handling.

I hope this helps.

3

Thank you @Ratko_Veprek!

However, the reason i asked was because the docs here mention the following:

Active data streams from the Daml Participant may abort and need to be re-established by the Daml application from a later offset than pruned, even if they are already streaming past it.

So if I have a transaction stream open, I wouldn’t need to worry about the above?

4

Thank you for your questions @Mr_Mannoroth .

In general pruning at a particular offset interferes with queries that attempt to access offsets “before” the pruning offset, so ideally we recommend avoiding issuing querying offset ranges that overlap with pruning.

In cases in which that is impractical, queries produce PARTICIPANT_PRUNED_DATA_ACCESSED errors:

  • fetching completion and transaction streams error when the “overall” start-range precedes the pruning offset - as the documentation says - even if the streaming has moved beyond the pruning offset.
  • fetching transactions via the active contract service is a bit more forgiving in that you’d only get a PARTICIPANT_PRUNED_DATA_ACCESSED error if the pruning offset is in the offset range still being streamed.

Best regards,
– Oliver

1 Like