Manual versus Automatic Pruning

Questions
,
1

I’m trying to understand the difference between the Participant Pruning section versus the Ledger Pruning section of the documentation.

  1. My current understanding is that the Participant Pruning is about manual pruning invocation via the Pruning Service available on the participant node’s Ledger API, while the Ledger Pruning section is Canton-specific and provides higher level mechanisms to schedule and auto-trigger the pruning process for convenience. Is my understanding correct?

  2. The Pruning Service can only be used to invoke Participant Pruning and there is no way to invoke sequencer and mediator pruning (manually) via the Ledger API - it can only be done via the automatic pruning approach. Is my understanding correct?

  3. If I use the Canton-specific mechanism to schedule/automate the pruning process based on this approach, then I don’t really need to invoke the Ledger API directly, right? The automatic pruning mechanism will do an under-the-hood conversion from the times I specify in the cron expression into offsets and other parameters needed to construct the PruneRequest to be sent to the Ledger API?

  4. Are there any situations in which I cannot use automatic pruning and must use manual pruning via the Pruning Service?

  5. Invoking the Pruning Service on the Ledger API requires administrative privileges. How would this work for enabling automatic pruning which has to be done on the Canton Console?

  6. How do you cancel automatic pruning that has already been scheduled but needs to be cancelled due to some circumstances? Any restrictions on this, eg, how long before the process it needs to be cancelled etc?

1 Like
2

Thank you for your questions, Mr_Mannoroth

We are in the process of reconciling the Participant and Ledger Pruning documentation pages, but your understanding of the points is correct.

Here is some more detail to your specific questions:

  1. Correct. The intention of introducing automatic pruning in the 2.6 release was to simplify the user experience by letting our users specify when to prune and have canton nodes automatically figure out the logistics such as internal batching and responding to failover.
  2. I should point out that you can also prune participants, sequencers, and mediators manually via the Canton console, but as mentioned on the “Ledger Pruning” page, we discourage this somewhat as you would otherwise have to handle various aspects yourself such as failover or as you mention in point 6 canceling. I encourage you to pursue automatic pruning, but for the sake of complete disclosure here are two links (1) to how you would have to perform canton-pruning prior to the implementation of automatic pruning: Ledger Pruning — Daml SDK 2.5.5 documentation and (2) an example of the command you would invoke manually on the sequencer Console Commands — Daml SDK 2.6.3 documentation .
  3. Correct, automatic pruning performs a number of things under the hood including mapping of times to offsets as well as following the active node in case of a failover mid-pruning.
  4. Our intention was to cover all scenarios with automatic pruning. At times an operator may want to perform pruning outside the scheduled pruning window and then it is perfectly fine to invoke pruning manually via the canton console or in the case of participant pruning via the Pruning Service. As for participant pruning, the Pruning Service prune endpoint and the canton console participant prune command Console Commands — Daml SDK 2.6.3 documentation invoke the same code and are interchangeable.
  5. Pruning via the canton console, either automatically or manually, also requires administrative privileges.
  6. Cancelling pruning is easier to perform when you use automatic pruning. You would do so using the clear_schedule command on one or all nodes on which you need to interrupt pruning. In terms of how long it takes for a cancellation to take full effect, the nodes typically stop pruning within one minute for sequencers and mediators. Participants can take longer - I have seen up to 15 minutes in our testing - to cancel as pruning participants involves more database tables and therefore the default “batch size” is 50000 rather than 1000.

Please let me know if I can assist you more on either automated or manual pruning.

Thanks,
– Oliver

2 Likes
3

Perfect - thank you so much @oliverse for the detailed explanation!