When you submit-and-wait for a command, the ledger API explicitly declares that RESOURCE_EXHAUSTED may be returned in certain circumstances.
service CommandService {
// Submits a single composite command and waits for its result.
// Returns ``RESOURCE_EXHAUSTED`` if the number of in-flight commands reached the maximum (if a limit is configured).
// Propagates the gRPC error of failed submissions including DAML interpretation errors.
rpc SubmitAndWait (SubmitAndWaitRequest) returns (google.protobuf.Empty);
No similar circumstances are documented for the pair of command submission and completion services.
// 1) ``INVALID_PARAMETER`` gRPC error on malformed payloads and missing required fields.
// 2) Failure communicated in the gRPC error.
// 3) Failure communicated in a Completion.
// 4) A Checkpoint with ``record_time`` > command ``mrt`` arrives through the Completion Stream, and the command's Completion was not visible before. In this case the command is lost.
// Identifies the exact type of the error.
// For example, malformed or double spend transactions will result in a ``INVALID_ARGUMENT`` status.
// Transactions with invalid time time windows (which may be valid at a later date) will result in an ``ABORTED`` error.
// Optional
google.rpc.Status status = 2;
However, @cocreature pointed out that there seems to be a chain of code in Sandbox that would yield RESOURCE_EXHAUSTED from the command submission service.
private def handleSubmissionResult(result: Try[SubmissionResult])(
implicit loggingContext: LoggingContext,
): Try[Unit] = result match {
case Success(Acknowledged) =>
logger.debug("Success")
Success(())
case Success(Overloaded) =>
logger.info("Back-pressure")
Failure(Status.RESOURCE_EXHAUSTED.asRuntimeException)
Yet that is only used from deduplicateAndRecordOnLedger, which would seem to imply much more evaluation than the command submission service should, at minimum, carry out. @cocreature reasonably suggested that I’m reading too much into it, but I think it is at least equally likely that Overloaded is an impossible state in the above function, handled to satisfy the exhaustiveness checker.
In a sense, that is all besides the point, because my goal is to be a well-behaved ledger API client, not one entangled with the foibles of this particular server. That yields two questions:
(1) When is it permissible for these services to return RESOURCE_EXHAUSTED? May a Ledger API server yield this from the submission call, as an element in the completion stream, or one of these at its discretion?
(2) What is required for these services to return with respect to RESOURCE_EXHAUSTED? For example, if a service accepts a command submission, is it then constrained in this respect with regard to related responses from the command completion service?
For context, the above arose during #7820.