Sequencer_backpressure

Canton Network
1

Hello,
For the last 24 hour i got this error on devnet:

{
“code”: “SEQUENCER_BACKPRESSURE”,
“cause”: “The sequencer is overloaded.”,
“correlationId”: null,
“traceId”: “6bf1ce089bd11e1dfd9497e8a45d9d26”,
“context”: {
“participant”: “participant”,
“reason”: “SendAsyncErrorGrpc(\n Request failed for sequencer.\n GrpcRequestRefusedByServer: ABORTED/SEQUENCER_OVERLOADED(2,6bf1ce08): Submission rate exceeds rate limit of 0/s.\n Request: send-async-versioned/f6b31b16-0cd7-4f30-88f8-10d04fb9b39d\n DecodedCantonError(\n code = ‘SEQUENCER_OVERLOADED’,\n category = ContentionOnSharedResources,\n cause = “Submission rate exceeds rate limit of 0/s.”,\n traceId = ‘6bf1ce089bd11e1dfd9497e8a45d9d26’,\n context = Seq(‘outer=>Overloaded’, ‘message1=>Submission rate exceeds rate limit of 0/s.’, ‘sequencer=>sequencer’, ‘definite_answer=>true’)\n)\n)”,
“synchronizerId”: “global-domain::1220be58c29e…”,
“category”: “2”,
“tid”: “6bf1ce089bd11e1dfd9497e8a45d9d26”,
“definite_answer”: “true”
},
“resources”: ,
“errorCategory”: 2,
“grpcCodeValue”: 10,
“retryInfo”: “1 second”,
“definiteAnswer”: null
}

Any hint how to fix it ?
Thanks

2

This happens when the sequencer is receiving more commands than it can handle. The command that you tried to submit was rejected because the sequencer is overloaded.

See here for the recommended resolution:

SEQUENCER_OVERLOADED

  • Explanation: The sequencer is overloaded and cannot handle the request.
  • Resolution: Retry with exponential backoff.
  • Category: ContentionOnSharedResources
  • Conveyance: This error is logged with log-level INFO on the server side and exposed on the API with grpc-status ABORTED including a detailed error message.
3

Thanks for you answer, I have tried different approaches but still get same error.

4

Can you compare your DevNet node’s version with the version listed here?

The current version is 0.5.1. There was a Synchronizer Upgrades with Downtime which took place yesterday.

5

It 0.4.20, do I need to upgrade? if so how ?

6

The page Synchronizer Upgrades with Downtime describes the upgrade process.

(PS. If you run into issues with the upgrade, let’s start a new thread. Thanks!)

1 Like