Sharding Canton Sequencers

Based on my understanding, canton performance scaling involves horizontally scaling participant nodes, then the mediator, and then the sequencer.

How would sequencer sharding work, given the sequencer performs transaction sequencing which is usually not a parallelisable task for the transactions within a particular domain? This might be an oversimplification, but assuming x transactions in a queue, how would n sequencers each sequencing x/n transactions correctly sequence across all x transactions?

2 Likes

There is no need for a queue that all sequencers share, because the order of messages to sequence is determined only as part of the sequencing and not before. So each sequencer has its own queue of messages that it wants to sequence. Abstractly, the order can be encoded by assigning some sort of counter to each message with the constraint that no counter is assigned multiple times. So if you partition the possibly assigned counters by sequencers (say if you have two sequencers, then one sequencer assigns only odd numbers and the other sequencer assigns only even numbers), then assigning numbers to messages can also be done in parallel. So we can actually write in parallel the messages with their counters into storage.

When you read the messages from the storage, you then have to make sure that you don’t read a message whose counter is higher than what any of the sequencers might still assign. For example, if the odd-counter sequencer is at 17 and the even-counter sequencer at 42 and the reader finds messages with counters 16, 17, 18, 24, 26, 40 in the storage, then it must not assume that there will not be a message with counter 23, because the odd-counter sequencer may still assign this counter. So a bit of synchronization is needed there, but in a high-throughput setting, we only need to synchronize the writers and readers every so often, not after each message.

2 Likes