I am setting up a Canton HA using examples/e04-high-availability. I would like to understand the mechanism of active-active on sequencers.
We have two different participant nodes, p1 and p2, already replicated and two sequencers seq_a and seq_b (on the same database). After bootstrap everything is well connected, and from the health status of the sequencers we observe that,
p1 to seq_a
p2 to seq_b
Assuming a transaction from p1 requires p2 confirmation. How seq_b knows this and sends to p2?
The sequencers have a shared database, and reading from any sequencer instance should return the same data. Ideally you should connect these participants to both sequencer instances (either with a load balancer or using the connect_ha admin command). This way if any of the instances go down the participant will quickly connect to another online sequencer.
Technically in the sequencer instances they just check the shared database a handful of times a second to see if any writes have occurred that need to be served to any participants currently connected to them. You can find a bunch more detail on exactly what’s happening in the HA architecture section in the user manual for the Sequencer.
To very specifically answer your question. The first sequencer writes an event to the shared database with an envelope addressing it to your second participant (the payload at the sequencer is encrypted so it has no idea it’s a tx confirmation request - to it it’s just a bunch of bytes it’s delivering to participants). The second sequencer instance will quickly notice that there’s a new event for a participant connected to it so will read it and push it over a grpc streaming subscription.