Pointing to private IP and port combination using domains.connect within Azure VMs on linked vnets

Questions
,
1

Hi all,

I’m trying to connect a participant to a domain node located on 2 different peered Vnets on Virtual Machines hosted with Azure using the

participant.domains.connect

command.

When I try to specify the privateIP:port combination in the command,
e.g domains.connect("domainAlias","10.2.3.4:5050")

I receive a an error
java.net.URISyntaxException: Illegal character in scheme name at index 0: 10.2.3.4:5050.

The private IP address cannot be used with a http:// prefix and it seems like the Canton console command does not accept the combination input syntax without a http:// prefix.

The privateIP:port combination is proven to work during remote console access when I include them as parameters in the Canton .conf file as such

address = 10.2.3.4
port = 5050

so i’m wondering how one can get domains.connect to accept it as well.

Is there a method to specify the port number when using a private IP address in the domains.connect command similar to how one would do so using a URL with a http:// prefix?
e.g domains.connect("domainAlias","http://localhost:5050")?

Thank you.

2

What error do you get if you try this?

domains.connect("domainAlias","http://10.2.3.4:5050")

I don’t see why a private ip address should not work there.

3

Using participant1.domains.connect(“”,“http://10.2.3.4:5050”)

ERROR c.d.c.c.EnterpriseConsoleEnvironment - Request failed for participant1.
  GrpcClientError: UNAVAILABLE/DOMAIN_IS_NOT_AVAILABLE(1,34fdba02): Cannot connect to domain Domain 'domain'
  Request: ConnectDomain(Domain 'domain',false)
  CorrelationId: 34fdba02896418e4a73843e7a5ba9ca6
  RetryIn: 1 second
  Context: HashMap(participant -> participant1, domain -> domain, reason -> Request failed for domain. Is the server running? Did you configure the server address as 0.0.0.0? Are you using the right TLS settings?
  GrpcServiceUnavailable: UNAVAILABLE/io exception
  Request: get domain id
  Causes: Connection refused: /http://10.2.3.4:443
    Connection refused, alias -> Domain 'domain')
  Command ParticipantAdministration$domains$.connect invoked from cmd1.sc:1
com.digitalasset.canton.console.CommandFailure: Command execution failed.

My bad, I just noticed that the request does get routed to 10.2.3.4 so it seems that it does detect it as a legitimate address, it just resolves to port 443 instead of port 5050. Would there be any reason this happens when connecting the participant node to the domain?

I didn’t experience this issue when bootstrapping the separatedomain nodes or when I accessed the individual nodes using a remote node, all on separate VMs with the same network connectivity privileges that exist between the participant node and domain node I am trying to connect to.

4

This sounds like there is already a configuration stored in either the console or database, you should probably try to follow the remote sequencer connection steps where you have full control over the proto, IP and port:

Section Connect Using Register
https://docs.daml.com/canton/usermanual/connectivity.html#connect-using-register

Edit: could you inspect the registered domains?

// Stored configuration
participant.domains.list_registered()
// Probably empty if you have no working connections
participant.domains.list_connected()
2 Likes
5

Thank you @davidd, it sounds about right. I attempted to establish a connection with a participant node with an in-memory storage configuration instead and it connected successfully.

I’ll drop all my databases, recreate them and retry again. Will update here on the results.

I encountered another issue below when I attempted a domain.setup.bootstrap command from the remote console connected to the sequencer, mediator and domain manager nodes.

ERROR c.d.c.c.EnterpriseConsoleEnvironment - Request failed for remoteDomain.
  GrpcRequestRefusedByServer: NOT_FOUND/NO_APPROPRIATE_SIGNING_KEY_IN_STORE(11,7d5dae9b): Could not find an appropriate signing key to issue the topology transaction
  Request: AuthorizeOwnerToKeyMapping(Add,None,SEQ::domain::122008a032ae...,122046a3a3c4...,signing,false)
  CorrelationId: 7d5dae9bfe722354e7dcf74793a8b93b
  Context: Map(candidates -> List(), domain-manager -> domain)
  Command SetupAdministration$Setup.bootstrap_domain invoked from cmd4.sc:1
com.digitalasset.canton.console.CommandFailure: Command execution failed.

This happened when I ran the bootstrap command shortly after recreating my databases and brought all my nodes back up.

I did not restart the remote console during this process.

Is this potentially caused by existing values in the in-memory storage of the remote console or references to missing values in the db which are used during the bootstrap which I assume are populated in the nodes’ dbs during spinning up of all the nodes?

I’ll reattempt a fresh connection regardless, just wanted to make sure I didn’t break anything while dropping the databases.

Thanks!

6

Were you successful when you restarted the remote console as well?

7

Thanks @davidd , that was indeed the issue! Using a fresh database indeed resolved the problem.

8

Hi @Mate_Varga yes that resolved the issue.
In addition to restarting the remote console, I performed a health check to ensure the nodes are reachable from the remote console before attempting the bootstrap as well.