SBT installation issue for daml-on-fabric

Hi Team

I ran into a similar issue as described here

https://githubmemory.com/repo/digital-asset/daml-on-fabric/issues/57

My environment is a Ubuntu Linux environment, not a mac

Any recommendation ?

Thanks

1 Like

Can you share the exact error message you’re seeing?

1 Like

Here is the outcome of the

sbt "run --port 6865 --role provision,time,ledger,explorer ../my-app/.daml/dist/my-app-0.1.0.dar"

Log below

myuser:/DAML-HLF/my-app/daml-on-fabric$ sbt "run --port 6865 --role provis                               ion,time,ledger,explorer ../my-app/.daml/dist/my-app-0.1.0.dar"
[info] welcome to sbt 1.4.7 (Ubuntu Java 11.0.10)
^[[D[info] loading settings for project daml-on-fabric-build from assembly.sbt,plugins.sbt ...
[info] loading project definition from /DAML-HLF/my-app/daml-on-fabric/project
[info] loading settings for project root from build.sbt ...
[info] set current project to DAML-on-Fabric (in build file:/DAML-HLF/my-app/daml-on-fabric/)
[info] running com.daml.DamlOnFabricServer --port 6865 --role provision,time,ledger,explorer ../my-app/.daml/dist/my                               -app-0.1.0.dar
Mar 08, 2021 2:14:39 PM org.hyperledger.fabric.sdk.helper.Config <init>
WARNING: Failed to load any configuration from: config.properties. Using toolkit defaults
Admin@org1.example.com private key: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/keystore/priv_sk
Admin@org1.example.com sign cert: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/signcerts/Admin@org1.example.com-cert.pem
Admin@org1.example.com private key: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/keystore/priv_sk
Admin@org1.example.com sign cert: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/signcerts/Admin@org1.example.com-cert.pem
Admin@org2.example.com private key: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp/keystore/priv_sk
Admin@org2.example.com sign cert: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp/signcerts/Admin@org2.example.com-cert.pem
Constructing channel mainchannel
Admin@org1.example.com private key: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/keystore/priv_sk
Admin@org1.example.com sign cert: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/signcerts/Admin@org1.example.com-cert.pem
Created channel mainchannel
Admin@org1.example.com private key: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/keystore/priv_sk
Admin@org1.example.com sign cert: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/signcerts/Admin@org1.example.com-cert.pem
Mar 08, 2021 2:14:43 PM org.hyperledger.fabric.sdk.Channel joinPeer
INFO: Channel{id: 9, name: mainchannel} joining Peer{ id: 2, name: peer0, channelName: null, url: grpcs://localhost:10051}.
Mar 08, 2021 2:14:43 PM org.hyperledger.fabric.sdk.Channel joinPeer
INFO: Peer Peer{ id: 2, name: peer0, channelName: mainchannel, url: grpcs://localhost:10051} joined into channel Channel{id: 9, name: mainchannel}
Peer peer0 grpcs://localhost:10051 joined channel mainchannel
Admin@org2.example.com private key: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp/keystore/priv_sk
Admin@org2.example.com sign cert: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp/signcerts/Admin@org2.example.com-cert.pem
Mar 08, 2021 2:14:43 PM org.hyperledger.fabric.sdk.Channel joinPeer
INFO: Channel{id: 9, name: mainchannel} joining Peer{ id: 5, name: peer0, channelName: null, url: grpcs://localhost:10056}.
Mar 08, 2021 2:14:43 PM org.hyperledger.fabric.sdk.Channel joinPeer
INFO: Peer Peer{ id: 5, name: peer0, channelName: mainchannel, url: grpcs://localhost:10056} joined into channel Channel{id: 9, name: mainchannel}
Peer peer0 grpcs://localhost:10056 joined channel mainchannel
Joined peers to channel mainchannel
Mar 08, 2021 2:14:43 PM org.hyperledger.fabric.sdk.Channel initialize
INFO: Channel Channel{id: 9, name: mainchannel} eventThread started shutdown: false  thread: null
No META-INF folder detected. Starting chaincode without any custom indexes
Orgs installs the chaincode on its peers.
Admin@org1.example.com private key: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/keystore/priv_sk
Admin@org1.example.com sign cert: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/signcerts/Admin@org1.example.com-cert.pem
Caught an exception running Channel mainchannel
org.hyperledger.fabric.sdk.exception.ProposalException: Status of install proposal did not ret ok for Peer{ id: 2, name: peer0, channelName: mainchannel, url: grpcs://localhost:10051}, FAILURE
        at org.hyperledger.fabric.sdk.LifecycleInstallChaincodeProposalResponse.getPackageId(LifecycleInstallChaincodeProposalResponse.java:36)
        at com.daml.FabricContext.lifecycleInstallChaincode(FabricContext.java:319)
        at com.daml.FabricContext.runChannel(FabricContext.java:244)
        at com.daml.FabricContext.initNetworkConfiguration(FabricContext.java:169)
        at com.daml.FabricContext.<init>(FabricContext.java:103)
        at com.daml.DAMLKVConnector.<init>(DAMLKVConnector.java:74)
        at com.daml.DAMLKVConnector.get(DAMLKVConnector.java:30)
        at com.daml.DamlOnFabricServer$.delayedEndpoint$com$daml$DamlOnFabricServer$1(DamlOnFabricServer.scala:65)
        at com.daml.DamlOnFabricServer$delayedInit$body.apply(DamlOnFabricServer.scala:48)
        at scala.Function0.apply$mcV$sp(Function0.scala:39)
        at scala.Function0.apply$mcV$sp$(Function0.scala:39)
        at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
        at scala.App.$anonfun$main$1$adapted(App.scala:80)
        at scala.collection.immutable.List.foreach(List.scala:392)
        at scala.App.main(App.scala:80)
        at scala.App.main$(App.scala:78)
        at com.daml.DamlOnFabricServer$.main(DamlOnFabricServer.scala:48)
        at com.daml.DamlOnFabricServer.main(DamlOnFabricServer.scala)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at sbt.Run.invokeMain(Run.scala:133)
        at sbt.Run.execute$1(Run.scala:82)
        at sbt.Run.$anonfun$runWithLoader$5(Run.scala:110)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
        at sbt.util.InterfaceUtil$$anon$1.get(InterfaceUtil.scala:17)
        at sbt.TrapExit$App.run(TrapExit.scala:258)
        at java.base/java.lang.Thread.run(Thread.java:834)
Admin@org1.example.com private key: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/keystore/priv_sk
Admin@org1.example.com sign cert: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/signcerts/Admin@org1.example.com-cert.pem
Admin@org2.example.com private key: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp/keystore/priv_sk
Admin@org2.example.com sign cert: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp/signcerts/Admin@org2.example.com-cert.pem
That's all folks!
Admin@org1.example.com private key: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/keystore/priv_sk
Admin@org1.example.com sign cert: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/signcerts/Admin@org1.example.com-cert.pem
[error] (run-main-0) com.daml.FabricContextException: response {
[error]   status: 500
[error]   message: "make sure the chaincode daml_on_fabric has been successfully defined on channel mainchannel and try again: chaincode daml_on_fabric not found"
[error] }
REST: Listening on port 8080 (try to query http://127.0.0.1:8080/blocks)
REST: Initializing background thread...
REST: Blocks reader: Starting...
[error] com.daml.FabricContextException: response {
[error]   status: 500
[error]   message: "make sure the chaincode daml_on_fabric has been successfully defined on channel mainchannel and try again: chaincode daml_on_fabric not found"
[error] }
[error]
[error]         at com.daml.FabricContext.queryChaincode(FabricContext.java:578)
[error]         at com.daml.FabricContext.queryChaincode(FabricContext.java:558)
[error]         at com.daml.FabricContext.queryChaincode(FabricContext.java:554)
[error]         at com.daml.DAMLKVConnector.getLedgerId(DAMLKVConnector.java:316)
[error]         at com.daml.DAMLKVConnector.<init>(DAMLKVConnector.java:90)
[error]         at com.daml.DAMLKVConnector.get(DAMLKVConnector.java:30)
[error]         at com.daml.DamlOnFabricServer$.delayedEndpoint$com$daml$DamlOnFabricServer$1(DamlOnFabricServer.scala:65)
[error]         at com.daml.DamlOnFabricServer$delayedInit$body.apply(DamlOnFabricServer.scala:48)
[error]         at scala.Function0.apply$mcV$sp(Function0.scala:39)
[error]         at scala.Function0.apply$mcV$sp$(Function0.scala:39)
[error]         at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
[error]         at scala.App.$anonfun$main$1$adapted(App.scala:80)
[error]         at scala.collection.immutable.List.foreach(List.scala:392)
[error]         at scala.App.main(App.scala:80)
[error]         at scala.App.main$(App.scala:78)
[error]         at com.daml.DamlOnFabricServer$.main(DamlOnFabricServer.scala:48)
[error]         at com.daml.DamlOnFabricServer.main(DamlOnFabricServer.scala)
[error]         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error]         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
Admin@org1.example.com private key: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/keystore/priv_sk
Admin@org1.example.com sign cert: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/signcerts/Admin@org1.example.com-cert.pem
[error]         at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error]         at java.base/java.lang.reflect.Method.invoke(Method.java:566)
Admin@org1.example.com private key: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/keystore/priv_sk
Admin@org1.example.com sign cert: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/signcerts/Admin@org1.example.com-cert.pem
[error] stack trace is suppressed; run last Compile / bgRun for the full output
Admin@org2.example.com private key: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp/keystore/priv_sk
Admin@org2.example.com sign cert: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp/signcerts/Admin@org2.example.com-cert.pem
Constructing channel mainchannel
Created channel mainchannel
Admin@org1.example.com private key: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/keystore/priv_sk
Admin@org1.example.com sign cert: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/signcerts/Admin@org1.example.com-cert.pem
Peer peer0 grpcs://localhost:10051 joined channel mainchannel
Admin@org2.example.com private key: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp/keystore/priv_sk
Admin@org2.example.com sign cert: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp/signcerts/Admin@org2.example.com-cert.pem
Peer peer0 grpcs://localhost:10056 joined channel mainchannel
Added peers to channel mainchannel
Mar 08, 2021 2:14:47 PM org.hyperledger.fabric.sdk.Channel initialize
INFO: Channel Channel{id: 25, name: mainchannel} eventThread started shutdown: false  thread: null
No META-INF folder detected. Starting chaincode without any custom indexes
That's all folks!
Admin@org1.example.com private key: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/keystore/priv_sk
Admin@org1.example.com sign cert: /DAML-HLF/my-app/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/signcerts/Admin@org1.example.com-cert.pem
REST: Blocks reader: Started.

@bartcant The issue we have seen previously is with the Hyperledger Fabric images when gRPC FUSE for file sharing is enabled in the docker client but it was a specific issue on OSX: have you made sure this is disabled?

Also what Docker client version are you running?

hi @sormeter
I indeed saw this comment on the message boards.

Difference is that I am not using Mac OSX. My environment is Ubuntu Linux

my docker version is
Docker version 20.10.3, build 48d30b5

Is there a setting on docker directly that allows me to disable this feature ? I don’t have any docker desktop version enabled on my linux box

@cocreature @sormeter Any further thoughts on a solution for a Linux based system ?

As far as I know the gRPC fuse is only a thing in docker for mac. I’m not sure what else could be causing it. Maybe @sormeter has an idea.

Hi @bartcant did you finally get this through? I just test the setup and finally make it working (takes me some time for troubleshooting).

hi @kctam,

Unfortunately, I am still stuck on the same issue.

I would be interested to know any issue and potential solutions you have encountered.

Hi @bartcant I just git clone the latest code. The only modification is on the config-local.yaml as I notice a timeout happens when the cc container is being built. By changing the invokeWaitTime to 120000 (was 60000) it works then. (though the name invokeWaitTime shouldn’t have impact on the image build. anyway, it works after the modification.)

I just retest again and it still works fine.

Thanks. I will make a similar change and see if it has any positive impact

A post was split to a new topic: Daml on Fabric quickstart example: package com.daml.ledger.rxjava does not exist

I am still researching my original error for installing daml-on-fabric

Maybe these additional logs from Hyperledger can shed some new light on the existing error

As a recap by starting the sbt "run --port 6865 --role provision,time,ledger,explorer ../quickstart/.daml/dist/quickstart-0.0.1.dar"

command I see the following error in my Linux Ubuntu machine

Admin@org1.example.com sign cert: /home/user1/daml-on-fabric/src/test/fixture/tmp/data/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/signcerts/Admin@org1.example.com-cert.pem
REST: Blocks reader: Started.
[error] com.daml.FabricContextException: response {
[error]   status: 500
[error]   message: "make sure the chaincode daml_on_fabric has been successfully defined on channel mainchannel and try again: chaincode daml_on_fabric not found"
[error] }
[error] 
[error] 	at com.daml.FabricContext.queryChaincode(FabricContext.java:578)
[error] 	at com.daml.FabricContext.queryChaincode(FabricContext.java:558)
[error] 	at com.daml.FabricContext.queryChaincode(FabricContext.java:554)
[error] 	at com.daml.DAMLKVConnector.getLedgerId(DAMLKVConnector.java:316)
[error] 	at com.daml.DAMLKVConnector.<init>(DAMLKVConnector.java:90)
[error] 	at com.daml.DAMLKVConnector.get(DAMLKVConnector.java:30)
[error] 	at com.daml.DamlOnFabricServer$.delayedEndpoint$com$daml$DamlOnFabricServer$1(DamlOnFabricServer.scala:65)
[error] 	at com.daml.DamlOnFabricServer$delayedInit$body.apply(DamlOnFabricServer.scala:48)
[error] 	at scala.Function0.apply$mcV$sp(Function0.scala:39)
[error] 	at scala.Function0.apply$mcV$sp$(Function0.scala:39)
[error] 	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
[error] 	at scala.App.$anonfun$main$1$adapted(App.scala:80)
[error] 	at scala.collection.immutable.List.foreach(List.scala:392)
[error] 	at scala.App.main(App.scala:80)
[error] 	at scala.App.main$(App.scala:78)
[error] 	at com.daml.DamlOnFabricServer$.main(DamlOnFabricServer.scala:48)
[error] 	at com.daml.DamlOnFabricServer.main(DamlOnFabricServer.scala)
[error] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[error] stack trace is suppressed; run 'last Compile / bgRun' for the full output

The Hyperledeger Log file shows the following error

[32mpeer0.org1.example.com_1  |e[0m e[31m2021-04-20 21:13:16.178 UTC [chaincode.platform] func1 -> ERRO 02ee[0m docker build failed: Error returned from build: 1 "go: github.com/golang/protobuf@v1.4.2: Get "https://proxy.golang.org/github.com/golang/protobuf/@v/v1.4.2.mod": EOF
e[32mpeer0.org1.example.com_1  |e[0m "
e[32mpeer0.org1.example.com_1  |e[0m e[31m2021-04-20 21:13:16.178 UTC [dockercontroller] buildImage -> ERRO 02fe[0m Error building image: docker build failed: Error returned from build: 1 "go: github.com/golang/protobuf@v1.4.2: Get "https://proxy.golang.org/github.com/golang/protobuf/@v/v1.4.2.mod": EOF
e[32mpeer0.org1.example.com_1  |e[0m "
e[32mpeer0.org1.example.com_1  |e[0m e[31m2021-04-20 21:13:16.179 UTC [dockercontroller] buildImage -> ERRO 030e[0m Build Output:
e[32mpeer0.org1.example.com_1  |e[0m ********************
e[32mpeer0.org1.example.com_1  |e[0m 
e[32mpeer0.org1.example.com_1  |e[0m ********************

Detailed Logs from HLF and SBT are attached
hlf and sbt logfile.zip (54.0 KB)

@cocreature @sormeter @kctam

The Hyperledger Fabric logs identify that a the docker build failed. Not sure if this is of any importance.

Do you have the log lines that identify which docker build failed? I’d actually be very surprised if anything worked at all after a docker build failure.

I found the first docker error on line 2784 in the hlf.log attached

From what I can see is that the Hyperledger system came up and is generating new output messages regularly, but is stuck on “height”:1

here are some docker outputs

docker ps --format=“table {{.ID}}\t{{.Names}}\t{{.Status}}”

CONTAINER ID   NAMES                                   STATUS
6508fe2e083b   damlonfabric_peer0.org2.example.com_1   Up About an hour
299f65725516   damlonfabric_peer0.org1.example.com_1   Up About an hour
58f68ed29a03   damlonfabric_configtxlator_1            Up About an hour
35fbb7791b04   damlonfabric_orderer.example.com_1      Up About an hour

docker exec damlonfabric_peer0.org1.example.com_1 peer channel list

2021-04-20 19:18:44.761 UTC [main] InitCmd -> WARN 001 CORE_LOGGING_LEVEL is no longer supported, please use the FABRIC_LOGGING_SPEC environment variable
2021-04-20 19:18:44.765 UTC [main] SetOrdererEnv -> WARN 002 CORE_LOGGING_LEVEL is no longer supported, please use the FABRIC_LOGGING_SPEC environment variable
2021-04-20 19:18:44.772 UTC [channelCmd] InitCmdFactory -> INFO 003 Endorser and orderer connections initialized
Channels peers has joined:
mainchannel

Same output is true for damlonfabric_peer0.org2.example.com

docker exec damlonfabric_peer0.org1.example.com_1 peer channel getinfo -c mainchannel

2021-04-20 19:25:43.450 UTC [main] InitCmd -> WARN 001 CORE_LOGGING_LEVEL is no longer supported, please use the FABRIC_LOGGING_SPEC environment variable
2021-04-20 19:25:43.452 UTC [main] SetOrdererEnv -> WARN 002 CORE_LOGGING_LEVEL is no longer supported, please use the FABRIC_LOGGING_SPEC environment variable
2021-04-20 19:25:43.456 UTC [channelCmd] InitCmdFactory -> INFO 003 Endorser and orderer connections initialized
Blockchain info: {"height":1,"currentBlockHash":"AsgkHN9kYeU65sJeUpj4FsvkHv3XSFenWPF44H7wkIM="}

Same output is true for damlonfabric_peer0.org2.example.com

I’m not seeing any attachment. Could you perhaps just copy lines 2780-2800?

added the zip again.

hlf and sbt logfile.zip (54.0 KB) Look at hlf.log

I’m not going to be able to help much with this, but these seem like the relevant log lines:

[error] com.daml.FabricContextException: response {
[error]   status: 500
[error]   message: "make sure the chaincode daml_on_fabric has been successfully defined on channel mainchannel and try again: chaincode daml_on_fabric not found"
[error] }

and

peer0.org1.example.com_1  | 2021-04-20 21:13:16.178 UTC [chaincode.platform] func1 -> ERRO 02e docker build failed: Error returned from build: 1 "go: github.com/golang/protobuf@v1.4.2: Get "https://proxy.golang.org/github.com/golang/protobuf/@v/v1.4.2.mod": EOF
peer0.org1.example.com_1  | "
peer0.org1.example.com_1  | 2021-04-20 21:13:16.178 UTC [dockercontroller] buildImage -> ERRO 02f Error building image: docker build failed: Error returned from build: 1 "go: github.com/golang/protobuf@v1.4.2: Get "https://proxy.golang.org/github.com/golang/protobuf/@v/v1.4.2.mod": EOF
peer0.org1.example.com_1  | "
peer0.org1.example.com_1  | 2021-04-20 21:13:16.179 UTC [dockercontroller] buildImage -> ERRO 030 Build Output:
peer0.org1.example.com_1  | ********************
peer0.org1.example.com_1  |
peer0.org1.example.com_1  | ********************

It seems plausible the SBT error is caused by the Docker build failure (i.e. because the container has not been built it did not get a chance to prepare the state SBT is trying to rely on) and the Docker build failure looks like it may be a transient network issue, so it may be worth just retrying the whole thing.

@Gary_Verhaegen I can recreate this every time in this environment. This error is not happening when I build using a clean and vanilla AWS or Azure Ubuntu environment.

Now since this test environment is sitting behind some restrictive corporate proxies (that may slow down some calls) I could see that something could not get retrieved or built on time.

Are there maybe any of the config-local.yaml file settings that could influence the Wait times ?

queryWaitTime: 30000
invokeWaitTime: 60000

I will increase theses timings by 100X and see what happens.