DAML image for multi platforms

Well we can’t get access to that haskell code right? So from my point of view I figured next step is to test anyway we can out here without being blocked.

Normal x86/arm64 linux containers run just fine inside of docker for mac’s linux vm. I’m pretty sure that rosetta still kicks in at the machine level to dynamically translate any native instructions that the VM layer might try to pass through.

I do believe you’re correct that Haskell has great cross compilation support and / or ability to build for arm.

That said though I have to emphasize that normal linux containers run just fine on osx so there is no reason we can’t make it work.

It would be great if I could somehow trap the instruction that causes an issue or inspect a core dump file perhaps?

It is open-source. All of it is in the daml repo.

As for the rest of your message I really don’t know what to tell you at this point. If you have an arm64 Linux machine available, perhaps you can create your own Docker image? The Dockerfile is here, though of course that one won’t help you directly, so let me connect the dots a little bit. You can easily take a look at the https://get.daml.com script by just curl’ing it, and you’ll see it ultimately “just” downloads the tarball and runs the installer.

Of course, if you use one of our published tarballs you’ll get the same Docker image we build and thus the same issue. So, instead, what you’d want to do at this point is build your own tarball, on an arm64 Linux machine, hoping that our build supports it (we’re not trying to prevent it but I’ve never tried so I can’t promise anything).

To build the tarball, you have to:

  • Clone the daml repo
  • Make sure you have direnv and nix installed
  • Run `bazel build //release:sdk-release-tarball-ce.tar.gz"
  • Wait. A long time, probably, since no binaries for arm64 Linux will be in the cache. I’d say budget about 3 hours for this to run through, though we never run without a cache so that’s just a guess. (Also this may not run all the tests so may be faster, if you’re lucky.)

Once that finishes successfully, you should have the tarball (among many other things) under bazel-bin/release/sdk-release-tarball.tar.gz. You can then use that in your Dockerfile: COPY the tarball, extract it, and run the install.sh script within.

Right, we tried exactly that earlier in the thread and it didn’t work out as expected. I agree though that it would be nice to have an ARM native build.

However I don’t think that has to be a hard requirement. We all use containers built for amd64 on m1/m2 macbooks it works. So I think we could also just try to debug why it segfaults.

With the latest version of Docker Desktop, it’s now possible to run x86_64/amd64 emulation with Rosetta on an Apple M1 chip which solves this issue when running daml commands in Docker on arm64 chipsets. Having tested this already, I can confirm that daml commands successfully execute using this emulation on an Apple M1 chip.

To enable the Rosetta emulation, follow the simple steps here. As per the instructions, run your image with the flag --platform linux/amd64 and the daml commands will execute successfully.

1 Like

Hey, thanks!

I did notice the release notes as well but when I gave it a shot it didn’t seem to help.

It appears I don’t have the Rosetta setting under features in development screen. However I believe that’s now enabled by default? I’m on v4.16.2

$ docker version
Client:
 Cloud integration: v1.0.29
 Version:           20.10.22
 API version:       1.41
 Go version:        go1.18.9
 Git commit:        3a2c30b
 Built:             Thu Dec 15 22:28:41 2022
 OS/Arch:           darwin/arm64
 Context:           default
 Experimental:      true

Server: Docker Desktop 4.16.2 (95914)
 Engine:
  Version:          20.10.22
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.9
  Git commit:       42c8b31
  Built:            Thu Dec 15 22:25:43 2022
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.14
  GitCommit:        9ba4b250366a5ddde94bb7c9d1def331423aa323
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

I think the ps output is showing that I’m using qemu instead of rosetta?

$ docker run --platform linux/amd64 -it --rm digitalasset/daml-sdk:2.4.0 sh -c 'ps -ef; daml test'
UID        PID  PPID  C STIME TTY          TIME CMD
daml         1     0  2 17:40 pts/0    00:00:00 /usr/bin/qemu-x86_64 /usr/bin/sh sh -c ps -ef; daml test
daml         8     1  0 16:03 ?        00:00:00 ps -ef
Segmentation fault

I tried with 2.4.0, 2.5.3, and 2.6.0-snapshot.

Could it be that it was discontinued and not merged in?

Beta features can be discontinued without notice. Learn more

Thanks for updating us!

I tested it and it works. You need to enable Apple Virtualization in General Settings and then Rosetta under Beta settings. This is on M1 with MacOS 13.2.1 and 4.16.2. I assume you also need Rosetta installed at the MacOS level as well as it is not installed by default.

I have Rosetta installed and Virtualization enabled.

However the option is not available under the Features/Beta section.

I’m on Docker Desktop 4.16.2 (95914) and reinstalled rosetta / restarted docker just to make sure.

Perhaps the exact tag (95914) matters?

Odd. I’m on Version 4.16.2 (95914).


I’m on Monterey 12.6.1 if that perhaps matters?

I ran this command and I get the following output:

docker run --platform linux/amd64 -it --rm digitalasset/daml-sdk:2.4.0 sh -c 'ps -ef; daml test'                                                                                  
UID        PID  PPID  C STIME TTY          TIME CMD
daml         1     0  0 20:43 pts/0    00:00:00 /rosetta/rosetta /usr/bin/sh -c ps -ef; daml test
daml         7     1  0 20:43 pts/0    00:00:00 /usr/bin/ps -ef
daml test: Not in project.

As per @nycnewman, I have the the version 4.16.2 (95914) and I had to enable both the Apple Virtualisation and Rosetta emulation settings manually.

Checking the release notes of Docker Desktop, this beta feature was released as part of version 4.16.0 and it states :

New Beta feature for MacOS 13, Rosetta for Linux, has been added for faster emulation of Intel-based images on Apple Silicon.

As you have alluded to already, you’ll need to update your OS to enable this beta feature in Docker Desktop.

I can confirm it now seems to work after upgrading to Ventura!

I’ll have to run through some more testing but this looks like it’s solved.

1 Like