Now I’ve noticed some rather strange behaviour. There is a common file, src/main/daml/Daml/Finance/Types.daml that has some common type definitions. It sits outside of the Interface or Implementation directories. Yet it is pulled into the builds (I can see this with damlc inspect).
I would like to better understand, the behaviour of source: in a daml.yaml file when using relative paths. How does it determine dependencies when building the package? In this case it seems to have pulled in something from outside the root of the interface package (one level below the source :)? Can the build-options : --include= somehow help here, and if so, what’s the difference?
I should add here, that in a previous approach, instead of using the source : with a relative path, I created symlinks from within src/interface and src/implementation pointing inside src/main/Daml/Finance/{Interface, Implementation}. And I guess my surprise comes from the fact that in this setup, the packages wouldn’t build, complaining of a missing Types.daml.
I suppose what I’m trying to say is : I would expect a symlink with a relative path to work the same way as a source : with a relative path in daml.yaml, but that’s not the case here.
As far as I can tell, the source field in a daml.yaml file defines either
a .daml file to use as an entry point for compilation, or
a directory, which is equivalent to having every .daml file in said directory (including nested files) simultaneously as entry points.
Then, during compilation, any imported modules are resolved either from dependencies or by looking at the file system - I’m not 100% sure on the details for this part, but the $DAML_PROJECT environment variable seems important - the important part is that damlc will look at the file system to find imported modules, even if the file it finds is outside the file(s) defined as the source.
For comparison, let’s consider a project without .. paths in its daml.yaml file,
# cat daml.yaml
name: abc
version: 1.0.0
source: Alfa/Bravo/Charlie.daml # this might as well be Alfa/Bravo/
dependencies:
- daml-prim
- daml-stdlib
-- cat Alfa/Bravo/Charlie.daml
module Alfa.Bravo.Charlie where
import Alfa.Delta.Echo ()
This compiles perfectly fine, even though the import of Alfa.Delta.Echo is outside of sources. This is because sources doesn’t (currently) define the root of the project, but rather an entry point for it (or multiple entry points in the case of a directory source).
This is still surprising behavior, don’t get me wrong, but it arises from the implicit assumption in the compiler that the source field in daml.yaml will always be a file or directory under or equal to the directory with the daml.yaml file, which breaks when a .. parent directory is used.
I think the language team should decide whether to make that assumption explicit, or to adjust the behavior to be less surprising in the edge case.
Maybe a bit of history helps understand the current behavior:
There are two concepts here which are orthogonal in the current implementation but from a user pov probably shouldn’t be.
The first is the list of roots modules that are included in the DAR produced by daml build. Initially, you could only have a single file. We later extended that to support a directory which means all files in that directory are considered root modules. This list of root modules is only used by daml build. Daml studio does not care about this at all.
The second concept is that of import paths: When we see an import, we need to locate that file somehow. For that, we search for that file in all import paths and take the first match. If it is not found, we search in dependencies and if it doesn’t exist, we fail. So what are the import paths? It is the list of paths specified via -I but there is one special case: For module A.B.C located in file path/to/A/B/C.daml we add path/to to the front of the import path.
So the first important take away from that is that the source directory has no influence on import paths and thereby where modules are found.
The second important take away is that trying to point source somewhere inside your module tree is usually a bad idea since you’ll add path/to into the import path and might include more than you expect.
Instead, if you want to be on the safe side, separate them at the top-level directories so something like
And then point source at those directories. That way path/to will never pull in other files. If you want to pull them in explicitly you can do so via -I just remember that they are only pulled in if they are referenced rather than considered roots in daml build.
I’m confused as to whether you are referring to A.B.C in the build-options : --include=path/to/A/B/C.daml, or just an import of A.B.C inside a daml file.
For instance, going back to our example, if I import Daml.Finance.Types in file src/main/daml/Daml/Finance/Interface/Transferable.daml, then is that causing src/main/daml (your equivalent of path/to) to be added to the import path? Or have I misunderstood?
Let’s say we’re trying to process module A.B.C from path/to/A/B/C.daml. This could be either because the file is a root itself (meaning it’s part of source) or because it was transitively imported from a root. Doesn’t really matter why we’re processing it.
Now that file has a bunch of imports, let’s say we have an import of module D.E.F and (assuming we haven’t already processed that file) we need to locate it. For that, we first search for path/to/D/E/F.daml and then to the same for all other include paths you specified until we find it. So in your example we’ll search for Daml.Finance.Types in src/main/daml. Note that this is local meaning it only affects the import resolution for a given file.