I’m trying to use git submodules with daml, but running into some problems.
I have two projects, both with a daml.yamlsource: daml folder. If I try to include one inside the other using git submodule add, I don’t think this will work, because the sdk expects the directory structure to match the module hierarchy. Is there some way to work around this?
I also thought I might be able to provide an array for the source: field, but that doesn’t work either . I’ve read this post:
which says I could use an individual file that imports the others, but then I would have a very weird import daml.x.y statement at the top; and I’m not sure I can include daml.yaml files in subdirs.
Has anyone had a similar experience and want to suggest how to work around this?
Here’s an example. Suppose you’re developing a website and creating Atom feeds. Instead of writing your own Atom-generating code, you decide to use a library. You’re likely to have to either include this code from a shared library like a CPAN install or Ruby gem, or copy the source code into your own project tree. The issue with including the library is that it’s difficult to customize the library in any way and often more difficult to deploy it, because you need to make sure every client has that library available. The issue with copying the code into your own project is that any custom changes you make are difficult to merge when upstream changes become available.
Git addresses this issue using submodules. Submodules allow you to keep a Git repository as a subdirectory of another Git repository. This lets you clone another repository into your project and keep your commits separate.
Looking at the usecase, it seems like you could still use a submodule and modify that library but rather than including it as a source, you have a two-step build process where you first build the (modified) dependency and then depend on the result.
Hmm you mean using something like dependencies : ../otherproject? But would this require an external build system (make) to detect changes to the dependency?
I’ve found a workaround, but it’s horrible - I’ve included the submodule in the root of the project, and then symlink the dirs from inside the (dependent) source: daml directory.
I’m not sure if this will horribly confuse github. We’ll see. Also, this only works as long as your module paths are unique.
Ideally my problem would be solved by allowing multiple source: dirs. I’ve no idea how much of a dev effort this would be though.
Honestly I’ve been using git since 2008-ish and I’ve yet to find any use case where submodules were not making things worse.
The specific alternative will depend on the details of the use-case, but in general you’re better off cloning the subrepo as part of your build process.
Yes that’s what I meant by two-step build process. Of course you can also just build your dependency manually if it changes rarely instead of relying on make.
I don’t understand how submodule makes things worse here. You would still have the same problem if you keep your code in a separate repo. Aren’t you just suggesting, like @cocreature , to use dependency: instead embedding the dependent source code directly into another project?
Create your dependency using git submodule add at the root of your project, i.e. outside the project source directory. Then symlink your dependent module directories from inside your source dir using relative paths. For instance, for a project ‘outer’ depending on ‘inner’:
outer/daml/Main.daml # imports module 'MyLib.MyDep'
outer/daml/MyLib # symlink to '../inner/daml/MyLib'
outer/.gitmodules # specifies that 'inner' is a git submodule
outer/inner/daml/MyLib/MyDep.daml
The limitation of this is that you can’t depend on multiple projects with the same paths.
I think that using a git submodule instead of a dependency: in a yaml file makes it easier to work on the two independent projects concurrently.
For example, if you have a dependency: in your daml.yaml file, then every time you make a change, you need to rebuild that project, possibly publish an artifact, and if you update the the dar version, then update your top-level daml.yaml file. Conversely, if you’re not updating the version, then it makes it difficult for another person working on the same project to correlate the two repos. With the git submodule you have a more granular and explicit way of doing this i.e. by not depending on the versioning scheme of your dar, but rather on an explicit hash that identifies history uniquely.
I’m not sure, but I also think you may run into problems with LSP server / IDEs if you update a dependency in the daml.yaml file, vs just updating code in a submodule. It’s only a suspicion though.
[edit] Another thing I just thought of is that (as far as I know) the SDK doesn’t handle packages for you. So if you specify a dependency: then you also have to figure out how to distribute it. The docs give an example of using a relative path:
And I would emphasize that this is only really useful if you’re actively developing the two repos in parallel. Once a project becomes stable, I think that using submodules becomes inappropriate.
So, just to share some more of our experience, @georg has been trying this in Windows and found that symlinks don’t work too well. I’ve found a SO post that shows that recent versions include symlinks as an installation option, but even then it requires admin rights so it’s not all plain sailing.
We’ve managed OK on unix & derivatives so far though.
. I’ve read this post: