Local database support for testing in a Local environment

Questions
, ,
1

Instead of using the sandbox database , is it possible to store the contracts generated in sandbox environment in a local database and connect this to a local UI environment

1 Like
2

The short answer is that you should not rely on anything else but the Ledger API (or something that uses the Ledger API, like the HTTP-JSON API) to access data on the ledger. The persistence model of the sandbox database is for internal use only and should not be depended on. Part of the storage is actually devoted to have the data laid out so that it can be accessed efficiently via the Ledger API, so by using it you gain in stability without sacrificing speed.

Would you mind giving me a better understanding of what you are trying to achieve? Perhaps there’s some other way to do it. :smiley:

1 Like
3

To add a bit to @stefanobaghino-da’s answer, there are two considerations against accessing the underlying database directly:

  1. The underlying schema may change. In future versions of the sandbox we will ensure that there is always a migration path/tool. Any UI connecting directly to the DB may break in future upgrades

  2. If and when you decide to port your application to a different DAML ledger you will not be able to rely on the underlying DB as the schemas differ greatly between different DAML ledgers and in many cases are stored as opaque blobs

As Stefano mentioned, to ensure application portability we recommend only using the Ledger API or a client of this API such as the HTTP-JSON API.

Multiple DAML users stream the grpc API into external data stores which are indexed and optimized for specific lookup patterns. As an example you can take a look at https://docs.daml.com/tools/extractor.html which streams ledger data into a Postgres database. Note that the extractor is in Early Access and should not be considered stable and supported at the moment.

1 Like
4

Got it, Thanks

so based on this recommendation, best approach would be to connect my local UI to the Dabble Ledger by

updating the "proxy: in the package.json and leverage the available API connecting to the Ledger

I assume then also that both will need to be on the same SDK level support to make the connections work.

5

or you can run your own instance of json-api.

$ daml json-api --help

Here is the documentation:
https://docs.daml.com/json-api/index.html

@bartcant the topic of your post is: “Local database support for testing in a Local environment”…

Please describe your use case. If you are talking about an automated testing, we have examples of how to start ledger and json-api from an integration test (for some ideas look into https://github.com/digital-asset/daml/blob/4c0ffdf4df2c1eead6490cf70a3d7756c2ced9f4/ledger-service/http-json/src/test/scala/com/digitalasset/http/AbstractHttpServiceIntegrationTest.scala).

6

In my first answer I understood you wanted to access the sandbox internal database directly. Reading your last message I believe I may have misunderstood your intention.

I’m not sure I have a clear picture of what you are trying to achieve, but perhaps are you trying to understand if you can somehow have a local development environment and then deploy it on DABL?

If that’s the case, you can use the sandbox (either with or without a persistent database backing it) and the HTTP-JSON API, which is the same exposed by DABL. You can test as long as you want locally and deploy at any time. daml start is a convenient command to spin up a sandbox and the HTTP-JSON API server locally (daml start --help for more info). When you are ready to deploy, you can do it on DABL, since the HTTP-JSON API is the same.

1 Like
7

Based on the discussions above I think I will follow the following approach:

  1. I will look into creating a persistent database store for Sandbox Ledger and connect my local UI to it
  2. I will look into leveraging some daml.scripts to seed the Sandbox Ledger to help further UI Testing purposes.
  3. Once all errors are cleared then I will deploy to Dabble with my local UI
    This gives me then the ability to create more contracts through the Live Data feauture and test further my UI
  4. Finally, when all is is complete I deploy both DAML and UI to project Dabble
2 Likes
8

Sounds very reasonable. :smiley: