Can we use JSON API to fetch a large amount of contracts (e.g: 1M, 10M,..?)?

Questions
1

Hi team,

Can we use JSON API to fetch a large amount of contracts (e.g: 1M, 10M,…?) ? Or the performance won’t be as good as using gRPC?

Best Regards,

Jean-Paul

2

It is technically feasible to fetch a large amount of contracts over the HTTP JSON API Service. Depending on the size of contracts, this may lead to problems, but you would need to test your application to understand how its behavior would scale.

Comparing the performance of the HTTP JSON API Service and the Ledger API is like comparing apples and… apple pies, I guess. They are different tools and depending on a variety of factors their performance characteristics can vary broadly as well. In general, it’s fair to say that you can more or less expect the HTTP JSON API Service to likely be slower than by custom tailoring your own application around the gRPC API, simply because of the fact that the HTTP JSON API Service is a layer built on top of it to answer to relatively generic application needs.

The main purpose of the HTTP JSON API Service is to get you quickly started writing your applications in Daml but I would question its use in a large-scale setup with specific performance expectations, as a custom built application that taps directly into the Ledger API and builds and maintains its own state is almost guaranteed to perform better, since the data layout of the query store can be tailored to the application’s needs as well.

There are ways in which you can make it scale but it requires some good design around the application that you want to serve, as well as the model by which you want to make it scale. Sharding could help in certain scenarios and I’m sure there’s more that can be done, but you would need to make an engineering investment in any case when using either the HTTP JSON API Service at scale or a custom-made application built around the Ledger API.

I like to think of the HTTP JSON API Service as the “Ruby On Rails” of the Daml ecosystem. Sometimes it’s a good idea to work on it and make it scale because the design of the application allows for it (StoryGraph is a nice success story about it), other times you have to rethink your approach, maybe even to a fundamental level (with Twitter being the most prominent example).

1 Like
3

Understood, thanks Stefano