Hi Folks,
If we built a DAML service to approve or disapprove certain transactions (data access approvals). Would the DAML contract have a limit on the amount of approvals or requests its able to do in a second or is it a matter of hardware, workflow etc? I remember the amount of 35,000 transactions per second once but i am talking more about a few million requests per second. Is DAMl cable of handling this type of load (message payload is minimal, small) ?
There’s no hard limit in Daml that would limit the amount of requests that you can handle in a unit of time, but with this kind of load in the millions of requests per second, special care should be given to design the application and load testing should be thorough and receive special care. It would also be interesting to hear the latency requirements.
To answer that question, we’d need to understand better what exactly you are trying to accomplish, millions of anything a second is a steep target so as @stefanobaghino-da says, it would require careful design. Let me explore some performance considerations here as I think they may be of general interest.
The first issue is contention on a single contract. Let’s say you wanted to implement a contract that counts requests. The contract then needs to be archived and re-created with an updated counter million times. The million requests would have to be synchronous, as you can’t make a request to update the counter from 3 to 4 before you know the contractId of the contract with count 3. To get to 1m requests per second here, the round-trip from client application to participant to network and back would have to be below 1microsecond. Network latency on 10Gbps networks is measured in the 10s of microseconds so you’d need to use extremely fancy networking tech or put everything on the same physical machine to have any chance. Even then, you’d need to squeeze Daml interpretation, commit to the underlying data store, and processing of the committed transaction into that time frame. Fast solid state drives have access times of around 30microseconds so there is no chance you can commit anything to disk in that timeframe. Lastly, the amount of data being shuffled around is pretty vast. The transaction and contract Ids alone are several bytes each, and each byte translates to a megabyte per second if you have 1m requests/s.
The above tells you that we have to parallelise. Ie there can’t be a single contract. But even so, as you suggest in your post, transaction throughput of most data stores, and that includes Daml, tends to be measured int he thousands of transactions per second at best. So you are going to have to parallelise heavily - maybe 100s of parallel processing pipelines. This won’t be practical.
The only way to make this work in practice is batching. Ie if we can reduce the 1m requests to 1000 transactions with 1000 updates each, we have a chance. Going back to the counting example above, writing 1000 contracts each saying “Here’s another batch of 1000” is possible.
However, if your 1000 requests are 1000 messages with 256 bytes of text each, you are committing at least 256 megabytes of data to your ledger each second. This will be tough again without parallelising.
So - if your data can be batched reasonably well, has manageable size, and can be sharded/parallelised, then yes, I think putting 1m requests a second on a Daml ledger is possible. As I said in the beginning, your exact case would need to be looked at carefully to figure out the best approach.
imagine a police station that has to approve every data request in tis jurisdiction. all data is encrypted E2E but if you want to touch it, you need approval from the police station as to the question if you can see the data but also what you can see from the data. these are the request types and we need to design the solution to handle millions of queries a second
Thank you all for your advice
I must say that I’m amazed by your number. Google says that they process 40000 search queries a second. I can’t quite picture what sort of request we are talking about if a police jurisdiction gets 25 times that volume.
Glancing at the high-level use case of managing data access to police data across a network of jurisdictions and other organisations, it sounds like a match made in heaven for Daml. It involves clear boundaries, between jurisdictions, sensitive data, privacy, provenance – all the topics Daml makes so much easier. And without your original question, performance wouldn’t have come to my mind as the primary concern.
That was a fascinating and informative answer, thank you 