Hi guys, we got a new problem here. The Java application we are building needs to read a large ACS. Is there anyway to process the ACS information and throw it away before the end of the ACS?
The GetActiveContractsResponse() in com.daml.ledger.javaapi.data appears to read the whole ACS and return it. Any one know if there is a different way to handle it? e.g. paging or streaming?
The ACS gRPC stream is split up based on the workflow ID. What happens if there is one workflow ID with a disproportionate number of contract? Is there any way we can split the stream up further? It can be very memory intensive if one workflow ID has a huge number of contracts. To clarify, our strategy is to stream the ACS, for each contract from the stream we call a function on it, producing an integer result which we store in memory (an integer being much smaller than the contract payloads). By consuming the stream message by message, we don’t need to read the entire ACS in memory (assuming the ACS is split up over many messages). We are using the rxjava bindings.
Ignore my comment on batching, I misread the code. it should always be a singleton list in the current implementation. This is not guaranteed though but I think you can reasonably expect that each stream element can fit in memory so it at most goes up to some reasonable batch size.
Let us try it on a later version. We were using 1.18 to test it. It probably behave differently now. @gerolf , would you know in which version the batch size was added?