What’s the recommended way to handle large queries, in the order of 10-100MM for e.g. reporting purposes? Would you query the ledger itself using the json or grpc api? would you store the txn data you know you want to query elsewhere (e.g. postgres) and later query that?
My recommended approach to enable analytics and reporting is to read the required data from the Ledger API into an appropriate query store (e.g. a relational database) and build your application around the query store. The decoupling allows you to evolve the data ingestion and analytics independently and the query store will be best suited to address your requirements (whereas the Ledger API doesn’t support efficient queries beyond the parties and template IDs dimensions).
2 Likes