Is there any exposed API for fetching archived contracts? If not, how can I fetch archived contracts in java application?
4 Likes
The short answer is no, you can’t fetch an inactive (archived) contract in Daml.
There are a number of reasons for this, the most pragmatic is that it would require storing and indexing every contract ever created since genesis—and that would be prohibitive. From a theoretical perspective, this would represent reasoning from false, which is not valid.
However these reasons only apply to Daml code, and impose no such restrictions on any applications built on top of the ledger-api. Any Java application will need to process the transaction service—even if it bootstraps from the active contract service. Generally you will extract and store some subset of the active-contract set; however, you are under no obligation to remove an entry from your application db just because it was archived.
In our Daml application, most of our ledger-api clients do in fact delete contract data once the contract is archived; however, not all of them do. Some of them (mostly reporting and diagnostic tools) mark contract data as inactive rather than deleting it, we do that precisely because we want these clients to be able to access an analyse inactive contracts.
3 Likes
However these reasons only apply to Daml code, and impose no such restrictions on any applications built on top of the ledger-api.
For instance if you use the daml navigator, there is a checkbox that allows you to display archived contracts.
3 Likes
Is there an answer to @Luciano’s question? I have a similar issue when knowing the Contract Identifier
a) querying the active contract by key
b) querying all instances archived and active of a contract by key
This is missing and need be exposed as an API for applications to analyse the data in the ledger
thanks in advance-
1 Like
Hey @Jean_Safar , I think there’s some confusion here. @Andrae is saying that yes, there is a way to get this from the API:
Any Java application will need to process the transaction service—even if it bootstraps from the active contract service. Generally you will extract and store some subset of the active-contract set; however, you are under no obligation to remove an entry from your application db just because it was archived.
I was just pointing out that the Navigator is an example of such an app - it allows you to filter archived contracts.
The only thing you can’t do, is call fetch on an archived contract from inside a choice, i.e. on-ledger.
1 Like
Thanks @Luciano ! However, I can’t find in the Java API a way to query those exact contracts by key active and/or archives.
If a system has a million contracts archived and active, I cannot be expected to loop through them all each time…
Is there such an API?
1 Like
That’s correct - replaying the entire transaction log in this case is the only way to get the archived contracts. My colleagues will step in to correct me here, but I believe the reason for this is to enable Ledger pruning, and hence this is a design choice which won’t be changed any time soon.
Allowing linear lookup through the API would require indexing those stale contracts, and this would mean the ledger would grow indefinitely.
1 Like
Thanks @Luciano for this !
This should really be considered closely as it is a major hurdle for any application using daml that intends to track the data …
1 Like
@Luciano … just one question though on the indexing side … isn’t Daml indexing the contract using the keys … and thus we should be able to access them easily, archived or not, without a full scan?
1 Like
Hi @Jean_Safar , we are actually looking into this topic right now. I think of this under the umbrella of “provenance”. As @Luciano and @Andrae said, you can get your hands on historic contracts by streaming the transaction service into your own data store and querying that. But you rightly point out that that is not user friendly. It adds development work to use-cases that want to expose ledger history or “provenance”.
Now the gRPC Ledger API does not have any real query capabilities, and deliberately so - maintaining indices is a heavy cost that affects ledger performance. But we do want to add just enough to make it possible to explore ledger history without maintaining an external data store.
The gap towards that is being able to follow “input contract” edges. Eg if you see an Exercise node, how can you find out where the contract originally came from? To close this, we are looking at a feature that allows you to find Create nodes from Contract IDs. That’s essentially querying by Contract Id.
This is probably not the full feature you had in mind. From your messages, I read that you want to be able to query ledger history like you can query ledger state on the JSON API. The major challenge with that is that ledger history grows linearly over time so with large applications one quickly runs into space constraints. This makes it a hard feature to get right in general. We attempted it once before with the Extractor, which gives you a queryable database including archived contracts, but it’s too crude. We will revisit that topic some time in the future. I run that under the umbrella term “Custom Views” as it’s really about letting devs maintain the specific queryable datasets that they need for their applications.
5 Likes
We do have indices, but at least until now, we “prune” them immediately when a contract gets archived. Ie the indices only cover active contracts. It’s possibly that that may change in the context of other work, in which case the indices may be available back to the last ledger pruning. I’ll bring this topic up in the relevant design discussion and see whether simple “FetchById” and “FetchByKey” endpoints would be cheap (in performance terms) to add.
2 Likes
Thanks a lot for all this @bernhard .
1 Like
Was there any advances related to this? I am currently building an application which would require to query archived contracts as well :).
Thank you for your attention and help 
Hi @LimitBreaker98 , there’s ongoing design work on on several fronts towards features that should make it easier to work with archived contracts. It’ll still be a while until they are available and stable, though.
Depending on the stage of your use-case, the Extractor linked above might be an option to get you off the ground. We do not recommend using that in a production setting, but if you get to that stage, building your own custom version of the Extractor is quite straightforward.
3 Likes
Thank you very much!
hey @bernhard is there an updated answer here? as the link to the Extractor is now broken and I’m guessing a lot has changed since Nov `21
We are planning to make it possible to fetch archived contracts by key or ID in an upcoming version. The obvious caveat to that is that it’ll only work for contracts that have not been pruned.
1 Like
@arya You can also use the new Custom Views labs feature to build a queryable store including archived contracts: Custom Views — Daml SDK 2.5.0 documentation
All feedback on that tool is welcome as we mature it towards production.
Curious if the 2.7.0 PQS feature is meant to address this?
Yes it is! PQS also exposes archived contracts. If you try it out let us know how it goes.