How to start Extractor from latest offset?

Questions
1

I use following command

daml extractor postgresql --user myuser --password mypass --connecturl jdbc:postgresql:test --party Alice -h localhost -p 6865 --from head --to follow

but getting error

23:56:09.157 INFO  c.d.e.Main$@[main] - Starting DAML Extractor...
23:56:10.467 INFO  a.e.s.Slf4jLogger@[kka.actor.default-dispatcher-4] - Slf4jLogger started
23:56:12.044 INFO  c.d.e.Extractor@[kka.actor.default-dispatcher-5] - Connected to ledger 6154cdfd-0e32-4ec0-a7dd-0f56a370b1d1


    23:56:12.062 INFO  c.d.e.w.PostgreSQLWriter@[kka.actor.default-dispatcher-5] - PostgreSQLWriter initializing...
    23:56:15.810 ERROR c.d.e.Extractor@[kka.actor.default-dispatcher-5] - FAILURE:
    {}.
    Exiting...
    com.daml.extractor.Types$DataIntegrityError: Current startup parameters are incompatible with the ones used when Extractor was first started:
    The current snapshot start parameter `LedgerOffset(Absolute(head))` is not equal to the one that was used when extraction started: `LedgerOffset(Boundary(LEDGER_BEGIN))
3 Likes
2

I had a quick look at the code, it seems that the head value is only supported for the --to option, while --from requires an absolute offset value.

This can probably be a valid feature request, would you mind sharing why you want Extractor to start from the latest offset? I would like to establish a use case so that we could have the elements to evaluate it as a possible addition. :slightly_smiling_face:

1 Like
3

If somehow my machine goes down and I don’t want to start from the beginning.

2 Likes
4

I believe that if you restart the extractor against an existing database (with the original parameters), it resumes streaming where it left off. So your use-case is covered by just restarting the Extractor with the original parameters.

4 Likes