What is the JSON API token for and how do I handle rotating it?

novusopt · July 7, 2020, 6:21am

Hi,

I have another question regarding the JSON API.
My understanding is that I have to provide via --access-token-file a file where token is store, that is used by the JSON API in order authenticate itself against the Ledger API. Now the question is, what should I do in case the Token Issuer issues token with max expiration of 1h. This would be quite cumbersome in terms of operating the JSON API. I need to fetch every 1h a new token update the file with the new token & restart the JSON API.

Could you please elaborate why the JSON API needs a token?
The documentation says “Note that this token is used exclusively for maintaining the internal list of known packages and templates, and that it will not be use to authenticate client calls to the HTTP JSON API: the user is expected to provide a valid authentication token with each call.” (https://docs.daml.com/json-api/index.html#with-authentication)
TBH I don’t understand that…

Thanks for any help!

stefanobaghino-da · July 7, 2020, 6:53am

From an authorization perspective, the JSON API uses the Ledger API to:

issue commands and read transactions and active contracts on the users behalf – for this the agent initiating the request must provide a token to authorize the request;
maintain internal state regarding the DAML packages available on the ledger – this is where the token passed at startup comes into play

Note that for the latter point, the endpoint used requires only a token with public claims, so the JSON API can be started with very little privileges to operate (you can see a rundown of the required claims for each service here – the JSON API book keeping only uses the ledger identity and package services, which both require only public claims, no need to be able to issue commands and read transactions as a party).

The JSON API should be able to automatically pick up a new token once you rotate it (i.e. no need to restart), so that should lower the maintenance burden. All you need is a process that periodically refreshes the token.

novusopt · July 7, 2020, 9:28am

Hi @stefanobaghino-da,

thx for your reply!

could you please elaborate regarding:
“maintain internal state regarding the DAML packages available on the ledger – this is where the token passed at startup comes into play”. For me it is still not clear what does this mean…

In terms of the structure of the token for the JSON API, could you please share which claims/information the token needs to have. I guess public: true & what else?

Just for my understanding, the JSON API does not need to get restarted manually when the token has been updated. That means the JSON API reads the file every time the JSON API submits a request to the Ledger API, is my understanding correct?

In order to update the token file, there has to be a process in place (implemented by the customer) for example a cronjob which updates the token automatically (getting a new token from the Token Issuer & store it in the token file), by implication this means the JSON API does not take care of refreshing the token, right?

Thanks a lot!

stefanobaghino-da · July 7, 2020, 9:41am

Packages

DAML code is compiled to packages that are uploaded to the ledger.

The Ledger API exposes an endpoint that allows consumers to download the compiled packages available on the ledger.

The compiled DAML code can be used to track information about templates.

The JSON API connects independently (as in, not on a user’s behalf) to the Ledger API to download the packages and use the information on templates for its own operation.

I hope this makes sense.

Token for public endpoints

Authorization is a ledger-specific concern. If you are using the sandbox, any valid JWT gives access to the public endpoints. You can read more here on the documentation about the token payload for sandbox JWTs. A token with admin: false and empty actAs and readAs claims will do while limiting the error surface.

Updating the token file

Yes.

EDIT: I believe a small clarification is due here. As I mentioned before, the token you provide when starting up is not used as part of any request performed on behalf of a user. The JSON API periodically queries the Ledger API for new packages so to maintain an up-to-date list of available templates. Every time it performs this operations, it does try to read the token. This ensure you don’t have to restart the JSON API to keep the token fresh. This does not mean however that the JSON API reads this file every time a request is sent to the Ledger API. User requests to issue commands and read transaction and active contract must come with the necessary authorization token. The token you pass at startup is not used for these.

Correct.

novusopt · July 8, 2020, 12:51pm

Hi,

thanks for the explanation. Just out of curiosity, since the JSON API does not need the JSON API token in order to route user requests to the Ledger API, is it possible to have the JSON API running without a dedicated token.
That means the JSON API would just forward user requests?
Is it required to have the JSON API fetching new packages?

Thanks a lot!

Leonid_Shlyapnikov · July 8, 2020, 2:15pm

JSON API does not need anything from a JWT token when serving /v1/packages endpoint. However it does check for the JWT in the header before redirecting the request to sandbox and if JWT token is missing, JSON API returns 401 status back to the client.

If you are running sandbox without authentication,

you don’t have to specify --access-token-file when starting JSON API
however your requests are still required to have a JWT token in the header, you can use any of the test JWT tokens provided on this page as a dummy token:
HTTP JSON API Service — Daml SDK 2.7.6 documentation

e.g. use this one:

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJodHRwczovL2RhbWwuY29tL2xlZGdlci1hcGkiOnsibGVkZ2VySWQiOiJNeUxlZGdlciIsImFwcGxpY2F0aW9uSWQiOiJmb29iYXIiLCJhY3RBcyI6WyJBbGljZSJdfX0.VdDI96mw5hrfM5ZNxLyetSVwcD7XtLT4dIdHIOa9lcU

or generate your own using:

You should NOT run ledger server in production without authentication enabled.

Regards!

Leonid_Shlyapnikov · July 8, 2020, 2:43pm

Two clarify a bit more.

There are two types of tokens:

service account token for ledger metadata access, required when ledger server is running with authentication. This token is used by JSON API internally to fetch ledger metadata needed for command processing/enrichment. That is the token you specify as --access-token-file when you start JSON API. This token is NOT required if ledger server is running with disabled authentication. Ledger operator is expected to created this token and start JSON API with that token specified as --access-token-file. Clients are not supposed to have access to this token, it is for JSON API service account only.
client account token, created for a specific ledger Party. This token used for command and request submission and you have to provide it in the header of the HTTP request, as described here: https://docs.daml.com/json-api/index.html#passing-token-with-http. This token is always required. This token is created by a client and not supposed to be shared with anyone else.

novusopt · July 8, 2020, 3:31pm

Hi @Leonid_Shlyapnikov thanks for your reply.

Talking about a production deployment (with postgres), then I think you start with authentication, right?
If yes, consequently you need to have a service account token in place for the JSON API…
I think you start the ledger without authentication only for “testing”…

Could you please elaborate on what will be checked if authentication is enabled?

Thanks a lot

Leonid_Shlyapnikov · July 8, 2020, 4:39pm

I would advice to start thinking about authentication and authorization from the very beginning, this would make your life easier when you need to deploy your ledger in UAT or PROD.

Here is the documentation explaining sandbox authentication options: DAML Sandbox — DAML SDK 1.2.0 documentation

Let’s clarify what postgres we are talking about.

You can run sandbox/ledger server with persistence/postgres backend enabled:

$ daml sandbox --sql-backend-jdbcurl ...

This is must have for any UAT/PROD deployment, else your ledger will be lost after the restart.

You can run json-api with optional query store enabled. Query store is a json-api search index, that uses postgres db to optimize queries over large active contract sets (ACS):

$ daml json-api --query-store-jdbc-config=driver=org.postgresql.Driver,url=jdbc:postgresql://localhost:5432/test?&ssl=false,user=postgres,password=password,createSchema=false" ...

If you are not planing to have a very large number of active contracts, you should be fine running json-api without --query-store-jdbc-config (query store is disabled). Where large number of active contracts depends on the JVM heap size allocated for your json-api. With default JVM heap settings you should not have any issues querying ACS as large as 1k - 5k active contracts.

My advice:

For local development, run sandbox without --sql-backend-jdbcurl, don’t specify any --auth-jwt-XXX options (authentication disabled) and json-api without --query-store-jdbc-config (query store is disabled).
json-api would force you to start thinking about authentication right away, it requires JWT token provided in the request header, so your client code will have to provide JWT in every HTTP request going to json-api.
For Integration env, run sandbox with --sql-backend-jdbcurl and with one of --auth-jwt-XXX options (that would enable authentication), run json-api wihtOUT --query-store-jdbc-config.
For UAT and Production, always run sandbox with --sql-backend-jdbcurl and with --auth-jwt-XXX (with enabled authentication); depending on the expected size of the ACS either enable --query-store-jdbc-config in json-api or increse JVM heap.

Hope this helps.

novusopt · July 10, 2020, 2:39pm

Hi @Leonid_Shlyapnikov,

thx a lot for your explanation. That makes completely sense for me.
Regarding the Token structure (ledgerID, applicationID, participantID, canActAs…) wich claims are required which not?

Thanks a lot!

Leonid_Shlyapnikov · July 10, 2020, 3:04pm

JSON API requires the following 3 fields in the token (HTTP JSON API Service — Daml SDK 2.7.6 documentation):

{
  "https://daml.com/ledger-api": {
    "ledgerId": "MyLedger",
    "applicationId": "foobar",
    "actAs": ["Alice"]
  }
}

applicationId – your application identificator, does not really matter what you put here
ledgerId – must match the Ledger ID that sandbox was started with: daml sandbox --ledgerid MyLedger
actAs – an array with a single element that contains your party ID

Keep in mind that sandbox’s JWT requirements are less restricting and documented here:
https://docs.daml.com/tools/sandbox.html#token-payload

novusopt · July 14, 2020, 7:34am

Hi @Leonid_Shlyapnikov

thanks a lot for you reply.
Does the JSON API token have the same structure or are there some other claims to consider…

Thanks a lot!

Leonid_Shlyapnikov · July 15, 2020, 1:36pm

Hi @novusopt!

JSON API JWT has the same structure as Sandbox JWT.
JSON API JWT is a subset of Sandbox JWT.
JSON API requires the above 3 fields in the payload, any additional fields will be passed to Sandbox, actually the entire JSON API JWT without any changes will be passed to Sandbox as is.

Topic		Replies	Views
Why is JSON API --access-token-file no longer needed Questions json-api	2	316	September 30, 2021
Use Auth Middleware with Sandbox Questions	2	164	May 16, 2022
How do I produce tokens for an authorized ledger? Questions json-api , ledger-api , daml-on-fabric	7	1020	August 13, 2020
Claims in JWT for multi-party submission using JSON API Questions daml , json-api	7	495	April 14, 2021
Allocation Of Parties With JWT Tokens on Ledger Questions daml , jwt	1	310	May 31, 2022

What is the JSON API token for and how do I handle rotating it?

Packages

Token for public endpoints

Updating the token file

Related topics