Daml-ui-template

Hi,

Using the UI-Template as an example, I build the React static files and can serve them with Nginx. I think I now need to have a reverse-proxy to the sandbox (with another nginx?), but I cannot find an example to follow.

Every papers I read talk about the development server with proxy: http//localhost:7575… and setupProxy.js…but how to do proxying for the production build? I follow some nginx reverse proxy examples on the web but none works.

Can someone share me a nginx.conf for this example? Is it… location /v1 {proxy_pass http://localhost:7575} or location /v1/stream { …} and follows by something about websocket??

I have no programming experience. Am I in the right direction or what should I do for the production build?

Thank you.

1 Like

Hey @EllisAllison,

I’d caveat my answer with none of our examples are intended to be used in production.

For the reverse proxy itself you would could do as the Nginx guide says:

location / {
    proxy_pass http://localhost:7575;
}

But a lot of this depends also on where/how you’re hosting your UI as well. Are both the UI and JSON API being hosted from the same machine?

Hi @EllisAllison,

We have an internal application called DAVL that is setup in that way, so hopefully its nginx configuration can serve as a reference point. These lines in particular seem to be what you’re looking for:

    location /v1/stream {
      proxy_pass http://${LEDGER_IP_PORT};
      proxy_http_version 1.1;
      proxy_set_header Upgrade \$http_upgrade;
      proxy_set_header Connection "Upgrade";
    }
    location /v1 {
      proxy_pass http://${LEDGER_IP_PORT};
    }

This is a Bash script that generates the actual nginx.conf file, so we expect substitutions for the LEDGER_IP_PORT env var. In a local environment (“production”-like), the resulting file looks like:

worker_processes auto;
pid /run/nginx.pid;

events {
  worker_connections 768;
}

http {
  sendfile on;
  tcp_nopush on;
  tcp_nodelay on;
  keepalive_timeout 65;
  types_hash_max_size 2048;
  include /etc/nginx/mime.types;
  default_type application/octet-stream;

  ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
  ssl_prefer_server_ciphers on;

  access_log /var/log/nginx/access.log;
  error_log /var/log/nginx/error.log;

  gzip on;

# <workaround>
# The UI currently does not support signing up, so we add a running Navigator
# to our setup. It will be served on 8080, so we also need to expose that port.
  server {
    listen 8080;
    server_name navigator;
    location / {
      proxy_pass http://navigator:4000;
    }
  }
# </workaround>

  # This serves https://davl.da-ext.net, with the load balancer doing TLS
  # termination.
  server {
    listen 80;
    server_name davl.da-ext.net;

    location /v1/stream {
      proxy_pass http://json-api:3000;
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "Upgrade";
    }

    location /v1 {
      proxy_pass http://json-api:3000;
    }

    root /app/ui;
    index index.html;
    location / {
      try_files $uri $uri/ =404;
    }

    # Disable all forms of caching on all requests.
    # A real production setup would require more granularity, but caching is
    # hard and we currently don't have big enough volumes.
    add_header Last-Modified $date_gmt;
    add_header Expires $date_gmt;
    add_header Cache-Control "no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0";
    add_header Pragme "no-cache";
  }

  # Load balancer redirects plain HTTP to this port.
  server {
    listen 8081;
    server_name _;
    return 307 https://davl.da-ext.net$request_uri;
  }
}

Note that this is a toy application, restricted at the network level to a trusted internal network, so I am not advocating this as a production-grade configuration. In particular, very little attention has been paid to security here. If you intend to put this in front of the internet, please consult a security expert; the internet is a scary place.

But hopefully it should show you how to redirect just the calls to the API (they all start with /v1) to another machine.

2 Likes

Thank you for your advice. I raised the question as I am new to programming and I simply want to understand how things work and fit together. Some other local IT teams have wasted us USD400K+ and almost a year with some other technologies on a similar project. I was not the CEO for that project but I personally sacked them all. I want to know clearly how and why before we spend a dollar more this time. Please allow me to take you some time to explain my situations and intentions.

As a semi-retired investment banker from Hong Kong, I am now working as an investment advisor advising some industries in Mainland China on their corporate finance, in particular supply chain finance. For years I have been thinking about how the industry players can trade with each other in a self-regulated environment so that we, acting as investment houses can provide financing to them. DLT + smart contracts appear to be the perfect solution.

I was introduced to Daml during a FinTech course and I find out that the HK Exchange is going to launch a settlement program in Daml in 2022. Being an executive of an investment bank, I trade in the HKEx everyday for half of my life and so I decided to take a deeper look into Daml.

I love DAML at first sight from the IOU example. So far I find DAML not too difficult to learn to build something really workable. I have tried to learn Haskell to understand DAML better and so far I can still manage. I build a simple trading and finance model for the FMCG industry as my first attempt during the covid19 quarantine period. I want to show the potential industry players, commercial banks as well as government and investment funds my POC before serious investments are deployed to study it further and build it. Everyone I talk to love my ideas and they have all heard something about blockchain. But nobody knows how and cannot be sure if DLT can really work for them. I want to present them my little works with a sandbox feel but for certain reasons, I cannot use DABL (another long story). Perhaps a local ECS service like Aliyun or AWS in China may do the job to give them a simple UI to try the demo.

As an amateur programmer, React is the major problem I am facing. The Daml-Ui-template is really good as it is not difficult for me to change the contents to work with the sandbox which is running the contracts I created. However, I cannot figure out from the React codes how api calls are made to the sandbox, except that the development server can magically do it for me. Having to learn and master React to understand this seems an overkill to me and I am just looking for a simple example to follow. In particular, how api works. This UI is just a transitional thing for demo purposes only. At the end of the days every user will have their own tailored made UI with many other local functions built-in.

I teach myself about docker and nginx and now I can set up the react development in docker to work with my Daml contracts running in the sandbox docker too. I have also tried the daml-on-sawtooth example. To complete the picture on understanding the whole set up, I tried to build the React and I think I need a proxy for it to “talk” to the sandbox. But how?

My present intention is simple. An UI to demonstrate how we can use DLT + Daml to solve the problems and how my ideas may work. It is not really a production build. I can either:

  1. put the React development and sandbox (+ triggers) in containers and set them up in ECS. The containers work locally but nothing show up in ECS (due to CORS problems or what?) BTW, Those are big docker containers consuming quite some 8G memory (but I don’t have to care too much on this at the moment); or

  2. build the react UI and serve it with Nginx in a container to work with the sandbox (or more aggressively, daml-on-sawtooth?) in containers. But I do not know how to set them up.

Excuse me for the long story and I would be much appreciate if you could give me some ideas how I could move forward quickly.

Thanks again for your time and valuable advices.

2 Likes

Many thanks Gary. I do really want to give you a big hug from China.

It is mid-night here but I will give it a try right now and revert to you again on my results.

Don’t worry it is just a demo, although I do want to put in up on ECS. I believe whoever could bring down Aliyun due to my fault will be rewarded by Alibaba. I will also talk to the government officials I know here to give the genius a medal. LOL.

1 Like

Hi @EllisAllison
We are a consulting firm focused on DAML smart contracts and have developed several solutions for banks, and healthcare firms deployed either on a local server, cloud server or on damlhub. You can check out one example for Equipment Financing use case that we created based on the ui-template and that we made open source
Feel free to take a look and reach out if you have any questions . (https://www.github.com/RethinkLedgers/ef-smartcontracts)

3 Likes

First off, I’ll point out that using React is by no means necessary. We provide three libraries to interact with a Daml ledger from JavaScript: @daml/types, @daml/ledger and @daml/react.

@daml/types is the substrate on which the codegen relies; @daml/ledger is a set of functions to facilitate working with the ledger and builds on top of both @daml/types and the types generated by the codegen. @daml/react builds on all the other to integrate with React. It is perfectly possible to use @daml/ledger directly if you do not want to use React.

As for how things work, exactly, let’s take a look at the Getting Started Guide, i.e. the code you get when you run daml new my-proj --template=create-daml-app.

First, we wrap the entire application in a DamlLedger pseudo-HTML element. This creates a Ledger object as defined in @daml/ledger and keeps it around in some sort of React ambient/global state/ I’m not particularly fond of this approach myself (I prefer my variables to be explicit), but that’s the React way.

Within that (dynamic) context, you can use the use* functions from @daml/react to access the underlying, hidden Ledger object. For example, calling useFetch will essentially just call fetch on that hidden Ledger object, plus some client-side wrapping of the results into a type that “plays well” with React.

The useEffect dance, without going into all the details, essentially allows us to say “make this API call, and then later on when you get the result, update the UI to reflect this data”.

Hope that helped make it seem less magical; happy to answer further questions.

Hopefully my nginx.conf above answers that, but happy to give more specific pointers if not.

The main issue here is that, for various (good, security-related) reasons, browsers don’t want to talk to multiple servers. So we make a single server (nginx) handle all of the requests, some directly itself, and some by just redirecting the requests to another server. Note that in this kind of setup it is necessary for the nginx server to have access to the JSON API, but it is not necessary for the JSON API to be connected to the internet.

With the same caveats about security etc., the DAVL example also has its full Terraform configuration availble, if you know how to read those.

For simple demo purposes you should be able to use much smaller containers. I can run the entire DAVL system locally on a VM with 4GB of RAM in total. That includes a container for PostgreSQL, a container for nginx, a container for the daml sandbox, a container for the JSON API, and a container for Navigator (definitely shouldn’t run in production, but can be useful locally to inspect the state of the ledger).

2 Likes

Hi @EllisAllison and thanks for that excellent post.

Without a doubt, as a non-professional like you, the React/UI is my Albatross also, however as @Gary_Verhaegen said, you can work around this using @daml/ledger

Welcome! :+1:t2:

1 Like

@Gary_Verhaegen I studied DAVL and tried to understand it as much as I could. I think it is too advance for my level. I am running the sandbox on my mac and I suppose the LEDGER_IP_PORT is http://localhost:7575? Anyway, I built the React daml-ui-template and tried to serve it with Nginx docker and I failed after many attempts. Here are my nginx.conf and my Dockerfile (Sorry, I don’t know how to paste codes here):

My nginx.conf

  server {
      listen 80 default_server;

      location / {
          root   /usr/share/nginx/html;
          index  index.html index.htm;
      }
      
      location /v1 {
        proxy_pass http://localhost:7575;
      }

      location /v1/stream {
        proxy_pass http://localhost:7575;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $host;
      }

      error_page   500 502 503 504  /50x.html;
      location = /50x.html {
          root   /usr/share/nginx/html;
      }
  }

My Dockerfile:

FROM nginx:alpine

COPY ./build /usr/share/nginx/html/

COPY nginx.conf /etc/nginx/conf.d/default.conf


EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

Hi @quidagis Thank you for your encouragement.

As an old banker trying to use DAML to solve some financial problems, I would say DAML is not too difficult a smart contract language to learn for a beginner like me. But building a UI is really taking me too much time to learn. God knows how I managed to learn some HTML/CSS as well as some docker and nginx just trying to figure out how to build an UI in the last three months. The daml-ui-template is a good template for my need but I really cannot figure out how this magic box works.

You and @Gary_Verhaegen have pointed out that I can use @daml/types, @daml/ledger and @daml/react to interact with the ledger (which I think is bundled in the sandbox?) but shamefully, I don’t know all these methods. Sometimes I wish if the documentation could demonstrate how to interact with the ledger with a simple HTML form for non-programmers like me to understand what is happening. I know IT professionals need more information to build their hard works but I think DAML is also a very good tool for finance and business executives. It would be a pity if they are frightened off by the underlying technologies while most business and finance solutions should be naturally initiated by these non-IT executives. So I will continue the DAML exploration. Being a non-English speaker and without IT background, I think I could still manage at my old age to join you all to make it better.

Thank you again. Best to you all and your families!

2 Likes

My thoughts exactly, and I have mentioned this to @nemanja who does great work in trying to help new Damlers learn better/faster.

Still, nothing great, is ever easy. Keep on coding, and asking questions here, you are in competent company, myself excluded :grinning:

1 Like

Thnx for the kind words @quidagis :smiley:

@EllisAllison Did you get a chance to look at our interactive tutorials showing step-by-step how to

  1. build an example social networking application with a React-based UI,
  2. add a new UI feature to it, and
  3. deploy it to Daml Hub

They might help with making UI progress and getting your application deployed without needing to configure a lot of things.

2 Likes

Unfortunately we do not currently support interactions with HTML forms directly. The issue here is that HTML <form> send data in a format called, appropriately, “form data” (formally application/x-www-form-urlencoded), which is not a format we currently support. At this point in time, your only option to interact with a Daml ledger from a browser is to go through the JSON API, which requires JSON data. And the only way to generate JSON requests from a browser is with JavaScript.

We could remove the React/Semantic-UI bits of the create-daml-app template, but it would still need to use JavaScript to manage interactions with the JSON API.

(Yes, sandbox is an in-memory, ephemeral ledger that exposes the same API as other Daml ledgers, so you can develop your code against it and then later on deploy the same code against a production, persistent one.)

My point was that, if you are comfortable with JavaScript (and HTML, and CSS, and the DOM API) but not with React, you could choose to not use React and therefore not use @daml/react while still benefitting from the codegen and still using @daml/types and @daml/ledger. However, if you are not familiar with JS, CSS, HTML & DOM (+ JS build tooling, I suppose) I’m afraid I don’t have a good answer for you at the moment. Web development has become awfully complicated these days.

The best I can give you at the moment (and I’m well aware it’s not much, so apologies for that) is an encouragement to read through our getting started guide if you haven’t already. It is using slightly different tools than daml-ui-template (Semantic UI instead of Material UI, npm instead of yarn) but the general principles are the same, and there is a lot more documentation. The create-daml-app documentation is also kept up-to-date, and testing that the template works as expected is part of our release process, so you are much less likely to encounter unexpected issues than with daml-template-ui, which does not seem to be actively maintained. (It’s currently 5 Daml releases late.)

Unfortunately the answer here is not straightforward. localhost is a fluent concept, as it literally means “the current machine”. So you can only be on localhost as seen from the same machine, where “machine” may not quite mean what you’d think it means.

Guessing a little bit, I am assuming that you have opened a terminal and typed daml sandbox (or daml start), so you have a sandbox running on your macOS “host” machine. At startup, the sandbox should output which port it’s listening to; something like:

INFO: Listening on localhost:6865 over plain text.

This is not what we’re interested in here. What you need to give to nginx is the hostname and port of the JSON API. If you’ve run daml start, it should automatically start (though that may depend on your daml.yaml); if so, it should print something like:

12:49:43.492 [http-json-ledger-api-akka.actor.default-dispatcher-7] INFO  com.daml.http.Main$ - Started server: ServerBinding(/127.0.0.1:7575)

This is the one we want. However, 127.0.0.1 is just as relative as localhost so that may not help us much. In particular, if you run nginx within Docker, things get a bit hairy. Docker only runs on Linux, which means that on macOS Docker has to run in a virtual machine. There are essentially two easy ways to get Docker up and running:

  • Docker for Mac, which is imo a terrible idea but the one promoted by Docker, so you’re probably running that.
  • docker-machine, which is a portable (non-macOS-specific) way to handle running Docker servers “outside” your local machine.

In both cases Docker will run in a Linux virtual machine; the main difference is that Docker for Mac tries to hide that fact and creates a very leaky abstraction that results in all sorts of hard-to-debug corner cases. Either way, the result will be that you have three networks at play here, and thus three possible meanings for localhost (== 127.0.0.1):

  1. The host, macOS machine.
  2. The Linux VM.
  3. The Docker container.

Visibility across these networks is not complete. Typically, 1 can see 2 but 2 cannot see 1, and 2 can see 3 but 3 cannot see 2 (or 1). The issue you’ll have here is that your nginx instance runs in 3 but your JSON API runs in 1, and therefore redirecting traffic from nginx (3) to JSON API (1) is not possible.

There are two workable options here. The first is to run the sandbox and JSON API as Docker containers too. That way, they’d be in the same network as nginx and therefore visible to it. To achieve that, you could use the (development-only, definitely not supported for production) daml-sdk image we provide. This would look something like:

FROM digitalasset/daml-sdk:1.12.0

USER root
RUN mkdir /app
COPY .daml/dist/ /app/
RUN chown -R daml:daml /app
USER daml

Build this with docker build -t my-daml . (from the root of your project), and then run it twice with commands along the lines of:

docker run --name sandbox my-daml daml sandbox --address 0.0.0.0 --ledgerid ledger /app/my.dar

to start a sandbox with your DAR file (i.e. your compiled daml code) loaded, and

docker run --name json --link sandbox my-daml daml json-api --ledger-host sandbox --ledger-port 6865 --http-port 7575 --address 0.0.0.0

to run the JSON API. And then you can run your nginx machine with the --link json option such that, from within the nginx container, the hostname of the JSON API is json and your configuration becomes:

...
      location /v1 {
        proxy_pass http://json:7575;
      }

      location /v1/stream {
        proxy_pass http://json:7575;
...

Hope that helps.

1 Like

@nemanja I had attempted all the tutorials and watched all the DAML Youtube videos. I have also tried deploying to the DAML Hub/DABL. (I personally has some reservations on using DAML Hub for demo purposes).

Anyway, thank you for your advice.

1 Like

@Gary_Verhaegen Thank you very much indeed for your detailed explanations to a beginner like me.

In fact, I can run the React development in a container with another container running the sandbox in the same docker network. I then greedily tried to run them on ECS but as expected, I cannot access it with my local browser. I searched for answers and it seems there are CORS problems and ppl would only put the production build on a remote server with Nginx. That was how I got here to you with all these naive questions.

I will try your recommendations and again, thank you for your time and guidance.

1 Like

Would love to hear more about what’s missing in the Your first feature tutorial and how we can improve it :slight_smile: (that tutorial is supposed to help in explaining how to develop a new feature in Daml and connect it to a UI). Happy to get your feedback either here or in a private message (whatever works better for you really).

1 Like

@nemanja The tutorial is great. However, I was not interested on it at the beginning as it is more like a Chat app while I was looking for business applications. In fact, it was the previous IOU example which made me love DAML at first sight. Anyway, I did the tutorial. I also like the daml-ui-template after watching a Youtube video on it and I could easily follow that video to modify a UI I want without having to know too much about React. The guidance to deploy to DAML Hub is very clear, including how to run triggers and initiated scripts. Those are all really great works.

My personal reservations on DABL/ DAML Hub are about users login. It is perfectly OK in real life as each participant can only have one account and one role in the real world. But for a MVP demo I think it is another story.

The power of DLT and DAML is users collaborations. i.e. a group of participants can interact with each other in a trusted and decentralised environment. So to enable a potential user to understand how a MVP demo might solve his needs, this potential user has to play different roles to see the whole picture. This is where I got some feedbacks from my friends and potential users:

  1. Not the whole world can access Google or has Gmail. I connect to DABL from from China using VPN. Further, my friends in the banks told me that they cannot use VPNs.

  2. To enable a user to play different roles to understand the MVP, this user got to login in with different account names/long party-codes and long token strings. Of course, I could create several dummy accounts for them but that was a crazy job. And they have to copy and paste all the long strings in order to login to play with the MVP. You can imagine how frustrating it is for them after several login and logout. They got lost completely and their attentions were drawn to these login things and not how the app solves their problems.

  3. The MVP use party-codes and names. I understand the reasons for this but you can imagine how confusing it would be if you have to trade with four to five different parties with long codes. We can tell Alice, Bob and Cindy easily but not their party-codes. Some have suggested to build a kind of dictionary to display user names in stead of party codes, but that would defeat the purposes of quick demo deployment. Besides, for a non-programmer like me it will be an overkill.

I was looking for a simple quick solution to show something like the sandbox to all potential parties. Because of the above, I decided to put up the MVP demo on ECS and that was how I got all these questions about the UI.

Thank you again for your kind advices.

3 Likes

Thanks a million for the very thorough feedback @EllisAllison, much appreciated! Another option could be to use the Daml Navigator, a simple UI that can show the workflow (it all depends on how nice the UI has to be and what you wanna show with it)

Hi @ellisAllison

You mentioned above you are looking for a more streamlined user management for different parties in an easy-to-use sample app. This is one of the reasons we further enhanced the daml-ui-template and created an “equipment finance” use case and implemented some enhancements in the daml-ui-template, including:

  • a more simple authentication models (eg. calculated JWT token passwords for login for any of the parties, which means you can log with no complicated JWT password required)
  • and more user friendly party names (using PartyIdHint and parties.json) within the application

You can find more information about this in our read.me file on https://github.com/RethinkLedgers/ef-smartcontract/blob/d80c2e325bebcc12a07f3648d92524ac6e1adcc9/README.md

2 Likes

@Gary_Verhaegen I am afraid I have to bother you again with more guidance.

I followed your advice and set up the sandbox and json containers as:

docker run --name sandbox my-daml daml sandbox --address 0.0.0.0 --ledgerid my-app /app/my-app-0.0.1.dar

docker run --name json --link sandbox my-daml daml json-api --ledger-host sandbox --ledger-port 6865 --http-port 7575 --address 0.0.0.0

I then run my nginx container as:

docker run --name nginx --link json -p 80:80 my-nginx

Here is my nginx.conf :

server {
listen 80;
location / {
     root   /usr/share/nginx/html;
     index index.html index.htm;
     try_files $uri $uri/ /index.html;
}
    location /v1 {
        proxy_pass http://json:7575;
      }
    location /v/stream {
    proxy_pass http://json:7575; 
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Host $host;
    proxy_cache_bypass $http_upgrade;
}
}

and the Dockerfiles:

FROM nginx:stable-alpine

COPY build /usr/share/nginx/html

COPY nginx.conf /etc/nginx/conf.d/default.conf

EXPOSE 80

CMD ["nginx", "-g", "daemon off;"]

After many attempts to modify my nginx.conf, I still have the following two errors on my Chrome:

WebSocket connection to 'wss://api/data/localhost/v1/stream/query' failed: 

useStreamQuery: WebSocket connection failed.

What have I missed?

1 Like