I understand what semaphores are for in the traditional sense, but within the context of DAML I am trying to understand the why and mechanics behind them.
Refer to danban
It appears like when I perform a fetchByKey on any contract, its the equivalent of applying a global lock on it then releasing the lock after the transaction completes?
How should I be thinking about this?
If I have more than 1 contract to update, then I should always be doing this locking semaphore pattern?
My confusion lies in whats the difference between an Update being an all or nothing atomic operation vs also doing this semaphore thing at the start of every choice which is just another way to lock everything to guarantee atomicity?
Disclaimer: I’m not too familiar with the Danban model so I might be missing some details.
Generally, this kind of pattern where you guard a choice by the existence of a key (if it doesn’t exist the fetchByKey will fail immediately and the transaction will be aborted) can be used to reduce contention. To make things a bit more concrete, in Danban this works as follows:
The Directory_AddObs choice modifies all user role contracts associated with the respective keys. This means that it is very likely to run into contention issues if any of the contracts are modified in parallel (if you have lots of users, this also becomes a large transaction and thereby necessarily slower which makes it even more problematic).
To avoid this, there is a PauseApp choice. This will archive the semaphore contract causing all the choices you discovered which could cause contention to fail immediately.
This is then used as follows:
Pause the app
Run the Transaction likely to run into contention issues which will now succeed since everything else fails immediately.
Unpause the app.
You can find an example of this at thttps://github.com/digital-asset/danban/blob/0064d0e41506baa1b86a6f24fb6ecc26c82892df/src/server/ledger.js#L169
You might notice that I link to an older model here. This is because as far as I know it is crucial to do the first step in a separate transaction. Otherwise, if pausing the app is part of the same transaction as the problematic choice you run into the exact same issue as the other transactions will only fail once the problematic one has been committed. So I don’t think the UserSessionAck choice in the current model is actually useful.
As far as I can tell, PauseApp is only ever used in the same transaction in the current model which makes this pattern useless. But I could definitely be missing something. Maybe @bernhard and @DFeichtinger can shed some light on this.
The reason I ask about this semaphore pattern is id like to assign parties to role groups. There has been a few threads on this
For example, my app is a phone book with 1000+ pages.
Lets assume its hosted on DABL so if it takes off everyone is their own unique party. I don’t know of a nice way to group N parties to 1 group type role without experiencing this phenomena I am coining the observer propagation quandary TM
A newly onboarded party is a basic user by default, I need to give them the ability to fetch every phone book page so my goal is to make them a stakeholder of all 1000+ phone book pages.
To achieve this, I need to do a bulk update and propagate this newly onboarded user to the observer list of all 1000+ phone book pages. (This is the part I am concerned over and if there needs to be some form of global lock in place).
I have other specific roles but these arent an issue because I have single role contracts for individual parties so they can do role membership checks by simple lookupByKey which I really like.
Is there a solution to this quandary or something to make me feel at ease that I am not doing something really dodgy?
My understanding is broadly the same, but just a minor point: when adding users they need to be added as observers to the semaphore. Unpausing after adding the user fetches the updated user list
@cocreature is spot on regarding the original purpose of the Semaphore contract. @Andreas_Lochbihler has just written an in-depth piece of documentation on DAML’s causality model. It explains the ordering guarantees a DAML Ledger gives, and consequently the types of contention you can run into.
Generally speaking, creates and archives are high contention, non-consuming actions are not. If you have 100 transactions all fetching or looking up a contract, and one archiving it, it doesn’t matter in which order the 101 transactions are received, the archive will always go through. It’s the fetches and lookups that come after which will fail.
Contention is controlled in Danban by sharding almost all data and using nonconsuming exercises where possible. The only times you have contention is if the same object is modified by two transactions - eg two updates to the same card.
The exception is the addition of new users. Because of the “observer propagation quandary”, a transaction adding a new user has contention with a lot of other operations. To have a chance of committing it, the operator has the Semaphore to pause the app. Because the lookupByKey is non-consuming, the archival/create of the Semaphore gets priority.
Right now, there is no good way to solve the “observer propagation quandary”, but it’s an issue high up in our priorities of problems to solve. The first solution we are planning is a feature which allows party-sharing. For your use-case, you’d create a directory or public party, and then give all users read-access to that party. When they query the Ledger, they’ll then get data for both their own as well as the directory party, and when they submit commands, the contracts known to directory can be fetched and looked up.
This mechanism relies on the directory party being hosted at least read-only on the same node, which makes it slightly more complicated in fully decentralized topologies, but we will also make it possible for a party to be hosted on multiple nodes, so you’d just give all relevant nodes read-access hosting for the party.
Thanks for the reply, I really look forward to a party sharing solution.
Is this achievable on DABL?
Just trying to establish a mental model for all the moving parts to make it work.
create a BasicUser party in DABL and lets say their unique party id is `ledger-party-basic-user’
This BasicUser is then a static observer added on any contract that has the ability to be available to basic users solving the observer propagation quandary.
if alice then uses my DABL app, she will get a token issued to her by DABL looking like this.
My question is, I need to append the BasicUser party to the actAs list for it to have a chance to work? There doesn’t seem to be a way in DABL to change it directly (yet)?
The feature is WIP and there’s still design work to be done. The only thing I’m sure of is that whatever we come up will work on DABL. It’s one of the main use cases Maybe @dtanabe or @dliakakos already have some idea what direction it’ll go.
On DABL, every ledger gets a “public party”; all users can freely acquire a read-only token for that Party and can then consequently read any data with that party as an observer.
