Tasks, Messages, & Transactions

Tasks, Messages, & Transactions – the holy trinity

Saturday, March 31st, 2007.

The discussion is picking up around disconnected Web Service interaction scenarios. Here’s a summary of what’s going on for those just joining us, but I would suggest reading the full posts as well:

Andres: “Basically, there is no disconnected mode… if you plan to build a multi-tier application with ADO.NET Orcas in the middle tier, you will need to hack your own change tracking mechanism in the client, send the whole changeset, and apply it in the middle tier.”

Udi: “[In] the UI, … tasks often corresponded very well to the coarse-grained messages we employed in terms of SOA.”

Jesse: “I have to disagree with Andres and agree with Udi.”

(Sorry, I just couldn’t resist. Here’s the important part)

“Andres rightly points out that you don’t want 5 different methods for updating an order. Although you technically could do it that way, you probably don’t want to be ignoring the transactional context of the conversation or locking tables for significant time periods.”

And, “The backend will still have all the update methods you are used to using, so you’re not writing a bunch of extra code in the DB. However, this approach is far more powerful. One reason why is that now you have an operation defined and you can control everything about that operation. You know when the call comes in through that method that you have a customer service rep trying to update an order and can build in logic based off of that.”

Andres: “Now, we need to send those changes to the server, together, because I want them executed in a single transaction. I cannot have a service called ‘ModifyOrderCustomerInformation’, another ‘AddALineToOrder’ and ‘DeleteLineFromOrder’. I need an ‘UpdateOrder’ service. You can use a diffgram to do it, or you could build your own change-serialization mechanism.”

And, “The only way I see is to map the UI to the service interface, so the user can ‘Change Address’ or ‘Update Marital Status’ as different operations in the UI layer, but I can’t let the user ‘Modify the Customer’. It’s a lot of work, and I seriously doubt that the users will like it.”

OK, now we’re all set.

Just to get something small out of the way, every Human-Computer Interaction (HCI) professional I have worked with has been in favor of task-based UIs. Every user that I have met that has used both styles of UI, task based and “grid” based, has reported that they were more productive when using the task based UI for “interactive work”. Data entry is not interactive work, so grids might be more suitable there, although improvements in bulk-loading technology and OCR have decreased the amount of “plain” data entry that I’m encountering. It also may be that I’m working less on systems where data entry is of significant importance.

First of all let’s agree that writing business logic for something like “DeleteLineFromOrder” or “UpdateMaritalStatus” is easier than for “UpdateOrder”, given that it fit with the overall system design.

Second, let’s agree that the business logic for something generic like “UpdateOrder” is composed of the various specific cases like “DeleteLineFromOrder” and “AddLineToOrder”.

Third, let’s agree that if the UI was task based, it would be easy for the client side to activate the correct web service/method – seeing as there is a high correlation between the tasks and the service methods.

Therefore, the question becomes how do we do all the above specific work in a single transaction? One answer may be to make the service statefull and using something like WS-Atomic Transaction to tie the various calls together. The problems in this approach are already well known. Another solution uses a messaging paradigm, something I wrote about before.

If we represent each of the tasks not as “web methods”, but rather as messages we could send all these messages together in one envelope to the server, and have it process all of them in a single transaction, activating the specific business logic pieces one after the other.

I’ll be using a code-first approach to describe the solution instead of an XML-first one since it abstracts away the technology, but it is still a contract-first approach.

Each message type is just a class that implements the “IMessage” interface. So we’d have a class called “DeleteLineFromOrderMessage” which is a Data-Transfer Object (DTO) whose members/properties are the data needed for processing – in this case, the order Id and the order line number. The same would be true for the class “AddOrderLineMessage”, it would contain the order Id and some other member for the order line data quite probably a DTO itself (so that we can use it in other places as well).

The client would generate an instance of the appropriate message class as the user finishes each sub-task, saving them up. When the user would click “confirm”, the client would send all these message objects to the server in one go with this API,

void IBus.Send(params IMessage[] messages);

Like so:

myBus.Send(addOrderLineMsg, delOrderLineMsg);

The bus would take all these objects and wrap them in a single SOAP envelope, and send them to the server, probably on a transactional channel, but that’s a configuration issue. At the server side, the bus would activate the transaction (because of the transactional channel) and start dispatching each of the specific messages to its message handler, one at a time, in the same thread.

So as you can see, there is no “transactional conversation”, and therefore we’re not locking tables in between calls, because we’ve gotten rid of “in between”. It’s just like the generic update in terms of transaction time and network hops, just with specific, simple business logic.

Andres might retort to that, “Even if Udi is right, and that’s the way applications should be built, I doubt most mere mortals in Earth will want to do it.” In fact, he did J If the frameworks supporting this style of development were supplied by Microsoft, I’m sure that most developers wouldn’t have a problem with it. When provided with such a framework, every developer I’ve worked with said that they wouldn’t want to go back to the “old way” (would that make Orcas outdated?). I’ll be putting up my frameworks soon, as well as examples on how to use them, it’s just taking longer than I expected.

Before closing, I just wanted to address the points Andres raised about concurrency:

“You could also need to know the previous values for optimistic concurrency checkings.”

I’ve written about how to do this before in regards to Realistic Concurrency, and the best example I have in terms of code is Better Domain-Driven Design Implementation.

This has gone a little longer than I planned, but I still don’t think I’ve covered everything in enough depth. I’d most appreciate investigative questions to help me shed light on the murkier parts of this, either as comments, posts on your own blog, or even via email. Let’s continue the conversation.

If you liked this article, you might also like articles in these categories:

If you've got a minute, you might enjoy taking a look at some of my best articles.
I've gone through the hundreds of articles I've written over the past 6 years and put together a list of the best ones as ranked by my 5000+ readers.
You won't be disappointed.

If you'd like to get new articles sent to you when they're published, it's easy and free.
Subscribe right here.

Follow me on Twitter @UdiDahan.

Something on your mind? Got a question? I'd be thrilled to hear it.
Leave a comment below or email me, whatever works for you.

» From Disconnected Scenarios to SOA Says:
April 1st, 2007 at 3:02 pm
[…] no way did I intend to discuss SOA related issues in my previous posts. Since I talk so much about SOA, and the use of the Bus Pattern and Asynchronous Messaging, I […]

Frans Bouma's blog : Why change-tracking has to be part of an entity object Says:
April 3rd, 2007 at 3:29 am
[…] inside the entity objects itself and gave some examples why that’s unfortunate. Udi pulled the discussion into SOA land, and I think that was unfortunate as well, because there’s a much wider used example which will […]

» Can Indigo be my bus? Says:
April 6th, 2007 at 4:58 pm
[…] How the Bus API handles the connection between messages and transactions […]

» [Podcast] Occasionally Connected Smart Clients and ADO.NET Sync Services Says:
April 17th, 2007 at 9:27 pm
[…] enjoying the recent discussion on Entity Framework, disconnected clients, Unit of Work, and messaging. A few weeks ago I wrote a note to self to “Ask Udi” about the new ADO.NET Sync […]

Darrel Miller Says:
May 4th, 2007 at 9:02 pm
Just to add another voice to the “Change Address” position instead of “Modify Customer”. I would go one step further to try and identify why the address is being changed. Are we “Correcting the Address” due to a clerical error, or changing the address because “Customer has Moved”? The actions taken based on those two different scenario’s could be significantly different.
Now… how you actually implement a UI in this style is a very different question. I look forward to someone trying.

thesoftwaresimplist Says:
May 5th, 2007 at 2:28 am
Darrel,

At the very simplest level, when right-clicking a customer, one of the options available is “Customer Moved”.

If there are relatively few actions that could be done on customer, some kind of outlook 2003 bar could be used.

John "Z-Bo" Zabroski Says:
May 17th, 2009 at 10:45 pm
Udi,

What is a grid-based UI?

Task-based UI – you give examples.

Grid-based UI – you throw in there like it is a common term.

Quite simply, Google Top 10 Search Results says the term is own you made up and didn’t define. I’d appreciate an explanation of what sort of symptoms Grid-based UIs have that you don’t like.

Look at it this way: You say, “let’s agree that if the UI was task based, it would be easy for the client side to activate the correct web service/method – seeing as there is a high correlation between the tasks and the service methods.”

This sets up a strawman for dismissing “Grid-based UI” (whatever that is). You actually never explain why Grid-based UI has a LOW correlation, or why correlation makes programmer’s life more difficult.

Also,

@Data entry is not interactive work, so grids might be more suitable there, although improvements in bulk-loading technology and OCR have decreased the amount of “plain” data entry that I’m encountering.

I don’t understand this, either. There is always a Suzy Creamcheese on the phone with a customer who needs to enter information somewhere about the call. Even if you have tasks that map to her cold and hot call scripts, at some point along the task she will be punching in numbers and names. If Suzy is a development personnel in a fundraising department working during a telethon, she needs a fast way to enter data.

udidahan Says:
May 18th, 2009 at 1:01 pm
John,

Sorry for not defining it – I did make an assumption there about people understanding its intent.

A grid-based UI is one where the user performs most/all activities in the app through a grid – specifically editing data. In this kind of user interaction model, the intent captured is a fairly technical “I want to edit this entity”, without the business context of “why” – like Darrel’s comment above describes.

Relating to “she will be punching in numbers and names”, I’m not arguing that that occurs, but that having additional business context is invaluable.

Some systems really are pure data entry in that the system doesn’t really have any validating to do, as the data coming in is being typed from a file of sorts which is already the authoritative source of information – nothing really that a computer could refute.

Does that make more sense?

John "Z-Bo" Zabroski Says:
May 22nd, 2009 at 6:23 am
You seem to be suggesting that there is no way for this “grid-based UI” to communicate a “change of address” message has been sent on UPDATE. I guess if your Complex Event Processor can only handle blobs of messages, then you need that level of granularity?

Also, what if the application doesn’t capture the business context. Instead, the business context is tied to a virtual event only known to complex event processing. Such is the case in a federated system controlled by messaging. You can see the messages flying back and forth, but without the ability to recognize events in a human understandable way, it’s just mud and you can’t actually see anything insightful.

The business context would be that there is a service waiting for a customer’s correct address.

So again, still I see a sort of strawman. Sorry. 🙁

John "Z-Bo" Zabroski Says:
May 22nd, 2009 at 1:57 pm
Actually, I think I know what you are trying to say.

I also think I can now better articulate what I am put-off by.

Sometimes users do need a “green screen terminal” do to their jobs well.

The scenarios you are talking about, with tasks like “Change XXX”, only make definitive sense when there is a service waiting on something. The service has a human dependency on XXX, and therefore the UI is Change XXX.

A good example is Amazon.com. Usually, they annoy me for two reasons.

1) A book I preordered has been delayed, and they want to confirm I am still interested.
2) A credit card I have on file with them is no longer valid, but my order has already taken place (but is now on lay-away until I change my CC#).

In both these scenarios, I get an email w/ a hyperlink to a secure URL (requiring login) that plants me on a page where I can take action to move the business process along. I don’t have to go looking for my order. They directly put the action item in front of me, and the screen tells me what my primary task is (“fix billing info” or “confirm you will wait until hell freezes over to get this book”).

I don’t like the phrase task-based UI here. Task-based UI means the user’s workspace is organized to improve completion of tasks, but doesn’t communicate the real usability concept: the UI *knows* what you need to do.

udidahan Says:
May 23rd, 2009 at 4:33 am
John,

Re: “the UI *knows* what you need to do.”

Exactly.

If you can find a better term to describe this, giving it a good contrast from what I described as “grid-based UIs”, I’d be most happy to hear it.

John "Z-Bo" Zabroski Says:
May 25th, 2009 at 9:26 pm
In UX/Interaction Design circles, the operative phrase is “workflow-centric”.

However, let me be very clear that technical jargon is virtually meaningless without the presence of examples (like the Amazon ones). I realize this is simply a blog, but the major failing of most blog entries on most blogs is a very dangerous myth: the myth that all programmers share the same psychology, when they do not.

Not only do programmers have different psychologies, but they also don’t share the same vocabulary.

Workflow-centric UIs are less common, in part because (a) you can’t simply use a code generation tool to build the UI for you (b) workflow is inherently hard to describe using human language, and as such there is no format for a compiler writer to use to help automate workflow-centric UI creation (c) workflows can often change, and so in workplaces where employee turnover is also high, human resources training officers have to work double duty to make sure people know what the hell their job is

With workflow UIs, you want to be sure that you only invest in them when you have a very stable description of the problem domain. If you properly design your application based on the real world, then any changes that occur in the real world should be not too dramatic, because people rarely dramatically change their workflow. Human beings, ya know, we’re risk adverse and don’t like change, and when we do change we tend to change in ways creating only subtle differences in the problem domain’s structure.

udidahan Says:
May 25th, 2009 at 10:21 pm
John,

Workflow-centric UI sounds good.

Sly Gryphon Says:
August 11th, 2009 at 6:50 am
Allowing multiple messages to be submitted is a good idea.

I have designed a system that way before, and it works very well. To indicate a transaction is required across multiple messages you can easily include the details in a wrapper (or header). (Or to get something working quickly even just initially assume multiple messages === transaction).

From memory the design was based on the IBM IFW (Information Framework, a framework for financial services), which uses a similar architecture (multiple messages can be sent in one transaction).

Sly

udidahan Says:
August 12th, 2009 at 7:50 am
Sly,

Thanks for your comments, I’ve found it a good option as well.
The thing I like is that the server can change its mind and process them one at a time.

Scott Whitlock Says:
September 17th, 2014 at 2:19 pm
I think you’re side-stepping the issue of optimistic concurrency, or you don’t understand what Andres means by saying “You could also need to know the previous values for optimistic concurrency checkings.” Your example of “Change Customer Address” doesn’t really highlight the concurrency problem because it’s so unlikely that two people would be changing the customer address at the same time to two different values. The real concurrency problem (which is admittedly rare) is when there’s a real chance of two instances updating the same thing at the same time and the second instance to commit would change its action if it knew what the first instance was doing. For example, consider a scheduling system where we schedule service calls. If you need to schedule a repair service at a customer location and you have many different repair crews, your logic might try to optimize which repair crew you schedule based on the location of other sites that crew is going to that day, the travel distances, and the already scheduled calls. When you scale this up, it’s quite reasonable to have cases where you’re making a scheduling decision on obsolete data and when you try to commit, sure it will stop you from making an “invalid” choice (like scheduling two repairs for the same crew at the same time) but it won’t stop you from making a non-optimal choice, like scheduling two different crews to go to the same area of town on the same day, when one crew could have done both more efficiently. When you’re working with a system like this, you have to take concurrency into account. This probably means moving the scheduling logic to the server side and having a “Schedule Optimal Crew and Time” service, and yet you still need to deal with the concurrency on the server side. Either you have to serialize the calls, which doesn’t scale up well, or you need to use optimistic concurrency on the server side and check that the assumptions you based your calculation on are still valid when you commit.

Tasks, Messages, & Transactions – the holy trinity

Your comment...