We had a great turnout yesterday at the Web Developer Community (not user group <grin/>). I passed on the presentation files and code samples to Noam but figured that the rest of my readers might enjoy them as well.

The (pdf) presentation is here: Asynchronous Systems Architecture for the Web

The code sample is here: Asynchronous User Management Code Sample

In the sample, you can see the use of sagas to manage the user registration process; store user email and hashed password, send a confirmation “email”, when the user clicks the “link”, the web server will take the saga id found in the url, and send a message with that id. This will cause the saga to complete and the user to be written to the “database”.

Since I didn’t have an email component on my laptop, and I’m guessing you don’t either, the saga just writes the url to the console. Copy and paste it from there into the browser, and you’re good to go.

A Word on TimeoutExceptions

One other thing that I want to call to your attention. When stepping-through the code in the debugger, you’re liable to spend more time than the Transaction Coordinator likes, which will cause it to rollback and try the message again. This is supposed to happen and occurs by design.

When you’re actually working with a database in a high performance environment, there will be cases where one transaction locks a page of a table and may cause other transactions to either timeout or be chosen as victims and just tossed. The behavior that best handles this scenario is just to retry the transaction.

However, you don’t have to write ugly code that checks for the specific error codes of each specific database for your code to work properly. The infrastructure will automatically do that for you – just let the exception happen. No need to write any try-catch code.

The sample is built on the newly released version of nServiceBus (1.6.1) but already contains all the binaries so you don’t have to set anything up yourself.

What’s coming for nServiceBus

We’re working towards a 2.0 release in the June-July timeframe which, beyond having the necessary documentation, web site, samples and everything any self-respecting open-source project has, is going to have some amazing grid-style features that will make all the message-priority & dynamic-routing stuff look “so last year”. Stay tuned.

In this podcast we’ll be discussing the issues around multi-threaded processing of messages by a service, specifically that the processing of message received second may be finished before that of the first. This scenario tends to rear its ugly head at higher levels of load and is critical for correctness in high-scalability environments.

Our long time listener Bill asks:

Hi Udi,

I have a question around processing of messages in proper order. When leveraging multiple threads to process messages in a message queue, it is possible for the second message in the queue to get processed before the first – especially if the first message is considerably larger than the second. I have taken a lot of care to make sure that messages are sent in the correct order, only to find that the receiving system can process them out of order anyway.

Consider a Policy Created notification, which must come before a Policy Approved notification. If both messages are sitting in the queue when the receiving service starts up, the approval message can be processed before the creation message. How can I make sure that message ordering is respected by the receiving system? I am using WCF/MSMQ as the underlying transport by the way. The only way I have found so far is to limit the receiving service to a single thread, which is by no means desirable.

Best Regards,

Bill

Download

Download directly here.

Additional References

• Blog post: In-Order Messaging a Myth?
• Blog post: Handling Messages out of Order

Want more?

Check out the “Ask Udi” archives.

Got a question?

Send Udi your question to answer on the show.

So, it turns out that Microsoft has quietly launched a new community-style site.

Titled "BizTalk Blogs", I wasn’t quite sure what my blog was doing there. It’s not that I never write about BizTalk – every once in a while I even find something nice to say about it 🙂 My quick post on BizTalk and Performance is one such example. But, let’s face it, a lot of the work I do is to provide BizTalk-like features like routing, transaction-management, and choreography (orchestration) without the actual product.

Apparently, I’m not the only non-BizTalk-only blogger there.

Including such names as Christian Weyer and Michelle Leroux Bustamante , there is a veritable who’s who in the Microsoft Connected Systems ecosystem and, quite frankly, I’m surprised the bouncer let me in the door.

So, this post is for my readers who, like me, have pretty much ignored anything looking like BizTalk for the past few years. Don’t let the name fool you. BizTalk Blogs is a valuable resource even for people who don’t care about BizTalk – and hey, you might even like what you start hearing about the future directions Microsoft is taking it.

But that’s enough of that. We’ll be back with your regularly scheduled BizTalk bashing right after this break…

As a part of my efforts to make clearer what place nServiceBus has in the Microsoft .NET ecosystem, I’ve decided to retire the term “workflow”. Almost every conversation about nServiceBus where the term “workflow” was brought up, the reaction was almost identical:

“What’s wrong with Workflow Foundation? Why aren’t you using it like you use WCF?”

There’s nothing wrong with Workflow Foundation. The thing is that nServiceBus doesn’t really need workflow in the general sense of the term. An older term that’s been used in the DBMS community might make more sense – “long-lived transactions”. You see, nServiceBus requires state management over many messaging interactions, and thus, some kind of long-lived transaction to maintain consistency.

In 1987, a different model was introduced for handling these scenarios – the Saga [GARCIA-MOLINA 87], and was further expanded in 1992 [CHRYSANTHIS 92] to allow the saga to commit if a non-vital subset of the sub-transactions abort. This is what is used in nServiceBus.

When used in distributed manner on top of one-way messaging, this results in a solution where each service runs its own “mini-workflow”, and coordinates its actions with other services via messages. This integration style is different from the traditional broker, man-in-the-middle approach found in products like Biztalk; and is known as “choreography”, and is in the process of standardization in the WS-* specs, known as WS-Choreography.

So, the bottom line is that the source, examples, and documentation of nServiceBus is being moved in this direction. The source and examples on the sourceforge site have already been brought forward. This is a breaking change.

I hope that this will make it clearer that nServiceBus is a higher-level set of abstractions than WCF/WF – limiting the generality found in these frameworks to enable one cohesive way of working that will result in a solution that is both scalable and robust. On the other hand, nServiceBus is not a fully integrated product like Biztalk and intends to tackle different kinds of problems.

Bigger than a WCF/WF, smaller than a breadbox Biztalk.

“Is it still valid to assume it is more expensive to design a scalable system?”

In Gavin Terrill’s news post on InfoQ, Big Architecture Up Front – A Case of Premature Scalaculation? he covers one of the questions so many of my clients deal with:

“How hard will it be to make it scale later?”

This is a valid question, especially for companies/products just beginning their lifecycle. When the product/web site isn’t bringing in any revenue yet, how much money should we spend on getting it ready for that future success?

The answer to that question lies in treating capacity and scalability differently (source).

What I mean by that is designing for scalability, yet separating out all technological aspects of the scaling from the core solution. That way, you can start with simple, low capacity technologies that won’t be too expensive. As you grow, upgrade that infrastructure and plug it in to your solution. Arnon’s recent post on Tier Splitting touches on a project we worked on together where we designed it in a way that we could scale down to a single process on a single machine and scale out to a server farm, all without changing the core system.

Let me take the design of nServiceBus as an example:

One primary property of scalable systems is the explicit treatment of all IO/communication. This can be seen in the one-way messaging exposed by the Bus object. There is no immediately evident way to do synchronous RPC-style request/response. This design decision is taken up front. However, the way that messages are passed around is abstracted behind an ITransport interface. You can deploy the first version of your system on MSMQ, and as load increases, switch to a more performant solution like RV or MQ, just by changing configuration. WCF does this kind of abstraction as well.

Another important element of the scalability of a system is how workflow instances are persisted. This behaviour is also abstracted behind an interface – IWorkflowPersister. Start out persisting workflows to a database. As you grow, swap that out of a replicated in-memory cache. In any case, the interaction between workflow and messaging at the logical level is set up front. All the pieces of the design are there. Up front. Helping you design your core application in a way that won’t limit your scalability in the future.

This is plain Separation-of-Concerns; code that works with your specific ESB kept out of your business logic.

Design first, scale with technology later.

[Full disclosure: I’m an editor for InfoQ]

I got this question the other day from one of my long-time readers Bill about nServiceBus and I thought I’d share:

I have a question around processing of messages in proper order.  When leveraging multiple threads to process messages in a message queue, it is possible for the second message in the queue to get processed before the first – especially if the first message is considerably larger than the second.  I have taken a lot of care to make sure that messages are sent in the correct order, only to find that the receiving system can process them out of order anyway.

Consider a Policy Created notification, which must come before a Policy Approved notification.  If both messages are sitting in the queue when the receiving service starts up, the approval message can be processed before the creation message. How can I make sure that message ordering is respected by the receiving system?  I am using WCF/MSMQ as the underlying transport by the way.  The only way I have found so far is to limit the receiving service to a single thread, which is by no means desirable.

Well, the solution is really quite simple (at first).

If you’ve received a message that you think has arrived out of order, just call:

this.bus.HandleCurrentMessageLater();

and that will put the message back at the end of the queue.

Once you start considering the fact that you don’t know when the first message is supposed to arrive, you might turn to using a workflow to handle the logic. The workflow would store the policy id, and then allow for N round-trips, before it decided that something bad had happened (like the Policy Created message getting lost), and then it could forward that to an operator, or possibly contact the first system and ask for a replay of the policy created message – or whatever automated fault resolution protocol you like.

In other words, message ordering is probably more trouble than its worth.

Often during my consulting engagements I run into people who say, "some things just can’t be made asynchronous" even after they agree about the inherent scalability that asynchronous communications pattern bring. One often-cited example is user authentication – taking a username and password combo and authenticating it against some back-end store. For the purpose of this post, I’m going to assume a database. Also, I’m not going to be showing more advanced features like ETags to further improve the solution.

The Setup

Just so that the example is in itself secure, we’ll assume that the password is one-way hashed before being stored. Also, given a reasonable network infrastructure our web servers will be isolated in the DMZ and will have to access some application server which, in turn, will communicate with the DB. There’s also a good chance for something like round-robin load-balancing between web servers, especially for things like user login.

Before diving into the meat of it, I wanted to preface with a few words. One of the commonalities I’ve found when people dismiss asynchrony is that they don’t consider a real deployment environment, or scaling up a solution to multiple servers, farms, or datacenters.

The Synchronous Solution

In the synchronous solution, each one of our web servers will be contacting the app server for each user login request. In other words, the load on the app server and, consequently, on the database server will be proportional to the number of logins. One property of this load is its data locality, or rather, the lack of it. Given that user U logged in, the DB won’t necessarily gain any performance benefits by loading all username/password data into memory for the same page as user U. Another property is that this data is very non-volatile – it doesn’t change that often.

I won’t go to far into the synchronous solution since its been analysed numerous times before. The bottom line is that the database is the bottleneck. You could use sharding solutions. Many of the large sites have numerous read-only databases for this kind of data, with one master for updates – replicating out to the read-only replicas. That’s great if you’re using a nice cheap database like mySql (of LAMP), not so nice if you’re running Oracle or MS Sql Server.

Regardless of what you’re doing in your data tier, you’re there. Wouldn’t it be nice to close the loop in the web servers? Even if you are using Apache, that’s going to be less iron, electricity, and cooling all around. That’s what the asynchronous solution is all about – capitalizing on the low cost of memory to save on other things.

The Asynchronous Solution

In the asynchronous solution, we cache username/hashed-password pairs in memory on our web servers, and authenticate against that. Let’s analyse how much memory that takes:

Usernames are usually 12 characters or less, but let’s take an average of 32 to be sure. Using Unicode we get to 64 bytes for the username. Hashed passwords can run between 256 and 512 bits depending on the algorithm, divide by 8 and you have 64 bytes. That’s about 128 bytes altogether. So we can safely cache 8 million of these with 1GB of memory per web server. If you’ve got a million users, first of all, good for you 🙂 Second, that’s just 128 MB of memory – relatively nothing even for a cheap 2GB web server.

Also, consider the fact that when registering a new user we can check if such a username is already taken at the web server level. That doesn’t mean it won’t be checked again in the DB to account for concurrency issues, but that the load on the DB is further reduced. Other things to notice include no read-only replicas and no replication. Simple. Our web servers are the "replicas".

The Authentication Service

What makes it all work is the "Authentication Service" on the app server. This was always there in the synchronous solution. It is what used to field all the login requests from the web servers, and, of course, allowed them to register new users and all the regular stuff. The difference is that now it publishes a message when a new user is registered (or rather, is validated – all a part of the internal long-running workflow). It also allows subscribers to receive the list of all username/hashed-password pairs. It’s also quite likely that it would keep the same data in memory too.

The same message can be used to publish both single updates, and returning the full list when using NServiceBus. Let’s define the message:

And the message that the web server sends when it wants the full list:

And the code that the web server runs on startup looks like this (assuming constructor injection):

And the code that runs in the Authentication Service when the GetAllUsernamesMessage is received:

And the class on the web server that handles a UsernameInUseMessage when it arrives:

When the app server sends the full list, multiple objects of the type UsernameInUseMessage are sent in one physical message to that web server. However, the bus object that runs on the web server dispatches each of these logical messages one at a time to the message handler above.

So, when it comes time to actually authenticate a user, this the web page (or controller, if you’re doing MVC) would call:

When registering a new user, the web server would of course first check its cache, and then send a RegisterUserMessage that contained the username and the hashed password.

When the RegisterUserMessage arrives at the app server, a new long-running workflow is kicked off to handle the process:

That UsernameInUseMessage would eventually arrive at all the web servers subscribed.

Performance/Security Trade-Offs

When looking deeper into this workflow we realize that it could be implemented as two separate message handlers, and have the email address take the place of the workflow Id. The problem with this alternate, better performing solution has to do with security. By removing the dependence on the workflow Id, we’ve in essence stated that we’re willing to receive a UserValidatedMessage without having previously received the RegisterUserMessage.

Since the processing of the UserValidatedMessage is relatively expensive – writing to the DB and publishing messages to all web servers, a malicious user could perform a denial of service (DOS) attack without that many messages, thus flying under the radar of many detection systems. Spoofing a guid that would result in a valid workflow instance is much more difficult. Also, since workflow instances would probably be stored in some in-memory, replicated data grid the relative cost of a lookup would be quite small – small enough to avoid a DOS until a detection system picked it up.

Improved Bandwidth & Latency

The bottom line is that you’re getting much more out of your web tier this way, rather than hammering your data tier and having to scale it out much sooner. Also, notice that there is much less network traffic this way. Not such a big deal for usernames and passwords, but other scenarios built in the same way may need more data. Of course, the time it takes us to log a user in is much shorter as well since we don’t have to cross back and forth from the web server (in the DMZ) to the app server, to the db server.

The important thing to remember in this solution is doing pub/sub. NServiceBus merely provides a simple API for designing the system around pub/sub. And publishing is where you get the serious scalability. As you get more users, you’ll obviously need to get more web servers. The thing is that you probably won’t need more database servers just to handle logins. In this case, you also get lower latency per request since all work needed to be done can be done locally on the server that received the request.

ETags make it even better

For the more advanced crowd, I’ll wrap it up with the ETags. Since web servers do go down, and the cache will be cleared, what we can do is to write that cache to disk (probably in a background thread), and "tag" it with something that the server gave us along with the last UsernameInUseMessage we received. That way, when the web server comes back up, it can send that ETag along with its GetAllUsernamesMessage so that the app server will only send the changes that occurred since. This drives down network usage even more at the insignificant cost of some disk space on the web servers.

And in closing…

Even if you don’t have anything more than a single physical server today, and it acts as your web server and database server, this solution won’t slow things down. If anything, it’ll speed it up. Regardless, you’re much better prepared to scale out than before – no need to rip and replace your entire architecture just as you get 8 million Facebook users banging down your front door.

So, go check out NServiceBus and get the most out of your iron.

I ran into this not so long ago besmirching Microsoft’s lack of “SOA Strategy”.

Straight off the bat, the slant is apparent:

“Microsoft doesn’t enjoy the best reputation in the tech community as what you might call a “team player.” Part of this is the open source community’s doing: They’re vocal and they value open code, which obviously hasn’t been a big priority for Microsoft.”

And the other big players that are buying everything that have an “S” in their name, that is called being a “team player”? Give me a break.

Other nameless bashers are brought to bear as well:

“The problem is, many technologists say Microsoft is being a bit vague and noncommittal about SOA. They wonder if Microsoft even has a real strategy for a service-based architecture.”

Seeing as the industry hasn’t even agreed on what is, or isn’t SOA, I’d say that the industry as a whole is “being a bit vague”. Is it about technology? Business? Both? In what way is it different from Enterprise Architecture? Solutions Architecture? Heck, we at the International Association of Software Architects [IASA] are still trying to help get a handle on just the “A” – Architecture.

Here’s some more:

“I think what’s confusing matters is Microsoft’s inability, thus far, to reconcile SOA’s demands for open code and standards with a business model that’s thrived on proprietary solutions.”

Given that we haven’t agreed what SOA is, stating that it “demands … open code” is something of a stretch. More so when we realize that architecture is primarily independent of technology. Code, open or otherwise, has little impact on whether a given architecture is service-oriented or not.

The piece also points to Eric Roch’s post which brings in the whole ESB debacle. Using yet another ill-defined TLA to bring clarity to SOA, yep, that’s the ticket.

Look, I’ve never been one to applaud Microsoft’s SOA efforts. But then again, I’ve never done that for any vendor or consulting firm. I wouldn’t trust any vendor with defining strategy for my clients, be that SOA strategy or anything else. You wouldn’t expect a vendor to come in, take a look around, and say, “Our products are simply incompatible with the way you do business here, goodbye.” even if that is the reality. So I really don’t understand what all the ruckus is about.

On one of the projects I’m consulting on they needed some special behavior to handle the following scenario:

Since the service needs to perform all request processing in near-real-time, it caches all data from the DB in memory (yes, that’s a lot of memory). Since the service needs to handle multiple requests concurrently, we’re using multiple threads (so far, so good). The problem is that we don’t want the service to handle messages received until it’s finished caching everything. Also, we don’t want that check to show up in every message handler (important when you have lots of message types).

This is actually quite easy to do with NServiceBus. Here’s how:

Have a thread-safe class, let’s call it Loader, for the API to the caching. Something along the lines of:

If (!Loader.HasCachedEverything)
  Loader.CacheEverything();

Obviously, the Loader will have internal logic for checking if it has already started loading things from the DB, so that it won’t do the same thing twice.

OK, now on to the interesting stuff.

We’d like to have the above code run no matter which kind of message we’ve received, so we just write a “generic” message handler – which handles “IMessage” like so:

When the message handler calls “HandleCurrentMessageLater”, the bus puts the current message in the back of the queue. If you’ve configured a transactional transport, this will be safe even in the case of a server restart.

Also, notice the “CanContinue = false”. This tells the bus that the message should not be passed on to any other message handlers, even if there are those that are configured to handle it.

We’ll also package this class up by itself, keeping it separate from the core logic of the service – making it easier to version these cross cutting concerns and the service logic. Let’s put it in “CrossCuttingConcerns.dll”

The final thing needed in order to achieve the behavior described above is to configure this message handler to run before any other handler. This is done in the config file of the process, under the “bus” object, in the “MessageHandlerAssemblies” property like so:

        <property name="MessageHandlerAssemblies">
          <list>
            <value>CrossCuttingConcerns</value>
            <value>ServiceLogic</value>
          </list>
        </property>

This is similar to the way HttpHandlers are (were?) configured in IIS – the order of the handlers defines the order in which the bus dispatches messages to them.

And that’s it.

We’re done.

If you have any questions you’d like to ask about NServiceBus, please feel free to send them my way: Questions@NServiceBus.com.

And just in closing I’d like to say that I don’t necessarily think you should be creating stateful services, but that there’s a time and place for everything.

The good folks from InfoQ interviewed me recently about NServiceBus. You can find the full interview here.

Here are some choice tidbits:

“For any developer who architects using SOA principles, on the basis for using NServiceBus over some other methodology or some other so-called enterprise service bus:”

NServiceBus makes it difficult to work in ways that will hurt your scalability. Since the asynchronous messaging patterns are brought to the fore, developers will, by default, avoid the temporal coupling so prevalent in most Web Service implementations. Other methodologies make so many other options just as easy to use that developers can make mistakes that will hurt their scalability and availability and only find out about those mistakes in production .

Another thing NServiceBus does differently is to totally isolate all workflow code from all technologies. This makes it easy to unit-test workflow classes, making it possible to iterate quickly on the development of key business processes. These portable .NET POJOs give developers the flexibility to host their workflows in whatever runtime they want.

“Having noticed NServiceBus is built with MSMQ in mind or as an option, when asked why this choice of direction: ”

The core of NServiceBus is not dependent on MSMQ. The extensibility points of NServiceBus allow you to plug in your own communication transport, subscription storage, and workflow persister. I’ve already implemented a transport over MSMQ, and another over WCF’s NetTCP binding. Developers can use those as-is or write their own. Many of the SOA products currently out there are much more tied to HTTP, so this is something of a break from the common practice.

I chose to use MSMQ since it is one of the two main Microsoft communications technologies that enables parties to communicate in a disconnected fashion (SQL Server Service Broker is the other). MSMQ has a much more accessible API in that it is directly available as a part of the .NET Framework whereas Service Broker currently isn’t. I consider disconnected communications a critical part of any SOA infrastructure since the Tenet of Service Autonomy does not allow us to assume that the service with which we wish to communicate is currently available.

Is there anything you’d like to know about NServiceBus? Drop me a comment below.

Thanks.