The other day I was at Juval’s presentation where the main message was WCF is a better .NET. In other words, if you use WCF on every one of your classes, you’ll benefit. I don’t know about you, but I’m quite wary of silver bullets – they tend to inflict quite a bit of pain when used indiscriminately. This post is my response to all the people who came up to me at the end of the presentation and wanted to know if I agreed with these far-reaching architectural statements.
First of all let me say that Juval is indeed a master presenter. The “looks like a class, walks like a class, quacks like a class” bit was excellent. I could tell that most people didn’t notice the speedy hands quickly deleting all attributes from the classes before the “looks like a class…” bit. At times, I got flashbacks from the Wizard of Oz – “pay no attention to the man behind the curtain”. If all attributes in WCF only went on the interfaces, then this might actually fly, but we all know that that’s not the case.
One of the interesting comparisons Juval made with WCF was the introduction of .NET. Few people in the audience seemed to remember (or maybe were just professionally younger than .NET’s 8 years), but when it came out .NET was marketed as being mainly about XML Web Services. Juval stated that this was done to play down the fact that .NET made the previous Windows programming technologies obsolete. He then drew the same conclusion about WCF – that it’s as much .NET 3.0 as .NET was the next version of MFC; besides being written in a language that resembles the previous technology, it’s really all different. I don’t think that anyone would argue the difference, but is it really a “plain .NET” killer?
The answer seemed to come around the overhead of WCF – yet Juval deftly deflected that issue with a demo showing WCF doing 200 calls a second. And everybody just bought it – I was shocked. That’s 5ms per call. If you actually take Juval’s advice and use WCF on all your classes, you’ve bought yourself one hell of a performance nightmare. Say you have around 20 of your objects involved in a sequence to handle a user action – not that many actually. With a 5ms lag per object interaction, that user action is going to take 100ms – not including any database or webservice stuff you might be doing. If you do that in a server environment, you’ll be doing roughly 10 concurrent users per core. And that’s not even doing any heavy calculations or anything. Moderately sized systems are running upwards of 1000 concurrent users – if they needed 100 cores (or dozens of servers) for that, I’m guessing that they’d be out of business.
Let’s cut this short – WCF everywhere doesn’t scale, doesn’t perform, isn’t maintainable, or testable either. In other words – don’t do it. I know Juval is a brilliant guy, and an amazing presenter – but I don’t believe he would be employing this with his own clients. This actually bears repeating. WCF is a fine technology for your application’s boundaries, but don’t be pushing it in.
After my recent post on using AOP for smart client development, my partner-in-crime, Arnon, suggested I explain a little bit more on the whole issue of multi-threading in the UI. This isn’t going to be another tired explanation of how you should only update controls on the main thread. This is going to be a post on the challenges multi-threading brings and how to address them.
Multiple Threads – Why?
One of the properties of smart clients is that they should be able to work offline. Sometimes that means an explicit action of taking data and bringing it to the local machine so that the user can work on it, and other times it just has to do with the fact that wireless connectivity can be flaky. More interesting scenarios include the submission of batch jobs and receiving notification on when they complete. The bottom line is that the user should be able to continue doing their interactive work uninterrupted as all this is occurring.
While the user is disconnected, obviously the data they are working on is local – the client is not calling the server to perform the work on its behalf. However, in the flaky connectivity scenario, this happens all the time. In other words, the design should be the same for handling all scenarios. One thread for interacting with the user, and (at least) on other thread for handling the issues of connectivity. The one thing that is clear is that we are going to have data on the client with which the user interacts on one thread, and which the background thread will also be updating as notifications arrive from the server.
Local Data and Multi-Threading – a Recipe for Disaster
When multiple threads are working with the same data, unless some specific code is in that object to make it thread-safe, there’s a good chance that object will end up in an inconsistent state. This can be catastrophic if we’re talking about air traffic control systems, dispensing medication, factory floor automation, etc. The problem is that we can’t just lock down the entire system whenever something needs to be done. While in some cases I’ve seen projects make each object thread-safe, implementing some base class for handling locking, that doesn’t work between objects and results in deadlocks.
Just as an example of the multi-object problem, consider the doctor prescribing medication for a patient. Now, the data about a patient in a smart client is not all on the Patient Object – there are lists of connected test results, medication already being taken, which nurses and doctors have treated this patient so far, etc. As a test result gets pushed to the client from the server, a new doctor unfamiliar with the history of the patient orders the same test again. These two objects under Patient – TestRequest and TestResult are being acted upon by different threads. One of the goals of the system was to eliminate duplicate testing for patients – cited as costing the hospital chain millions of dollars a year. Good thing we addressed the cross-object multi-threaded locking thing properly 🙂
Always Switch from Background to Foreground?
One solution I’ve seen used successfully is to always change threads from background to foreground whenever an update comes in from the server. This works when you don’t have many updates or the work performed on the client as a result of an update doesn’t take long. Always keep in mind, though, the flaky connectivity scenario. What tends to happen is that server notifications bunch up and then hit the client all together. When this occurs, clients developed this way are rendered unusable.
It would seem like this solution isn’t valid because of the above, but don’t dismiss it so quickly. This is an easy solution to implement that may address your specific environment and, as such, be cost effective. A large part of the work I’ve been doing is to make the more complex environments just as easy to develop as these simpler ones.
Infrastructure-level, Safe, Multi-Object Locking
Luckily for us developers, in the .net framework there is a class that handles this for us – well, 2 actually: ContextBoundObject and SynchronizationAttribute. When using these two classes, we can create something known as a Synchronization Domain which acts as a global lock for all objects belonging to the synchronization domain. What this means is that if the user thread is trying to add a test request object while the background thread is already creating a test result object, the user thread will block automatically until the background thread completes its work.
There is only one teensy-weensy problem – ContextBoundObjects are really heavy-weight. The last thing you want is having millions of these running around in your client – you’ll end up with a multi-threading safe unusable UI. Also, the creation of an object inheriting from ContextBoundObject takes quite a bit longer than a plain-old.net object. In other words, technology by itself will not solve our problems – we need some patterns for the correct use of the technology so that we can maintain a reasonable level of performance while taking care of safety.
MVC and Threading – Controllers
Regardless of which flavour of MVC you prefer (I’m in the Supervising Controller camp for smart clients), the logic controlling what goes on in the client is found in the controllers. What this means is that actions from the user as well as background notifications will need to go through these controllers. It is important that these controllers be thread-safe since they are state-full – managing which windows are open, which step in a given process a user is currently doing, etc.
The characteristics of these controller objects which make them best suited to inherit from ContextBoundObject are that there are only a handful of these objects at any point in time and that they are created at startup – they’re singletons (in the “only-one-of-them” sense of the word).
The only special thing that controllers need to do in terms of threading is to dispatch calls to view objects on the foreground thread, even if the thread currently running is the background thread. For example, popping up a “toast” that a test result has arrived when a notification from the server comes in.
These elements – inheriting from ContextBoundObject, use of the SynchronizationAttribute, and thread-switching can be pulled up in to a BaseController class:
using System;
using System.ComponentModel;
using System.Runtime.Remoting.Contexts;
namespace ControllerFramework
{
[Synchronization(SynchronizationAttribute.REQUIRED)]
publicclass BaseController : ContextBoundObject
{
protected ISynchronizeInvoke invoker;
public ISynchronizeInvoke Invoker
{
get { return invoker; }
set { invoker = value; }
}
protectedvoid MarshalToUiThread(Delegate toCall, paramsobject[] parameters)
{
if (this.invoker == null)
return;
if (this.invoker.InvokeRequired)
invoker.BeginInvoke(toCall, parameters);
else
toCall.DynamicInvoke(parameters);
}
}
}
Well, I think that this is long enough for a single blog post. In the next instalment of this series I’ll be talking about how model objects and views fit into the multi-threaded smart client. After that, we’ll be seeing how service agents, messaging, and service contract design need to be done in this style. While all this blogging will be going on, I’ll be getting a software factory up that will tie all these patterns and frameworks together so that all developers will be able to write thread-safe, high-performance smart clients without needing a doctorate in computer science – not that I have one 🙂
It’s one of the things that sometimes drives me mad about the YAGNI philosophy of Agile.
We need to stop throwing out the baby with the bath water.
Jay really liked that statement with relation to my previous post “Scalability – you wish you’re gonna need it“, so I thought I’d put up a logo for this movement. Anyone feeling like joining in, leave a comment, link, or whatever.
I understand that we don’t need to over-engineer everything, putting in every possible kind of extensibility point, so I accept that part of YAGNI. That is not a license to not think about the extensibility points you do need.
<Remarks>
This is a somewhat tongue-in-cheek post, and I do not want the pendulum to swing to far back the other way. But I do think it’s time it took a step back from the over-zealous “we’ll TDD our way there” thinking. Maybe Ron can pull it off. I’ve yet to see anyone else succeed.
I wanted to follow up on my recent post, “In order messaging a myth?” by showing the exact code that solves the issue. I have a podcast waiting to come online that deals with the specifics, so keep your eye out for that too.
The important thing to note is that if we just automatically return the message to the queue, we may get “stuck” with that message if the first PolicyCreatedMessage never arrived. This opens us up to a Denial-of-Service attack by quite simply flooding us with a bunch of messages that never get cleaned up.
Anyway, the general idea is to first try the regular happy path, and only if we see that prerequisite data isn’t available, do we see if another thread may be working on that data. This is done by decreasing the isolation level of our transaction from the regular ReadCommitted to ReadUncommitted. This will enable our thread to see if some other thread inserted the policy in to the Policies table but hasn’t committed its transaction yet.
using (ITransaction tx = s.BeginTransaction(IsolationLevel.ReadCommitted))
{
Policy p = s.Get<Policy>(message.PolicyId);
if (p != null)
{
p.Approve();
tx.Commit();
}
else
policyExists = false;
}
if (!policyExists) // check to make sure
using (ISession s = OpenSession())
using (ITransaction tx = s.BeginTransaction(IsolationLevel.ReadUncommitted))
{
Policy p = s.Get<Policy>(message.PolicyId);
if (p != null) // another thread hasn’t committed its tx yet, so try message again later
this.bus.HandleCurrentMessageLater();
else
this.bus.Return((int)ErrorCodes.PolicyNotFound);
}
}
}
The next step will be how we take this code and make it generic, so that we don’t have write the same code over and over again for the different kinds of message handlers we have.
But that will have to wait until the next installment 🙂
This is a valid question, especially for companies/products just beginning their lifecycle. When the product/web site isn’t bringing in any revenue yet, how much money should we spend on getting it ready for that future success?
The answer to that question lies in treating capacity and scalability differently (source).
What I mean by that is designing for scalability, yet separating out all technological aspects of the scaling from the core solution. That way, you can start with simple, low capacity technologies that won’t be too expensive. As you grow, upgrade that infrastructure and plug it in to your solution. Arnon’s recent post on Tier Splitting touches on a project we worked on together where we designed it in a way that we could scale down to a single process on a single machine and scale out to a server farm, all without changing the core system.
Let me take the design of nServiceBus as an example:
One primary property of scalable systems is the explicit treatment of all IO/communication. This can be seen in the one-way messaging exposed by the Bus object. There is no immediately evident way to do synchronous RPC-style request/response. This design decision is taken up front. However, the way that messages are passed around is abstracted behind an ITransport interface. You can deploy the first version of your system on MSMQ, and as load increases, switch to a more performant solution like RV or MQ, just by changing configuration. WCF does this kind of abstraction as well.
Another important element of the scalability of a system is how workflow instances are persisted. This behaviour is also abstracted behind an interface – IWorkflowPersister. Start out persisting workflows to a database. As you grow, swap that out of a replicated in-memory cache. In any case, the interaction between workflow and messaging at the logical level is set up front. All the pieces of the design are there. Up front. Helping you design your core application in a way that won’t limit your scalability in the future.
This is plain Separation-of-Concerns; code that works with your specific ESB kept out of your business logic.
I got this question the other day from one of my long-time readers Bill about nServiceBus and I thought I’d share:
I have a question around processing of messages in proper order. When leveraging multiple threads to process messages in a message queue, it is possible for the second message in the queue to get processed before the first – especially if the first message is considerably larger than the second. I have taken a lot of care to make sure that messages are sent in the correct order, only to find that the receiving system can process them out of order anyway.
Consider a Policy Created notification, which must come before a Policy Approved notification. If both messages are sitting in the queue when the receiving service starts up, the approval message can be processed before the creation message. How can I make sure that message ordering is respected by the receiving system? I am using WCF/MSMQ as the underlying transport by the way. The only way I have found so far is to limit the receiving service to a single thread, which is by no means desirable.
Well, the solution is really quite simple (at first).
If you’ve received a message that you think has arrived out of order, just call:
this.bus.HandleCurrentMessageLater();
and that will put the message back at the end of the queue.
Once you start considering the fact that you don’t know when the first message is supposed to arrive, you might turn to using a workflow to handle the logic. The workflow would store the policy id, and then allow for N round-trips, before it decided that something bad had happened (like the Policy Created message getting lost), and then it could forward that to an operator, or possibly contact the first system and ask for a replay of the policy created message – or whatever automated fault resolution protocol you like.
In other words, message ordering is probably more trouble than its worth.
If you’ve read my recent post on the threading issues I’ve been dealing with in Smart Client Applications, then you’re probably beginning to get the picture that its fairly complex. To tell you the truth, it is. And up until this point I haven’t been able to find anything that’ll help – and that includes the CAB/SCSF. But yesterday I had my epiphany. The answer was in AOP.
You see, the main problem that I hadn’t been able to solve was that in order for the code to be thread-safe, you had to make sure that no code in the views would/could change entity data. One solution is not to use data-binding, which sucks, but isn’t enough to be sure. Another solution is to have all supervising-controllers clone an entity before they give it to a view. Even if you could possibly code review every line of those classes, the new guy (or old guy who forgot) will, by accident, write one new line of code that could pass an entity to a view without cloning it first. That’s not a very sustainable solution.
This thing has been bothering me for a couple of months now and I hadn’t found a way around it. Until yesterday, like I said. I was talking to somebody about threading stuff, and somehow my unconscience lobbed me this thought about AOP. Now I’m not the sharpest pencil in the pack, but I know to listen when my unconscience “speaks”.
So I set about going over what I knew about AOP – interceptors, advisors, advice, introductions, etc, etc. And then it dawned on me. I could intercept all calls to any object that implemented IView, check the parameters of those calls, and if they implemented IEntity, to clone them before passing them through.
<Homer-style WOOHOO />
The great thing is that developers don’t need to remember to clone entities – it happens automatically. The even greater thing is that this will lead developers to writing the correct kind of interaction between their views and supervising controllers.
Together with nServiceBus, this is going to make the extremely difficult problem of writing thread-safe smart clients possible.
I’ve never made use of AOP in a framework before so I’d like to get the broader community’s feedback on this before incorporating this in production. I’ve spoken with some serious AOP folks who have allayed most of my uncertainties, but I’d like to hear more. Anyway, here’s the proof of concept (that makes use of Spring).
If this turns out to be a viable solution, I think we’ll have a solid environment for building a software factory on top of. That is something that I’m really excited about. In this multi-core future (present) that is upon us, multi-threading on the client is pretty much a necessity. We need a way to get things safe and stable by default without requiring a member of the CLR team to hold our hand.
Anybody who’s interested in helping, drop a comment below.
Last week I was at a client in their test lab and saw a strange bit of behavior. The system could be described as something like an air traffic control system, showing things moving around on a map. For just a second, a fraction of a second, one of the “planes” disappeared from the map and then reappeared again.
When I asked if anybody else saw it, one of the developers said, “Yeah, that happens sometimes – but it fixes itself right after that.”
“What if the user sends a command to the server making use of that location?”, the PM asked. “Could that cause them to collide?”
You could hear a pin drop.
After everyone got passed the preliminary shock, we got down to work. I asked if I could look at the logs, but after more than an hour, I found nothing. No reason to explain the strange behavior. I suggested doing some more instrumentation so that whenever a location changed on the client-side entities, we’d write that to the log.
After that, we ran the system again in the lab under the expected load (several hundred things moving every second, and the user doing the expected activity) and didn’t notice anything. An intern “volunteered” to keep working the system while the rest of us went to lunch. When we came back, he told us that everything seemed to be working OK.
These Heisenbugs are the things that keep me up at night.
“Watching the system changes its behavior”, one of the older devs nodded his head sagely.
Just as we were about to leave the lab another one of the developers gave a shout, “It did it again!”. We quickly stopped the system. Opened the (rather huge) log files and looked for the latest entries.
There it was.
A context switch between setting the latitude and longitude of an entity.
That should not have happened. Not that context switches don’t happen, but rather that it should have been impossible by design. We had made use of synchronization domains and the appropriate patterns so that two threads could never concurrently be working on the same instance of an entity. The synchronization features baked in to nServiceBus had taken care of everything up to that point.
Before getting into the threading solution, I want to address a specific alternate patch that was deployed in the meantime:
The solution for the long/lat problem was simple – just make Location a value object and use a single setter for it rather than one for Latitide and another for Longitude. We were still worried about other bits of data that were correllated in the domain – things that couldn’t be solved the same way.
After getting 3 grizzled C++ veterans in the room, we did a code walkthrough of the threading model of nServiceBus. We went through the nitty gritty details of synchronization domains, how the Bus object was outside of the domain, why that was important for user experience, how the message handlers couldn’t be ContextBoundObjects because of the performance impact of creating and destroying them at a high rate, why they couldn’t just be singletons, why they still had to run in the synchronization domain, so that the UI thread couldn’t work on the same (or related) objects at the same time, etc, etc.
And then it hit me.
The bus was communicated directly with the message handlers.
After the Object Builder created the message handler, the bus dispatched the message to the handler directly. And since the bus was outside the synchronization domain, then the thread calling into the handler wouldn’t have locked the domain, leaving the UI thread open to go in and touch those very same objects.
They say that really understanding the problem is 90% of the solution. I’m hoping to meet them some day, because they’re really smart.
All that we needed to do was have the Object Builder dispatch the message to the handler instead of the bus – since the builder was configured to be in the synchronization domain (on the client side). Something as simple as just adding the method:
We redeployed the system to the lab, ran all the functional, stress, load, etc tests and everything appeared to be stable. The system has been under scrutiny for the past 4 days by batteries of testers instructed specifically to look for those strage kinds of behavior. Other developers are running scripts on the log files looking for other kinds of context switches that may have been missed by the testers. I am happy to report that they haven’t found anything.
Not that this means that the problem isn’t there. We really can’t be sure. However, the PM has decided that we are stable enough to go into pilot mode – deploying into production beside the current system; having users work on both systems at the same time. I’m optimistic.
I’m personally involved in two more production-projects that are making use of nServiceBus in similarly high-end situations and we’ve never had these threading problems – now two years running.
You can now find my talk from TechEd – ARC401 Designing High Performance, Persistent Domain Models – online. You have to log in to the TechEd site first, otherwise the DRM will block the video.
Often during my consulting engagements I run into people who say, "some things just can’t be made asynchronous" even after they agree about the inherent scalability that asynchronous communications pattern bring. One often-cited example is user authentication – taking a username and password combo and authenticating it against some back-end store. For the purpose of this post, I’m going to assume a database. Also, I’m not going to be showing more advanced features like ETags to further improve the solution.
The Setup
Just so that the example is in itself secure, we’ll assume that the password is one-way hashed before being stored. Also, given a reasonable network infrastructure our web servers will be isolated in the DMZ and will have to access some application server which, in turn, will communicate with the DB. There’s also a good chance for something like round-robin load-balancing between web servers, especially for things like user login.
Before diving into the meat of it, I wanted to preface with a few words. One of the commonalities I’ve found when people dismiss asynchrony is that they don’t consider a real deployment environment, or scaling up a solution to multiple servers, farms, or datacenters.
The Synchronous Solution
In the synchronous solution, each one of our web servers will be contacting the app server for each user login request. In other words, the load on the app server and, consequently, on the database server will be proportional to the number of logins. One property of this load is its data locality, or rather, the lack of it. Given that user U logged in, the DB won’t necessarily gain any performance benefits by loading all username/password data into memory for the same page as user U. Another property is that this data is very non-volatile – it doesn’t change that often.
I won’t go to far into the synchronous solution since its been analysed numerous times before. The bottom line is that the database is the bottleneck. You could use sharding solutions. Many of the large sites have numerous read-only databases for this kind of data, with one master for updates – replicating out to the read-only replicas. That’s great if you’re using a nice cheap database like mySql (of LAMP), not so nice if you’re running Oracle or MS Sql Server.
Regardless of what you’re doing in your data tier, you’re there. Wouldn’t it be nice to close the loop in the web servers? Even if you are using Apache, that’s going to be less iron, electricity, and cooling all around. That’s what the asynchronous solution is all about – capitalizing on the low cost of memory to save on other things.
The Asynchronous Solution
In the asynchronous solution, we cache username/hashed-password pairs in memory on our web servers, and authenticate against that. Let’s analyse how much memory that takes:
Usernames are usually 12 characters or less, but let’s take an average of 32 to be sure. Using Unicode we get to 64 bytes for the username. Hashed passwords can run between 256 and 512 bits depending on the algorithm, divide by 8 and you have 64 bytes. That’s about 128 bytes altogether. So we can safely cache 8 million of these with 1GB of memory per web server. If you’ve got a million users, first of all, good for you 🙂 Second, that’s just 128 MB of memory – relatively nothing even for a cheap 2GB web server.
Also, consider the fact that when registering a new user we can check if such a username is already taken at the web server level. That doesn’t mean it won’t be checked again in the DB to account for concurrency issues, but that the load on the DB is further reduced. Other things to notice include no read-only replicas and no replication. Simple. Our web servers are the "replicas".
The Authentication Service
What makes it all work is the "Authentication Service" on the app server. This was always there in the synchronous solution. It is what used to field all the login requests from the web servers, and, of course, allowed them to register new users and all the regular stuff. The difference is that now it publishes a message when a new user is registered (or rather, is validated – all a part of the internal long-running workflow). It also allows subscribers to receive the list of all username/hashed-password pairs. It’s also quite likely that it would keep the same data in memory too.
The same message can be used to publish both single updates, and returning the full list when using NServiceBus. Let’s define the message:
[Serializable] public class UsernameInUseMessage : IMessage { private string username; public string Username { get { return username; } set { username = value; } }
private byte[] hashedPassword; public byte[] HashedPassword { get { return hashedPassword; } set { hashedPassword = value; } } }
And the message that the web server sends when it wants the full list:
[Serializable] public class GetAllUsernamesMessage : IMessage {
}
And the code that the web server runs on startup looks like this (assuming constructor injection):
public class UserAuthenticationServiceAgent { public UserAuthenticationServiceAgent(IBus bus) { this.bus = bus; bus.Subscribe(typeof(UsernameInUseMessage)); bus.Send(new GetAllUsernamesMessages()); }
}
And the code that runs in the Authentication Service when the GetAllUsernamesMessage is received:
public class GetAllUsernamesMessageHandler : BaseMessageHandler<GetAllUsernamesMessage> { public override void Handle(GetAllUsernamesMessage message) { this.Bus.Reply(Cache.GetAll<UsernameInUseMessage>()); } }
And the class on the web server that handles a UsernameInUseMessage when it arrives:
public class UsernameInUseMessageHandler : BaseMessageHandler<UsernameInUseMessage> { public override void Handle(UsernameInUseMessage message) { WebCache.SaveOrUpdate(message.Username, message.HashedPassword); } }
When the app server sends the full list, multiple objects of the type UsernameInUseMessage are sent in one physical message to that web server. However, the bus object that runs on the web server dispatches each of these logical messages one at a time to the message handler above.
So, when it comes time to actually authenticate a user, this the web page (or controller, if you’re doing MVC) would call:
public class UserAuthenticationServiceAgent { public bool Authenticate(string username, string password) { byte[] existingHashedPassword = WebCache[username]; if (existingHashedPassword != null) return existingHashedPassword == this.Hash(password);
return false; } }
When registering a new user, the web server would of course first check its cache, and then send a RegisterUserMessage that contained the username and the hashed password.
[Serializable] [StartsWorkflow] public class RegisterUserMessage : IMessage { private string username; public string Username { get { return username; } set { username = value; } }
private string email; public string Email { get { return email; } set { email = value; } }
private byte[] hashedPassword; public byte[] HashedPassword { get { return hashedPassword; } set { hashedPassword = value; } } }
When the RegisterUserMessage arrives at the app server, a new long-running workflow is kicked off to handle the process:
public class RegisterUserWorkflow : BaseWorkflow<RegisterUserMessage>, IMessageHandler<UserValidatedMessage> { public void Handle(RegisterUserMessage message) { //send validation request to message.Email containing this.Id (a guid) // as a part of the URL }
/// <summary> /// When a user clicks the validation link in the email, the web server /// sends this message (containing the workflow Id) /// </summary> /// <param name="message"></param> public void Handle(UserValidatedMessage message) { // write user to the DB
That UsernameInUseMessage would eventually arrive at all the web servers subscribed.
Performance/Security Trade-Offs
When looking deeper into this workflow we realize that it could be implemented as two separate message handlers, and have the email address take the place of the workflow Id. The problem with this alternate, better performing solution has to do with security. By removing the dependence on the workflow Id, we’ve in essence stated that we’re willing to receive a UserValidatedMessage without having previously received the RegisterUserMessage.
Since the processing of the UserValidatedMessage is relatively expensive – writing to the DB and publishing messages to all web servers, a malicious user could perform a denial of service (DOS) attack without that many messages, thus flying under the radar of many detection systems. Spoofing a guid that would result in a valid workflow instance is much more difficult. Also, since workflow instances would probably be stored in some in-memory, replicated data grid the relative cost of a lookup would be quite small – small enough to avoid a DOS until a detection system picked it up.
Improved Bandwidth & Latency
The bottom line is that you’re getting much more out of your web tier this way, rather than hammering your data tier and having to scale it out much sooner. Also, notice that there is much less network traffic this way. Not such a big deal for usernames and passwords, but other scenarios built in the same way may need more data. Of course, the time it takes us to log a user in is much shorter as well since we don’t have to cross back and forth from the web server (in the DMZ) to the app server, to the db server.
The important thing to remember in this solution is doing pub/sub. NServiceBus merely provides a simple API for designing the system around pub/sub. And publishing is where you get the serious scalability. As you get more users, you’ll obviously need to get more web servers. The thing is that you probably won’t need more database servers just to handle logins. In this case, you also get lower latency per request since all work needed to be done can be done locally on the server that received the request.
ETags make it even better
For the more advanced crowd, I’ll wrap it up with the ETags. Since web servers do go down, and the cache will be cleared, what we can do is to write that cache to disk (probably in a background thread), and "tag" it with something that the server gave us along with the last UsernameInUseMessage we received. That way, when the web server comes back up, it can send that ETag along with its GetAllUsernamesMessage so that the app server will only send the changes that occurred since. This drives down network usage even more at the insignificant cost of some disk space on the web servers.
And in closing…
Even if you don’t have anything more than a single physical server today, and it acts as your web server and database server, this solution won’t slow things down. If anything, it’ll speed it up. Regardless, you’re much better prepared to scale out than before – no need to rip and replace your entire architecture just as you get 8 million Facebook users banging down your front door.
So, go check out NServiceBus and get the most out of your iron.
We brought him on site to give our development staff the 5-day “Advanced Distributed System Design” training. The course profoundly changed our understanding and approach to SOA and distributed systems.
Consider some of the evidence: 1. Months later, developers still make allusions to concepts learned in the course nearly every day 2. One of our developers went home and made her husband (a developer at another company) sign up for the course at a subsequent date/venue 3. Based on what we learned, we’ve made constant improvements to our architecture that have helped us to adapt to our ever changing business domain at scale and speed If you have the opportunity to receive the training, you will make a substantial paradigm shift.
If I were to do the whole thing over again, I’d start the week by playing the clip from the Matrix where Morpheus offers Neo the choice between the red and blue pills. Once you make the intellectual leap, you’ll never look at distributed systems the same way.
Beyond the training, we were able to spend some time with Udi discussing issues unique to our business domain. Because Udi is a rare combination of a big picture thinker and a low level doer, he can quickly hone in on various issues and quickly make good (if not startling) recommendations to help solve tough technical issues.” November 11, 2010
Ian Robinson, Principal Consultant at ThoughtWorks
"Your blog and articles have been enormously useful in shaping, testing and refining my own approach to delivering on SOA initiatives over the last few years. Over and against a certain 3-layer-application-architecture-blown-out-to- distributed-proportions school of SOA, your writing, steers a far more valuable course."
Shy Cohen, Senior Program Manager at Microsoft
“Udi is a world renowned software architect and speaker. I met Udi at a conference that we were both speaking at, and immediately recognized his keen insight and razor-sharp intellect. Our shared passion for SOA and the advancement of its practice launched a discussion that lasted into the small hours of the night. It was evident through that discussion that Udi is one of the most knowledgeable people in the SOA space. It was also clear why – Udi does not settle for mediocrity, and seeks to fully understand (or define) the logic and principles behind things. Humble yet uncompromising, Udi is a pleasure to interact with.”
Glenn Block, Senior Program Manager - WCF at Microsoft
“I have known Udi for many years having attended his workshops and having several personal interactions including working with him when we were building our Composite Application Guidance in patterns & practices. What impresses me about Udi is his deep insight into how to address business problems through sound architecture. Backed by many years of building mission critical real world distributed systems it is no wonder that Udi is the best at what he does. When customers have deep issues with their system design, I point them Udi's way.”
Karl Wannenmacher, Senior Lead Expert at Frequentis AG
“I have been following Udi’s blog and podcasts since 2007. I’m convinced that he is one of the most knowledgeable and experienced people in the field of SOA, EDA and large scale systems.
Udi helped Frequentis to design a major subsystem of a large mission critical system with a nationwide deployment based on NServiceBus. It was impressive to see how he took the initial architecture and turned it upside down leading to a very flexible and scalable yet simple system without knowing the details of the business domain.
I highly recommend consulting with Udi when it comes to large scale mission critical systems in any domain.”
Simon Segal, Independent Consultant
“Udi is one of the outstanding software development minds in the world today, his vast insights into Service Oriented Architectures and Smart Clients in particular are indeed a rare commodity. Udi is also an exceptional teacher and can help lead teams to fall into the pit of success. I would recommend Udi to anyone considering some Architecural guidance and support in their next project.”
Ohad Israeli, Chief Architect at Hewlett-Packard, Indigo Division
“When you need a man to do the job Udi is your man! No matter if you are facing near deadline deadlock or at the early stages of your development, if you have a problem Udi is the one who will probably be able to solve it, with his large experience at the industry and his widely horizons of thinking , he is always full of just in place great architectural ideas.
I am honored to have Udi as a colleague and a friend (plus having his cell phone on my speed dial).”
Ward Bell, VP Product Development at IdeaBlade
“Everyone will tell you how smart and knowledgable Udi is ... and they are oh-so-right. Let me add that Udi is a smart LISTENER. He's always calibrating what he has to offer with your needs and your experience ... looking for the fit. He has strongly held views ... and the ability to temper them with the nuances of the situation. I trust Udi to tell me what I need to hear, even if I don't want to hear it, ... in a way that I can hear it. That's a rare skill to go along with his command and intelligence.”
Eli Brin, Program Manager at RISCO Group
“We hired Udi as a SOA specialist for a large scale project. The development is outsourced to India. SOA is a buzzword used almost for anything today. We wanted to understand what SOA really is, and what is the meaning and practice to develop a SOA based system.
We identified Udi as the one that can put some sense and order in our minds. We started with a private customized SOA training for the entire team in Israel. After that I had several focused sessions regarding our architecture and design.
I will summarize it simply (as he is the software simplist): We are very happy to have Udi in our project. It has a great benefit. We feel good and assured with the knowledge and practice he brings. He doesn’t talk over our heads. We assimilated nServicebus as the ESB of the project. I highly recommend you to bring Udi into your project.”
Catherine Hole, Senior Project Manager at the Norwegian Health Network
“My colleagues and I have spent five interesting days with Udi - diving into the many aspects of SOA. Udi has shown impressive abilities of understanding organizational challenges, and has brought the business perspective into our way of looking at services. He has an excellent understanding of the many layers from business at the top to the technical infrstructure at the bottom. He is a great listener, and manages to simplify challenges in a way that is understandable both for developers and CEOs, and all the specialists in between.”
Yoel Arnon, MSMQ Expert
“Udi has a unique, in depth understanding of service oriented architecture and how it should be used in the real world, combined with excellent presentation skills. I think Udi should be a premier choice for a consultant or architect of distributed systems.”
Vadim Mesonzhnik, Development Project Lead at Polycom
“When we were faced with a task of creating a high performance server for a video-tele conferencing domain we decided to opt for a stateless cluster with SQL server approach. In order to confirm our decision we invited Udi.
After carefully listening for 2 hours he said: "With your kind of high availability and performance requirements you don’t want to go with stateless architecture."
One simple sentence saved us from implementing a wrong product and finding that out after years of development. No matter whether our former decisions were confirmed or altered, it gave us great confidence to move forward relying on the experience, industry best-practices and time-proven techniques that Udi shared with us.
It was a distinct pleasure and a unique opportunity to learn from someone who is among the best at what he does.”
Jack Van Hoof, Enterprise Integration Architect at Dutch Railways
“Udi is a respected visionary on SOA and EDA, whose opinion I most of the time (if not always) highly agree with. The nice thing about Udi is that he is able to explain architectural concepts in terms of practical code-level examples.”
Neil Robbins, Applications Architect at Brit Insurance
“Having followed Udi's blog and other writings for a number of years I attended Udi's two day course on 'Loosely Coupled Messaging with NServiceBus' at SkillsMatter, London.
I would strongly recommend this course to anyone with an interest in how to develop IT systems which provide immediate and future fitness for purpose. An influential and innovative thought leader and practitioner in his field, Udi demonstrates and shares a phenomenally in depth knowledge that proves his position as one of the premier experts in his field globally.
The course has enhanced my knowledge and skills in ways that I am able to immediately apply to provide benefits to my employer. Additionally though I will be able to build upon what I learned in my 2 days with Udi and have no doubt that it will only enhance my future career.
I cannot recommend Udi, and his courses, highly enough.”
Nick Malik, Enterprise Architect at Microsoft Corporation
“You are an excellent speaker and trainer, Udi, and I've had the fortunate experience of having attended one of your presentations. I believe that you are a knowledgable and intelligent man.”
He accompanied us in all stages of our development cycle and helped us put vision into real life distributed scalable software. He brought fresh thinking, great in depth of understanding software, and ongoing support that proved as valuable and cost effective.
Udi has the unique ability to analyze the business problem and come up with a simple and elegant solution for the code and the business alike. With Udi's attention to details, and knowledge we avoided pit falls that would cost us dearly.”
Børge Hansen, Architect Advisor at Microsoft
“Udi delivered a 5 hour long workshop on SOA for aspiring architects in Norway. While keeping everyone awake and excited Udi gave us some great insights and really delivered on making complex software challenges simple. Truly the software simplist.”
Motty Cohen, SW Manager at KorenTec Technologies
“I know Udi very well from our mutual work at KorenTec. During the analysis and design of a complex, distributed C4I system - where the basic concepts of NServiceBus start to emerge - I gained a lot of "Udi's hours" so I can surely say that he is a professional, skilled architect with fresh ideas and unique perspective for solving complex architecture challenges. His ideas, concepts and parts of the artifacts are the basis of several state-of-the-art C4I systems that I was involved in their architecture design.”
We’d been meaning to delve into messaging at Eleutian after multiple discussions with and blog posts from Greg Young and Udi Dahan in the past. We weren’t entirely sure where to start, how to start, what tools to use, how to use them, etc. Being able to sit in a room with Udi for an entire week while he described exactly how, why and what he does to tackle a massive enterprise system was invaluable to say the least.
We now have a much better direction and, more importantly, have the confidence we need to start introducing these powerful concepts into production at Eleutian.”
Gad Rosenthal, Department Manager at Retalix
“A thinking person. Brought fresh and valuable ideas that helped us in architecting our product. When recommending a solution he supports it with evidence and detail so you can successfully act based on it. Udi's support "comes on all levels" - As the solution architect through to the detailed class design. Trustworthy!”
Chris Bilson, Developer at Russell Investment Group
“I had the pleasure of attending a workshop Udi led at the Seattle ALT.NET conference in February 2009. I have been reading Udi's articles and listening to his podcasts for a long time and have always looked to him as a source of advice on software architecture. When I actually met him and talked to him I was even more impressed. Not only is Udi an extremely likable person, he's got that rare gift of being able to explain complex concepts and ideas in a way that is easy to understand. All the attendees of the workshop greatly appreciate the time he spent with us and the amazing insights into service oriented architecture he shared with us.”
Alexey Shestialtynov, Senior .Net Developer at Candidate Manager
“I met Udi at Candidate Manager where he was brought in part-time as a consultant to help the company make its flagship product more scalable. For me, even after 30 years in software development, working with Udi was a great learning experience. I simply love his fresh ideas and architecture insights. As we all know it is not enough to be armed with best tools and technologies to be successful in software - there is still human factor involved. When, as it happens, the project got in trouble, management asked Udi to step into a leadership role and bring it back on track. This he did in the span of a month. I can only wish that things had been done this way from the very beginning. I look forward to working with Udi again in the future.”
Christopher Bennage, President at Blue Spire Consulting, Inc.
“My company was hired to be the primary development team for a large scale and highly distributed application. Since these are not necessarily everyday requirements, we wanted to bring in some additional expertise. We chose Udi because of his blogging, podcasting, and speaking. We asked him to to review our architectural strategy as well as the overall viability of project.
I was very impressed, as Udi demonstrated a broad understanding of the sorts of problems we would face. His advice was honest and unbiased and very pragmatic. Whenever I questioned him on particular points, he was able to backup his opinion with real life examples.
I was also impressed with his clarity and precision. He was very careful to untangle the meaning of words that might be overloaded or otherwise confusing. While Udi's hourly rate may not be the cheapest, the ROI is undoubtedly a deal.
I would highly recommend consulting with Udi.”
Robert Lewkovich, Product / Development Manager at Eggs Overnight
“Udi's advice and consulting were a huge time saver for the project I'm responsible for. The $ spent were well worth it and provided me with a more complete understanding of nServiceBus and most importantly in helping make the correct architectural decisions earlier thereby reducing later, and more expensive, rework.”
The class was very well put together. The materials were clear and concise and Udi did a fantastic job presenting it. It was a good mixture of lecture, coding, and question and answer. I fully expected that I would be taking notes like crazy, but it was so well laid out that the only thing I wrote down the entire course was what I wanted for lunch. Udi provided us with all the lecture materials and everyone has access to all of the samples which are in the nServiceBus trunk.
Now I know why Udi is the "Software Simplist." I was amazed to find that all the code and solutions were indeed very simple. The patterns that Udi presented keep things simple by isolating complexity so that it doesn't creep into your day to day code. The domain code looks the same if it's running in a single process or if it's running in 100 processes.”
Ian Cooper, Team Lead at Beazley
“Udi is one of the leaders in the .Net development community, one of the truly smart guys who do not just get best architectural practice well enough to educate others but drives innovation. Udi consistently challenges my thinking in ways that make me better at what I do.”
Liron Levy, Team Leader at Rafael
“I've met Udi when I worked as a team leader in Rafael. One of the most senior managers there knew Udi because he was doing superb architecture job in another Rafael project and he recommended bringing him on board to help the project I was leading. Udi brought with him fresh solutions and invaluable deep architecture insights. He is an authority on SOA (service oriented architecture) and this was a tremendous help in our project. On the personal level - Udi is a great communicator and can persuade even the most difficult audiences (I was part of such an audience myself..) by bringing sound explanations that draw on his extensive knowledge in the software business. Working with Udi was a great learning experience for me, and I'll be happy to work with him again in the future.”
Adam Dymitruk, Director of IT at Apara Systems
“I met Udi for the first time at DevTeach in Montreal back in early 2007. While Udi is usually involved in SOA subjects, his knowledge spans all of a software development company's concerns. I would not hesitate to recommend Udi for any company that needs excellent leadership, mentoring, problem solving, application of patterns, implementation of methodologies and straight out solution development. There are very few people in the world that are as dedicated to their craft as Udi is to his. At ALT.NET Seattle, Udi explained many core ideas about SOA. The team that I brought with me found his workshop and other talks the highlight of the event and provided the most value to us and our organization. I am thrilled to have the opportunity to recommend him.”
Eytan Michaeli, CTO Korentec
“Udi was responsible for a major project in the company, and as a chief architect designed a complex multi server C4I system with many innovations and excellent performance.”
Carl Kenne, .Net Consultant at Dotway AB
“Udi's session "DDD in Enterprise apps" was truly an eye opener. Udi has a great ability to explain complex enterprise designs in a very comprehensive and inspiring way. I've seen several sessions on both DDD and SOA in the past, but Udi puts it in a completly new perspective and makes us understand what it's all really about. If you ever have a chance to see any of Udi's sessions in the future, take it!”
Avi Nehama, R&D Project Manager at Retalix
“Not only that Udi is a briliant software architecture consultant, he also has remarkable abilities to present complex ideas in a simple and concise manner, and... always with a smile. Udi is indeed a top-league professional!”
Ben Scheirman, Lead Developer at CenterPoint Energy
“Udi is one of those rare people who not only deeply understands SOA and domain driven design, but also eloquently conveys that in an easy to grasp way. He is patient, polite, and easy to talk to. I'm extremely glad I came to his workshop on SOA.”
Scott C. Reynolds, Director of Software Engineering at CBLPath
“Udi is consistently advancing the state of thought in software architecture, service orientation, and domain modeling.
His mastery of the technologies and techniques is second to none, but he pairs that with a singular ability to listen and communicate effectively with all parties, technical and non, to help people arrive at context-appropriate solutions.
Every time I have worked with Udi, or attended a talk of his, or just had a conversation with him I have come away from it enriched with new understanding about the ideas discussed.”
Evgeny-Hen Osipow, Head of R&D at PCLine
“Udi has helped PCLine on projects by implementing architectural blueprints demonstrating the value of simple design and code.”
Rhys Campbell, Owner at Artemis West
“For many years I have been following the works of Udi. His explanation of often complex design and architectural concepts are so cleanly broken down that even the most junior of architects can begin to understand these concepts. These concepts however tend to typify the "real world" problems we face daily so even the most experienced software expert will find himself in an "Aha!" moment when following Udi teachings.
It was a pleasure to finally meet Udi in Seattle Alt.Net OpenSpaces 2008, where I was pleasantly surprised at how down-to-earth and approachable he was. His depth and breadth of software knowledge also became apparent when discussion with his peers quickly dove deep in to the problems we current face. If given the opportunity to work with or recommend Udi I would quickly take that chance. When I think .Net Architecture, I think Udi.”
Sverre Hundeide, Senior Consultant at Objectware
“Udi had been hired to present the third LEAP master class in Oslo. He is an well known international expert on enterprise software architecture and design, and is the author of the open source messaging framework nServiceBus.
The entire class was based on discussion and interaction with the audience, and the only Power Point slide used was the one showing the agenda.
He started out with sketching a naive traditional n-tier application (big ball of mud), and based on suggestions from the audience we explored different solutions which might improve the solution. Whatever suggestions we threw at him, he always had a thoroughly considered answer describing pros and cons with the suggested solution. He obviously has a lot of experience with real world enterprise SOA applications.”
Raphaël Wouters, Owner/Managing Partner at Medinternals
“I attended Udi's excellent course 'Advanced Distributed System Design with SOA and DDD' at Skillsmatter. Few people can truly claim such a high skill and expertise level, present it using a pragmatic, concrete no-nonsense approach and still stay reachable.”
Nimrod Peleg, Lab Engineer at Technion IIT
“One of the best programmers and software engineer I've ever met, creative, knows how to design and implemet, very collaborative and finally - the applications he designed implemeted work for many years without any problems!”
Jose Manuel Beas
“When I attended Udi's SOA Workshop, then it suddenly changed my view of what Service Oriented Architectures were all about. Udi explained complex concepts very clearly and created a very productive discussion environment where all the attendees could learn a lot. I strongly recommend hiring Udi.”
Daniel Jin, Senior Lead Developer at PJM Interconnection
“Udi is one of the top SOA guru in the .NET space. He is always eager to help others by sharing his knowledge and experiences. His blog articles often offer deep insights and is a invaluable resource. I highly recommend him.”
Pasi Taive, Chief Architect at Tieto
“I attended both of Udi's "UI Composition Key to SOA Success" and "DDD in Enterprise Apps" sessions and they were exceptionally good. I will definitely participate in his sessions again. Udi is a great presenter and has the ability to explain complex issues in a manner that everyone understands.”
Eran Sagi, Software Architect at HP
“So far, I heard about Service Oriented architecture all over.
Everyone mentions it – the big buzz word.
But, when I actually asked someone for what does it really mean, no one managed to give me a complete satisfied answer.
Finally in his excellent course “Advanced Distributed Systems”, I got the answers I was looking for.
Udi went over the different motivations (principles) of Services Oriented, explained them well one by one, and showed how each one could be technically addressed using NService bus.
In his course, Udi also explain the way of thinking when coming to design a Service Oriented system.
What are the questions you need to ask yourself in order to shape your system, place the logic in the right places for best Service Oriented system.
I would recommend this course for any architect or developer who deals with distributed system, but not only.
In my work we do not have a real distributed system, but one PC which host both the UI application and the different services inside, all communicating via WCF.
I found that many of the architecture principles and motivations of SOA apply for our system as well. Enough that you have SW partitioned into components and most of the principles becomes relevant to you as well.
Bottom line – an excellent course recommended to any SW Architect, or any developer dealing with distributed system.”