With all the talk of CQRS, the area that doesn’t get enough treatment (in my opinion) is that of queries. Many are already beginning to understand the importance of task-based UIs and how that aligns to the underlying commands being sent, validated, and processed in the system as well as the benefits of messaging-centric infrastructure (like NServiceBus) for handling those commands reliably. When it comes to queries, though, it isn’t nearly as well understood what it means for a query to be “task based”.

Starting with CRUD

Let’s start with a traditional CRUD application and work our way out from there.

In these environments, we often see users asking us to build “excel-like” screens that allow them to view a set of data as well as sort, filter, and group that data along various axes. While we might not get this requirement right away, after some time users begin to ask us to allow them to “save” a certain “query” that they have set up, providing it some kind of name.

That, right there, is a task-based query and it is the beginning of deeper domain insight.

Pattern matching

Any time a user is repeatedly running the same query (this can be once a day or some other unit of time) there is some scenario that the business is trying to identify and is using that user as a pattern-matching engine to see if the data indicates that that scenario has occurred.

It’s quite common for us to get a requirement to add some field (often a boolean or enum) to an entity which defaults to some value and then see that same field used in filtering other queries. These measures are sometimes instituted as a temporary stop-gap while a larger feature is being implemented, though (as the saying goes) there is nothing more permanent than a temporary solution.

Where we developers go wrong

The thing is, many developers don’t notice these sorts of things happening because we don’t actually look at the kinds of queries users are running.

One excellent technique to better understand a domain is to sit down with your users while they’re working and ask them, “what made you run that query just now?”, “why that specific set of filters?”.

What I’ve noticed over the years is that our users find very creative ways to achieve their business objectives despite the limitations of the system that they’re working with. We developers ultimately see these as requirements, but they are better interpreted as workarounds.

I’ll talk some more about how a software development organization should deal with these workarounds in a future post, but I want to focus back in on the queries for now.

Oh, and don’t get me started on caching or NoSQL, not that I think that those tools don’t provide value – they do, but they’re only relevant once you know which business problem you’re solving and why.

Not all queries are created equal

Even before bringing up the questions I described in the previous section, any time you get query-centric requirements the first question to ask is “how often will the user be running this specific query?”.

If the answer is that the specific query will be run periodically (every day, week, etc), then drill deeper to see what pattern the user will be looking for in the data. If the person you’re talking to doesn’t know to answer that question, then go find someone who does. Every periodic query I’ve seen has some pattern behind it – and in my conversations with thousands of other developers over the years, I’ve seen that this is not just my personal experience.

But there is a case where a query does get run repeatedly without there being a pattern behind it.

I know this sounds like I’m contradicting myself, but the distinction is the word “specific” that I emphasized above.

There are certain users who behave very differently from other users – these users are often doing what I call research, i.e. the “I don’t know what I’m looking for but I’ll know it when I see it” people.

These researchers tend to repeatedly query the data in the system however they tend to run different queries all the time. This is the reason why traditional data warehouse type solutions don’t tend to work well for them. Data warehouses are optimized for running specific queries repeatedly.

Keeping the Single-Responsibility Principle in mind – we should not try to create a single query mechanism that will address these two very different and independently evolving needs.

And now on to Search

Search is a feature that is needed in many systems and whose complexity is greatly underestimated.

While the developer community has taken some decent strides in understanding that search needs to be treated differently from other queries, the common Lucene/Solr solutions that are applied are often overwhelmed by the size of the data set on which the business operates.

The problem is compounded by our user population being spoiled by Google – that simple little text box and voila, exactly what you’re looking for magically appears instantaneously. They don’t understand (or care) how much engineering effort went into making that “just work”.

Lucene and Solr work well when your data set isn’t too large, and then they become pretty useless as the quality of their results degrades. The thing is that many of us in IT tend to work on projects where we have an unrealistically small data set that we use to test the system and, at these volumes, it looks like our solutions work great. But if you have 20 million customers, do you think a full text search on “Smith” is going to find just the right one?

Larger data sets require a relevance engine – something that feeds off of what users do AFTER the query to influence the results of future queries. Did the user page to the next screen? That needs to be fed back in. Did they click on one of the results? That needs to be fed back in too. Did they go back to the search and do another similar search right after looking at a result – that should possibly undo the previous feedback.

And that’s just relevance for beginners.

You know what makes Google, you know, Google? It’s that they have this absolutely massive data set of what users do after the query that informs which results they return when. You probably don’t have that. That and search is/was their main business for many years – I’m betting that it’s not your main business.

You should discuss this with your stakeholders the next time they ask for search functionality in your system.

In closing

I know that the common CQRS talking points tell you to keep your queries simple, but that doesn’t mean that simple is easy.

It takes a fair bit of domain understanding to figure out what the queries in the system are supposed to be – what tasks users are trying to achieve through these queries. And even when you do reach this understanding, convincing various business stakeholders to change the design of the UI to reflect these insights is far from easy.

It often seems like the reasonable solution to give our users everything, to not limit them in any way, and then they’ll be able to do anything. What ends up happening is that our users end up drowning in a sea of data, unable to see the forest for the trees, ultimately resulting in the company not noticing important trends quickly enough (or at all) and therefore making poor business decisions.

Even if your company doesn’t believe itself to be in “Big Data” territory, I’d suggest talking with the people on the “front lines” just in case. Many of them will report feeling overwhelmed by the quantity of stuff (to use the correct scientific term) they need to deal with.

It’s not about Lucene, Solr, OData, SSRS, or any other technology.

It’s on you. Go get ’em.

Feeling a little bit rant-y today, as I just saw some more abuse of remote calls, this time on the Java side of things.

JPA is the Java Persistence API – a kind of ORM, as you’d expect. Luckily, a lot of the web services stuff was already on the way out by the time that EclipseLink DBWS came out. DBWS allowed you to expose database artifacts as web services.

I mean, it’s not like we have any other interoperable ways of accessing data, right?

Anyway, like I said, that didn’t take off, but now they’re reinventing it – this time with REST!

In case you had any doubts, REST is pure awesomeness and adding it to anything else makes it awesome too. Lest anybody take this out of context (it’s happened before), I’m being sarcastic.

Here it is.

God knows they couldn’t let Microsoft totally dominate this area with OData coming out quite some time ago. In case you were wondering, OData was designed to provide standard CRUD access of a data source over HTTP.

Of course, none of these support any transactions so if you actually wanted to do some meaningful business logic on top of this CRUD, you wouldn’t have any consistency. And, let’s face it, if you’re not doing any meaningful business logic, just basic persistence, you just do it. That problem’s been solved a long time ago.

Can we please stop reinventing SQL already?

One of the things that surprises some developers that I talk to is that you don’t always get consistency even with end-to-end synchronous communication and a single database. This goes beyond things like isolation levels that some developers are aware of and is particularly significant in multi-user collaborative domains.

The problem

Let’s start with an image to describe the scenario:

The main issue we have here is that the values transaction 2 gets for A and B are those from T0 – before either transaction 1 or 3 completed. The reason this is an issue is that these old values (usually together with some message data) are used to calculate what the new state of C should be.

Traditional optimistic concurrency techniques won’t detect any problem if we don’t touch A or B in transaction 2.

In short, systems today are causing inconsistency.

Some solutions

1. Don’t have transactions which operate on multiple entities (which probably isn’t possible for some of your most important business logic).

2. Turn on multi-version concurrency control – this is called snapshot isolation in MS Sql Server.

Yes, you need to turn it on. It’s off by default.

The good news is that this will stop the writing of inconsistent data to your database.
The bad news is that it will probably cause your system many more exceptions when going to persist.

For those of you who are using transaction messaging with automatic retrying, this will end up as “just” a performance problem (unless you follow the recommendations below). For those of you who are using regular web/wcf services (over tcp/http), you’re “cross cutting” exception management will likely end up discarding all the data submitted in those requests (but since that’s what you’re doing when you run into deadlocks this shouldn’t be news to you).

The solution to the performance issues

Eventual consistency.

Funny isn’t it – all those people who were afraid of eventual consistency got inconsistency instead.

Also, it’s not enough to just have eventual consistency (like between the command and query sides of CQRS). You need to drastically decrease the size of your entities. And the best way of doing that is to partition those entities across multiple business services (also known in DDD lingo as Bounded Contexts) each with its own database.

This is yet another reason why I say that CQRS shouldn’t be the top level architectural breakdown. Very useful within a given business service, yes – though sometimes as small as just some sagas.

Next steps

It may seem unusual that the title of this post implies that SOA is the solution, yet the content clearly states that traditional HTTP-based web services are a problem. Even REST wouldn’t change matters as it doesn’t influence how transactions are managed against a database.

The SOA solution I’m talking about here is the one I’ve spent the last several years blogging about. It’s a different style of SOA which has services stretch up to contain parts of the UI as well as down to contain parts of the database, resulting in a composite UI and multiple databases. This is a drastically different approach than much of the literature on the topic – especially Thomas Erl’s books.

Unfortunately there isn’t a book out there with all of this in it (that I’ve found), and I’m afraid that with my schedule (and family) writing a book is pretty much out of the question. Let’s face it – I’m barely finding time to blog.

The one thing I’m trying to do more of is provide training on these topics. I’ve just finished a course in London, doing another this week in Aarhus Denmark, and another next month in San Francisco (which is now sold out). The next openings this year will be in Stockholm, London; Sydney Australia and Austin Texas will be coming in January of next year. I’ll be coming over to the US more next year so if you missed San Francisco, keep an eye out.

I wish there was more I could do, but I’m only one guy.

Hmm, maybe it’s time to change that.

One of the things I cover early on in my course is the problem with traditional layered architecture driving people to create a business logic layer made up of a bunch of inter-related entities. I see this happening a lot, even though nowadays people are calling that bunch of inter-related entities a “domain model”.

Let me just say this upfront – most inter-related entity models are NOT a domain model.
Here’s why: most transactions don’t respect entity boundaries.

That being said, you don’t always need a domain model.
The domain model pattern’s context is “if you have complicated and everchanging business rules” – right there on page 119 of Patterns of Enterprise Application Architecture.

Persisting the customer’s first name, last name, and middle initial – and later reading and showing that data does not sound either complicated or that it is really going to change that much.

Then there are things like credit limits, that may be on the customer entity as well. It is likely that there are business requirements that expect that value to be consistent with the total value of unpaid orders – data that comes from other entities.

The problem that is created is one of throughput.

Since databases lock an entire row/entity at a time, if one transaction is changing the customer’s first name, the database would block another transaction that tried to change the same customer’s credit limit.

The bigger your entities, the more transactions will likely need to operate on them in parallel, the slower your system will get as the number of transactions increases. This feeds back in on itself as often those blocked transactions will have operated already on some other entity, leaving those locked for longer periods of times, blocking even more transactions.

And the absurd thing is that the business never demanded that the customer’s first name be consistent with the credit limit.

What if we didn’t have a single Customer entity?

What if we had one that contained first name, last name, middle initial and another that contained things like credit limit, status, and risk rating. These entities would be correlated by the same ID, but could be stored in separate tables in the database. That would do away with much of the cascading locking effects drastically improving our throughput as load increases.

And you know what? That division would still respect the 3rd normal form.

Which of these entities do you think would be classified by the business under the “complicated and everchanging rules” category?

And for those entities that are just about data persistence – do you think it’s justified to use 3 tiers? Do we really need a view model which we transform to data transfer objects which we transform to domain objects which we transform to relational tables and then all the way back? Wouldn’t some simpler 2-tier programming suffice – dare I say datasets? Ruby on Rails?

Are we ready to leave behind the assumption that all elements of a given layer must be built the same way?

So, I’ve gotten back from a most enjoyable couple of days in Sweden where I gave two half-day tutorials, the first being the SOA and UI composition talk I gave at the European Virtual ALT.NET meeting (which you can find online here) and the other on DDD in enterprise apps (the first time I’ve done this talk).

I’ve gotten some questions about my DDD presentation there based on Aaron Jensen’s pictures:

Yes – I talk with my hands. All the time.

That slide is quite an important one – I talked about it for at least 2 hours.

Here it is again, this time in full:

You may notice that the nice clean layered abstraction that the industry has gotten so comfortable with doesn’t quite sit right when looking at it from this perspective. The reason for that is that this perspective takes into account physical distribution while layers don’t.

I’ll have some more posts on this topic as well as giving a session in TechEd Europe this November.

Oh – and please do feel free to already send your questions in.

After reading Ayende’s post advocating against “soft deletes” I felt that I should add a bit more to the topic as there were some important business semantics missing. As developers discuss the pertinence of using an IsDeleted column in the database to mark deletion, and the way this relates to reporting and auditing concerns is weighed, the core domain concepts rarely get a mention. Let’s first understand the business scenarios we’re modeling, the why behind them, before delving into the how of implementation.

The real world doesn’t cascade

Let’s say our marketing department decides to delete an item from the catalog. Should all previous orders containing that item just disappear? And cascading farther, should all invoices for those orders be deleted as well? Going on, would we have to redo the company’s profit and loss statements?

Heaven forbid.

So, is Ayende wrong? Do we really need soft deletes after all?

On the one hand, we don’t want to leave our database in an inconsistent state with invoices pointing to non-existent orders, but on the other hand, our users did ask us to delete an entity.

Or did they?

When all you have is a hammer…

We’ve been exposing users to entity-based interfaces with “create, read, update, delete” semantics in them for so long that they have started presenting us requirements using that same language, even though it’s an extremely poor fit.

Instead of accepting “delete” as a normal user action, let’s go into why users “delete” stuff, and what they actually intend to do.

The guys in marketing can’t actually make all physical instances of a product disappear – nor would they want to. In talking with these users, we might discover that their intent is quite different:

“What I mean by ‘delete’ is that the product should be discontinued. We don’t want to sell this line of product anymore. We want to get rid of the inventory we have, but not order any more from our supplier. The product shouldn’t appear any more when customers do a product search or category listing, but the guys in the warehouse will still need to manage these items in the interim. It’s much shorter to just say ‘delete’ though.”

There seem to be quite a few interesting business rules and processes there, but nothing that looks like it could be solved by a single database column.

Model the task, not the data

Looking back at the story our friend from marketing told us, his intent is to discontinue the product – not to delete it in any technical sense of the word. As such, we probably should provide a more explicit representation of this task in the user interface than just selecting a row in some grid and clicking the ‘delete’ button (and “Are you sure?” isn’t it).

As we broaden our perspective to more parts of the system, we see this same pattern repeating:

Orders aren’t deleted – they’re cancelled. There may also be fees incurred if the order is canceled too late.

Employees aren’t deleted – they’re fired (or possibly retired). A compensation package often needs to be handled.

Jobs aren’t deleted – they’re filled (or their requisition is revoked).

In all cases, the thing we should focus on is the task the user wishes to perform, rather than on the technical action to be performed on one entity or another. In almost all cases, more than one entity needs to be considered.

Statuses

In all the examples above, what we see is a replacement of the technical action ‘delete’ with a relevant business action. At the entity level, instead of having a (hidden) technical WasDeleted status, we see an explicit business status that users need to be aware of.

The manager of the warehouse needs to know that a product is discontinued so that they don’t order any more stock from the supplier. In today’s world of retail with Vendor Managed Inventory, this often happens together with a modification to an agreement with the vendor, or possibly a cancellation of that agreement.

This isn’t just a case of transactional or reporting boundaries – users in different contexts need to see different things at different times as the status changes to reflect the entity’s place in the business lifecycle. Customers shouldn’t see discontinued products at all. Warehouse workers should, that is, until the corresponding Stock Keeping Unit (SKU) has been revoked (another status) after we’ve sold all the inventory we wanted (and maybe returned the rest back to the supplier).

Rules and Validation

When looking at the world through over-simplified-delete-glasses, we may consider the logic dictating when we can delete to be quite simple: do some role-based-security checks, check that the entity exists, delete. Piece of cake.

The real world is a bigger, more complicated cake.

Let’s consider deleting an order, or rather, canceling it. On top of the regular security checks, we’ve got some rules to consider:

If the order has already been delivered, check if the customer isn’t happy with what they got, and go about returning the order.

If the order contained products “made to order”, charge the customer for a portion (or all) of the order (based on other rules).

And more…

Deciding what the next status should be may very well depend on the current business status of the entity. Deciding if that change of state is allowed is context and time specific – at one point in time the task may have been allowed, but later not. The logic here is not necessarily entirely related to the entity being “deleted” – there may be other entities which need to be checked, and whose status may also need to be changed as well.

Summary

I know that some of you are thinking, “my system isn’t that complex – we can just delete and be done with it”.

My question to you would be, have you asked your users why they’re deleting things? Have you asked them about additional statuses and rules dictating how entities move as groups between them? You don’t want the success of your project to be undermined by that kind of unfounded assumption, do you?

The reason we’re given budgets to build business applications is because of the richness in business rules and statuses that ultimately provide value to users and a competitive advantage to the business. If that value wasn’t there, wouldn’t we be serving our users better by just giving them Microsoft Access?

In closing, given that you’re not giving your users MS Access, don’t think about deleting entities. Look for the reason why. Understand the different statuses that entities move between. Ask which users need to care about which status. I know it doesn’t show up as nicely on your resume as “3 years WXF”, but “saved the company $4 million in wasted inventory” does speak volumes.

One last sentence: Don’t delete. Just don’t.

My article on “employing the domain model pattern” has been published in the August edition of MSDN Magazine.

Here’s a short excerpt:

“In this article, we’ll go through the reasons to (and not to) employ the domain model pattern, the benefits it brings, as well as provide some practical tips on keeping the overall solution as simple as possible.”

Continue reading…

I’ve been hearing from people that have had a great deal of success using the Domain Event pattern and the infrastructure I previously provided for it in Domain Events – Take 2. I’m happy to say that I’ve got an improvement that I think you’ll like. The main change is that now we’ll be taking an approach that is reminiscent to how events are published in NServiceBus.

Background

Before diving right into the code, I wanted to take a minute to recall how we got here.

It started by looking for how to create fully encapsulated domain models.

The main assertion being that you do *not* need to inject anything into your domain entities.

Not services. Not repositories. Nothing.

Just pure domain model goodness.

Make Roles Explicit

I’m going to take the advice I so often give. A domain event is a role, and thus should be represented explicitly:

   1:  public interface IDomainEvent {}

If this reminds you of the IMessage marker interface in nServiceBus, you’re beginning to see where this is going…

How to define domain events

A domain event is just a simple POCO that represents an interesting occurence in the domain. For example:

   1:  public class CustomerBecamePreferred : IDomainEvent 

   2:  {

   3:      public Customer Customer { get; set; }

   4:  }

For those of you concerned about the number of events you may have, and therefore are thinking about bunching up these events by namespaces or things like that, slow down. The number of domain events and their cohesion is directly related to that of the domain model.

If you feel the need to split your domain events up, there’s a good chance that you should be looking at splitting your domain model too. This is the bottom-up way of identifying bounded contexts.

How to raise domain events

In your domain entities, when a significant state change happens you’ll want to raise your domain events like this:

   1:  public class Customer

   2:  {

   3:      public void DoSomething()

   4:      {

   5:          DomainEvents.Raise(new CustomerBecamePreferred() { Customer = this });

   6:      }

   7:  }

We’ll look at the DomainEvents class in just a second, but I’m guessing that some of you are wondering “how did that entity get a reference to that?” The answer is that DomainEvents is a static class. “OMG, static?! But doesn’t that hurt testability?!” No, it doesn’t. Here, look:

Unit testing with domain events

One of the things we’d like to check when unit testing our domain entities is that the appropriate events are raised along with the corresponding state changes. Here’s an example:

   1:  public void DoSomethingShouldMakeCustomerPreferred()

   2:  {

   3:      var c = new Customer();

   4:      Customer preferred = null;

   5:   

   6:      DomainEvents.Register<CustomerBecamePreferred>(

   7:          p => preferred = p.Customer

   8:              );

   9:   

  10:      c.DoSomething();

  11:      Assert(preferred == c && c.IsPreferred);

  12:  }

As you can see, the static DomainEvents class is used in unit tests as well. Also notice that you don’t need to mock anything – pure testable bliss.

Who handles domain events

First of all, consider that when some service layer object calls the DoSomething method of the Customer class, it doesn’t necessarily know which, if any, domain events will be raised. All it wants to do is its regular schtick:

   1:  public void Handle(DoSomethingMessage msg)

   2:  {

   3:      using (ISession session = SessionFactory.OpenSession())

   4:      using (ITransaction tx = session.BeginTransaction())

   5:      {

   6:          var c = session.Get<Customer>(msg.CustomerId);

   7:          c.DoSomething();

   8:   

   9:          tx.Commit();

  10:      }

  11:  }

The above code complies with the Single Responsibility Principle, so the business requirement which states that when a customer becomes preferred, they should be sent an email belongs somewhere else.

Notice that the key word in the requirement – “when”.

Any time you see that word in relation to your domain, consider modeling it as a domain event.

So, here’s the handling code:

   1:  public class CustomerBecamePreferredHandler : Handles<CustomerBecamePreferred>

   2:  { 

   3:     public void Handle(CustomerBecamePreferred args)

   4:     {

   5:        // send email to args.Customer

   6:     }

   7:  } 

This code will run no matter which service layer object we came in through.

Here’s the interface it implements:

   1:  public interface Handles<T> where T : IDomainEvent

   2:  {

   3:      void Handle(T args); 

   4:  } 

Fairly simple.

Please be aware that the above code will be run on the same thread within the same transaction as the regular domain work so you should avoid performing any blocking activities, like using SMTP or web services. Instead, prefer using one-way messaging to communicate to something else which does those blocking activities.

Also, you can have multiple classes handling the same domain event. If you need to send email *and* call the CRM system *and* do something else, etc, you don’t need to change any code – just write a new handler. This keeps your system quite a bit more stable than if you had to mess with the original handler or, heaven forbid, service layer code.

Where domain event handlers go

These handler classes do not belong in the domain model.

Nor do they belong in the service layer.

Well, that’s not entirely accurate – you see, there’s no *the* service layer. There is the part that accepts messages from clients and calls methods on the domain model. And there is another, independent part that handles events from the domain. Both of these will probably make use of a message bus, but that implementation detail shouldn’t deter you from keeping each in their own package.

The infrastructure

I know you’ve been patient, reading through all my architectural blah-blah, so here it is:

   1:  public static class DomainEvents

   2:  { 

   3:      [ThreadStatic] //so that each thread has its own callbacks

   4:      private static List<Delegate> actions;

   5:   

   6:      public static IContainer Container { get; set; } //as before

   7:   

   8:      //Registers a callback for the given domain event

   9:      public static void Register<T>(Action<T> callback) where T : IDomainEvent

  10:      {

  11:         if (actions == null)

  12:            actions = new List<Delegate>();

  13:   

  14:         actions.Add(callback);

  15:     }

  16:   

  17:     //Clears callbacks passed to Register on the current thread

  18:     public static void ClearCallbacks ()

  19:     {

  20:         actions = null;

  21:     }

  22:   

  23:     //Raises the given domain event

  24:     public static void Raise<T>(T args) where T : IDomainEvent

  25:     {

  26:        if (Container != null)

  27:           foreach(var handler in Container.ResolveAll<Handles<T>>())

  28:              handler.Handle(args);

  29:   

  30:        if (actions != null)

  31:            foreach (var action in actions)

  32:                if (action is Action<T>)

  33:                    ((Action<T>)action)(args);

  34:     }

  35:  } 

Notice that while this class *can* use a container, the container isn’t needed for unit tests which use the Register method.

When used server side, please make sure that you add a call to ClearCallbacks in your infrastructure’s end of message processing section. In nServiceBus this is done with a message module like the one below:

   1:  public class DomainEventsCleaner : IMessageModule

   2:  { 

   3:      public void HandleBeginMessage() { }

   4:   

   5:      public void HandleEndMessage()

   6:      {

   7:          DomainEvents.ClearCallbacks();

   8:      }

   9:  }

The main reason for this cleanup is that someone just might want to use the Register API in their original service layer code rather than writing a separate domain event handler.

Summary

Like all good things in life, 3rd time’s the charm.

It took a couple of iterations, and the API did change quite a bit, but the overarching theme has remained the same – keep the domain model focused on domain concerns. While some might say that there’s only a slight technical difference between calling a service (IEmailService) and using an event to dispatch it elsewhere, I beg to differ.

These domain events are a part of the ubiquitous language and should be represented explicitly.

CustomerBecamePreferred is nothing at all like IEmailService.

In working with your domain experts or just going through a requirements document, pay less attention to the nouns and verbs that Object-Oriented Analysis & Design call attention to, and keep an eye out for the word “when”. It’s a critically important word that enables us to model important occurrences and state changes.

What do you think? Are you already using this approach? Have you already tried it and found it broken in some way? Do you have any suggestions on how to improve it?

Let me know – leave a comment below.

The ability to map entity relationships is broadly supported by many O/RM tools. For some reason, though, many developers run into issues when trying to map a many-to-many relationship between entities. Although much has already been written about the technological aspects of it, I thought I’d take more of an architectural / DDD perspective on it here.

Value Objects Don’t Count

While the canonical example presented is Customer -> Address, and has a good treatment here for nHibernate, it isn’t architecturally representative.

Addresses are value objects. What this means is that if we have to instance of the Address class, and they both have the same business data, they are semantically equivalent. Customers, on the other had, are not value objects – they’re entities. If we have two customers with the same business data (both of them called Bob Smith), that does not mean they are semantically equivalent – they are not the same person.

All Entities

Therefore, for our purposes here we’ll use something different. Say we have an entity called Job which is something that a company wants to hire for. It has a title, description, skill level, and a bunch of other data. Say we also have another entity called Job Board which is where the company posts jobs so that applicants can see them, like Monster.com. A job board has a name, description, web site, referral fee, and a bunch of other data.

A job can be posted to multiple job boards. And a job board can have multiple jobs posted. A regular many to many relationship. At this point, we’re not even going to complicate the association.

This is simply represented in the DB with an association table containing two columns for each of the entity tables’ ids.

In the domain model, developers can also represent this with the Job class containing a list of JobBoard instances, and the JobBoard class containing a list of jobs.

It’s intuitive. Simple. Easy to implement. And wrong.

In order to make intelligent DDD choices, we’re going to first take what may seem to be a tangential course, but I assure you that your aggregate roots depend on it.

Moving forward with our example

Let’s say the user picks a job, and then ticks off the job boards where they want the job posted, and clicks submit.

For simplicity’s sake, at this point, let’s ignore the communication with the actual job sites, assuming that if we can get the association into the DB, magic will happen later causing the job to appear on all the sites.

Our well-intentioned developer takes the job ID, and all the job board IDs, opens a transaction, gets the job object, gets the job board objects, adds all the job board objects to the job, and commits, as follows:

   1:          public void PostJobToBoards(Guid jobId, params Guid[] boardIds)

   2:          {

   3:              using (ISession s = this.SessionFactory.OpenSession())

   4:              using (ITransaction tx = s.BeginTransaction())

   5:              {

   6:                  var job = s.Get<Job>(jobId);

   7:                  var boards = new List<JobBoard>();

   8:   

   9:                  foreach(Guid id in boardIds)

  10:                      boards.Add(s.Get<JobBoard>(id));

  11:   

  12:                  job.PostTo(boards);

  13:   

  14:                  tx.Commit();

  15:              }

  16:          }

In this code, Job is our aggregate root. You can see that is the case since Job is the entry point that the service layer code uses to interact with the domain model. Soon we’ll see why this is wrong.

** Notice that in this service layer code, our well-intentioned developer is following the rule that while you can get as many objects as you like, you are only allowed one method call on one domain object. The code called in line 12 is what you’d pretty much expect:

   1:          public void PostTo(IList<JobBoard> boards)

   2:          {

   3:              foreach(JobBoard jb in boards)

   4:              {

   5:                  this.JobBoards.Add(jb);

   6:                  jb.Jobs.Add(this);

   7:              }

   8:          }

Only that as we were committing, someone deleted one of the job boards just then. Or that someone updated the job board causing a concurrency conflict. Or anything that would cause one single association to not be created.

That would cause the whole transaction to fail and all changes to roll back.

Rightly so, thinks our well-intentioned developer.

But users don’t think like well-intentioned developers.

Partial Failures

If I were to go to the grocery store with the list my wife gave me, finding that they’re out of hazelnuts (the last item on the list), would NOT buy all the other groceries and go home empty handed, what do you think would happen?

Right. That’s how users look at us developers. Before running off and writing a bunch of code, we need to understand the business semantics of users actions, including asking about partial failures.

The list isn’t a unit of work that needs to succeed or rollback atomically. It’s actually many units of work. I mean, I wouldn’t want my wife to send me to the store 10 times to buy 10 items, so the list is really just a kind of user shortcut. Therefore, in the job board scenario, each job to job board connection is its own transaction.

This is more common than you might think.

Once you go looking for cases where the domain is forgiving of partial failures, you may start seeing more and more of them.

Aggregate Roots

In the original transaction where we tried to connect many job boards to a single job, we saw that the single job is the aggregate root. However, once we have multiple transactions, each connecting one job and one job board, the job board is just as likely an aggregate root as the job.

We can do   jobBoard.Post(job);    or     job.PostTo(jobBoard);

But we need just a bit more analysis to come to the right decision.

While we could just leave the bi-directional/circular dependency between them, it would be preferable if we could make it uni-directional instead. To do that, we need to understand their relationship:

If there was no such thing as “job”, would there be meaning to “job board” ? Probably not.

If there was no such thing as “job board”, would there be meaning to “job” ? Probably. Yes. Our company can handle the hiring process of a job regardless of whether the candidate came in through Monster.com or not.

From this we understand that the uni-directional relationship can be modelled as one-to-many from job board to job. The Job class would no longer have a collection of Job Board objects. In fact, it could even be in an assembly separate from Job Board and not reference Job Board in any way. Job Board, on the other hand, would still have a collection of Job objects.

Going back to the code above we see that the right choice is   jobBoard.Post(job);   

Job Board is the aggregate root in this case. Also, the many-to-many mapping has now dissolved, leaving behind a single one-to-many mapping.

Let that sink in a second.

But Wait…

While the GUI showing which jobs are posted on a given job board are well served by the above decision (simply traversing the object graph from Job Board to its collection of Jobs), that’s not the whole story. Another GUI needs to show administrative users which Job Boards a given Job has been posted to. Since we no longer have the domain-level connection, we can’t traverse myJob.JobBoards.

Our only option is to perform a query. That’s not so bad, but not as pretty as object traversal.

The real benefit is in chopping apart the Gordian M-to-N mapping knot and getting a cleaner, more well factored domain model.

That gives us much greater leverage for bigger, system-level decomposition.

We’re now all set to move up to a pub/sub solution between these aggregate roots, effectively upgrading them to Bounded Contexts. From there, we can move to full-blown internet-scale caching with REST for extra scalability on showing a job board with all its jobs.

In Closing

We often look at many-to-many relationships just like any other relationship. And from a purely technical perspective, we’re not wrong. However, the business reality around these relationships is often very different – forgiving of partial failures, to the point of actually requiring them.

Since the business folks who provide us with requirements rarely think of failure scenarios, they don’t specify that “between these two entities here, I don’t want transactional atomicity” (rolling our technical eyes – the idiots [sarcasm, just to make sure you don’t misread me]).

Yet, if we were to spell out what the system will do under failure conditions when transactionally atomic, those same business folks will be rolling our eyes back to us.

What I’ve found surprises some DDD practitioners is how critical this issue really is to arriving at the correct aggregate roots and bounded contexts.

It’s also simple, and practical, so you won’t be offending the YAGNI police.

Related Content

From CRUD to Domain-Driven Fluency

[Podcast] Domain Models, SOA, and The Single Version of the Truth

Update: The next post in this series is now online here.

My previous post on how to create fully encapsulated domain models introduced the concept of events as a core pattern of communication from the domain back to the service layer. In that post, I put up enough code to get the idea across but didn’t address issues like memory leaks and multi-threading. This post will show the solution to those two critical points.

I’ve snipped out one of the events in the previous example for brevity.

Previous API

The previous API looked like this:

   1:  public static class DomainEvents

   2:  {

   3:       public static event EventHandler GameReportedLost;

   4:       public static void RaiseGameReportedLostEvent()

   5:       {

   6:             if (GameReportedLost != null)

   7:                 GameReportedLost(null, null);

   8:       }

   9:   

  10:       public static event EventHandler CartIsFull;

  11:       public static void RaiseCartIsFull()

  12:       {

  13:             if (CartIsFull != null)

  14:                 CartIsFull(null, null);

  15:       }

  16:  }

One thing that we want to keep in the solution is that all the code to define events, their names, and the parameters they bring will be in one place – in this case, the DomainEvents class. One thing that we’d like to fix is the amount of code needed to define an event.

Previous Service Layer

Here’s what our previous service layer code looked like:

   1:  public class AddGameToCartMessageHandler :

   2:      BaseMessageHandler<AddGameToCartMessage>

   3:  {

   4:      public override void Handle(AddGameToCartMessage m)

   5:      {

   6:          using (ISession session = SessionFactory.OpenSession())

   7:          using (ITransaction tx = session.BeginTransaction())

   8:          {

   9:              ICart cart = session.Get<ICart>(m.CartId);

  10:              IGame g = session.Get<IGame>(m.GameId);

  11:   

  12:              Domain.DomainEvents.GameReportedLost +=

  13:                gameReportedLost;

  14:              Domain.DomainEvents.CartIsFull +=

  15:                cartIsFull;

  16:   

  17:              cart.Add(g);

  18:   

  19:              Domain.DomainEvents.GameReportedLost -=

  20:                gameReportedLost;

  21:              Domain.DomainEvents.CartIsFull -=

  22:                cartIsFull;

  23:   

  24:              tx.Commit();

  25:          }

  26:      }

  27:   

  28:      private EventHandler gameReportedLost = delegate { 

  29:            Bus.Return((int)ErrorCodes.GameReportedLost);

  30:          };

  31:   

  32:      private EventHandler cartIsFull = delegate { 

  33:            Bus.Return((int)ErrorCodes.CartIsFull);

  34:          };

  35:      }

  36:  }

Another thing that should be improved is the amount of code needed in the service layer.

Raising an event, though, should still be fairly simple – one line of code similar to DomainEvents.RaiseGameReportedLost().

New API

Here’s what the new API looks like:

   1:  public static class DomainEvents

   2:  {

   3:       public static readonly DomainEvent<IGame> GameReportedLost = 

   4:                                            new DomainEvent<IGame>;

   5:   

   6:       public static readonly DomainEvent<ICart> CartIsFull=

   7:                                            new DomainEvent<ICart>;

   8:  }

It looks like we’ve managed to bring down the complexity of defining an event.

Raising an event is slightly different, but still only one line of code (“this” refers to the Cart class that is calling this API): DomainEvents.CartIsFull.Raise(this);

New Service Layer

The advantage of having a disposable domain event allows us to use the “using” construct for cleanup.

   1:  public class AddGameToCartMessageHandler :

   2:      BaseMessageHandler<AddGameToCartMessage>

   3:  {

   4:      public override void Handle(AddGameToCartMessage m)

   5:      {

   6:          using (ISession session = SessionFactory.OpenSession())

   7:          using (ITransaction tx = session.BeginTransaction())

   8:          using (DomainEvents.GameReportedLost.Register(gameReportedLost))

   9:          using (DomainEvents.CartIsFull.Register(cartIsFull))

  10:          {

  11:              ICart cart = session.Get<ICart>(m.CartId);

  12:              IGame g = session.Get<IGame>(m.GameId);

  13:   

  14:              cart.Add(g);

  15:   

  16:              tx.Commit();

  17:          }

  18:      }

  19:   

  20:      private Action<IGame> gameReportedLost = delegate { 

  21:            Bus.Return((int)ErrorCodes.GameReportedLost);

  22:          };

  23:   

  24:      private Action<ICart> cartIsFull = delegate { 

  25:            Bus.Return((int)ErrorCodes.CartIsFull);

  26:          };

  27:      }

  28:  }

I also want to mention that you don’t necessarily have to have the same service layer object handle these events as that which calls the domain objects. In other words, we can have singleton objects handling these events for things like sending emails, notifying external systems, and auditing.

The Infrastructure

The infrastructure that makes all this possible (in a thread-safe way) is quite simple and made up of two parts, the DomainEvent that we saw being used above, and the DomainEventRegistrationRemover which handles the disposing:

   1:  using System;

   2:  using System.Collections.Generic;

   3:   

   4:  namespace DomainEventInfrastructure

   5:  {

   6:      public class DomainEvent<E> 

   7:      {

   8:          [ThreadStatic] 

   9:          private static List<Action<E>> _actions; 

  10:   

  11:          protected List<Action<E>> actions 

  12:          {

  13:              get { 

  14:                  if (_actions == null) 

  15:                      _actions = new List<Action<E>>(); 

  16:   

  17:                  return _actions; 

  18:              }

  19:          }

  20:   

  21:          public IDisposable Register(Action<E> callback) 

  22:          {

  23:              actions.Add(callback);

  24:              return new DomainEventRegistrationRemover(delegate

  25:                  {

  26:                      actions.Remove(callback);

  27:                  }

  28:              ); 

  29:          }

  30:   

  31:          public void Raise(E args) 

  32:          {

  33:              foreach (Action<E> action in actions) 

  34:                  action.Invoke(args);

  35:          }

  36:      }

  37:  }

  38:   

Note that the invocation list of the domain event is thread static, meaning that each thread gets its own copy – even though they’re all working with the same instance of the domain event.

Here’s the DomainEventRegistrationRemover – even simpler:

   1:  using System;

   2:   

   3:  namespace DomainEventInfrastructure

   4:  {

   5:      public class DomainEventRegistrationRemover : IDisposable 

   6:      {

   7:          private readonly Action CallOnDispose;

   8:   

   9:          public DomainEventRegistrationRemover(Action ToCall) 

  10:          {

  11:              this.CallOnDispose = ToCall; 

  12:          }

  13:   

  14:          public void Dispose() 

  15:          {

  16:              this.CallOnDispose.DynamicInvoke();

  17:          }

  18:      }

  19:  }

For your convenience, I’ve made these available for download here.

I also want to add that if you haven’t looked at the comments on the original post – there’s some really good stuff there (36 comments so far). Take a look.