Jak Charlton - Insane World : DDD

What is Domain Driven Design?

Jak Charlton — Mon, 16 May 2011 02:15:00 GMT

"... the key to expert performance in many fields is domain knowledge rather than intelligence."
Don Reinertsen

Domain Driven Design is a software development methodology, intended to achieve a software system closely modelled on and aligned with real business processes.

Traditionally development tends to be a technically led process, where requirements are passed from the business to the development teams, and then the development teams go off and produce their best guesstimate at what the requirements said.

In a waterfall approach this leads to large requirements documents that are diligently collated, analysed, reviewed and approved. These documents are then given to development teams to turn into working software.

Agile approaches can also accept the requirements documents that waterfall tends to produce, but for them to actually progress forwards they must be broken to small tasks and stories which can then be queued ready for development.

Domain Driven Design to a large degree steps back from these two distinct ends of the spectrum, and looks at how the requirements are gathered in the first place - if you like, it bridges the gap between doing everything up front and everything at the last minute.

DDD understands that requirements are never 'done', but exist as a living document. More importantly, the living document in question is actually the software itself - all other documents are an artefact and reflection of the code.

As the software system develops and grows, deeper understanding of the problem grows too - DDD is all about the discovery of the solution through deeper understanding of the problem.

Where DDD really differs though is that it sees the software system as a reflection of the business process, an enabler rather than the driver. DDD is deeply concerned about the business processes, business terminology and practices. Technical concerns are largely secondary and a means to an end.

The Ubiquitous Language is at the centre of DDD - this is a shared and growing language. A negotiated language derived from the business terminology and enriched by the development teams. If a business person doesn't understand a term in the UL, then the chances are the UL needs an evolution. If a technical person doesn't understand a term in the UL then the chances are they need a conversation with a Domain Expert.

The Domain Experts are the second essential component of DDD - people who deeply understand the business domain, and the business itself. These people are integral to the development process. They may not be required "full time" like the traditional Product Owners of some Agile methodologies, but they must be accessible continually, and be prepared to invest time in the process. Domain Experts are not to be considered outsiders, but as the core of the DDD process - they are as much a part of the development team as any developer or tester.

DDD doesn't begin and end - it is a constant process of re-evaluation, refactoring, remodelling and redesign - every conversation should bring you to a closer understanding of the problem. At no point is DDD 'done' - it is always there; the Ubiquitous Language grows and develops, the Domain Models change as understanding changes, code is restructured and refactored to better represent that understanding.

Artefacts will come and go, but only one really matters - the code base. It is the only expression of the solution that is free of abstraction. While documents may try to explain or describe the system, only the code does so without losing information. This means that in DDD the code must remain of high quality, be clear and expressive, be free of technical abbreviations or jargon, and as far as possible should make at least some sense to a Domain Expert when explained to them.

Clever code doesn't work in DDD, nor do magical processes or "you don't need to know" components. Domain Experts shouldn't need to become developers to understand what the key parts of the software system is doing for them. They also shouldn't need to think in terms of databases or batch jobs or other technical concerns.

DDD is the ultimate expression of Agile - it is about dealing with a constantly shifting and evolving set of requirements - because as anyone who has ever been involved with a software project knows - it is very rare for a requirement to remain intact from the beginning to the end of a project, and almost impossible for it to do so as the business grows and changes.

Through constant conversations, DDD will guide you to an ever more accurate software representation of your business process.

CQRS - The Cult of Shiny Things

Jak Charlton — Fri, 29 Oct 2010 00:12:00 GMT

CQRS has become another casualty of the buzzword culture and the cult of 'shiny things'

What started as a collection of some reasonably good principles has now turned into an almost religious mantra for some, with more and more outrageous claims, and almost no basis in fact or experience.

Scalability

Claims like "almost infinitely scalable systems" associated to CQRS are ludicrous, especially as none of the currently "near infinitely scalable systems" use CQRS or have probably even heard of it. The largest systems on the planet have no need of CQRS, they use many tricks and techniques that have been associated with CQRS - but they don't use it, nor are they ever likely to use it, because when you have really large systems, things like CQRS become totally irrelevant.

The scaling benefits of CQRS come from understanding eventual consistency and messaging - concepts that were around long before CQRS, and that will be around long after it too.

Task Based UIs

Well, yes CQRS may require a task based UI, but task based UIs were again around long before CQRS, and will again outlast it. This is just a different way of thinking about how people interact with computers - this is a UX issue, not a technical one.

In fact, if anything, CQRS can be a major cause of issues here - when you really have a CRUD style system, CQRS is going to cause you all sorts of issues you will need to resolve. And much as I dislike CRUD based UIs, the reality is we have abused users to work this way in many cases - so sometimes that is just how we have to make our next UI work.

Side Effect Free

Another aspect of CQS (and by extrapolation CQRS) is that functions employing CQS are side effect free - you don't accidentally introduce subtle bugs by writing and reading in one operation.

And while this is nice in theory, the scale of system where this preventative measure would have benefit is the one where this problem is just as easily solved by many other simpler means, like just writing smart code. Larger systems have so many points in them where read and write are already being mixed, CQRS here just lends an illusion of safety.

Denormalised Read Models

A valuable benefit of CQRS is the concept that the views have specific read models that are denormalised from the (presumably) complex domain model.

This has a number of benefits, though these are largely boiled down to improved efficiency of querying (and action that tends to happen frequently). These models can also be extrapolated out to provide reporting and MI models too.

But again, we have been doing this for many years already, and usually to account for overloaded database servers that we don't wish to query against, particularly with MI.

And querying these days, even with horribly complex joins on highly complex models is becoming faster and faster. Push querying to a slave server, and you gain all the benefits of a rich model, and fast queries. And for even more speed you can look to NoSQL solutions - which are effectively based on the principle that you denormalised your data anyway.

Performance Considerations

Part of the theoretical benefit of CQRS is that your queries will be faster as your denormalised datastores are updated at write time, so you don't have to do horrible joins on queries (which in theory on most systems happen far more frequently than writes)

These days, the performance of relational databases and especially NoSQL databases is advancing rapidly. The new breed of databases takes things like sharding and replication very seriously, and as first class features, not afterthoughts like the databases of old did.

Replication in MySQL takes roughly 0.0002 of a second (see http://nr-content.s3.amazonaws.com/railslab/videos/17-ScalingRails-Scaling-Your-Database-Part-1.mp4 at 6 minutes in) - compare that with how long a message based approach like CQRS will take to push that to your secondary datastores and you are talking orders of magnitude faster for the dumb replication approach. Apparently it operates at this speed for up to 10 slave databases before degrading.

Now, think about this bit, the 37 Signal systems (a huge system) operates on a single MySQL instance running with 105Gb of memory caching. And 37 Signal could be considered the archetype of an Active Record approach, after all it is the reason Ruby on Rails exists. Are you writing a system that big?

Maybe you are so let's examine a larger system - Wordnik stores over 12 billion documents, 3Tb of data per node, 500k requests per hour and 4 times that at peak load, and yet their fetch time is around 60ms. (http://blog.wordnik.com/12-months-with-mongodb)

Let's not even begin to discuss Twitter, Ebay, Facebook or other similar "mega systems" - they still respond in milliseconds with tens or hundreds of millions of users - think that's down to CQRS?

Dealing with Complexity

Yep - CQRS deals with complexity to one degree - it forces you to explicitly think about what each call is doing - "Is this writing data or reading data?" - and that thought process is a good thing.

But it introduces another layer of complexity that is more subtle and harder to deal with - the replication to secondary datastores is complex in itself - it effectively needs to do that horrible join you are trying to avoid in your queries so that it can update your denormalised tables. That in itself isn't simple.

But it also needs to be able to maintain those secondary datastores if, for example they are corrupted and need recreating or if they miss some messages. In normal replication this is trivial, drop the slave, add a new slave - wait a short while - someone already wrote all that for you on a database system - you are going to need to write it all yourself with CQRS.

So Why Have I Implemented CQRS Before?

Well, at the time I first implemented CQRS I had two reasons for doing so - and oddly neither had to do with any of the aforementioned technical aspects. In fact, one was a technical problem and one was a people problem.

Firstly this was nearly two years back, and the system that we were writing a layer over was a very old and crumbling legacy database system. It had no real normalised schema, and was open to abuse from just about every development team in the organisation without warning to others - it was also horribly prone to outages.

One project had just "failed" (though I'm sure some would say it was just a lesser success), primarily due to trying to put a domain model on top of the legacy database.

So, the problem we had was to create a disconnected system that could operate independently of the legacy system, but also use the legacy system as it's true authority for the information it used. Because of the horrible complexity in the legacy system, using CQRS allowed us to write what were frankly quite horrible SQL statements for reading data from legacy, while using a pseudo-document database for the daily operations. We then used horrible SQL statements to replicate this data back into legacy.

Of course, what we were really writing was a standard message based architecture, but the term CQRS encompassed that quite nicely.

We could have approached this without the CQRS badge - but it served as a useful political tool - a number of the existing team had heard of it and were quite enthusiastic, and management would be confounded enough by it to not get in the way too much. It also broke the mental model the development teams had with "put an ORM over the database".

So the secondary benefit of CQRS here was a people problem - it was a new buzzword - it got new enthusiasm from a team that was demotivated from the previous project - and more than anything - that motivation was going to be the key to the success or failure of the new project.

So Why Have I Presented On It Before?

Well two reasons really, the first being that I have always tried to dispel the mythology around CQRS and to explain the real benefits of a flexible and scalable architecture - it just so happens that people want the buzzword explaining - you have to admit it's a pretty vague and fuzzy term with a myriad of definitions and explanations.

The secondary reason being, as my last presentation at DDD Sydney explained - my objective was to make you think. There's a lot of good things hiding under the CQRS banner, and I really would like people to think about better ways of writing software.

Conclusion

In the few years since CQRS was "invented" it has got a bit of a cult buzzword status. It is talked about as though it fixes a multitude of problems.

But in that time, NoSQL has really taken off. RBDMS' have got far more powerful. And memory and processing power has increased significantly. Database caching is exceptionally impressive. REST gives us proxy caching for near enough free. Memcached and Velocity make light of massive data loads. We even have the nefarious "cloud" now, where processing power and memory is cheaper than the effort involved in maintain complex architectures.

Scaling systems is now a pretty commonplace business problem for most of the startups taking off -and they don't need CQRS - there is a very good chance you don't either. We are past that point in time where we had to prematurely optimise things like database queries.

If it's about thinking differently, then I can tell you all you need to know about CQRS in a simple paragraph:

Learn to think of systems as being inconsistent at all times, no matter what illusion they present of 'real time'. Learn to embrace messaging, REST, caching, replication, sharding, and a hundred other techniques to solve these problems. Embrace user centric task based UIs, embrace strong UX, embrace strong domain driven design, embrace business led functionality and delivering systems that match real requirements. Embrace thinking differently.

CQRS is no silver bullet, in fact it has almost now become a solution to a problem that doesn't exist any more. If you can give me a problem that CQRS claims to solve, I'm fairly sure I can solve it more easily than I could have done 2 years ago when I started using CQRS. I'm sure I'll still use a lot of the things that CQRS encapsulates, because they are after all very good principles - I'm just pretty much over playing buzzword bingo!

My Objective Today is to Make You Think

Jak Charlton — Mon, 19 Jul 2010 00:08:00 GMT

"Maybe There is a Better Way"

I recently presented at DeveloperDeveloperDeveloper in Sydney, and although my talk was Stuff About CQRS, I opened with the slide

My Object Today Is to Make You Think ... 'Maybe There is a Better Way'

(slides here)

The real focus of this was around how normal people think, and how unlike normal people we developers really are. My role in development is all about enabling better communication, because fundamentally I believe the real value a developer brings to a project is not technical, but is in the way they interact with the team, and more importantly with the normal people they are actually creating software for.

Obviously some of this has roots in DDD, the Ubiquitous Language is obviously an attempt to traverse this chasm that seems to exist between us.

Then I touched on user interfaces, and how they are so rarely designed the way people think - normal people think about their objectives and goals, not in terms of data like we developers do. Inductive UIs are focused on tasks, unlike the traditional data driven UIs that we tend to throw at users - users and people don't think in grids and columns and rows, they think "I want to change my address", not "open my customer record, edit the three fields under address and save to the database" - only a sadist or a developer would think that way.

And finally I touched on things like NoSQL databases, which neatly solve a communication problem - they stop us thinking about How to store information and let us focus on What we are storing and Why.

And lastly I tried to show the link between CQRS and these business problems - how it made you focus on the language, on tasks and objectives and how it let you detach the How from the What and Why.

But most importantly, what I was trying to do in that presentation was to throw some non-mainstream ideas out into the audience, to spark discussion and debate, and to get people to think - Maybe There is a Better Way

If only a few of those in the audience went away and Googled some of the ideas I was talking about, it will be another step towards moving development away from it's heavy focus on technology and technological solutions - and towards a people and business driven focus, where technology is an artifact, not the deciding factor.

ThinkDDD Resource Site

Jak Charlton — Tue, 04 May 2010 00:08:00 GMT

For a while now I have been wanting to replace the creaking DDDStepByStep.com site - I originally set it up using Community Server, but it soon became obvious after deployment that CS just wasn't up to the job - the server regularly ran out of resources just running the vanilla product and became a pain to keep rebooting.

So, to try yet again to sort out the information, I setup a new website thinkddd.com and used it as an opportunity to play with Ruby, and the Radiant CMS. Deployment to Heroku for such a low bandwidth site also worked out as free - so the new site cost basically nothing except for time.

The slides from my recent presentation to the Sydney alt.net user group are available from here.

Hopefully it is a lot clearer and easier to navigate, so if you are interested in Domain Driven Design - enjoy.

Solving Complexity with Message Based Architectures

Jak Charlton — Thu, 04 Feb 2010 07:55:00 GMT

Ultimately most complexity in software comes not from the requirements, the business logic, or even the underlying systems. Most complexity comes out of a poorly considered and managed architecture, and this is commonly seen in tightly coupled systems that rapidly degrade into Big Balls of Mud.

The key to simplifying architectures is to decouple their component parts, making each more specialised and less dependent on the others. This allows every component to focus on a specific task without risk of those tasks becoming compromised by, or interfering with, others.

The basic premise of most architectures that succeed in this way is that they are message based, rather than being glorified CRUD systems pulling data to and from databases. Operations are central to the system, data is ultimately just the storage for the results of those operations.

This separation of concerns allows development teams to concentrate on component parts with little impact upon other teams, and no more reliance upon them than an agreement on the messages they will listen for and those they will publish. While messages share some characteristics of service interfaces, they differ mostly in their granularity and lack of dependence upon one another. Messages allow true decoupling, where traditional service layers only promise this and usually end up creating even more tightly coupled systems.

"Message based" does not inherently mean SOA, and in many ways differs substantially. Although both can be considered message based, SOA brings significant overhead and generalisation, along with an organisation wide remit. Simple message based systems operate on the basis of message within the application rather than across an enterprise.

Poor software architecture ends up impacting more than just the software system under development, and frequently the impact can be felt across an organisation; in the way development teams approach problems, in the way they respond to new requirements, and throughout the SDLC.

More often than not, the SDLC is a reflection of the weaknesses in software architecture, and is frequently trying to play 'catch up' to attempt to deal with the issues weak architecture brings. These compromises and aspects of the SDLC then start to propagate and impact other areas of the development and support process. Often decisions about software architecture become the underlying drivers for the entire IT department, and the SDLC turns into a point of conflict as it becomes more about trying to restrain and control a big ball of mud than it does about providing a framework for providing business value.

We Are Not Doing DDD – Part Two - CQS

Jak Charlton — Mon, 22 Jun 2009 16:32:58 GMT

From the beginning of my current project we have been working under some horrible constraints, many imposed by legacy systems, many by late decisions that would have speeded things immensely if made earlier, and many imposed by decisions that are outside of my control.

This lead us early on to make decisions on architecture that eliminated any possibility of using DDD heavily, and as I mentioned in my original post about this project it is also essentially a glorified CRUD system - we read data from databases, we modify it a bit, and we put it back again.

What We Are Doing – Command Query Separation

The first architectural choice that I made, and in hindsight got accepted with relative ease, was to employ CQS to simplify the system.

Earlier projects had suffered quite badly from being abstractions of legacy systems with new functionality bolted on. As

far as possible I wanted to eliminate this problem from our project, as I was all too aware that this was an easy trap for the team to fall into, and would burn a lot of development time very fast.

To make the system simpler I chose to separate our query mechanisms from the "domain" type stuff, our commands and data writing/updating code.

CQS is a pretty simple concept. One path through your system is responsible for querying of anything much more than "Get By ID" type calls. This side of our system is responsible for reading data from legacy systems, reading data from legacy web services, and for reading data from our application database.

Querying

The Query side deals largely in DataTables, DataSets and a very very limited number of DTOs. This data is largely and essentially used for display purposes, it is the contents of search results, the results of postcode lookups, the information to populate dropdown lists and tables.

The Query side of CQS does not need strongly typed entities, nor does it require strongly typed DTOs - as it is largely ad-hoc data maintaining these entities and DTOs would consume a disproportionate amount of development time for something a DataTable can deal with more than adequately.

Our Query side actually uses the Query Object pattern, most of which encapsulate a call to an Oracle DB, some of which wrap simple NHibernate criteria and a few of which sit around web services. These are fronted by a WCF facade, which allows us to put these queries out of process, and to cache them aggressively should be need.

What we do not have in the system anywhere, and especially in the Query side, is any big entity model – we have two very small entity models (5 entities in total) used for very specific functions within our application.

Command

The Command side of the CQS equation is a little different. Firstly, it deals with anything that is essentially a request to "go do something". In our case these are commands like "SaveDraft" and "CancelPolicy"

The Command side can write data to the databases, the Query side is not allowed to.

If we were doing DDD, this is where our Domain would sit - and in fact we have been referring to this as "the domain" for most of this project. But, we aren’t using DDD and we don't really have a Domain Model. It is however where most stuff we could describe as "logic" lives. This is primarily a bunch of application services, most of which operate in response to messages fired over MSMQ using NServiceBus.

This side of the architecture has a number of listeners that are responsible for pulling messages from various MSMQs and for dealing with the specific requests, for example pushing data back to our legacy systems, for logging, and eventually for notifications.

Benefits

Early on, the use of CQS paid for itself easily. By removing the complications of maintaining read and write models together, we quickly managed to get working prototype up and running, with no need for a "domain" or entity model up front.

As the project has progressed, this decision has helped simplify decisions that otherwise would have involved structural or fundamental changes to the architecture – but more importantly, it has helped all of the developers think in terms of a command driven system, rather than viewing the system as CRUD over a database.

Commercial Suicide - Integration at the Service Level

Jak Charlton — Tue, 19 May 2009 07:03:00 GMT

In my previous post I described the problems with trying to integrate your organisation at the database level, and the fallacies surrounding the idea of the One True Authority Database. I also alluded to this being a problem with services too.

When you try to create a monolithic and authoritarian database that knows all, sees all, and brings everything into a single coherrent whole, you are heading down a slippery slope. Different applications have different requirements, and even within the same job role, the meaning of Customer can have a totally different meaning in two different contexts. I also argued that data has no value without context, and the context is the application that is meeting the business requirements, be these user interaction, system interaction, or reporting.

SOA to the Rescue!

I'm not arguing without ample evidence behind me here - the idea of integration at a database level was one that was well discredited many years ago. I grant you many organisations haven't got the message yet, and that many vested interests still keep trying to make the magic solution work, but fundamentally a long time back we in IT recognised that the One True Authority Database (OTADB) was a "bad thing" - and this lead us to the promised land - initially Web Services and then on to Service Oriented Architectures.

SOA is an attempt to resolve the OTADB issue - take all those database calls, and even calls to different databases, and unify them through a set of coherrent and cohesive services - SOA can provide us with the abstraction over out data stores that is required - and now we can have the One True Authority Service Layer.

With SOA we can define services to wrap our databases, services to update data, service to read and query data, and even services to unify those "different" Customers into a single coherrent and canonical data model.

And that's where it all goes wrong again.

Anti-Pattern: The Canonical Data Model

Back when our database guys were trying to integrate all this information, what they were trying to achieve, but maybe didn't realise, was a canonical data model - this is the "holy grail" of SOA. Martin Fowler describes it thus:

How can you minimize dependencies when integrating applications that use different data formats? …Design a Canonical Data Model that is independent from any specific application. Require each application to produce and consume messages in this common format

Sounds perfect! We now have a way of abstracting our systems from each other, and our canonical data model will pull this information all together at the end.

So, I'm going out on a limb here, disagreeing with Martin Fowler is rarely a smart move.

Except I'm not disagreeing with Martin Fowler so much as I am disagreeing with those that try to implement his ideas and principles. I don't recollect anywhere that Martin says that all data can be squeezed into this model, or that this model is the "one true view". In fact, later evolutions of Martin's work have included things like Domain Driven Design, which is almost the opposite of a canoncial data model - DDD says that you have a different model for every domain or context, each specifically engineered to solve the problem at hand. Given the choice, I am certainly in the DDD camp here - after all data without context is worthless, and it is the domain (and DDD) that provides the context.

The key and important part of a canonical data model is it provides a mapping between inconsistent formats and differing views of data.

So as an ideal, the canonical data model is fine - in practice it is often a major anti-pattern.

There is still no One True Authority, and SOA really never promised this, nor did the canonical data model - what they promised was enough abstraction to allow applcaitions to work independently, and to fulfill their specific requirements independently. And then to leave it to the service layers to unify this data.

So Service Oriented Architectures HAVE Come to the Rescue?

Not quite - again my argument here isn't so much about SOA, or any of the things around it, but more the people who try to implement it, often without understanding what they are trying to achieve, the problems that SOA or a service layer solves, or even the amount of effort that is required to get a canonical data model right.

The key part here is that a canonical data model is a massive excercise to do right. Essentially it means analysing every applciation, every service and every database within your organisation and extracting their individual requirements and current functionality, and trying to distill that down to a set of common definitions.

While this may be eventually achievable, in practice it is almost never the case. This "holy grail" of SOA is much like the "holy grail" of integration at the database level, eventually it boils down to "the problem is too complex, too fast moving and too indeterminate to define". In development we figured out long ago that we needed Agile to help us move quickly as requirements shifted and changed, but the OTADB and the SOA canonical data model do not allow this to happen, they seek to solidify the design early.

SOA done "right" may well make this problem smaller, but SOA is rarely done right.

Can Domain Driven Design Help?

DDD was largely an attempt to address these problems - by identifying that individual applications and contexts required different data, DDD allowed us to recognise that square pegs and round holes do not match. Eventually with a big enough hammer you may be able to force it through, but there is going to be a lot of effort and a lot of collaterel damage.

Domain Driven Design includes terms like "Domain", "Bounded Context" and "Anti Corruption Layer" specifically to deal with the issues around trying to use a "one solution fits all" hammer to crack what is quite a simple nut.

DDD is largely unconcerned with what other things Domains talk to, or even how, all that matters is that within a Domain there is one logical view of the world, tailored to the requirements of that specific Domain. Beyond that Domain, there may be a large central database, there may be a perfect canonical data model, but at application level, we have the Domain, Bounded Contexts and Anot Corruption Layers to protect us from all of that.

DDD gives us a loosely coupled way of thinking.

So SOA is Evil and the One True Authority Database is Evil?

Well hardly, but unfortunately we live in an imperfect world, and the people implementing these noble ideas are often far from perfect. They do not have mystical powers of future vision, and they do make mistakes.

By creating properly decoupled systems, and in this I certainly do not mean "systems that use web services", we can isolate our applications from external change and influence, and ensure we meet the requirements we have today, and can react quickly to future requirements as they appear.

Big Design Up Front is still BDUF, whether it is done at a database level, a service level, or in a canonical data model.

Commercial Suicide - Integration at the Database Level

Jak Charlton — Thu, 14 May 2009 06:44:00 GMT

There are many ways you can commit commercial suicide, but there is possibly no slower and more agonising death than that produced by attempting that great architectural objective, the single authoritative database to which all applications talk.

The theory is good, if we have a single database then we have all our business information in one place, accessible to all, easy to report against, reduced maintenance costs, consistency across all applications, and a host of other good objectives.

However all these noble ideals hide a more fundamental problem, that the single database does not solve any of them, and makes most of them into far bigger problems.

All Information In One Place - The Single Authority

On the face of it this sounds like a great objective - after all developers try to live by the maxim of Don't Repeat Yourself - and data in many place is a clear violation of that principle.

Data that appears across many applications and across many storage mechanisms leads to all sorts of massive problems; inconsistency, duplication, replication, duplicated business logic and code, essentially all boiling down to - you end up with spaghetti data. Spaghetti data is much like spaghetti code - it sprawls, gets tangled up, and becomes hard to pull apart without covering yourself in pasta sauce. This is obviously a "bad thing".

So what is wrong?

Well the first and most obvious thing that is wrong is that all applications have different requirements, and different "world views". Although there may in theory be some concept of a "Customer" for the organisation as a whole, even this most basic of data items varies widely between individual departments and even individual applications within a single department.

You could approach this problem, as many database centric people would do, and say "that is the problem right there, we need to standardise all these applications to use the One True Customer". But that is missing the really important word in the definition of the problem ... "requirements" ... this is not accidental that the Customer is different to different parts of the business, it really *is* different.

Your database guys could say "well your Customer in your application may be different as you have different requirements, but you will just have to fit it into our One True Customer", but this is then like trying to put snakes into a plastic bag - they really don't want to be in the bag, they don't fit too well in the bag, and sooner or later while putting one in some others are going to escape or bite your hand. And when your other department starts putting his snakes in the bag too you will be fighting for who gets to not be bitten.

And worse still - now you are trying to map from your requirements based Customer to the One True Customer, and spend an inordinate amount of time maintaining this translation layer in your application. When that One True Customer changes, as he undoubtedly will as new applications require he is expanded to deal with more data they require, every previous application needs to be revisited, large parts of it need ot be re-written, and the whole application needs to be regression tested again. And you have to do this for every single application talking to your single authority database.

You could just skip this stuff, and rely on your applications ignoring this new data, and rely on the database not caring if they correctly updated new data - but this really will come back to bite you - when that bag of snakes starts getting really large and really full - you really don't want to be the one trying to get new snakes into it.

The Truth About the One True Authority

There isn't one.

There - I've said it - I have upset all those database guys, probably upset a large number of SOA guys (I'll cover Commercial Suicide - Integration at Service Level in a later post), and have totally disagreed with noble business objectives.

The truth is, data must have context - without context, data is worthless, absolutely and totally worthless. Data stored in a database has no context, and therefore has no value. Context is provided by the applications that read and write that data, and therefore they are the only thing that matters, and their requirements are the only thing that matters. That means, they need data that is specific to their application, structured in a way that makes that application meet the business objectives, and in a way that makes that application meet non-functional requirements like resilience, reliability and consistency.

So Why Does Anyone Want the One True Authority Database?

Well, in legacy terms it is easy to understand why database admins and database developers want it - it is their lifeblood, their whole raison d'etre. More importantly, it is the culture in which they were brought up - the data is the important thing, the data is the centre of the universe, the data must be consistent, uniform and pure.

But leaving database developers aside, more importantly why would a corporation want the One True Authority Database (OTADB)? After all, the title of this post says this is "Commercial Suicide", so why hasn't this got through to management?

Well - the promise of OTADB is that it will reduce errors in duplication, reduce waste, reduce duplicated effort and reduce maintenance costs - all highly desirable business objectives. And indeed, from those that advocate this approach (those database admins and developers again), the OTADB sounds mighty attractive. On the face of it, it achieves all of these objectives.

Where it falls down is that this holy grail of software development is always just out of reach, they never quite manage to achieve it. Each application that is developed starts to make the OTADB worse, people start to hack things into it to get it to meet business requirements, not because developers want to hack things together, but precisely becasue the restriction of the OTADB *forces* them to do it that way if they are to deliver any kind of functionality at all.

They blame these hacks for ruining their vision of the One True Authority Database, the database admins tell them that they have to fight the application teams to stop them messing up their nice database, but that the OTADB no longer meets the noble objectives as those pesky development teams have messed it up for everyone.

Wait a Minute - What is the BUSINESS Objective Behind the One True Authority Database?

If anyone was to step back from those noble objectives and ask a far more fundamental question, the solution might actually be a lot more obvious than it may seem. While they are all noble objectives, largely actually made worse specifically by the OTADB approach, they are not the real business objective.

Underlying all the other requirements, the ultimate business requirement that drives people (in particular database admins) to want a single database is so that they can see what their company looks like, in other words - so they can produce Management Information - reports to you and me.

This is the single and most fundamental requirement for a business - to have a clear, consistent, accurate and up to date picture of what their company looks. This is what management needs, it is what allows them to make decisions, allows them to identify problems and allows them to spot opportunities.

So, we are going to all this effort, and believe me it is extensive and significant effort, all to support some reporting tools at the end of the day. Reporting tools have problems with data in different formats, with data that is inconsistent, with data that is disparate and distributed. So at some point in the past, the "accepted truth" became "we need one true authority database to be able to produce good reports"

Reporting is a Context - and data only has purpose and relevance in context

If the Elephant in the Room is Actually Reporting, How Do We Solve The Elephant Problem?

This is almost so easy to deal with, it is silly. Perhaps it is because it is so obviously simple that is has been overlooked by many and rejected by others. Especially as it violates another one of those noble objectives ... to provide quality reporting information, we duplicate more of our data.

Yep - we duplicate it - after all reporting data is read only, so it doesn't matter if it is just a copy of other data. Reporting requirements are also very different to transactional requirements too, so we get the added benefit of being able to optimise that duplicated data for the reporting functions.

Data in relational databases is actually very poor for query and reporting purposes, and there is a constant compromise to make it fast for all those applications to write to, that makes it poor to report on - and vice-versa.

How this data gets into the reporting database isn't my direct concern in this blog post, suffice to say the "easy" way is to publish messages with data changes, and have a reporting application pick those up and persist them. My point here - is that splitting the reporting functions from the day-to-day business functions pays massive dividends.

Now We Have a New Problem

That still leaves us with one problem - what happens when disparate applications really do need to know about data in other applications? What happens when my call centre operatives are asked to update the address for one of the Customers. Now as each application has it's own view of the world, and it's own data stores, my accounting applciation does not have access to that change.

Well, the solution to the "how does data get in the reporting databases" question is exactly the same one here - you published messages from your application when you have changes that the rest of the corporation may be interested in. Fire off a message saying "CustomerAddressUpdated" and any other applciation that is concerned can now listen for that message and deal with it as it sees fit.

As It Sees Fit

And this is the real business objective we were trying to achieve in the beginning ... avoiding Corporate Suicide.

When applications are each responsible for their own data, their own actions, and are only responsible for letting the "enterprise" know they have made some changes that other things may need to know about - then you have your solution.

In good development terms, we have proper separation of concerns ... applications are responsible for their data, and their data only. They decide if they care about data from other applications - they are not forced to use it, nor to work around it.

DDD: Videos from Øredev

Jak Charlton — Sun, 26 Apr 2009 18:36:14 GMT

Leonard Petrica kindly pointed out that the videos from Øredev might be useful to those interested in Domain-Driven Design. They include talks from Jimmy Nilsson, Randy Stafford, Dan Berg Johnsson, Einar Landre and Eric Evans.

So if you are interested, check them out

We Are Not Doing DDD

Jak Charlton — Wed, 08 Apr 2009 20:34:00 GMT

Recently I joined a new project, and within reason, and excluding some legacy systems we have to talk to, we have the “luxury” of an almost greenfield project. Probably as greenfield as you realistically get anyway.

I was brought in partially as a good old fashioned coder (the project needed another person cutting code) and partly as guys in the team knew my blog and general approach to software development and wanted to inject some of those ideas into the team. I’m not sure exactly what they were expecting, and I’m not sure if they got what they were expecting, but suffice to say the first two weeks has been turbulent and frenzied.

I’m certainly not known for my subtlety, nor for my coding skills which are by no means the best you can buy in. What I feel is my key strength is my ability to look at the wider picture than sometimes teams have the luxury to do, and to push some of my enthusiasm and energy into a project. I’m sure I’ll blog in the near future about the actual impact my presence has had on the team and the project, but this blog is primarily about why we are not going to be doing DDD.

But Casey Does Domain-Driven Design

One of the factors that made the client hire me was my knowledge around DDD, or at least my projection of that knowledge on the web. It was certainly something they felt could have helped some of their past and ongoing projects, and something they thought would be a real help to them. I don’t think anyone suspected I would come in and say “Hey guys, lets all do full on DDD”, so luckily for them I didn’t, what may have surprised them is that I did pretty much the opposite. In fact my early assessment was this project really didn’t have any need for most of the stuff in DDD – fundamentally it, like the other projects in progress by the same team, was a CRUD application at heart. Data out of the DB, a bit of manipulation, some input by users and services, and dump the data back to the DB.

Many out there disagree with my general statement that CRUD applications don’t make good candidates for DDD – but not only do I stand by that statement, but two of the other client’s projects nearing completion showed some of the real pitfalls of DDD. While previous projects had attempted to use parts of DDD to help simplify their applications, it had not all gone according to plan.

What Goes Wrong With DDD Over CRUD – The DB Domain Model

The first most obvious weakness of the systems that existed was the Domain Model. It had started with all the best intentions – there was a legacy database that acted as the primary store for the business, and many different systems used and updated this database. So NHibernate was chosen as a way to create a Domain Model over the database, to abstract the system away from the DB and to treat it as a legacy system.

Unfortunately one of the primary requisites of DDD is direct involvement and commitment from the business to the project and in this case that wasn’t part of the project. The team had to come up with their own Domain Model – and in doing so had fallen into the classic pitfall of creating a “domain model” that mirrored the legacy database. Now not only was there a legacy database, but essentially there was a legacy domain model too. A large amount of work was put into getting this NHibernate model right, and in hindsight it has probably consumed disproportionate resources to get this model overlaying the DB.

In addition, although at present it simplifies many operations against the DB, it does not abstract the code base from the database, it just acts as a heavyweight data access layer. Worse still, many of the “hacks” that had to be made to get NHibernate to work with the (frankly terrible) underlying database structure, had actually made the system very fragile, and fairly rigid.

Lesson to be learned: Do not drive your domain model from the database, drive it from the Domain Experts, user stories and use cases. Without significant commitment from your business to provide you with those experts, you are at risk of creating a significant amount of code, with no quantifiable benefit.

What Goes Wrong With DDD – Where is The Ubiquitous Language?

A somewhat related problem that is now obvious, again in hindsight, is that there was no clear attempt made to extract any kind of Ubiquitous Language out of the (non-existent) domain experts, nor from the business analysts. The language between the teams has fallen into the classic pitfall of a language confused across development teams, technical support teams, legacy systems, and business terminology that was not clearly defined nor understood.

Lesson to be learned: Get your language sorted out early on. Insist upon a basic glossary of terms at the very minimum, regardless of whether you do DDD or not. This will save countless hours of mistakes and confusion, and will get new developers and business people up to speed on the project in record time.

What Goes Wrong With DDD – Services Services Services!

I sat down early on with numerous team members and the architect who was in the unenviable position of making all this stuff work, and ensuring all the new stuff learned from previous mistakes and moved forward.

One of the things that became apparent to me was that while discussing architectures, the term “Services” was in frequent use, more than frequent to be honest, it permeated through everything. It was a term that business analysts and project managers were comfortable with, and perhaps that was the reason that “Services” was the terminology for decoupling things. From that it soon became obvious that Services of all kinds had become intrinsic to the current codebase, from Web Services, to Application Services, to Domain Services, and on and on.

Now, there is nothing wrong with “Services” per se, but interestingly when I suggested a new architecture for the new project, and discussed this with various people, I managed to explain the entire new architecture with almost no use of the word “Services”.

When I pointed this out, it soon became clear we could discuss a decoupled system, with little direct reference to “Services”, as fundamentally Services are an implementation detail, and have little important to the overall architecture of a system. Now instead of talking about Service being called, we can discuss messages being moved around the system, without any need to worry about what consumes them, or how they are delivered. The final architecture includes services for sure, but they are the implementation of a much more abstract concept.

Lesson learned: Technical language can become the norm, and then it tends to be the solution to every problem. Step back from things once in a while, and see if the terminology you are using is driving your architecture and design. Try to avoid using implementation terminology in relation to things like architecture, or it will drive the design.

So Where Are We Now?

The one really big thing that has happened over the past two weeks is that the introduction of some new blood to the team, along with some new ideas, has really revitalised the team. Everyone has new found energy. We also have a broad consensus of what didn’t go right on the last two projects, and the things that went well – we now have a strong idea of how to build on the strengths of the existing systems, and where we need to compensate for the things that didn’t turn out as planned.

Luckily the teams involved are all really eager to learn, and really keen to make each and every system better – and that alone will make a huge impact to how successful this project will be.

Well, this post has turned into a bit of an EPIC … and is far too long to continue without boring everyone to death … so I will post the concluding part soon. And with luck I may explain my opening statement … “We Are NOT Doing DDD”.

del.icio.us Tags: DDD,Domain Driven Design,Architecture

DDD: Invariants or Contextual Validation?

Jak Charlton — Wed, 11 Mar 2009 08:10:44 GMT

Greg Young made a good point to me regarding my last post about Validation, Consistency and Immutability, specifically around validation – even more specifically he thought I may have simplified it too far.

I was trying to cover the basics of the subject, but may have made the bit around validation a little fuzzy – over the series I have been deliberately taking baby steps so as not to lose anyone early. A lot of the confusion I see around DDD is largely around people jumping right into the deep end without the basic stuff clear first.

The last blog post had a very long section that I removed – it was entitled “Invariants” – but after I proof read it, it made absolutely no sense to me, so I figured it would make no sense to anyone reading it either. Turns out, by removing it, I may have made less sense too! I’ll try again. (disclaimer: suffering from really rotten head cold, so this may make no sense either – feel free to tell me if so)

Invariants

As the name says, an Invariant is something that cannot change – it cannot be varied.

Greg’s point was:

not to get philosophical but... consider Plato's ideals. What attributes of an object make it that object? Must be valid!

This is what I believe Greg likes to refer to as an invariant – or what I was trying to explain around using constructors to enforce “valid” entities.

This then splits our validation into two distinct camps – the invariants and the contextual validation.

An invariant is something that cannot be changed about our object – using our previous example, it must have a CustomerNumber, a FirstName and a LastName, and these must always be valid individually and when evaluated together – there can be no variance from that rule – and so we enforce that by only allowing setting these via the constructor or a strongly named method. No Customer can ever exist without these being valid.

The things that make our Customer a Customer are those attributes.

Contextual Validation

In my previous example, I used the concept that a Customer may have a Policy – let’s say this is an insurance scenario. I also said that the fact that the Customer didn’t have a Policy, it didn’t mean the Customer was invalid, it just meant we would have to have some kind of validation at the point we needed to have a Policy

This is what I believe Greg refers to as Contextual Validation – when we use the entity we may need to validate it is in a fit state for the specific operation we wish to carry out, but we shouldn’t have to validate everything about it.

In Conclusion

Greg said he is doing up a blog post of his own on the subject at some point in the future, which is good as the issue of validation seems to be a sticking point in DDD and in development generally. I definitely look forward to reading it. Keep an eye on his blog at Codebetter for it.

By defining the types of validation more accurately, we can try and avoid some of the confusion which I was introducing in my last post on the subject.

Invariants are the rules that cannot ever be broken about an entity, aggregate or VO

Contextual validation is a check that for a particular action, the entity, aggregate or VO is valid at that point.

del.icio.us Tags: DDD,Validation,Invariants

DDD: Sample Application First Steps

Jak Charlton — Mon, 09 Mar 2009 16:39:14 GMT

As mentioned in my last post, the DDD Parcel Service is opening up for business soon. Also as mentioned, the first thing I did was to grab my Domain Expert and start some initial conversations around the first areas I am approaching – the Booking service.

All of the documentation I am generating is to be included in the SVN (SVN public access to follow in the near future please be patient), so hopefully as the design evolves, and the UL and model change with it, this will be evident to some degree within that documentation. I’m going to get boring again, but the documentation is far more important in DDD than many of us Agilists would traditionally like – at least the resulting artifacts are, so it is important to me to show in the documentation why I am making code decisions.

The Tools

I am primarily using two nifty tools at the moment for capturing the information I need – the ever excellent MS Office OneNote 2007, and a new addition to my arsenal – Balsamiq

OneNote provides and excellent dumping group for notes and ideas as they get fired at me, and quickly allows me to form those into some coherent format.

But the true star of this process so far is Balsamiq.

User Interface Lead Design?

I wanted to validate the User Stories I had extracted from my DE, by showing how these could take form within the UI. To some this may seem like heresy, after all DDD is all about the domain right?

Well, I actually see DDD as being lead by a combination of the Ubiquitous Language on one hand, and the User Stories and Use Cases on the other.

Extracting the UL from the initial User Stories was part of the first task, but validation of these was far easier to do with some visual aid – so a mockup was needed. I could have done this in HTML, but luckily Balsamiq recently became a devlicio.us partner and I was keen to try it in anger on a “real” project.

Mockups Made Simple

The initial user story we worked out was around booking a parcel delivery, and after some discussion we got to:

The Customer wants to be able to send a Consignment , so they log onto the Client website to Book a Delivery:

They choose their Pickup Location
They choose their Destination
They choose their required Pickup Date
They choose their Delivery Window options
They choose the Consignment details
They choose any Special Instructions
They receive a Quotation for delivery if the system is capable of quoting, or a Callback for a Non-Standard Consignment
If they wish to proceed they click Confirm

Firing up Balsamiq was quick, and within a few minutes of drag and drop, and a few moments of cursing when I kept forgetting to lock items, I had a good starting point for the UI:

Less than 30 minutes later I had the whole story mocked up and ready to validate. Not unsurprisingly, it was really easy for my DE to look at these screens and envisage the process in action. This lead to a few changes in the UL, and a couple of extra questions arising – exactly the kind of validation that was needed. Had I done this in HTML or code I could have spent a good many hours doing something not half as representative.

If you want to see the initial modelling you can download them as PDFs here:

Kudos to both OneNote and Balsamiq for excellent applications to let me gather and organise the results of my initial conversations!

del.icio.us Tags: DDD,Balsamiq,OneNote,Modelling

DDD: The Beginning of My Sample Application

Jak Charlton — Sat, 07 Mar 2009 19:08:20 GMT

The Domain Driven Design Parcel Service is a small competitor in a hugely competitive market. With such competitors as UPS, FedEx and DHL, we are at a competitive disadvantage, and need a solution to cover our interactions with our customers, and to streamline our internal processes. This software will be at the heart of our expansion plans and needs to grow as the business does. Although we are currently only a national carrier in the UK, our long term vision has much wider reach.

The Domain Vision Statement (Evans page 416):

The DDD Parcel Service Booking Domain is to be responsible for Customer interactions. It should reflect the processes and practices our Customers want to use. This should reflect the relationship we want to have with our Customers, and easily allow them to deal with booking processes, managing their accounts with us, our discount and promotion schemes, and allow them to see a historical record of their business with us.

And So It Begins

I needed something concrete to start putting some of the concepts from this series into, to try and show how the patterns of DDD can be used in a “real” code base. In addition, I have spent a long time thinking about the specific domain for the example application, as it had to be something that had real meat to it, and real behaviour.

I could have gone down the shopping site route, but expanded it to be more like Amazon than a small one man operation – but as shopping sites have been done to death I had to find something else. Unfortunately the original example application for DDD also exists in a similar sphere to my choice. I did consider this carefully, and have weighed up many options.

The domain chosen had to be:

Easy to understand when you put yourself in the shoes of the business
Easy to understand when in the shoes of a customer of that business
Simple enough to not get swamped in technical details
Complex enough to show a number of aspects around the things DDD brings real benefits to
Have real behavioural process rather than just another database and a pretty UI

In addition, I had to be able to deal with the domain with little friction – although it may have been fun to learn some of the domains that various people suggested to me (including the Oil Drilling Industry!) – I only have a limited amount of spare time, and my girlfriend has a big priority on most of that.

The DDD Parcel Service fulfilled all of my simplicity and complexity requirements, it is a pretty common domain to most of us in everyday life, at least from the Customer perspective. It also happens to be a domain that I have touched on in the past from a business perspective.

It happens that with my choice of a Parcel Service domain I had easy access to a Domain Expert of sorts – my girlfriend has a long family history in the haulage business, and on a daily basis manages the haulage for a customer just like those that the DDD Parcel Service would have.

This Morning’s Progress

Despite both myself and my girlfriend suffering bad colds, I pinned her down for 20 minutes to start the first session on extracting some of the user stories for the DDD Parcel Service

I then started hammering away at a few screen mockups in Balsamiq … and I think on that point I’ll start a new blog post for tomorrow or the day after.

So Where Is The Code?

Don’t be in too much of a hurry! I have started a project, and have the core framework starting to form up. But more importantly to me, and to DDD is the design, the extraction of the UL, the agreement of the User Stories, and the way we get to the code. Over the next week or so I will try and start turning the initial stories into a beginning code model – and at that point you can all rip it to pieces!

Technorati Tags: DDD,User Stories,Balsamiq

Make Your Own RSS Feeds With Yahoo Pipes

Jak Charlton — Fri, 06 Mar 2009 15:11:14 GMT

When setting up the DDD series on http://dddstepbystep.com I wanted to bring it various information from across the web, to try and consolidate a lot of Domain Driven Design info all in one place.

My first thought was I would setup a blog, and whenever I found something that I thought was a good fit, I would copy a summary to the new site, and link back to the original piece. This was a great idea, apart from the fact that it was going to turn out to be horribly time consuming.

The next step was finding a better way of doing this, hence one of the reasons I hose to host the new site in Community Server – it has a “Mirrors” setting for each blog, where you can add any number of RSS feeds, and it will bring in a summary for you every time the job runs.

After a lot of messing around getting CS set up, I found one of the major limitations of CS is the “Mirrors” functionality … although it is good at actually bringing in the feeds, it has no way of filtering, combining or correlating feeds. It also has an arbitrary way of detecting duplicate entries, which appears to be based solely on the title – which is less than simplistic.

After a bit of searching and trying various options, I came across Yahoo Pipes – this has got to be one of the coolest tools on the web right now.

With Pipes I was able to set up a complex set of feeds, followed by a series of transformations that allowed me to sort out all the discrepancies that CS was unable to deal with.

The “DDD on the Web” Blog Feed

Firstly I was able to combine half a dozen feeds of blogs I know regularly have great information on DDD subjects. This was a simple matter of giving the Fetch Feeds source the URL of each RSS.

Then I needed to filter out any blog posts by myself – as one of the feeds is coming from devlicio.us I didn’t want to duplicate my posts on each site, as I post the full blog directly to the new site as well as here. With a quick drag and drop of the Filter operator, my posts were gone.

Next I wanted to remove all the posts that didn’t mention DDD or Domain Driven Design – another Filter in place and I was closer to what I needed.

The Unique operator allowed me to then remove any duplicate posts based on their content – the CS one would have allowed duplicated content if the title was different.

Next I wanted to add the original author name into the title … to turn “This is my Blog Post” into “This is my Blog Post by Dave D Spatch”. Things then got a little tricky. As I was bringing in feeds from various sites, and they were inconsistent in their naming, the author fields were often coming through in the wrong place. Using a combination of the Rename and Regex operators allowed me to sort this out and provided me with a much better title. A last Regex allowed me to replace a number of “username” type fields with the full names of those I knew – so for example I replace “colinjack” with “Colin Jack”.

And finally – we get a single feed that I pass on to Community Server for it to import. The last step was a simple cosmetic change, I changed the URL from some horrible id string, a nice human readable one: http://pipes.yahoo.com/dddstepbystep/ontheweb

All of which is now brought in correctly to the blog at: http://dddstepbystep.com/blogs/ontheweb/ where I can still manually add anything of interest the feed has missed.

The “DDD at Yahoo” Blog Feed

I wanted to also bring in the feed from the domaindrivendesign group at Yahoo – where a lot of general discussion around DDD takes place.

This had some similar elements to the other Pipe, but also required me to work around the limitation of Community Server I mentioned earlier – it detects a duplicate item based on the title alone – and skips it if it finds an existing match.

As the DDD Yahoo group has a lot of “Re: Your Subject” titles, CS discards all but the first it finds if you use the RSS directly.

Pipes again allowed me to resolve this, by appending the author to the beginning of the title, and adding the time to the end to the title, I can get almost uniqueness. The title looks a bit ugly, but I couldn’t find a better way to fool CS into doing it right.

In Conclusion

Yahoo Pipes is a superb tool – it can be a bit tricky to understand at first, and some things still seem convoluted to me … but it has allowed me to work around some tricky limitations with relative ease.

If you find a need to make your RSS feeds just a little better – then take a look. Better still, it can work with data from all over the web, and turn that into an RSS feed for you.

del.icio.us Tags: DDD,Yahoo,Yahoo Pipes

DDD: Validity, Consistency and Immutability

Jak Charlton — Wed, 04 Mar 2009 16:47:05 GMT

There seems to be some confusion around these and similar concepts, so I thought it might be an idea to provide some clarification. Now these things aren’t specific to DDD, but they certainly have a lot of relevance there, and often provoke furious debate.

In this series I first mentioned these things back when I introduced Entities and Value Objects, I said that you should try and ensure Entities are valid at all times, and that Value Objects should be immutable. The discussion around Aggregates and Aggregate Roots has also proved to be of some confusion, the definition from Eric Evans of an Aggregate says “A set of consistency rules applies within the Aggregate's boundaries”.

Download the ebook of the series so far (also published on dddstepbystep.com)

Validity

The term “valid” has many meanings in many contexts. When it was mentioned with regards to an Entity, I suggested that you should attempt to ensure that your Entities are always in a valid state, and cannot become invalid. I followed up that this could be achieved by removing property accessors, and only using either constructors or strongly named methods.

Let us go back to our Customer example, and presume that for a Customer to be valid within our Domain, it must have a CustomerNumber, a FirstName, and a LastName. It may also have half a dozen other properties.

So we can ensure this Entity never becomes invalid by removing the property accessors for the CustomerNumber, FirstName, and LastName, and providing a single constructor that allows us to create a new Customer … Customer(customerNumber, firstName, lastName)

Now there is no possibility that this Customer can become invalid – it either has these three properties or it cannot be created. If we want to be able to change the Customer’s name, we can create a new method .UpdateName(firstName, LastName)

By taking this approach, we ensure that the entity can never be put into an invalid state, for example a developer setting a .FirstName property, but forgetting to set the LastName at the same time.

Some have taken this concept too far, and assume that as a Customer also has a Policy, that it must always have one of these or it is invalid. Of course, there are things we may want to do with our Customer that will require it to have a Policy, but that does not constitute an invalid Customer, it constitutes a rule that must be applied before we attempt to take the action.

The purpose of ensuring the Entity is always valid prevents us having to do messy things like checking an .IsValid() method every time we want to persist our entity, or we write a function that accepts an instance of our entity. Enforcing an “always valid” approach will mean that we do not have to code multiple guard clauses scattered across our applications.

Consistency

The term consistency has come up in this series and around DDD mailing lists. In this respect, consistency is the idea that one piece of data in the system is consistent with another. Here is the bad news:

You cannot ever achieve total consistency

OK – headline over. The point is that something in your application will always be inconsistent. The best example is that when your users have chosen to edit the Customer on their screen they have a copy of that data. By the time they have received that data it will potentially be inconsistent with the domain, and with your persistence store – after all this is why we have the concepts of optimistic and pessimistic locking. When they submit a change to the Customer, it is probably minutes out of date.

Back to our headline, when you accept that it is impossible to achieve total consistency, you can deal with the real problem – how you achieve eventual consistency. Eventually messages will make their way through our domain, eventually data will be persisted, eventually you will have consistency.

While our reporting domain may lag behind our primary domain, does it honestly matter? How important is it that they are both totally consistent? You can certainly achieve a close facsimile of total consistency, but it will come at a high price – does the business want to pay for this?

When you let go of the premise that your data is ever really consistent, it now just becomes an issue of an SLA agreement, if the business really wants data that accurate, then explain the costs and tradeoffs and go with it. If after an explanation they understand that “almost consistent” is good enough for almost all applications, then they just have to make a decision about how important this is to them.

Consistency also applies at a much more granular level, with the domain for example. Entities and Aggregates may be valid, but not consistent with each other at all times. Eventually they should become consistent, but ensuring they are all consistent at all times is very tricky to say the least.

While an Aggregate Root is responsible for ensuring consistency with the Aggregate, it is acceptable, and almost inevitable that Aggregates will be inconsistent with each other at some point in their lifetime.

Again, letting go of the myth of total consistency here will give you far more flexibility.

Immutability

I would hope this is familiar to all – but perhaps some of you saw me mention it and didn’t quite get my context.

The concept of immutability is pretty much what it says – the thing cannot be mutated – or in context, an Immutable Value Object can be created, but can never be changed.

If you want to change an immutable object, then you can create a new one and replace the original.

By making objects immutable, we avoid many problems with validation and consistency, only the constructors on the object matter, and they only need to validate the parameters passed in. It is far easier to check all the parameters at a single time than to check individual properties against each other at a later stage.

del.icio.us Tags: Validation,DDD,Domain Driven Design,Immutability,Consistency