Jak Charlton - Insane World : Mocking

Make It Easy To Refactor

Jak Charlton — Mon, 21 Sep 2009 15:17:00 GMT

I'm currently looking at a code base that could do with some refactoring, overall it's a pretty good code base, and the test coverage is pretty good too. But looking through it gave me some ideas on what makes a code base tricky to safely refactor. Here are some tips to make sure your code is in a good state to be refactored with safety and confidence.

Provide Good Test Names

Test methods should have good, meaningful names; this is of course exactly the same as all code - naming is very important to give context and meaning. I am looking at a lot of tests that make sense to the original developer, and within reason make sense to me, but they do not properly describe what the test tries to do and what it expects.

A test named "ValidHardware()" is less than ideal for telling me the intent of the test. If the method had been correctly named it would have been called:

ensure_that_hardware_detector_returns_true_when_valid_hardware_is_present()

Now you tell me, which is easier to understand? And which is going to give me more confidence in refactoring that hardware detector?

Make Sure Variables Are Well Named

Probably blindingly obvious given the above section, but if you use poor variable names, I am probably going to waste a lot of time trying to figure out exactly which thing is which, and worse still, I am probably going to miss subtle bugs.

Cover The Right Things

Beware of high coverage that does not actually test either all the aspects of a component, or that makes a lot of assumptions.

Getting 100% coverage is likely to be impossible, and probably your return on investment over 75% is going to be low, but don't achieve the 75% by testing the trivial and easy to test things only - it's going to be those tricky to test components that will cause most bugs when refactoring.

If something is tricky to test, perhaps you have the wrong level of abstraction applied, or perhaps the test is trying to tell you something. Skipping it and going for some simpler tests may pay off in the short term, but in the long term will hurt you.

Avoid Fragile Tests

There is a lot of debate between the "classicist and mockist schools" about what to test, and personally I veer towards state based testing for most components - but regardless of where your personal impulses lie, avoid creating fragile tests. Any test that is likely to break with trivial refactoring could be considered fragile, and those tests are going to cost you a lot of time and cause many potential bugs when refactoring.

One of the main causes of fragile tests are an overreliance on mocks, and specifically using mocks over stubs. By all means mock where necessary, but try to limit mock usage to very specific test cases, and try to favour stubs over mocks and favour "Setup.ResultFor" over "Expect.Call" type semantics.

Avoid [Setup] and [Teardown]

Yes, I know they make it easier to create lots of similar tests, but they make your tests fragile and interdependent.

James Newkirk (who was on the original NUnit team) made a good blog post on this when he moved to writing xUnit, where Setup and Teardown were removed.

They make it hard to read your tests (you have to read code in two or three locations to understand what the test does) and can cause Setup/Teardown getting too many responsibilities, leading to fragility or code bloat.

Either switch to setup within the individual test, or favour a Context based testing approach such as with BDD

Conclusion

Tests are code, and code has to be maintained too. Make sure your tests are of a similar or higher standard than the rest of your codebase, make sure they test the right things, in the right way, and you will make your life much easier, and that of the guy who comes after you much easier too

The Language of Mocks (or is that Test Doubles)

Jak Charlton — Wed, 11 Jun 2008 09:25:00 GMT

Ian Cooper brought up an interesting point on the altdotnet mailing list recently, around the language used to describe Mocks, or more accurately his preference for the term Test Double over my use of the generic term Mock.

In most areas of development I can often be considered a language perfectionist, for example calling something a Unit Test when it is *clearly* an Integration Test drives me nuts - but when it comes to Mocks, Stubs, Fakes, Spys, or the rather more generic Test Doubles ... I just can't get passionate about the language.

This all started a while ago, when people started using the term mock, but really it wasn't a mock - or more often they used the term mock when referring to what many mock frameworks call a Strict Mock, just one type of mock.

Martin Fowler long ago posted on a clarification of the terminology around mocking, and xUnitPatterns has a fuller description. The distinction that Gerard Meszaros introduced was to use the term Test Double to describe all of these things we use to help us test our code. Fakes, Spys, Mocks, Dummys ... all just a type of Test Double.

Now language is important. Very important. As Ian pointed out in his email, most arguments centre around a misunderstanding of the terminology involved - I'm sure the world would be a nicer place if language was universal and could not be interpreted differently by different people, but then it would be a linguistically poorer place for it too.

In development, language is something that is vital to good communication of what are often very complex technical concepts. We evolved the languages of patterns and principles to allow us to avoid confusion and misunderstandings, and most areas of development have their own specific language terminology.

But, with this one area of testing I have a problem with the terminology, because the term that I hear from developers on a daily basis is "mocking", not "test doubles". And when terminology enters the common language, it is very hard to replace it with a new term just because you prefer it. I'm sure that Test Double describes the concept better, and I'm sure that it is important to have some distinction between Mocks and Fakes, but in my experience, that is not how developers refer to these things.

It is much easier to say "we can mock this out" than it is to say "we can create a test double here", the first is just more natural in the English language. So the term "mock" has become almost a defacto term for what Martin Fowler et al refer to as a Test Double, whilst also being a Mock with a capital M for describing the Mock that most mock frameworks create.

So, much as I might like to be a language perfectionist on this one, I tend to just use the term mock, and leave the decision as to whether it is a Mock, Fake, Stub or Whatever to the point I actually code it. So far nobody has misconstrued Mock to mean "you must use a mocking framework", or at least not anyone to whom the concepts of mocking are not entirely new anyway, in which case much more clarification is needed in any case.

Is Alt.net Duplicitous?

Jak Charlton — Wed, 21 May 2008 09:54:00 GMT

Roy makes a number of accusations about the Alt.net community, particularly centred around their espoused ideals clashing with their comments and blogs. I respect Roy greatly, his blog was one of my first real sources for great TDD information, and he has consitently put out great blog posts and software. Largely his complaints come down to TypeMock (who he now works for) not getting an overly warm welcome by many people in the Alt.net world.

I don't for one second question Roy's sincerity, he really has shifted his view on how testing and mocking should work, for what he sees as all the right reasons, and now he seeks to convert others to the same cause.

I'm not what you might call a "paid up member" of the Alt.net movement, my blog address appears on altnetpedia.com, I frequent the Alt.net mailing list, but I have never attended one of the conferences, nor do I put the Alt.net logo on my site. I read a few dozen blogs, probably a third would be classed as Alt.net people, most would not be. I guess I have a foot in Alt.net but dance with the devil.

I am however one of the people that has commented a number of times on TypeMock and its suitability as a mocking framework (for my needs), and Roy's obvious frustration comes down largely to his perception that people are bashing TypeMock unfairly, or out of dogma, and not seeing his vision for a better way to write software that TypeMock can help with.

That is perhaps the case with some people. TypeMock has some fervent supporters, and there are some people that think TypeMock encourages bad practices and should be avoided - software for some reason is one of those things people become almost religious about. I suspect that most people, myself included, are somewhere in the middle.

TypeMock is to me largely irrelevant. It has some plus sides (debugger support, mocking legacy and framework classes easily), and it has some down sides (it makes it "ok" to couple your code, or to use concretions over abstractions)

But mostly I just cannot see the point of TypeMock as it exists. It is, if you exclude the plus and minus points I just listed for a moment, basically the same as Moq or Rhino Mocks. It does more or less the same things, in more or less the same way, except both of those options are free, and TypeMock is one of the most costly development tools you could own.

I'm certainly not against commercial software, I own personal licences for all the software I use on a daily basis, but those products I choose to purchase are the best in their class, or solve a particular problem I face. I also weigh up the cost to value benefit.

The best comparison would be against unit testing frameworks, which are all to the best of my knowledge open source. There are commercial tools which make them easier to use, but there is no need for a commercial unit testing framework, as they are at least 95% feature complete.

The same could be said of mocking frameworks, Rhino and Moq are at least 95% feature complete, and pretty much (in all day to day ways) the same as TypeMock. There is for me, no one "killer feature" that TypeMock has that justifies it's price or putting a dependency upon it within a code base.

So is Alt.Net guilty of the duplicitous charge that Roy has made?

Expecting all people who frequent the Alt.net world to be constantly flicking between technology and tool choices is not going to hold water. A typical software project may last weeks, months and possibly years. You cannot switch and change that frequently, you have to make choices and go with them. That doesn't stop Alt.Net people wanting to become aware of more options, but it also means that new options have to get compared to existing options, and even when a better option comes along, it is frequently not going to be used because of historical reasons. People still have their favourites, they still have their personal biases, and they still consider themselves as capable individuals who can evaluate these things.

I honestly don't think TypeMock in it's current form is that revolutionary that we should be sitting up and rushing to use it. The few new things it brings to the mocking party are different, not much better or that much worse. So picking the right tool for the job, means most of the time I prefer to write a wrapper or abstraction (a few wasted minutes) as opposed to changing an entire mocking strategy, all for the sake of a shinier toy. Others make take another approach. When I encounter a real world scenario where I need the power that TypeMock has, I'll re-evaluate.

Sometimes, the newer tool isn't the right tool for the job.

Reflector Magic - How Hard is it to Abstract a Legacy System?

Jak Charlton — Fri, 02 May 2008 11:55:00 GMT

As it sort of came up via a previous post, and on the altdotnet mailing list ... I thought I would share a little secret with you ... abstracting out legacy systems can be pretty darn simple with a bit of Reflector magic!

Step One: Get Reflector

You can download it here if you don't already have it (how could you be writing code without it???)

Step Two: Get some Addins

There are some really cool ones at CodePlex

Make sure you get the Doubler Addin

Step Three: Run the Wrapper / Interface Generator

Pick your favourite legacy code base (MOSS is my particular favourite), pick your favourite nasty classes (SPList is just so damn sucky) ... and let Doubler work it's magic.

Voila!!! ... no hard to write or maintain decorators, no painful legacy system to mock, and frankly, the Doubler code is so light that you don't even need to test it - just use MOSS as though it was a really well written piece of software (OK maybe that last bit is pushing the trade descriptions act a bit far).

Now you can use your favourite mocking framework, no need for the magic that some provide ...

TypeMock - Powerful, but Needed?

Jak Charlton — Thu, 01 May 2008 14:33:00 GMT

A while back Roy Osherove posted an offer to the altdotnet mailing list on behalf of TypeMock:

Reminder: TypeMock has offered a free Typemock Isolator ent. license for one year for anyone who has registered to the current ALT.NET site.  Also, if you're an MVP you can get the same license for free.

We do ask that once you use it, you write in your site\blog at least ONE thing that you don't like about it, or that could be better (it has to be constructive so that we can learn from it!) and hopefully things you like about it.

A fine offer, how could I possibly pass it up? Hopefully this post covers that "ONE thing".

I have looked into TypeMock before and found it to be "good" but not "good enough to make me want to switch tools". My previous comparison was fairly trivial, a comparison to Rhino which I was using, but in a problem domain that included MOSS 2007, a perfect candidate for TypeMock, as it is the pure epitome of a legacy system - yes it is possible to have a legacy system upon release.

Microsoft Makes Software That Only TypeMock Can Make Mockable!

MOSS makes life hard for developers due to a very simple, and very silly, design decision ... Microsoft like statics, they like enums, they like Internal, and they like Sealed. Far be it from me to judge their reasons behind these things, but suffice to say it makes testing a right royal pain in the posterior.

TypeMock is the only mocking framework that can help with this kind of problem, as it uses the .NET profiler to intercept the parts that other frameworks cannot reach, notably it relishes mocking things like MOSS. Unfortunately, when I was working on MOSS stuff and could have used it, there was enough resistance to testing in general, that proposing we spend a not insignificant sum on a mocking framework was tantamount to bending over and asking to be kicked very hard. I couldn't find any compelling reason why we should try to use TypeMock, over testing pretty close to the MOSS wire, but not right up to it. The solution I came up with was an abstraction of SharePoint.

No Separation of Concerns - TypeMock to the Rescue?

Now I have moved on to a new client, where my role is somewhat different. As the "Lead/Technical/Solutions/Whatever Architect" it is within my remit to recommend better ways of doing things, and in theory mocking is one of those concerns. The particular project I am overseeing is sorely lacking testing, and the concept of mocking is just not within reach.

But again I find myself with no compelling reason to recommend TypeMock, and a few reasons why not to.

Firstly, the thing it would have going for it ... the principle of Dependency Injection, or of Inversion of Control are, or at least have been, alien concepts to this code base. So a lot of code exists where in an ideal world we would be writing tests that would need to mock concreted dependencies, the "killer feature" of TypeMock.

But this needs to be weighed against some other factors.

Is Using TypeMock Helping You Pretend Your Code Doesn't Suck?

There is a lack of testing in general, and for the few tests that do exist, they are largely integration type tests, almost all actually reading and writing data into test databases. This is the mindset that the team has got themselves into - you test against a database. They therefore see little reason to test in terms of small "units" and hence do not really see why they should mock the ADO.NET components (for example) to allow them to make those same tests without a database in place - in fact they largely see the tests as being tests of the stored procedures they have written.

Then there is the concept of mocking in general. I often see this scenario, and again it is repeated, mocking is not a basic concept, before you can really get into mocking or at least can see real benefit in it, you need to have gone a long way down the unit testing road, and preferably down the test driven development road. Until you have a firm grasp on those concepts, mocking is quite a hard thing to "get" ... why would I want to mock something? Why don't I just test these 5 classes together? More experienced practitioners of TDD might give a list of reasons, but testing is an art, not a science, and until you "get" unit testing for yourself ... it just doesn't seem worth the effort.

When you do "get" unit testing, you start to think of code in a different way, you start to think of small identifiable software components that interact together, and therefore can be tested. But unless you get DI and IoC as principles, it still seems more obvious to test the 2, 3 or 4 classes that are tightly coupled as a single unit. You need to next make that step where you see that it is more beneficial to pass dependencies into classes, rather than have them contained within the class, and often you gain this insight from an inability to write good unit tests.

TypeMock To The Rescue!!!

"You know those pesky classes that you can't test because they have dependencies, well TypeMock can make those easily testable for you!"

And, indeed this is true. It can indeed save you from all those silly DI and IoC principles - why use a Strategy pattern when there will only ever be one Strategy?

Well for one thing, the tests that you found hard to write without DI/IoC were telling you something ... you had tightly coupled code, and possibly issues around the Single Responsibility Principle and Separation of Concerns.

At this point I can hear many screaming "Oh no, more damn patterns", or as a developer at a previous client said to me "I'm sick of all these acronyms".

Why The Pain You Felt Was Good Pain

The foundation of good software is understanding the basic principles that developers have come to espouse over many years, and many successful and failed projects. And your hard to test classes were flashing a big red warning light over your monitor telling you that you had some potential violations.

Now, with any pattern or principle, there are times when you can and should break it. But until you understand it to a fairly detailed degree, you are probably safer by choosing to follow it rather than ignore it out of ... well, ignorance.

Back to TypeMock saving us earlier, and allowing us to test our classes anyway ... indeed it could have done, but for a lot of developers, it would have shielded them from potentially poor code design, and just stretched their learning process out, encouraging them to write poorer quality code, as they have a tool to get around the problem they perceive (inability to test), and not realising there is a large problem lurking (their software is rigid and will be hard to extend and evolve in future)

I Don't Need No Damn IoC ... I Created This Dependency On Purpose

As mentioned above, there are times when rules are made to be broken - so I am not saying TypeMock should never be used in this scenario, but unless you know the rule you are breaking, and the reason you are breaking it, and have expressed that clearly within your code base ... just don't.

So, on this basis a mocking framework like Rhino or Moq, with their limitation on only being able to test code that is properly decoupled, would show you where your design was weak.

Experience on the Job Market

Then we have the experience factor, if I recommend a tool, I want to have some confidence that developers hired on the project will have experience with it. If someone has experience with unit testing, and with mocking, the high likelihood is that their experience is around Rhino Mocks, the least likely framework they will have used is TypeMock.

How Much???

Then there is the real deciding factor for me recommending TypeMock, cost.

Cost isn't a factor on a per developer, or per project basis, the cost of 350-450 Euros certainly isn't cheap, but on any serious software project it is a small drop in the ocean, at least with a good business and technical justification.

Cost is a major factor as the alternatives, Rhino, Moq and others, give you 95% of the functionality, can be argued to be better as they do *not* allow mocking of tightly coupled code, and more importantly are free.

On any project, when picking tools, you have to pick your fights. Do I want to expend energy getting someone to write a cheque for ReSharper for every developer, or for TypeMock? Do I want to expend energy getting agreement to use Windsor and IoC in our code base, or argue why we need to pay for TypeMock? Do I want to get agreement on NUnit over MSTest, or spend that time arguing that investment in TypeMock will pay off?

Fundamentally, a mocking framework is a very small and insignificant factor in the overall project success or failure, or even of the day to day working practices. Most other tools come as a higher priority in the decision making chain. And as such I find it hard to use (my valuable) time preparing business justifications for a tool like TypeMock, knowing full well that when asked why Rhino or Moq are not adequate for the task, that I will also be struggling for a good technical reason too.

So Where Does That Put TypeMock?

So, my conclusion is, that unless I have a legacy system to mock (put your favourite MS enterprise product in here), I would be very hard pressed to choose TypeMock as my mocking framework. Where I did have a legacy system to mock, I would be hard pressed to justify why we do not just buy a single TypeMock licence for our continuous integration server and treat tests against the legacy system as integration tests using TypeMock, and use another framework like Rhino for the tests that developers would be writing on a daily basis.

So, how can TypeMock make itself more attractive and viable? A very good, and rhetorical, question. Dropping the cost significantly may help, but presumably they have a reasonable number of customers willing to pay that price, or they wouldn't be in business, and I am thinking of a more reasonable price being in the 30-50 euro range, the level at which cost becomes insignificant as a decision point, but presumably this is just not an option. TestDriven.Net is around 130 euros, and bang-for-buck is far ahead of TypeMock in the value game.

And still ... I wouldn't want inexperienced developers wielding TypeMock at their code - I would prefer they stopped, thought and figured out why they can't write their test without it. When they figured it out, then they can ignore good principles from a position of strength.