Derik Whittaker

Syndication

News


Images in this post missing? We recently lost them in a site migration. We're working to restore these as you read this. Should you need an image in an emergency, please contact us at imagehelp@codebetter.com
Strict Mocking Semantics Is the only way to Mock

Recently I was giving a Mocking presentation and I made the statement that I prefer strict mocking semantics over loose semantics.  At the end of presentation one of the members of the audience raised point to my liking strict over loose.  We had a great discussion and I thought I would regurgitate it here and give my reasons for liking strict over loose.

Definition of Strict Replay Semantics:
Only the methods that were explicitly recorded are accepted as valid. This mean that any call that is not expected would cause an exception and fail the test. All the expected methods must be called if the object is to pass verification.

Definition of Loose Replay Semantics:
Any method call during the replay state is accepted and if there is no special handling setup for this method a null or zero is returned. All the expected methods must be called if the object is to pass verification.

Pros of Strict Replay

  • Developer creating the test must know EXACTLY what the codes does they are testing
  • Will cause the test to fail in the future if anyone changes the underlying code
  • Creates a 'what you see is what you get' scenario in your tests.  Meaning that if you do not setup an expectation you will NOT have success.

Cons of Strict Replay

  • May appear to make the test brittle
  • Requires the developer to know EXACTLY what the code does (yes this is also a pro).  Sadly a lot of developers are too lazy to completely understand the code they are creating, hence this being a con.
  • May create a lot of 'noise' (I.e. a lot of mock setups) if your code is not loosely coupled (this is a con, but is really a result of a code smell)

Replay of the conversation
During my presentation I had made comment that I prefer strict semantics over loose and simply left it at that.  I did not really go into detail, but at the end of the session one of the audience members wanted to dive deeper into the why, so we did.

He made a very good observation in regards to strict mocking and that was that my test will fail if it was originally setup to expect a call to the 'db mock' 2 times, but later the method was modified to only need 1 call.  I agreed with his observation, but told him I actually see this as a good thing. 

When we create a test we are testing our code base at that point in time.  We are also testing business rules at that point in time.  When ever I setup my mocks (create expectations) I am doing so with knowledge that a given dependency will ONLY be called N number of times.  If I need to refactor/enhance that method/logic and I change the number of times my dependency will be called, I want my test to fail.  This failing test should prompt me to review my test logic to ensure that I am still testing the intent of the method.

By this logic, strict mocking semantics is the best way to go.  If I had used loose semantics my original test would still pass when it should fail potentially hiding flaws in my test logic.  Remember green tests does not equal good code coverage.

Till next time,

***** Updated *****
Removed a misinterpreted statement from Ayende over at Rhino
***** Updated *****


Posted 05-19-2008 8:27 AM by Derik Whittaker
Filed under:

[Advertisement]

Comments

Davy Brion wrote re: Strict Mocking Semantics Is the only way to Mock
on 05-19-2008 12:26 PM

I don't really understand why you think strict replay will be the norm with rhino mocks 3.5.... CreateMock has been replaced with StrictMock but i think it was done to make the difference between CreateMock and DynamicMock more clear.  The choice is still yours, and it is now a very explicit one, but one is not preferred over the other in Rhino Mocks 3.5

Bill Sorensen wrote re: Strict Mocking Semantics Is the only way to Mock
on 05-19-2008 1:23 PM

Ayende may not share your sentiments; please see

ayende.com/.../Rhino-Mocks-Futures.aspx

I have found strict mocking leads to brittle tests. You have to have intimate knowledge of the code under test; it's no longer a black box. If there are two methods on an interface (dependency) that can produce the same results (Parse and TryParse, for instance), refactoring the underlying code can break the tests. Even when there's a code smell involved, the developer may not have the ability to change the interface (which could be provided by a third party) or even aspects of the code under test.

Derik Whittaker wrote re: Strict Mocking Semantics Is the only way to Mock
on 05-19-2008 1:44 PM

@Bill,

All i got out of that post is the following 'Using strict mocks by default was a bad design decision on my part.'  He is not saying that using strict mocks is bad, just making it the implied default was bad.

As for making your tests brittle, i guess that is a matter of opinion, but if you ask me, this is a good think.  This makes you think about your tests when you are making code changes and this is good.

Finally, nowhere did i say using dynamic mocks is bad, they are good and useful.  But I like strict mocks better.  If you have the need (like testing 3rd party api's) then us dynamic mocks.

Scott Bellware wrote re: Strict Mocking Semantics Is the only way to Mock
on 05-19-2008 8:02 PM

Strict mocks is often a design smell.  It causes tests to express knowledge of an operation's internals, which is a violation of encapsulation of teh system under test.

Any large code base that depends on strict mocking by default will suffer from unnecessary productivity loss due to breaking tests that have to be fixed where those tests suffer from inappropriate intimacy.

An API's internals are supposed to be allowed to change independently of its tests.  This is an essential quality of an agile codebase.

Strict mocking by default is a rather extreme stance.  Agile succeeds by finding those places where strictness can be slackened without causing any observable and categorical loss in integrity.  Opting for the strictest possible default runs contrary to this heuristic.

Derik Whittaker wrote re: Strict Mocking Semantics Is the only way to Mock
on 05-19-2008 8:40 PM

@Scott,

See I disagree with your stance that strict mocks are design smell and cause brittle tests.  Yes they may cause tests to break, but that may not be a bad thing.  

These by the internals changing and your tests not breaking can cause your tests to become stale and non-useful.

Also, with loose mocks simply returning the default for the given type (ie null for string, 0 for int, etc) you may have breaking tests as well.  However, this time you may have no knowledge of why the test broke.

And finally, i never said that strict mocking is the only way, what I said was that I like strict mocks over loose mocks.

Reflective Perspective - Chris Alcock » 2008 » May » 20 wrote Reflective Perspective - Chris Alcock » 2008 » May » 20
on 05-20-2008 3:10 AM

Pingback from  Reflective Perspective - Chris Alcock  » 2008 » May » 20

Zubair Khan wrote re: Strict Mocking Semantics Is the only way to Mock
on 05-20-2008 10:14 AM

I would agree with posters that too many strict mocks create a (bad) brittle test. It leads many to break encapsulation and make refactoring a nightmare.  I would suggest that Strict mocks are only to be used in test that test how a class uses a dependency, rather than for mocks used to fill dependencies / isolate a test.

If you need to ensure that a service is called once and once only, then use a strict mock. The less strict you are about mocks the more resilient to change your tests are, and probably the more focused on what you are trying to achieve they are. Rather than how you achieve it.

Inappropriate intimacy. Good term!

Pass Drug Test Blog wrote Pass Drug Test Blog
on 06-13-2008 2:08 PM

...

Community Blogs wrote Strict Mocks
on 10-23-2008 2:03 AM

I am extremely excited to be a part of a team that has deliberately decided to follow the Agile path

About The CodeBetter.Com Blog Network
CodeBetter.Com FAQ

Our Mission

Advertisers should contact Brendan

Subscribe
Google Reader or Homepage

del.icio.us CodeBetter.com Latest Items
Add to My Yahoo!
Subscribe with Bloglines
Subscribe in NewsGator Online
Subscribe with myFeedster
Add to My AOL
Furl CodeBetter.com Latest Items
Subscribe in Rojo

Member Projects
DimeCasts.Net - Derik Whittaker

Friends of Devlicio.us
Red-Gate Tools For SQL and .NET

NDepend

SlickEdit
 
SmartInspect .NET Logging
NGEDIT: ViEmu and Codekana
LiteAccounting.Com
DevExpress
Fixx
NHibernate Profiler
Unfuddle
Balsamiq Mockups
Scrumy
JetBrains - ReSharper
Umbraco
NServiceBus
RavenDb
Web Sequence Diagrams
Ducksboard<-- NEW Friend!

 



Site Copyright © 2007 CodeBetter.Com
Content Copyright Individual Bloggers

 

Community Server (Commercial Edition)