Images in this post missing? We recently lost them in a site migration. We're working to restore these as you read this. Should you need an image in an emergency, please contact us at imagehelp@codebetter.com
Choosing A Mocking Framework

If you're serious about testing before long you'll bump into the need to do some mocking. You can of course chose to write your own mocks but that is hardly time effective or trivial.  More than likely you'll want to use a framework, but how do you choose between the most popular options?

There's been some discussion on Twitter between myself and Daniel Cazzulino (@kzu), the creator of Moq (pronounced "Mock-you" or just "Mock"), a newer and popular .NET mocking, about mocking frameworks and what goes into choosing a mocking framework.  Daniel and I share a passion in mocking and by extension testing.  As the creator of Moq, I hold him in very high regard.

The discussion centered around some work that Andrew Kazyrevich has been doing recently.  Andrew is passionate about mocking.  As such he's created an open source project which compares the "big dawgs" of .NET mocking frameworks.  In his introductory post he explains the intent, which is message I support and want to help spread:

I've started a small open source project. It provides a unified set of tests written against Moq, NMock2, Rhino Mocks and Typemock Isolator, so that you can easily compare the frameworks and make an informed decision when picking one up.

In Andrew's most recent post he compares the execution time of the various frameworks.  My response on the RhinoMocks mailing list was terse but sincere:

Curious about speed in your tests, is the a first level concern when choosing a testing framework?

Daniel answered the question in a tweet:

IMO, API design, usability and expressiveness is a much more valuable comparison. #moq is not optimized for runtime perf.

To which I responded:

I would also throw in documentation, discoverability, and community into your equation.

So, fine blog reader, I have a question for you.  If you are using a mocking framework, what are your first-level concerns when you chose your framework?  For those who have yet to utilize a framework, what will you base your decision on when you do choose?

  • Execution Speed
  • API Design
  • Usability
  • Expressiveness
  • Documentation
  • Discoverability (possibly coincides with API Design)
  • Community
  • Other?

Posted 03-19-2009 11:02 PM by Tim Barcz

[Advertisement]

Comments

Colin Goudie wrote re: Choosing A Mocking Framework
on 03-20-2009 12:08 AM

These are my 3

Usability

Expressiveness

Documentation

Refactorability

I've yet to have projects with 100,000's of tests, so I haven't run into speed issues. I more frequently run into bad documentation or mocking that on first glance, doesn't make any sense at all.

Tim Barcz wrote re: Choosing A Mocking Framework
on 03-20-2009 12:23 AM

@Colin...

Dude...there's four in your list.... :-)

Can you explain "Refactorability"?  I would see that incumbent upon the code under test moreso than the mocking framework itself....no?

Hernan Garcia wrote re: Choosing A Mocking Framework
on 03-20-2009 12:27 AM

In order of importance:

Documentation

Community

Usability / Discoverability (I guess both tied to API design)

Expressivenes

Documentation is my main concern, mostly on usage examples of non trivial features. A big and active community can made for lack of documentation at a certain level.

Usability, discoverability and api design they are kind of at the same level. If the documentation is clear and can make with some obscure or not evident usage. At equal documentation better usability wins.

I guess that speed is not a concern (for me) when choosing a mocking framework. I actually never though about it.

I guess It had to be really bad and make a significant impact on my CI build for me to worried about it.

By the record me and my team switched resently from NMock2 to Rhino.Mocks and we are very pleased, specially with the new AAA syntax.

Scott Muc wrote re: Choosing A Mocking Framework
on 03-20-2009 12:29 AM

As I am a newbie when it comes to mocking the first framework I picked up was Rhino.Mocks. I believe this is mainly because I respect Ayende a lot and read his blog religously and because it's quite prevalent in the .Net community.

Speed is probably the least of my concerns. As an inexperienced mocker (please don't ask me to differentiate between a fake, and a stub ;-)) the thinks I value are a low barrier to entry, but sophisticated options once I get comfortable. Next is documentation along with a large community.

DotNetKicks.com wrote Choosing a mocking framework
on 03-20-2009 12:29 AM

You've been kicked (a good thing) - Trackback from DotNetKicks.com

mysite wrote re: Choosing A Mocking Framework
on 03-20-2009 3:42 AM

I also can't understand what is "Refactorability" like you say.

Tormod wrote re: Choosing A Mocking Framework
on 03-20-2009 4:00 AM

I think expressiveness is the most important. Tests are to be read and understood.

But also it is very important that the component that I'm developing has an intuitive API. An expressive mocking framework applied on a poorly designed component API gives low readability anyhow.  

I have a problem seeing how something can be expressive without having high useability. Nevertheless, useability is second.

Again, API goes hand in hand with the two above. I don't see how you can score high on the two withhout a good API.

I care about performance, but only to the extent that it is noticeable to me, not some millisec benchmarking.

Reflective Perspective - Chris Alcock » The Morning Brew #311 wrote Reflective Perspective - Chris Alcock » The Morning Brew #311
on 03-20-2009 4:25 AM

Pingback from  Reflective Perspective - Chris Alcock  » The Morning Brew #311

Derik Whittaker wrote re: Choosing A Mocking Framework
on 03-20-2009 6:10 AM

I agree with the good API design/expressiveness, but not for reasons have been mentioned.  For me it is about ramp time for a new developer who is learning a isolation framework.

Using isolation frameworks is not a trivial thing for the new person, the simpler and easier  it is for them to use the fast more people will adopt them.

Emil Cardell wrote re: Choosing A Mocking Framework
on 03-20-2009 6:45 AM

I'm new to mocking and I've been looking to choose a framework. And I think the most important stuff is

For the product I'm using I want:

API Design

Usability

Expressiveness

And hoppfully the community will grow of the product that has requerments and then the documentation will grow.

Now I'm standing between rihino mocks and MOQ.

chrissie1 wrote re: Choosing A Mocking Framework
on 03-20-2009 6:46 AM

Discoverability

Documentation (but if it is discoverable than you won't need it as much)

Community.

Last on the list is execution speed but it must be within reason of course.

sergiopereira wrote re: Choosing A Mocking Framework
on 03-20-2009 8:50 AM

My main concern would be the usability, which would reflect a good API design and diminish the need for extensive documentation.

But, to be honest, I think the most of the times people will choose whatever product is being talked about the most. Doing the research and trying all the different options, although being the right thing to do, isn't something everyone wants or has time to do. I'm sure I have done that same thing when choosing products in other segments.

Bruno Martínez wrote re: Choosing A Mocking Framework
on 03-20-2009 9:31 AM

IMO, mocking frameworks are needed because the base language is not expresive enough. I would rank then Usability as the first concern.

DotNetShoutout wrote Choosing A Mocking Framework - Tim Barcz
on 03-20-2009 11:39 AM

Thank you for submitting this cool story - Trackback from DotNetShoutout

Colin Goudie wrote re: Choosing A Mocking Framework
on 03-20-2009 9:50 PM

haha, that's what you get when you add something at the last minute. I did actually mean to say 4 things :)

By refactorability I mean ensuring that the mocking framework is using first class objects rather than say, strings, to represent functions. (I think JMock 1 did that). It was a major pain because changing your production code didn't cause the tests to change correctly

Tim Barcz wrote re: Choosing A Mocking Framework
on 03-20-2009 10:30 PM

@Colin,

Thanks for the explanation of "refactorability", makes perfect sense now.  Also, I have to agree with the assessment of strings.  I believe it was out of this frustration that Ayende created RhinoMocks.

Matt Hinze wrote re: Choosing A Mocking Framework
on 03-22-2009 8:38 PM

1. API design (which encapsulates several other options)

2. Features

Weekly Links #45 | GrantPalin.com wrote Weekly Links #45 | GrantPalin.com
on 03-22-2009 8:53 PM

Pingback from  Weekly Links #45 | GrantPalin.com

Tim Barcz wrote re: Choosing A Mocking Framework
on 03-22-2009 9:31 PM

@Matt,

What do you mean by "Features"?  Can you give some examples in terms of mocking frameworks you have used before?

#.think.in wrote #.think.in infoDose #22 (16th Mar - 20th Mar)
on 03-24-2009 6:50 AM

#.think.in infoDose #22 (16th Mar - 20th Mar)

Matt Hinze wrote re: Choosing A Mocking Framework
on 03-30-2009 8:49 PM

Sure.  Things like Rhino Mocks' comprehensive constraints, ordered playback, and AAA syntax are reasons why I've not felt compelled to switch.  I consider proxies a feature and stubs / mocks differentiation.

Kenali dan Kunjungi Objek Wisata di Pandeglang wrote re: Choosing A Mocking Framework
on 10-22-2009 5:20 PM

This is actually really interesting regarding your fact article here, This article is very informative.

kenali dan kunjungi objek wisata di pandeglang wrote re: Choosing A Mocking Framework
on 10-26-2009 6:54 AM

Thanks for taking the time to discuss this, I feel strongly about it and love learning more on this topic. If possible, as you gain expertise, would you mind updating your blog with more information? It is extremely helpful and beneficial to your readers.

Tim Ellison wrote re: Choosing A Mocking Framework
on 11-11-2009 1:32 PM

As my own use of mocking frameworks has evolved over the last couple of years, I have to rank expressiveness first followed by documentation.  Speed is also not a concern given I have yet to find a mocking framework that is excessively slower or faster than others.

I have to admit that RhinoMocks playback feature is very nice.  Having recently downloaded Mock You, I'm very interested in trying out its .NET 3.5 capabilities.  

Great article.  

bakıcı oyunları wrote re: Choosing A Mocking Framework
on 05-03-2010 1:38 AM

Very very nice. Respect

<a href="www.bakicioyunlari.net/">bak%C4%B1c%C4%B1 oyunları</a>

devlicio.us wrote re: Choosing A Mocking Framework
on 05-24-2011 1:52 PM

Choosing a mocking framework.. Awesome :)

devlicio.us wrote re: Choosing A Mocking Framework
on 06-03-2011 8:23 AM

Choosing a mocking framework.. Tiptop :)

devlicio.us wrote re: Choosing A Mocking Framework
on 06-05-2011 8:39 AM

Choosing a mocking framework.. Awful :)

7hpb xxx u7us wrote re: Choosing A Mocking Framework
on 07-02-2011 2:30 PM

Choosing a mocking framework.. Slap-up :)

Treadmills for Sale wrote re: Choosing A Mocking Framework
on 07-08-2011 8:08 AM

q3OjYb I read online (computer problems) positive feedback about your resource. Didnt even believe it, and now saw myself. It turned out that I was not fooled.

rlfxzfyehx wrote re: Choosing A Mocking Framework
on 09-25-2011 12:28 PM
aqexyivwje wrote re: Choosing A Mocking Framework
on 09-25-2011 1:36 PM
lcubcxjltc wrote re: Choosing A Mocking Framework
on 09-25-2011 2:43 PM
gwfhkkdqfe wrote re: Choosing A Mocking Framework
on 09-25-2011 3:49 PM
zefzhxktbh wrote re: Choosing A Mocking Framework
on 09-25-2011 4:37 PM

soqkmefwmjdjp, http://www.smwlimlpla.com mgbuysfybu

Stream2020 wrote re: Choosing A Mocking Framework
on 10-06-2011 11:44 AM

I've started a small open source project. It provides a unified set of tests written against Moq, NMock2, Rhino Mocks and Typemock Isolator, so that you can easily compare the frameworks and make an informed decision when picking one up

Stream2020 wrote re: Choosing A Mocking Framework
on 10-06-2011 11:50 AM

I've started a small open source project. It provides a unified set of tests written against Moq, NMock2, Rhino Mocks and Typemock Isolator, so that you can easily compare the frameworks and make an informed decision when picking one up

Seo Company wrote re: Choosing A Mocking Framework
on 11-24-2012 12:18 PM

fG93hb I cannot thank you enough for the article post. Awesome.

seo social bookmarking wrote re: Choosing A Mocking Framework
on 01-17-2013 9:26 AM

iY5RL0 Awesome post.Really looking forward to read more. Great.

generic viagra discount wrote re: Choosing A Mocking Framework
on 01-27-2013 3:12 AM

z5w908 Thank you for your blog post.Really looking forward to read more. Much obliged.

pills for lose weight wrote re: Choosing A Mocking Framework
on 02-01-2013 6:17 AM

45kyar Looking forward to reading more. Great blog.Really thank you! Will read on...

buy viagra online wrote re: Choosing A Mocking Framework
on 02-02-2013 9:01 AM

cQuS70 Say, you got a nice blog article.Really looking forward to read more. Keep writing.

buy stendra online wrote re: Choosing A Mocking Framework
on 02-14-2013 9:46 PM

vvaMdB Im obliged for the article.Much thanks again. Cool.

buy stendra online wrote re: Choosing A Mocking Framework
on 02-15-2013 2:08 AM

og9vrg Thanks so much for the blog. Cool.

buy cialis discount wrote re: Choosing A Mocking Framework
on 02-24-2013 5:52 AM

OvhGqM Really enjoyed this blog post.Really looking forward to read more. Really Great.

clomiphene 25 mg wrote re: Choosing A Mocking Framework
on 02-27-2013 12:42 PM

OijeJ4 Major thankies for the article post.Thanks Again. Awesome.

amazing site wrote re: Choosing A Mocking Framework
on 03-03-2013 6:58 PM

X6LUuU I really liked your blog post.Thanks Again. Great.

bookmaring service wrote re: Choosing A Mocking Framework
on 03-13-2013 1:18 PM

57ojJs Im thankful for the article. Fantastic.

social bookmarks wrote re: Choosing A Mocking Framework
on 03-14-2013 7:44 AM

SgfbeZ Fantastic blog post.Really thank you! Much obliged.

buy social bookmarks wrote re: Choosing A Mocking Framework
on 03-23-2013 8:10 AM

uT2iTc Very informative blog post. Cool.

Social bookmarks wrote re: Choosing A Mocking Framework
on 03-23-2013 3:15 PM

uSwidN I really liked your blog post.Thanks Again. Really Cool.

social bookmarking service wrote re: Choosing A Mocking Framework
on 04-04-2013 12:01 AM

fxyFDG Say, you got a nice article post.Really looking forward to read more. Great.

stoner wrote re: Choosing A Mocking Framework
on 04-06-2013 1:01 AM

I really liked your article.Thanks Again. Really Great.

buy social bookmarks wrote re: Choosing A Mocking Framework
on 04-14-2013 6:01 AM

EM4vd9 Great blog article.Much thanks again. Really Cool.

buy social bookmarks wrote re: Choosing A Mocking Framework
on 04-28-2013 12:41 AM

ZAAXvA I appreciate you sharing this post.Thanks Again. Want more.

digital camera guide wrote re: Choosing A Mocking Framework
on 05-14-2013 3:55 AM

HFJi8f Major thanks for the blog article. Awesome.

social bookmarking service wrote re: Choosing A Mocking Framework
on 06-05-2013 1:16 PM

BxnCLA Thanks a lot for the article.Really thank you! Keep writing.

good social bookmarks wrote re: Choosing A Mocking Framework
on 06-20-2013 10:50 PM

cXjFNv I value the article.Thanks Again. Awesome.

news and many more wrote re: Choosing A Mocking Framework
on 07-04-2013 9:00 AM

Mut0PF Really appreciate you sharing this blog.Much thanks again. Great.

moldavian news wrote re: Choosing A Mocking Framework
on 07-08-2013 1:01 PM

lJSALQ Very good blog article.Much thanks again. Really Cool.

moldova news wrote re: Choosing A Mocking Framework
on 07-12-2013 8:53 PM

73SjH8 Really enjoyed this blog article.Really looking forward to read more. Really Cool.

news wrote re: Choosing A Mocking Framework
on 07-26-2013 10:59 AM

iCvWHW Appreciate you sharing, great blog post.Much thanks again. Fantastic.

buy social bookmarks wrote re: Choosing A Mocking Framework
on 07-28-2013 11:59 AM

PPPE74 Muchos Gracias for your blog.Much thanks again. Cool.

news wrote re: Choosing A Mocking Framework
on 08-02-2013 11:28 AM

xccxYq Wow, great blog. Really Great.

amazing news wrote re: Choosing A Mocking Framework
on 08-03-2013 1:52 PM

7IURvB Thanks-a-mundo for the blog post.Really thank you! Really Great.

awesome links for you wrote re: Choosing A Mocking Framework
on 08-19-2013 9:31 AM

qmVG4z Looking forward to reading more. Great post.Much thanks again. Really Cool.

great link buildng wrote re: Choosing A Mocking Framework
on 08-19-2013 10:31 PM

LwN3XO A round of applause for your article.Much thanks again. Awesome.

great link buildng wrote re: Choosing A Mocking Framework
on 08-21-2013 10:56 AM

LY4c2y Im thankful for the article.Really looking forward to read more. Much obliged.

the best seo service wrote re: Choosing A Mocking Framework
on 09-06-2013 12:17 PM

8wAvos Major thanks for the article.Much thanks again.

make money online wrote re: Choosing A Mocking Framework
on 09-11-2013 6:38 PM

s244Dm Looking forward to reading more. Great article post.Much thanks again. Will read on...

only for 5 dollars wrote re: Choosing A Mocking Framework
on 09-13-2013 2:19 PM

lTmNHU Thanks so much for the blog article.Much thanks again. Great.

cheap link building wrote re: Choosing A Mocking Framework
on 09-24-2013 1:05 PM

LUBhTO A big thank you for your post.Really looking forward to read more. Will read on...

check out seo wrote re: Choosing A Mocking Framework
on 09-29-2013 6:42 PM

EF4k0Q Fantastic article post.Really looking forward to read more. Much obliged.

awesome linkbuilding site wrote re: Choosing A Mocking Framework
on 10-01-2013 12:58 AM

8RoVYN wow, awesome blog article.Really thank you! Want more.

check out these guys! wrote re: Choosing A Mocking Framework
on 10-15-2013 1:04 PM

JqUwL2 Really appreciate you sharing this blog article.Really thank you! Great.

take a look at it! wrote re: Choosing A Mocking Framework
on 10-24-2013 8:36 PM

Q8jkLh Thanks-a-mundo for the article. Fantastic.

top seo guys wrote re: Choosing A Mocking Framework
on 11-01-2013 10:26 AM

knY09Y Hey, thanks for the blog article.Thanks Again. Great.

cheap backlinks wrote re: Choosing A Mocking Framework
on 07-18-2014 4:47 PM

aDYL3Q Im thankful for the blog post.Really thank you! Keep writing.

Add a Comment

(required)  
(optional)
(required)  
Remember Me?

About The CodeBetter.Com Blog Network
CodeBetter.Com FAQ

Our Mission

Advertisers should contact Brendan

Subscribe
Google Reader or Homepage

del.icio.us CodeBetter.com Latest Items
Add to My Yahoo!
Subscribe with Bloglines
Subscribe in NewsGator Online
Subscribe with myFeedster
Add to My AOL
Furl CodeBetter.com Latest Items
Subscribe in Rojo

Member Projects
DimeCasts.Net - Derik Whittaker

Friends of Devlicio.us
Red-Gate Tools For SQL and .NET

NDepend

SlickEdit
 
SmartInspect .NET Logging
NGEDIT: ViEmu and Codekana
LiteAccounting.Com
DevExpress
Fixx
NHibernate Profiler
Unfuddle
Balsamiq Mockups
Scrumy
JetBrains - ReSharper
Umbraco
NServiceBus
RavenDb
Web Sequence Diagrams
Ducksboard<-- NEW Friend!

 



Site Copyright © 2007 CodeBetter.Com
Content Copyright Individual Bloggers

 

Community Server (Commercial Edition)