Using ConventionTests

About conventions

I’m a big fan of using conventions when developing applications. I blogged about it in the past (here, here and later here). I also gave a talk about my experience with this approach and how I currently use it at NDC last week (slides are available here, video should be posted in a few weeks).

 

One problem I faced when trying to build convention validation tests was lack of simple API that would allow me to build the validation rules for my tests. I built a spike of such library (called Norman) last year. It was meant to be full-featured generic convention-validation engine, built on top of Mono.Cecil that would be able to validate rules inside method bodies (think rules like “no ViewModel can call data access code”). I quickly abandoned it, not being able to come up with simple and concise API that at the same time would be generic enough to cover every possible scenario.

 

About ConventionTests

Having failed with Norman’s approach on a later project I approached the problem from a completely different angle. This is based on observation regarding conventions I tend to use – they, in majority of cases, tend to be based on information about types, and internal models of frameworks I tend to use. In other words – information that can be easily obtained via plain Reflection.

Also limiting myself to very few scenarios allowed me to come up with a focused, simple and concise API. That’s how ConventionTests project was born.

 

Using ConventionTests

ConventionTests is a simple code-only Nuget that provides very minimalistic and limited API enforcing certain structure when writing convention tests and integrating with NUnit (more on that below). Installing it will add two .cs files to the project and a few dependencies (NUnit, Windsor and ApprovalTests).

image

ConventionTests.NUnit file is where all the relevant code is located and __Run file is the file that runs your tests (why that is, later).

The approach is to create a file per convention and name them in a descriptive manner, so that you can learn what the conventions you have in the project by just looking at the files in your Conventions folder, without having to open them.

 

Each convention test inherits (directly or indirectly from IConventionTest) interface. There’s an abstract implementation of the interface, ConventionTestBase and a few specialized implementations for common scenarios provided out of the box: Type-based one (ConventionTest) and two for Windsor (WindsorConventionTest, non-generic and generic for diagnostics-based tests).

 

Type-based convention tests

The most common and most generic group of conventions are ones based around types and type information. Conventions like “every controller’s name ends with ‘Controller’”, or “Every method on WCF service contracts must have OperationContractAttribute” are examples of such conventions.

You write them by creating a class inheriting ConventionTest, which forces you to override one method. Here’s a minimal example:

public class Controllers_have_Controller_suffix_in_type_name : ConventionTest
{
    protected override ConventionData SetUp()
    {
        return new ConventionData
            {
                Types = t => t.IsConcrete<IController>(),
                Must = t => t.Name.EndsWith("Controller")
            };
    }
}

Hopefully the code is self-explanatory (it was designed to be). You specify two predicates. One to find types that your convention is concerning, and the other specifying what your convention demands. If you now run tests in your project (or in __Run.cs file) You will see the result similar to this:

image

In this case I had a single class implementing IController, named Foo, which obviously doesn’t follow the convention that’s why the test failed, giving me generic failure message which just lists the types that failed the test.

To make it more readable I may want to make the test failure message more descriptive and actionable so that whoever sees a failing test like this will be able to know why their code is wrong and how to fix it.

protected override ConventionData SetUp()
{
    return new ConventionData
        {
            Types = t => t.IsConcrete<IController>(),
            Must = t => t.Name.EndsWith("Controller"),
            FailDescription = "Routing will not work unless controllers are named {something}Controller",
            FailItemDescription = t => t.ToString() + ". Name should be " + t.Name + "Controller perhaps?"
        };
}

The two additional properties will cause the failure to look like this:

image

This tells the developer who sees that test not only were the problem is, but also why it is a problem, and suggests a solution.

For scenarios where the convention is not all black and white and there are some legitimate exceptions to the rule there is built in integration with ApprovalTests framework (I blogged about ApprovalTests here). I know that the convention I used here as an example is pretty black and white but assuming it wasn’t changing the code to:

protected override ConventionData SetUp()
{
    return new ConventionData
        {
            Types = t => t.IsConcrete<IController>(),
            Must = t => t.Name.EndsWith("Controller"),
            FailDescription = "Routing will not work unless controllers are named {something}Controller",
            FailItemDescription = t => t.ToString() + ". Name should be " + t.Name + "Controller perhaps?"
        }.WithApprovedExceptions("Let's say Foo has special routing or something");
}

will change it to approval test where approved file will contain the output of the test, so in this case:

image

Windsor-based convention tests

Another common set of convention tests I tend to write are tests regarding my IoC container (and I tend to use Castle Windsor, therefore that’s the one supported out of the box). The structure of the tests and API is similar, with difference being instead of types we’re dealing with Windsor’s component Handlers.

public class List_classes_registered_in_Windsor : WindsorConventionTest
{
    protected override WindsorConventionData SetUp()
    {
        return new WindsorConventionData(new WindsorContainer()
                                                .Install(FromAssembly.Containing<AuditedAction>()))
            {
                FailDescription = "All Windsor components",
                FailItemDescription = h => BuildDetailedHandlerDescription(h)+" | "+
                    h.ComponentModel.GetLifestyleDescription(),
            }.WithApprovedExceptions("We just list all of them.");
 
    }
}

In this case we’re dealing with slightly different types but we follow the same pattern. We initialize the container and then specify predicates (if necessary) on its handlers. We don’t need to do that in which case all handlers from the container will be considered. This is useful to create (approval) test like this one that just lists all the components in the container, along with the list of services they expose and their lifestyle. With that you get quick insight into what’s in your container by just looking at the approved file. Moreover every change to the list of components will cause the test to fail, drawing your attention, to hopefully validate that the changes happening are indeed what you were expecting.

Windsor diagnostics-based convention tests

Windsor as some built in diagnostics and there’s a base class for tests that take advantage of that. For example the following test will list all potentially misconfigured components.

public class List_misconfigured_components_in_Windsor : 
    WindsorConventionTest<IPotentiallyMisconfiguredComponentsDiagnostic>
{
    protected override WindsorConventionData<IHandler> SetUp()
    {
        return
            new WindsorConventionData<IHandler>(
                new WindsorContainer().Install(FromAssembly.Containing<AuditedAction>()))
                {
                    FailDescription = "The following components come up as potentially unresolvable",
                    FailItemDescription = MisconfiguredComponentDescription
                }.WithApprovedExceptions("it's *potentially* for a reason");
    }
}

In this case instead of inspecting all handlers we go through handlers returned by the diagnostic (there’s another base class, with two generic parameters for diagnostics that do not deal with handlers). We make it an approval test so we can have an approved list of false-positives.

 

Those are the basic scenarios provided out of the box, however you can extend it to easily support more custom scenarios. For example, one case that happened on my current project just this week, was this. We’re using NHibernate and FluentNHibernate (by convention) and the default cascading on collections is set to “cascade all delete orphans”. This is not always the valid choice and for those cases we use mapping overrides.

 

Custom convention tests

We wanted to create a convention test (with approval) that lists all our collections where we do cascade deletes, so that when we add a new collection the test would fail reminding us of the issue, and forcing us to pay attention to how we structure relationships in the application. To do this we could create a base NHibernateConventionTest and NHiibernateConventionData to create similar structure, or just build a simple one-class convention like that:

public class List_collection_that_cascade_deletes:ConventionTestBase
{
    public override void Execute()
    {
        // NH Bootstrapper is our custom class to set up NH
        var bootstrapper = new NHibernateBootstrapper();
        var configuration = bootstrapper.BuildConfiguration();
 
        var message = new StringBuilder("Collections with cascade delete orphan");
        foreach (var @class in configuration.ClassMappings)
        {
            foreach (var property in @class.PropertyIterator)
            {
                if(property.CascadeStyle.HasOrphanDelete)
                {
                    message.AppendLine(@class.NodeName + "." + property.Name);
                }
            }
        }
        Approve(message.ToString());
    }
}

This is obviously much more imperative and much less readable than the other tests, therefore I suggest in most casese you make the effort and try to make the tests as terse, readable and declarative as possible so that they truly read like a good executable documentation.

 

And if you’re not sure how something works, remember the whole thing is a code-only nuget so you can just read the code.

 

Get it here.


Posted 06-14-2012 10:19 AM by Krzysztof Koźmic
Filed under: ,

[Advertisement]

Comments

Tor Hovland wrote re: Using ConventionTests
on 06-19-2012 6:08 AM

I like the idea of using tests to do things like this, and allowing for approved exceptions makes it an intriguing tool for real-world use. This was one of the most useful talks at NDC, in my opinion.

Krzysztof Koźmic wrote re: Using ConventionTests
on 06-22-2012 8:23 PM

Thanks Tor :)

Mehdi Khalili wrote re: Using ConventionTests
on 06-23-2012 10:03 PM

Nice one Krzysztof. I like the readability and simplicity of the API. The code-only nuget package for smaller frameworks is a nice idea too.

So ... have you completely abandoned Norman?!

Krzysztof Koźmic wrote re: Using ConventionTests
on 06-26-2012 11:04 PM

@Mehdi Khalili

It is abandoned for now. Ideally I would like to pick it up as I think the idea has a lot of potential and could bring a lot of value but for now I have higher priority issues and too little time to do it :)

About The CodeBetter.Com Blog Network
CodeBetter.Com FAQ

Our Mission

Advertisers should contact Brendan

Subscribe
Google Reader or Homepage

del.icio.us CodeBetter.com Latest Items
Add to My Yahoo!
Subscribe with Bloglines
Subscribe in NewsGator Online
Subscribe with myFeedster
Add to My AOL
Furl CodeBetter.com Latest Items
Subscribe in Rojo

Member Projects
DimeCasts.Net - Derik Whittaker

Friends of Devlicio.us
Red-Gate Tools For SQL and .NET

NDepend

SlickEdit
 
SmartInspect .NET Logging
NGEDIT: ViEmu and Codekana
LiteAccounting.Com
DevExpress
Fixx
NHibernate Profiler
Unfuddle
Balsamiq Mockups
Scrumy
JetBrains - ReSharper
Umbraco
NServiceBus
RavenDb
Web Sequence Diagrams
Ducksboard<-- NEW Friend!

 



Site Copyright © 2007 CodeBetter.Com
Content Copyright Individual Bloggers

 

Community Server (Commercial Edition)