.NET OSS dependency hell

Paul, whom some of you may know as the maintainer of Horn project, left a comment on my blog, that was (or to be more precise – I think it was) a continuation of series of his tweets about his dissatisfaction with the state of affairs when it comes to dependencies between various OSS projects in .NET space, and within Castle Project in particular.

paul_twitter

I must say I understand Paul, and he’s got some valid points there, so let’s see what can be done about it.

Problems

One of the goals of Castle Project from the very beginning has been modularity of its elements. As castle main page says:

Offering a set of tools (working together or independently) and integration with others open source projects, Castle helps you get more done with less code and in less time.

How do you achieve modularity. Say you have two projects, Foo and Bar that you want to integrate. You could just reference one from the other.

eb1c2cc[1]

This however means that whenever you use Foo, you have to drag Bar with you. For example, whenever you want to use MonoRail, you’d need to drag ActiveRecord with it, along with entire set of its dependencies, and their dependencies, etc.

Instead you employ Dependency Inversion (do not confuse with Dependency Injection). You make your components depend on abstractions, not the implementation. This however means, that in .NET assembly model, you need to introduce third assembly to keep the abstractions in.

51067fb6[1]

Now we have 3 assemblies instead of 2 to integrate two projects. Within Castle itself common abstractions are being kept in Castle.Core.dll. But what if we want to take more direct advantage of one project in another project still maintaining the decoupled structure? We need to extract the functionality bridging the two projects to yet another assembly. Tick – now we have 4 of them.

607f68d4[1]

In this case the FooBar project would be something like ActiveRecord integration facility, which integrates ActiveRecord with Windsor.

When you mix multiple projects together you enter another problem – versioning.

Say you want to integrate few projects together, some of which are interdependent (via bridging assemblies, not shown here for brevity)

69f20a13[1]

Now, once a new version of one of the projects is released, you either have to wait for all the other projects to update their dependency to the latest version, do it yourself (possibly with some help from Horn), or stick to the old version. The situation gets even more complicated when there were some breaking changes introduced, in which case plain recompilation will not do – some actual code would need to be written to compensate for that.

These are the main issues with this model, let’s now look at possible solutions.

Solutions

First thing that comes to mind – if having some assemblies means you’ll need even more assemblies, perhaps you should try to minimize that number? This has already come to our minds. With last wave of releases we performed some integration of projects. EmailSender got integrated into Core, one less assembly. Logging adapters for log4net and nlog were merged into core project, which means they still are separate assemblies (as they bridge Castle with 3rd party projects) but they’re now synced with Core and are released with it, which means this is one less element in your versioning matrix for you to worry about. Similar thing happened with Logging Facility, which now is versioned and released with Windsor itself.

For the next major version, there are suggestions to take this one step further. Merge DynamicProxy with DictionaryAdapter and (parts of) Core into a single assembly; Merge Windsor and MicroKernel (and other parts of Core) into an other assembly. With that you get from 5 assemblies to 2.

That reduces Castle’s internal dependencies, but what about other projects that depend on it? After the recent release, we started a log of breaking changes, along with brief explanation and suggested upgrade paths, to make it easier for applications and frameworks to upgrade. We have yet to see how this plays out.

What else can be done?

This is the actual question to you? What do you think can be done, for Castle specifically, but more broadly – for entire .NET OSS ecosystems to make problems Paul mentioned go away, or at least make them easier to sort out?


Posted 02-25-2010 8:11 PM by Krzysztof Koźmic
Filed under: ,

[Advertisement]

Comments

Scott Bellware wrote re: .NET OSS dependency hell
on 02-25-2010 4:36 PM

I have a Rails project that uses 15 open source libraries and servers. I don't seem to have this problem. I also don't have that problem when I upgrade apps to new versions of Rails itself. I don't think the problem with dependencies, but with packaging and package management. Maybe the issue with Castle has more to do with packaging technologies. I know that I've had a much easier go at this problem when not using .NET.

Ultimately, the dependency problem is more of a class coupling and versioning problem than an assembly problem. It just so happens that .NET classes are pre-packaged as .NET assemblies. Moving all of the classes into a single assembly solves the assembly problem, but then it just shows that the assembly problem isn't really a problem at all.

The real problem here is that .NET uses static coupling, regardless of whether the coupling is to a virtual type or a non-virtual type. Stop using static coupling and the problem becomes much less problematic.

Keith Nicholas wrote re: .NET OSS dependency hell
on 02-25-2010 5:12 PM

what did you use to make your diagrams? :)

Rik Hemsley wrote re: .NET OSS dependency hell
on 02-25-2010 5:47 PM

I don't see these issues as being a problem with OSS libraries - they're simply the usual day to day issues that arise when dealing with modular systems. Surely even if you use no external libraries in a solution, your own set of projects ends up with a similar structure?

I don't see how reducing the number of assemblies helps at all. Surely EmailSending doesn't belong in Core?

Paul Batum wrote re: .NET OSS dependency hell
on 02-25-2010 7:31 PM

To be honest I'm a little confused about Paul Cowan's comment because I thought Horn was supposed to help with these issues. Assuming their aren't any breaking changes, shouldn't Horn rebuild my stack for me when a new castle assembly has a feature I want? What am I missing?

@Keith

http://yuml.me/

@Rik

Why not? Consider: System.Net.Mail.SmtpClient can be found in System.dll

Paul Cowan wrote re: .NET OSS dependency hell
on 02-26-2010 8:53 AM

@Paul Batum

My point is that maybe with .NET's assembly model, the solution might not be a 100% technical solution.  Common sense and community should also play a factor to try and make some sense of it.

As every OSS (horn included) works in a silo the problem will never be fixed.

My point is that attempts like horn will always be swimming up stream with the way things currently are.

Horn's descriptor model takes buy in across the board.

Buy in we did not received.

Jeremy Miller has been posting about providing a gems like experience.

While I applaud the motives, talk is cheap and I simply have no idea how anyone can resolve dependencies as stands without building from the source.

I challenge anyone to come up with a better solution than horn.

Then again, people are busy writing their own IoC containers etc.

Robert Rudduck wrote re: .NET OSS dependency hell
on 02-26-2010 9:44 AM

One of the ways I have managed the non-breaking changes is through using a BindingRedirect. Not always the prettiest solution if you have a bunch of them but it allows you to redirect bindings at runtime to newer assembly versions.

Doesn't fix things when the changes are breaking though...

uberVU - social comments wrote Social comments and analytics for this post
on 02-26-2010 5:36 PM

This post was mentioned on Twitter by huyrua: .NET OSS dependency hell - devlicio.us/.../net-oss-dependency-hell.aspx

Krzysztof Koźmic wrote re: .NET OSS dependency hell
on 02-27-2010 5:51 AM

@Paul Cowan  

What do you mean by "Common sense and community should also play a factor to try and make some sense of it."?

As Castle team we're pretty open about our release schedule - we publish upcoming release dates as the approach (exact date few weeks before that) castleproject.org/.../projects.html

- we also discuss releases on Castle group so that every member of projects depending on Castle can join the discussion, and talk to us about changes. We're pretty happy to help them with update process, but you know what - hardly anyone ever does that.

- When we were releasing DynamicProxy 2.2 I suggested Fabio (from NHibernate team) to update their bytecode provider to new DP, but he didn't find that necessary. We ship NHibernate with updated bytecode provider as part of ActiveRecord.

Please expand this thought. I meant this post as a means to ignite a discussion, so I'll be very happy to hear your opinion on community/common sense aspect of it.

Steve wrote re: .NET OSS dependency hell
on 03-18-2010 9:26 PM

My problem is , a facility , ie. wcf is created against a certain version of Windsor.  I pull it down, and have to 'magically' figure out which one.  Build this, get this, try this, etc..

It's a MESS.

The solution is simple - if you code a product against a particular version then

Does this blog support email notification - I'd like to see if anything comes out of this ?

1. document what you built it against

2. or include the assemblies you used.

Here is example - I am hunting around for the missing interfaces in the current wcf integration build, then I have to figure out how to build the entire castle stack to get the right version.  

Then that breaks the mvccontrib castle version, etc.. ad nauseam.

The sad part is -the work done is fantastic.  But then it's a major PITA to use it - it's then created more for the guys who made it, not for those trying to simply figure out how to use it.

Please consider this suggestion above - because I'm growing weary of playing 'find the right dll' game.

Great work, wish I could use it  :)

Krzysztof Koźmic wrote re: .NET OSS dependency hell
on 03-22-2010 1:36 PM

Steve,

Isn't WCF Facility bound to Windsor v2.1.1?

I know it's a PITA, which mostly is due to fact that this facility has no released version as of yet (although it's close). When it gets a release it will be against current stable version of its dependencies, so the problem should go away.

About The CodeBetter.Com Blog Network
CodeBetter.Com FAQ

Our Mission

Advertisers should contact Brendan

Subscribe
Google Reader or Homepage

del.icio.us CodeBetter.com Latest Items
Add to My Yahoo!
Subscribe with Bloglines
Subscribe in NewsGator Online
Subscribe with myFeedster
Add to My AOL
Furl CodeBetter.com Latest Items
Subscribe in Rojo

Member Projects
DimeCasts.Net - Derik Whittaker

Friends of Devlicio.us
Red-Gate Tools For SQL and .NET

NDepend

SlickEdit
 
SmartInspect .NET Logging
NGEDIT: ViEmu and Codekana
LiteAccounting.Com
DevExpress
Fixx
NHibernate Profiler
Unfuddle
Balsamiq Mockups
Scrumy
JetBrains - ReSharper
Umbraco
NServiceBus
RavenDb
Web Sequence Diagrams
Ducksboard<-- NEW Friend!

 



Site Copyright © 2007 CodeBetter.Com
Content Copyright Individual Bloggers

 

Community Server (Commercial Edition)