The Bolla Blog

Sponsors

The Lounge

Wicked Cool Jobs

Syndication

News

  • Settling into a new job/life in the Pittsburgh area.
Images in this post missing? We recently lost them in a site migration. We're working to restore these as you read this. Should you need an image in an emergency, please contact us at imagehelp@codebetter.com
A Better Continuous Integration + Version Control System?

Imagine this workflow...

  1. You open your project in your IDE.
  2. The IDE automatically updates your working copy with any new changes on the server.
  3. You begin your work, using TDD or "test after".
  4. You run your tests and they pass.
  5. The locally passing tests trigger the process of your changes being sent to the CI server.
  6. The CI server does a full build + test run.
  7. If the full build/test completes, your changes are automatically committed to the VC server. The commit message is the concatenation of the new test names.
  8. Your teammate's IDE polls the VC server for changes once per minute, and automatically updates his working copy when new changes are available.
  9. His IDE is smart enough to merge changes in a way that does not disrupt his typing. Projects are updated "silently" and merges are performed successfully even with files that are open with unsaved changes.
  10. This process is repeated thoughout the day between all team members.


The concept here, is that passing tests trigger integration. The benefits are less excise of having to do updates and commits, as well as a better sense of "trust" in the latest version because it is always compiling and passing tests. The test-driven commit encourages effective test writing.

 This obviously puts the burden on the codebase to be testable. Some parts of the application, UI for example, are notoriously difficult to test. This "Continuously Integrating Version Control System" (CIVCS) would need ways to manually trigger commits as well. Plus for updating non-functional parts of the application, such as your spinning company logo graphic.

Also we need an IDE that can load changes to a project file without interrupting the user. VS 2005 is horrible at this. One way to cheat this would be for the CIVCS IDE plugin to simulate the changes being made manually though the UI.

There are a few places where our currently tools aren't quite ready to make this a reality, but I don't think we're that far either. I image it would be rather easy to add a feature to Resharper's test runner to trigger a process if all the tests pass. TeamCity already has a feature to do a "pre-tested commit." Determining new tests should be as easy as comparing all the test names to the previous build's list of tests. Polling for changes... trivial. Merging changes without interrupting the user... ok we need to work on this one. The "only tested code" concept could be handled with a branch that gets sync'd with the main branch. Most of the puzzle pieces are there. We just need someone (*cough* Jetbrains *cough*) to put them all together. Thoughts?

 


Posted 04-03-2008 9:39 PM by Jim Bolla
Filed under: ,

[Advertisement]

Comments

keith wrote re: A Better Continuous Integration + Version Control System?
on 04-03-2008 11:18 PM

automatic commits and updates is such a horrible idea.

developers need a stable base upon which to make changes over time, which they sync up to the trunk when they are prepared to, not when their IDE chooses to.  just b/c tests pass doesn't mean your changes are compatible with my changes.  have you never worked on a project where multiple ppl are changing the same files?  

the goal of CI isn't to automate your entire workflow.  that mindset is a slippery slope towards code generation and robots programming.

Derik Whittaker wrote re: A Better Continuous Integration + Version Control System?
on 04-04-2008 6:31 AM

I don't like the idea of a passing test updating VC or kicking off CI.  Many times my tests 'pass', but I am not done with either the test or code under test.  I could have logic that is mid cycle and checking everything in could foobar the next guy trying to use my code.

I see your point, but I think that check-ins have to be a manual, with intent process.

Jim Bolla wrote re: A Better Continuous Integration + Version Control System?
on 04-04-2008 9:10 AM

"developers need a stable base upon which to make changes over time"

I agree. But I think that a code base that always compiles and passes a substantial battery of tests IS a stable base.

"just b/c tests pass doesn't mean your changes are compatible with my changes"

Assuming adequate test coverage and all the tests pass, then there's a high chance that the changes ARE compatible, because passing tests mean that the code still meets all of the assertions established in the tests.

"have you never worked on a project where multiple ppl are changing the same files?"

One scenario is when two people are editing a file in independent ways. Dev A is changing methods A, B,C and Dev B is changing methods X, Y, Z. This type of change should be transparently mergeable. The other scenario is when Dev A and Dev B are editing the same block of code, which leads to merge conflicts. In this scenario "CIVCS" wound't be able to not automerge the changes but instead could indicate the merge conflict by putting a "merge confilct" icon in the gutter to the left of the relevant lines of code.

"the goal of CI isn't to automate your entire workflow."

Perhaps *my* goal is in fact to automate my workflow. There are different workflows that work well for different teams. I'm simply suggesting an alternative that I think has some potential.

" that mindset is a slippery slope towards code generation and robots programming."

Code generation can be a powerful tool when used appropriately. For example, we generate DTO classes based on our database schema as part of our data access layer. This gives us a certain level of compile time validation of our data access. Our NUnit tests cover the rest. Reshaper templates are another form of code generation that is quite handy on the micro scale. And a web control is nothing more that a code construct (C#) that generates other code constructs (HTML + JS)

Jim Bolla wrote re: A Better Continuous Integration + Version Control System?
on 04-04-2008 9:40 AM

"Many times my tests 'pass', but I am not done with either the test or code under test. "

If you're writing new code that is not yet integrated into the entire system, then your teammate receiving this partially complete (but compiling and test passing) code should not disrupt him.

If you're modifying existing code by refactoring or adding functionality but the tests all pass, then this should still be safe code for your teammate to use.

If you're changing the behavior of code, and then changing the assertions in the tests for that code, then there is a chance that these changes may cause problems for your teammate.

Derik, can you think of any other scenarios where a committed change may be disruptive to another teammember?

So far I'm seeing two negative reactions:

"I don't want my changes automatically committed"

and

"I don't want my working copy automatically updated/"

I think auto-committing is a great idea. It means tested changes are available for integration sooner.

I can feel how auto-updating could be disrupted. This is probably where the tools need to work well to not disrupt the developer. I get this. But from my experience, at least 90% of the time changes could be merged in silently and successfully.

Steve Campbell wrote re: A Better Continuous Integration + Version Control System?
on 04-04-2008 11:02 AM

I'm a little wary of the auto-commit idea - explicit commits are important for lots of reasons, not least of which is identification of atomic changes that need to be integrated to other branches of the code.  

The idea of the working copy being auto-updated is ok for me, but may not work on larger projects that contain multiple solutions, in such a way that for the new working copy to function, I would need new binaries too, and possibly even a database migration script.  Also, if a programmer is spiking on something, then they may throw away changes - integrating too early can be a bad thing then.

That said, I can see scenarios where the idea has merit - perhaps in the beginning stages of the project, where we are growing from nothing to something.  Often, team members have to wait around while someone else works on a core feature.  In that case, perhaps it would be good to get a more rapid integration cycle.  

keith wrote re: A Better Continuous Integration + Version Control System?
on 04-04-2008 2:21 PM

"The other scenario is when Dev A and Dev B are editing the same block of code, which leads to merge conflicts. In this scenario "CIVCS" wound't be able to not automerge the changes but instead could indicate the merge conflict by putting a "merge confilct" icon in the gutter to the left of the relevant lines of code."

but then i'm forced to resolve the conflict _now_ instead of when i'm ready to (when my code is finished and ready, by my definition of ready, for integration)

disruption and context switches lower productivity substantially, on average.

Greg Young wrote re: A Better Continuous Integration + Version Control System?
on 04-04-2008 5:57 PM

While this sounds pretty good and reasonable you are asking your development environment to do the equivalent of realizing when you want a beer and walking over to the kitchen to bring you one.

btw: commit on tests pass only works if I write tests first ;-)

Sean Chambers wrote re: A Better Continuous Integration + Version Control System?
on 04-06-2008 11:51 AM

I feel the need to chime in here as well. Interesting ideas but like everyone else I think auto-commit is a very bad idea.

The last step of the TDD mantra is refactor. Therefore, if an auto-commit is happening every green step, then you never have an opportunity to refactor your tests/code to support the new test. In addition, I write 10-20 tests between commits because  one commit from my workstation often includes a set of tests that test a specific portion of the project, but just a single test per commit.

Some of your ideas are great though, opening your IDE and it automatically getting the most current revision from SC is a good idea, maybe with an alert window in case you don't want the most recent revisions however.

Add a Comment

(required)  
(optional)
(required)  
Remember Me?

About The CodeBetter.Com Blog Network
CodeBetter.Com FAQ

Our Mission

Advertisers should contact Brendan

Subscribe
Google Reader or Homepage

del.icio.us CodeBetter.com Latest Items
Add to My Yahoo!
Subscribe with Bloglines
Subscribe in NewsGator Online
Subscribe with myFeedster
Add to My AOL
Furl CodeBetter.com Latest Items
Subscribe in Rojo

Member Projects
DimeCasts.Net - Derik Whittaker

Friends of Devlicio.us
Red-Gate Tools For SQL and .NET

NDepend

SlickEdit
 
SmartInspect .NET Logging
NGEDIT: ViEmu and Codekana
LiteAccounting.Com
DevExpress
Fixx
NHibernate Profiler
Unfuddle
Balsamiq Mockups
Scrumy
JetBrains - ReSharper
<-- NEW Friend!

 



Site Copyright © 2007 CodeBetter.Com
Content Copyright Individual Bloggers

 

Community Server (Commercial Edition)

CodeBetter.Com