Rob Reynolds - The Fervent Coder : UppercuT, HowTo

Published Applications AKA _PublishedApplications

Rob Reynolds — Wed, 23 Mar 2011 02:23:12 GMT

Less maintenance. Less work to package during your automated builds. Too easy.

Remember Our Old Friend _PublishedWebsites?

You’ve probably seen the _PublishedWebsites folder when building websites in automated builds. If not you can stop paying attention now.

Still with me? Great! So you know how it packages up everything nicely with content files going where they should with nearly ZERO cost to your build scripts. All you need to do is override the output directory (OutDir) and you get this feature. This behavior is arguably one of the best things that web projects do during automated builds.

So how many of you, like me, just accepted the fact that this was just for web, and not other applications, and went on maintaining your build scripts to package up your other application types? Yeah, me too. Painful…and we looked at complicated ways of solving this problem. There has to be a better way.

Introducing _PublishedApplications

Recently David Keaveny came up with an idea to take the Microsoft.Web.targets file (which produces the _PublishedWebsites folder – it is imported by the csproj/vbproj file), change up a couple of variables and rename it to the Microsoft.Application.targets file. When I saw this I thought this was an awesome idea! How come I didn’t think of that? Why didn’t anyone else see the simple solution?

The concept is simple – You add the .targets file to your application. Then you edit the csproj/vbproj file by hand to add in the import for the .targets file.

And you get a nice folder with your application bits nicely packaged for you.

Note: The _PublishedApplications folder ONLY shows up when you override the output directory during an automated build or command line access to building a solution. The rest of the time your builds go to their normal folders (i.e. bin\Debug).

PublishedApplications Is On NuGet

I loved this idea so much I did the work to figure out how to automate the imports part of the idea so it was a simple command to use Microsoft.Application.targets. And this is on NuGet: http://nuget.org/list/packages/publishedapplications

Install-Package PublishedApplications

Want A Demonstration?

Why talk when we can just look at a short video that demonstrates this idea?

Install-Package PublishedApplications

Special thanks to David Keaveny for this great idea!

AppHarbor - Azure Done Right AKA Heroku for .NET

Rob Reynolds — Wed, 16 Feb 2011 12:49:23 GMT

Easy and Instant deployments and instant scale for .NET?

Awhile back a few of us were looking at Ruby Gems as the answer to package management for .NET. The gems platform supported the concept of DLLs as packages although some changes would have needed to happen to have long term use for the entire community. From that we formed a partnership with some folks at Microsoft to make v2 into something that would meet wider adoption across the community, which people now call NuGet. So now we have the concept of package management. What comes next?

Heroku

Instant deployments and instant scaling. Stupid simple API. This is Heroku. It doesn’t sound like much, but when you think of how fast you can go from an idea to having someone else tinker with it, you can start to see its power. In literally seconds you can be looking at your rails application deployed and online. Then when you are ready to scale, you can do that. This is power. Some may call this “cloud-computing” or PaaS (Platform as a Service).

I first ran into Heroku back in July when I met Nick of RubyGems.org. At the time there was no alternative in the .NET-o-sphere. I don’t count Windows Azure, mostly because it is not simple and I don’t believe there is a free version. Heroku itself would not lend itself well to .NET due to the nature of platforms and each language’s specific needs (solution stack). So I tucked the idea in the back of my head and moved on.

AppHarbor Enters The Scene

I’m not sure when I first heard about AppHarbor as a possible .NET version of Heroku. It may have been in November, but I didn’t actually try it until January. I was instantly hooked. AppHarbor is awesome! It still has a ways to go to be considered Heroku for .NET, but it already has a growing community. I created a video series (at the bottom of this post) that really highlights how fast you can get a product onto the web and really shows the power and simplicity of AppHarbor.

Deploying is as simple as a git/hg push to appharbor. From there they build your code, run any unit tests you have and deploy it if everything succeeds. The screen on the right shows a simple and elegant UI to getting things done.

The folks at AppHarbor graciously gave me a limited number of invites to hand out. If you are itching to try AppHarbor then navigate to: https://appharbor.com/account/new?inviteCode=ferventcoder

After playing with it, send feedback if you want more features. Go vote up two features I want that will make it more like Heroku.

Disclaimer: I am in no way affiliated with AppHarbor and have not received any funds or favors from anyone at AppHarbor. I just think it is awesome and I want others to know about it.

From Zero To Deployed in 15 Minutes (Or Less)

Now I have a challenge for you. I created a video series showing how fast I could go from nothing to a deployed application. It could have been from Zero to Deployed in Less than 5 minutes, but I wanted to show you the tools a little more and give you an opportunity to beat my time.

And that’s the challenge. Beat my time and show it in a video response. The video series is below (at least one of the videos has to be watched on YouTube). The person with the best time by March 15th @ 11:59PM CST will receive a prize.

Ground rules:

.NET Application with a valid database connection
Start from Zero
Deployed with AppHarbor or an alternative
A timer displayed in the video that runs during the entire process
Video response published on YouTube or acceptable alternative
Video(s) must be published by March 15th at 11:59PM CST.
Either post the link here as a comment or on YouTube as a response (also by 11:59PM CST March 15th)

From Zero To Deployed In 15 Minutes (Or Less) Part 1

From Zero To Deployed In 15 Minutes (Or Less) Part 2

From Zero To Deployed In 15 Minutes (Or Less) Part 3

UppercuT v1.2–NuGet Support

Rob Reynolds — Mon, 24 Jan 2011 02:18:57 GMT

For those that have not yet heard, NuGet went v1 recently along with a whole slew of tools from the Microsoft folks. I’ve been lucky to be a part of the NuGet project and see it take shape over the past few months with community input and contributions. Even though v1.0 was released, we are already moving forward with getting ideas and prioritizing features for the next version.

To follow the announcement, UppercuT (UC) v1.2 now includes support for NuGet out of the box. Plus, it will handle versioning the nuspec file for you, a highly requested feature for those that worked with other package managers.

UppercuT + NuGet == Can Packaging Get Any Easier?

It can. UC exists to take the pains out of builds. To go from zero to your first NuGet package with the goodness of UC, just read on and follow the directions. If you are already creating nuget packages, UC can help update the version in the nuspec for you automatically. Read on…

Upgrading?

1. For those upgrading, you bring over the entire contents of the build directory like before. Please see the downloads (or the ReadMe ReleaseNotes section) for any items you need to change between your previous version and the latest version of UC you are downloading.

2. You will want to add the following to your UppercuT.config file:

<property name="app.nuget" value="..${path.separator}${folder.references}${path.separator}NuGet${path.separator}NuGet.exe" overwrite="false" />

2. Include the NuGet folder in your lib directory from the UC distribution (or get the latest NuGet.exe and drop it in a NuGet folder under lib):

3. Add the nuget directory at your top level (next to the build.bat file) from the UC distribution:

New to Uppercut?

1. There is a nice write up on how to get UC set up on your project in less than 5 minutes (or you can try the gems approach)! It really shows how little is needed to get a fully conventional build with the ability to upgrade in seconds instead of hours for your builds.

Getting Some Nuget-ty Goodness To Your Builds

1. Head into the top level nuget folder. Rename the __NAME__.nuspec file to the name of your nugget (most likely your project name if there are no other packages named the same). Here we are working with SidePOP, so I named it sidepop.nuspec.

2. Let’s open that file and take a look.

<?xml version="1.0"?>
<package xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <metadata>
    <id>__REPLACE__</id>
    <version>DO_NOT_EDIT</version>
    <authors>__REPLACE__</authors>
    <owners>__REPLACE__</owners>
    <summary>__REPLACE__</summary>
    <description>__REPLACE__</description>
    <!--<projectUrl>__REPLACE__</projectUrl>
    <licenseUrl>__REPLACE__</licenseUrl>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <tags>space delimited</tags>
    <iconUrl>32x32.png</iconUrl>
    <dependencies>
      <dependency id="something" version="1.0.0.0" />
    </dependencies>-->
  </metadata>
</package>

NOTE: Notice the metadata attribute doesn’t have the xmlns in it (xmlns="http://schemas.microsoft.com/packaging/2010/07/nuspec.xsd"). This needs to be removed for xpath to be able to replace the version. If you are bringing an existing nuspec in, you will want to remove that.

3. Look how most things say __REPLACE__. The version attribute does not. Do not edit version (if you do, it will just be replaced during the build).

4. Let’s set up the nuspec for SidePOP. SidePOP has a dependency on log4net version 1.2.10.

<?xml version="1.0"?>
<package xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <metadata>
    <id>sidepop</id>
    <version>DO_NOT_EDIT</version>
    <authors>Rob Reynolds, Tim Hibbard</authors>
    <owners>Rob Reynolds</owners>
    <summary>SidePOP gives your app the ability to receive email</summary>
    <description>SidePOP allows your application the ability to receive email</description>
    <projectUrl>http://sidepop.googlecode.com</projectUrl>
    <licenseUrl>http://www.apache.org/licenses/LICENSE-2.0</licenseUrl>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <tags>email</tags>
    <!--<iconUrl>32x32.png</iconUrl>-->
    <dependencies>
      <dependency id="log4net" version="1.2.10" />
    </dependencies>
  </metadata>
</package>

5. We also have content transformation for the config file, so we’ll include that in the nuget folder.

6. In that we will create our transforms for the web.config and app.config.

7. Now we go back up to our top level folder and run build.bat.

8. If we head down to our code_drop/nuget folder, you can see we have a nupkg with the right version on it!

9. Now all we have to do is look to be sure we have everything we want and we can upload. In my case I need to exclude the log4net file I have, so I need to delete that one from the directory during the build process. How do I get in? UC Extension Points of course!

10. If I drop into the build folder and open a command line I can type: customize nugetPrepare.step post

11. This will create an extension point file for me in build.custom.

12. Open that file and let’s remove the log4net file from the nuget drop directory prior to the nuget build. This is how I set up that file:

<?xml version="1.0" encoding="utf-8"?>
<project name="CUSTOM POST NUGETPREPARE" default="go">
  <!-- Project UppercuT - http://projectuppercut.org -->
  <property name="build.config.settings" value="__NONE__" overwrite="false" />
  <include buildfile="${build.config.settings}" if="${file::exists(build.config.settings)}" />
  <property name="path.separator" value="${string::trim(path::combine(' ', ' '))}" />
  <property name="dirs.current" value="${directory::get-parent-directory(project::get-buildfile-path())}" />
  <property name="path.to.toplevel" value=".." />
  <property name="folder.code_drop" value="code_drop" overwrite="false" />
  <property name="dirs.drop" value="${dirs.current}${path.separator}${path.to.toplevel}${path.separator}${folder.code_drop}" overwrite="false" />
  <property name="dirs.drop" value="${path::get-full-path(dirs.drop)}" />
  <property name="folder.nuget" value="nuget" overwrite="false" />
  <property name="dirs.drop.nuget" value="${dirs.drop}${path.separator}${folder.nuget}" overwrite="false" />
  
  <target name="go">
    <echo message="Removing log4net from '${dirs.drop.nuget}\lib'" />
    <delete>
      <fileset basedir="${dirs.drop.nuget}\lib" >
        <include name="log4net.*" />
      </fileset>
    </delete>
  </target>
  
</project>

13. Now looky here, log4net is no longer in my package.

14. Now I am ready to test my package before I push it out by putting it in a local feed and grabbing it.

15. All good. Now I can upload and put my package on the official feed!

NOTE: SidePOP is in Alpha. If you want to use it to check email, it works, but be prepared for possible errors.

Conclusion – UppercuT and NuGet – A Good Team!

You can see how fast we went from ZERO NuGet to up on the http://nuget.org site! In 4 steps I was already building a NuGet package, and in less than 15 steps it was production ready! What are you waiting for? Go UppercuT your code now!

If you have any UC related questions when getting it all NuGet-ty, feel free to contact the list: http://groups.google.com/group/chucknorrisframework

With this knowledge, you shall build.

How To – UppercuT and Gems

Rob Reynolds — Tue, 20 Jul 2010 14:00:00 GMT

In a previous post I mentioned how I was going to show you how UppercuT (UC) has the ability to make gems stupid simple to create and publish. You ask if gems can get any easier and to that I answer, “Why YES, they can!” How about just filling out the information for the gemspec, running a build and having a nice, shiny new gem ready for publishing?

Rock The Gems

Basically you want to get the latest release of UppercuT. You can download it or grab the source and compile.

There are already instructions out there for how to get UC in your project, so I’m not going to concentrate on that.

Once you upgrade (or add and get everything else set up), you want to have this gems folder at your top level (just under trunk or branch name).

In that gems folder you are going to find a file named something like the file below. Rename that file to your new gemname.gemspec.

Note: Once you have a gemspec file in a gems folder, your build server NOW needs to also have ruby and gems installed.

Open that file in your favorite text editor and fill in the details. Here’s a good post on how to do that.

Then just for having the gems folder with a gemspec in it, UC will automatically try to build the gem for you (the code in your code_drop/projectname folder is brought over to code_drop/gems/lib folder).

Removing All of the Other Output After the Gem is Built

Once we are good with what we are getting back for the gem, we can start cleaning up. So we go into our build.custom (don’t have one? create it right next to the build folder) folder and create a file named gemsBuild.post.step.

Let’s open the file and insert this:

<?xml version="1.0" encoding="utf-8"?>
<project name="CUSTOM POST GEMSBUILD" default="go">
  <!-- Project UppercuT - http://projectuppercut.org -->
  <property name="build.config.settings" value="__NONE__" overwrite="false" />
  <include buildfile="${build.config.settings}" if="${file::exists(build.config.settings)}" />
  <property name="dirs.current" value="${directory::get-parent-directory(project::get-buildfile-path())}" />
  <property name="path.to.toplevel" value=".." />
  <property name="folder.code_drop" value="code_drop" overwrite="false" />
  <property name="dirs.drop" value="${dirs.current}\${path.to.toplevel}\${folder.code_drop}" overwrite="false" />
  <property name="folder.gems" value="gems" overwrite="false" />
  
  <target name="go" depends="run_tasks" />
  
  <target name="run_tasks">
    <delete>
      <fileset basedir="${dirs.drop}/${folder.gems}" >
        <exclude name="*.gem" />
        <include name="**/*" />
      </fileset>
    </delete>
  </target>
  
</project>

Note: Don’t like NAnt? You can also use Ruby or PowerShell instead of NAnt to write your custom extensions.

Now when we run our build again, we have a nice clean folder.

What If I Want to Change What Goes Into my Gem?

Interested in influencing what goes INTO your gem in the first place? That’s a pretty good thing to be concerned with so that you don’t have all of your referenced assemblies sitting in there. Read about how to set up dependencies. Then you will create a file next to gemsbuild.post.step named gemsPrepare.post.step.

In that file, you will insert something similar to the following (roundhouse file):

<copy todir="${dirs.drop}\${folder.gems}\lib">
  <fileset basedir="${dirs.drop}\${folder.gems}\lib\MSBuild">
    <include name="**/*.*" />
  </fileset>
</copy>

<copy todir="${dirs.drop}\${folder.gems}\lib">
  <fileset basedir="${dirs.drop}\${folder.gems}\lib\NAnt">
    <include name="**/*.*" />
  </fileset>
</copy>

<delete>
  <fileset basedir="${dirs.drop}\${folder.gems}\lib" >
    <include name="ConsoleApp/**" />
    <include name="MSBuild/**" />
    <include name="NAnt/**" />
  </fileset>
</delete>

Learn More

With this knowledge, you shall build. Interested in more UppercuT? Check out the ChuckNorris framework and join the group.

Before you comment about “cluttering” the ruby community, please be sure to read this (we’re with you on this): http://devlicio.us/blogs/rob_reynolds/archive/2010/07/19/gems-for-net-community-response.aspx

Gems - Package Management for .NET

How To – Gems & .NET & How To – Gems & .NET - Dependencies (References)

The Future is Now!

How To – Gems And .NET – Dependencies (References)

Rob Reynolds — Sat, 17 Jul 2010 13:04:00 GMT

In my last post I didn’t mention dependencies. Dependencies are their own animal. They require a couple more things to be in place. Let’s talk about those things.

In the .NET world, the dependency for compiled bits is usually an exact version of a reference.

Let me explain. So for example, you have a reference to log4net, and you don’t ILMerge it into your assembly. You now have a dependency that the DLL needs to be there and a particular version (outside of redirecting the bindings). So what I’m getting at is that you require an exact version of a particular DLL. And what you really need is an exact name, version, culture, and public key token of a DLL. But let’s keep things simple. It’s really the version and the name when culture is neutral (and the key shouldn’t change in the same version). So just the name and version.

Adding a Reference as a Dependency

For each reference you have to a library, you find out what version it is (assembly version) and then add that as a dependency. You can do that by cracking open reflector and taking a look at the actual assembly version.

Don’t use the properties. Neither file version or product version are going to be accurate here:

There is nothing out there that says that assembly, file and informational (also known as product) versions have to be the same. .NET relies on the assembly version for referencing. It makes sense that we should as well. Here’s a better example where things are different:

So what would I put in my gemspec? If your reference was to log4net version 1.2.10.0, then you need to assign a dependency to that exact version. Done like so:

spec.add_dependency('log4net','= 1.2.10.0')

I believe you add each referenced dependency to it’s own line.

Gem Exists?

Now to the sanity check. Before you even add it as a dependency, you want to ensure that the gem exists.

Go to http://rubygems.org and in the top right there is a search box. Search for your reference there.

So let’s search for log4net to be sure it’s there.

Sweet! I can move on to my next reference because the right version of the gem exists.

Keep in mind that the name of the gem may not be the one you are looking for and/or the name may be slightly different. For example. I have a gem for UppercuT. The gem is named uppercutbuild because there was already a gem named uppercut.

Gem Doesn’t Exist?

Now if it’s not there, you can add it. When the actual authors want to start managing the gem, you can just add them as owners so they can push their own gems.

To check the owners of a gem you type:

gem owner gemname

To add someone, according to the gem docs, you issue this command (all on one line):

gem owner gemname --add users.confirmed.email.address.for.ruby.gems@wherever.com

And that’s it.

You see how I am listed as the owner of the log4net gems? I am not really the developer, when I created the gem, I tied it as closely as I could to the apache project and the committers. When those guys are ready to own the gem, I have the specs for both 1.2.9 and 1.2.10 (both are commonly referred to without the last version octet) and I can just add them as owners.

Gems - Package Management for .NET & How To – Gems & .NET

Walkthrough - Create Gems Even Easier With a Conventional Build (UppercuT)!

The Future is Now!

How To - Gems And .NET

Rob Reynolds — Fri, 16 Jul 2010 15:30:00 GMT

In my last post I showed gems being used for .NET. Now let’s talk about How. Most of this stuff I’ve learned over the past two days, so if I have a mistake here or you have a better idea, please don’t hesitate to offer a better solution.

The GemSpec

The Gem::Specification reference is your friend.

In order to create a gem, you need to define a gem specification, commonly
called a “gemspec”.

A gemspec consists of several attributes. Some of these are required;
most of them are optional.

From here you learn what is required and what will just get you there.

Setup

1. Create a folder named gems in your top level source directory.

2. In that folder we are going to put our gemspec and version files. This is where we will store the files in source control (and one of them may become autogenerated).

3. We will bring our gems folder to our compiled source folder after we build. Then we can add in the compiled output.

GemSpec for .NET

1. Create a file named project.gemspec. In our example it is roundhouse.gemspec. This is the most important file for this entire process.

2. Open the gemspec in your favorite notepad editor. Copy the below in and change it for you needs.

version = File.read(File.expand_path("../VERSION",__FILE__)).strip

Gem::Specification.new do |spec|
  spec.platform    = Gem::Platform::RUBY
  spec.name        = 'roundhouse'
  spec.version     = version
  spec.files = Dir['lib/**/*']

  spec.summary     = 'RoundhousE - Professional Database Change and Versioning Management'
  spec.description = 'RoundhousE is a Professional Database Change and Versioning Management tool'
  
  spec.authors           = ['Rob "FerventCoder" Reynolds','Pascal Mestdach','Jochen Jonckheere','Dru Sellers']
  spec.email             = 'chucknorrisframework@googlegroups.com'
  spec.homepage          = 'http://projectroundhouse.org'
  spec.rubyforge_project = 'roundhouse'
end

3. Just about everything with tick marks above you will edit to suit your needs. spec.name and spec.rubyforge_project (and the gemspec file name) should all match and not be an already existing project name on RubyForge.

4. If you are a singular author, instead of spec.authors, replace it with

spec.author = 'somebody'

You can also set up description for multiple lines.

spec.description = <<-EOF
   Rake is a Make-like program implemented in Ruby. Tasks and
   dependencies are specified in standard Ruby syntax.
 EOF

Dependencies On Other Libraries

What we call references. You have a dependency on them existing for your library to run. See this post.

VERSION file

This file is stupid simple. It’s a version number. I believe you can put whatever you want in here. Use the Assembly Version number here and stick with the .NET 4 octets of numbers (like 0.0.0.0) for version.

1. Create a file named VERSION.

2. Open it in your favorite editor and put the version you want here.

Libraries

1. Create a folder called lib.

2. Copy YOUR compiled DLLs into here. Your references (or dependencies) should not go here. See How To: Gems and .NET - Dependencies.

Documentation

Create a docs folder. In that folder goes your documentation. This could be release notes, a ReadMe, actual documentation. This area is open. Just make sure you add the docs folder to the specification.

spec.files = Dir['lib/**/*'] + Dir['docs/**/*']

Executables

If you want to give someone the ability to run an executable from the command line after installing your application with gems, and you use .NET, this is how you do it.

1. Create a folder named bin as a subdirectory next to the gemspec.

2. Put your executable (and dependencies) into the bin directory.

3. Create your shim file (it’s named what you would type at the command line). I’ve called mine rh. The one above allows ruby to be able to actually execute the Windows executable. Let’s open it and see what it looks like. This, we learned from an answer to a post on stack overflow. This shim also goes in source control

result = system(File.dirname(__FILE__) + "/rh.exe " + ARGV.join(' '))
exit 1 unless result

4. Don’t forget that space at the end of your executable name.

5. Open your gemspec. We need to make sure we have spec.executables filled out and our new directory added to our Files.

version = File.read(File.expand_path("../VERSION",__FILE__)).strip

Gem::Specification.new do |spec|
  spec.platform    = Gem::Platform::RUBY
  spec.name        = 'roundhouse'
  spec.version     = version
  spec.files = Dir['lib/**/*'] + Dir['docs/**/*'] + Dir['bin/**/*']
  spec.bindir = 'bin'
  spec.executables << 'rh'

  spec.summary     = 'RoundhousE - Professional Database Change and Versioning Management'
  spec.description = 'RoundhousE is a Professional Database Change and Versioning Management tool'
  
  spec.authors           = ['Rob "FerventCoder" Reynolds','Pascal Mestdach','Jochen Jonckheere','Dru Sellers']
  spec.email             = 'chucknorrisframework@googlegroups.com'
  spec.homepage          = 'http://projectroundhouse.org'
  spec.rubyforge_project = 'roundhouse'
end

Build And Push

You must already have Ruby (1.8.6 or better) and RubyGems installed and/or updated to at least 1.3.7 (gem update –system). You will want to use a RubyInstaller version of Ruby under the Ruby on Windows section.

1. Open a command line and type

gem build project

2. If there are no issues, you should have a gem for upload.

3. Create an account with RubyGems.org.

4. Ensure your project name isn’t already taken by searching for it. If it is you will need to rename your gemspec file, spec.name, and spec.rubyforge_project to a name that is not taken.

5. Type the following command:

gem push project_someversionnumbers.gem

6. Let it finish. Head out to rubygems.org and look at your shiny, new gem!

7. Test it.

gem install project

gem uninstall project

FollowUp

In my next post, I’ll show you how to make it stupid simple. UppercuT will support this feature natively so all you have to do is have a directory called gems (at the top level of your source) with the gemspec. During the build, it will create the VERSION file and copy your output the the lib folder (custom step for bin folder will be necessary). Then it will execute a step that builds the gem. All for the price of “build”. That’s coming up in the next blog post. In the meantime, feel free to ask any questions you have. Stay tuned…

Gems - Package Management for .NET & How To – Gems & .NET - Dependencies (References)

Walkthrough - Create Gems Even Easier With a Conventional Build (UppercuT)!

The Future is Now!

Deciding On Features For Open Source

Rob Reynolds — Sat, 15 May 2010 23:16:48 GMT

“Open source feature selection is subjective.”

An interesting question was posed to me recently at a presentation - “How do you decide what features to include in the [open source] projects you manage?”

Is It Objective?

I’d like to say that it’s really objective and that we vote on features and look at what carries the most interest of the populace. Actually no I wouldn’t. I don’t think I would enjoy working on open source (OSS) as much if it someone else decided on what features I should include. It already works that way at work. I don’t want to come home from work and work on things that others decide for me unless I’m being paid for those features.

So how do I decide on features for our open source projects? I think there are at least three paths to feature selection and they are not necessarily mutually exclusive.

Feature Selection IS the Set of Features For the Domain

Your product, in whatever domain it is in, needs to have the basic set of features that make it answer the needs of that domain. That is different for every product, but if you take for example a build tool, at the very least it needs to be able to compile source. And these basic needed features are not always objective either. Two people could completely disagree what makes for a required feature to meet a domain need for a product. Even one person may disagree with himself/herself about what features are needed based on different timeframes. So that leads us down to subjective.

Feature Selection IS An Answer To Competition

Some features go in because the competition adds a feature that may draw others away from your product offering. With OSS, there are all free alternatives, so if your competition adds a killer feature and you don’t, there isn’t much other than learning (how to use the other product) to move your customers off to the competition. If you want to keep your customers, you need to be ready to answer the questions of adding the features your competition has added.

Sometimes it’s about adding a feature that your competition charges for, but you add it for free. That draws people to the free alternative – so sometimes that adds a motivation to select a feature. Sometimes it’s because you want those features in your product, either to learn how you can answer the question of how to do something and/or because you have a need for that feature and you want it in your product. That also leads us down the road to subjective.

Feature Selection IS Subjective

I decide on features based on what I want to see in the product I am working on. Things I am interested in or have the biggest need for usually get picked first, with things that do not interest me either coming later or not at all. Most people get interested in an area of OSS because it solves a need for them and/or they find it interesting. If one of these two things is not happening and they are not being paid, it’s likely that person will move on to something else they find interesting or just stop OSS altogether.

OSS feature selection is just that – subjective. If it wasn’t, it wouldn’t be opinionated and it wouldn’t have a personality about it. Most people like certain OSS because they like where the product is going or the personalities behind the product.

For me, I want my products to be easy to use and solve an important problem. If it takes you more than 5-10 minutes to learn how to use my product, I know you are probably going somewhere else. So I pick features that make the product easy to use and learn, and those are not always the simplest features to work on. I work for conventions and make the product opinionated, because I think that is what makes using a product easier, if it already works with little setup. And I like to provide the ability for power users to get in and change the conventions to suit their needs. So those are required features for me above and beyond the domain features. I like to think I do a pretty good job at this. Usually when I present on something I’ve created, I like seeing people’s eyes light up when they see how simple it is to set up a powerful product like UppercuT.

Patches And/Or Donations

But remember before you say I’m a bad person or won’t use my product, I’ll always accept patches or I might like the feature that you suggest. If you like using the products I provide and they solve a problem for you the two biggest compliments you can provide are either a patch or a donation. If you think the product is great, but if it could do this one other thing, it would be awesome(!), then consider contacting me and providing a patch, or consider contacting me with a donation and a request to put the feature in. And alternatively, if it’s a big feature, you could hire me to work on the product to make it even better.

What If There Are Multiple Committers?

In the question of multiple committers, I choose that someone always makes the ultimate decision to select whether a feature should be part of a product or not. But for other OSS project maybe this is not the case. If there is not an ultimate decision maker, then there is the possibility of either adding every feature suggested or having a deadlock on two conflicting features.

So let me pose this question. If you work on Open Source, how do you decide on what features to put in your open source projects? How do you decide what doesn’t belong? What do you do when there are conflicting features?

UppercuT – Custom Extensions Now With PowerShell and Ruby

Rob Reynolds — Sun, 14 Mar 2010 01:43:27 GMT

Arguably, one of the most powerful features of UppercuT (UC) is the ability to extend any step of the build process with a pre, post, or replace hook. This customization is done in a separate location from the build so you can upgrade without wondering if you broke the build.

There is a hook before each step of the build has run. There is a hook after. And back to power again, there is a replacement hook. If you don’t like what the step is doing and/or you want to replace it’s entire functionality, you just drop a custom replacement extension and UppercuT will perform the custom step instead.

Up until recently all custom hooks had to be written in NAnt. Now they are a little sweeter because you no longer need to use NAnt to extend UC if you don’t want to. You can use PowerShell. Or Ruby. Let that sink in for a moment. You don’t have to even need to interact with NAnt at all now.

Extension Points

On the wiki, all of the extension points are shown. The basic idea is that you would put whatever customization you are doing in a separate folder named build.custom. Each step Let’s take a look at all we can customize:

The start point is default.build. It calls build.custom/default.pre.build if it exists, then it runs build/default.build (normal tasks) OR build.custom/default.replace.build if it exists, and finally build.custom/default.post.build if it exists. Every step below runs with the same extension points but changes on the file name it is looking for. NOTE: If you include default.replace.build, nothing else will run because everything is called from default.build.
- policyChecks.step
- versionBuilder.step NOTE: If you include build.custom/versionBuilder.replace.step, the items below will not run.
  - svn.step, tfs.step, or git.step (the custom tasks for these need to go in build.custom/versioners)
- generateBuildInfo.step
- compile.step
- environmentBuilder.step
- analyze.step NOTE: If you include build.custom/analyze.replace.step, the items below will not run.
  - test.step (the custom tasks for this need to go in build.custom/analyzers) NOTE: If you include build.custom/analyzers/test.replace.step, the items below will not run.
    - mbunit2.step, gallio.step, or nunit.step (the custom tasks for these need to go in build.custom/analyzers)
  - ncover.step (the custom tasks for this need to go in build.custom/analyzers)
  - ndepend.step (the custom tasks for this need to go in build.custom/analyzers)
  - moma.step (the custom tasks for this need to go in build.custom/analyzers)
- package.step NOTE: If you include build.custom/package.replace.step, the items below will not run.
  - deploymentBuilder.step

Customize UppercuT Builds With PowerShell

UppercuT can now be extended with PowerShell (PS). To customize any extension point with PS, just add .ps1 to the end of the file name and write your custom tasks in PowerShell.

If you are not signing your scripts you will need to change a setting in the UppercuT.config file. This does impose a security risk, because this allows PS to now run any PS script. This setting stays that way on ANY machine that runs the build until manually changed by someone. I’m not responsible if you mess up your machine or anyone else’s by doing this. You’ve been warned.

Now that you are fully aware of any security holes you may open and are okay with that, let’s move on.

Let’s create a file called default.replace.build.ps1 in the build.custom folder.

Open that file in notepad and let’s add this to it:

write-host "hello - I'm a custom task written in Powershell!"

Now, let’s run build.bat.

You could get some PSake action going here. I won’t dive into that in this post though.

Customize UppercuT Builds With Ruby

If you want to customize any extension point with Ruby, just add .rb to the end of the file name and write your custom tasks in Ruby. Let’s write a custom ruby task for UC. If you were thinking it would be the same as the one we just wrote for PS, you’d be right!

In the build.custom folder, lets create a file called default.replace.build.rb.

Open that file in notepad and let’s put this in there:

puts "I'm a custom ruby task!"

Now, let’s run build.bat again.

That’s chunky bacon.

UppercuT and Albacore.NET

Just for fun, I wanted to see if I could replace the compile.step with a Rake task. Not just any rake task, Albacore’s msbuild task. Albacore is a suite of rake tasks brought about by Derick Bailey to make building .NET with Rake easier. It has quite a bit of support with developers that are using Rake to build code.

In my build.custom folder, I drop a compile.replace.step.rb. I also put in a separate file that will contain my Albacore rake task and I call that compile.rb.

What are the contents of compile.replace.step.rb?

rake = 'rake'
arguments= '-f ' + Dir.pwd + '/../build.custom/compile.rb'

#puts "Calling #{rake} " + arguments
system("#{rake} " + arguments)

Since the custom extensions call ruby, we have to shell back out and call rake. That’s what we are doing here. We also realize that ruby is called from the build folder, so we need to back out and dive into the build.custom folder to find the file that is technically next to us.

What are the contents of compile.rb?

require 'rubygems'
require 'fileutils'
require 'albacore'

task :default => [:compile]

puts "Using Ruby to compile UppercuT with Albacore Tasks"
desc 'Compile the source'
msbuild :compile do |msb|
  msb.properties = { :configuration => :Release, :outputpath => '../../build_output/UppercuT' }
  msb.targets [:clean, :build]
  msb.verbosity = "quiet"
  msb.path_to_command = 'c:/Windows/Microsoft.NET/Framework/v3.5/MSBuild.exe'
  msb.solution = '../uppercut.sln'
end

We are using the msbuild task here. We change the output path to the build_output/UppercuT folder. The output path has “../../” because this is based on every project. We could grab the current directory and then point the task specifically to a folder if we have projects that are at different levels. We want the verbosity to be quiet so we set that as well.

So what kind of output do you get for this? Let’s run build.bat

custom_tasks_replace:

     [echo] Running custom tasks instead of normal tasks if C:\code\uppercut\build\..\build.custom\compile.replace.step exists.
     [exec] (in C:/code/uppercut/build)
     [exec] Using Ruby to compile UppercuT with Albacore Tasks
     [exec] Microsoft (R) Build Engine Version 3.5.30729.4926
     [exec] [Microsoft .NET Framework, Version 2.0.50727.4927]
     [exec] Copyright (C) Microsoft Corporation 2007. All rights reserved.

If you think this is awesome, you’d be right!

With this knowledge you shall build.

Warmup – Getting Started

Rob Reynolds — Mon, 01 Feb 2010 14:25:00 GMT

What if there was a tool out there that could let you specify a structure for a project (visual studio solution + everything else) and save you up to 3+ hours of work every time you started a new project?

Warmup was an idea by Dru Sellers to remove all of the setup work required every time you set up a new project. You know, create the solution, add projects, put in your references, etc. Then how about getting the infrastructure for your service/website/console set up as well with things like IoC, etc? What about patterns and other pet items that you put in any project?

Yeah – there’s an app for that. And it’s pretty simple to use. Plus you can change your templates when you have new ideas, so it’s totally rockstar!

The first thing you do is to set up templates somewhere in source control (svn or git). Then you specify where that is to the configuration and what type of source control.

<warmup
  sourceControlWarmupLocation="git://github.com/ferventcoder/warmup-templates.git"
  sourceControlType="git"
  />

Then you run a simple command. If base was one of the folders below the directory above my source control, then that is what I would specify as the first argument.

warmup.exe base nameOfProject

I specify nameofProject. That is what I want my project to be named when I am complete.

Templating

Let’s start by taking a look at the base template. The basic idea here is simple. Place __NAME__ everywhere you want to be replaced when running Warmup. In the same way UppercuT does token replacement with ConfigBuilder, DocBuilder, SqlBuilder, and DeployBuilder, Warmup does token replacement for an entire solution.

If we were to look at some of the files you would see the absolute depth of how naming really can be replaced. Let’s take a look at the solution though, that will hold quite a bit of meaning for you.

Now that we have our template all set up, we have one thing to do. Open our .sln file in Notepad and delete the first line (not sure what happens here, but this works). Microsoft should be on the first line when we are done.

And now our template is all ready, so we check our changes into source. If we are using git, we need to push back to the repository we are looking at after we finish committing.

Run Warmup

Now that we’ve seen our template, let’s run Warmup.

Let’s call the new project Alpha.

warmup.exe base Alpha

This is the output:

Hardcore git cloning action to: C:\code\warmup\code_drop\warmup\Alpha
Running: cmd /c git clone git://github.com/ferventcoder/warmup-templates.git C:\code\warmup\code_drop\warmup\Alpha
Initialized empty Git repository in C:/code/warmup/code_drop/warmup/Alpha/.git/

replacing tokens

This project already has UppercuT, RoundhousE, and others already in it. Take a look at the lib folder:

I open a command line and run build.bat. I get a successful build with 31 passing tests! If I were to go to the code_drop folder, I’m all ready to deploy if I had my deployment framework already here.

I can already run RoundhousE and create my database from here:

Conclusion

I have an entire structure that allows me to just concentrate on the stories at hand. Warmup may not be the best thing since sliced bread, but it’s going to save you oodles of time! If you know someone else that has created a template you want to use and it’s shared publicly, you can just edit the config file to point there. We have been using warmup for close to 3 months now and it saves us quite a bit of time. Plus we find more and more things we can put back into the templates to save us time. I believe this aspect of learning and growing your templates over time is the intention of warmup. Plus your template may not be the same as mine and that’s completely cool!

Rob Reynolds - The Fervent Coder : UppercuT, HowTo

Published Applications AKA _PublishedApplications

Remember Our Old Friend _PublishedWebsites?

Introducing _PublishedApplications

PublishedApplications Is On NuGet

Want A Demonstration?

AppHarbor - Azure Done Right AKA Heroku for .NET

Heroku

AppHarbor Enters The Scene

From Zero To Deployed in 15 Minutes (Or Less)

UppercuT v1.2–NuGet Support

UppercuT + NuGet == Can Packaging Get Any Easier?

Upgrading?

New to Uppercut?

Getting Some Nuget-ty Goodness To Your Builds

Conclusion – UppercuT and NuGet – A Good Team!

How To – UppercuT and Gems

Rock The Gems

Removing All of the Other Output After the Gem is Built

What If I Want to Change What Goes Into my Gem?

Learn More

Related Posts

How To – Gems And .NET – Dependencies (References)

Adding a Reference as a Dependency

Gem Exists?

Gem Doesn’t Exist?

Related Posts

How To - Gems And .NET

Setup

GemSpec for .NET

Dependencies On Other Libraries

VERSION file

Libraries

Documentation

Executables

Build And Push

Related Posts

Deciding On Features For Open Source

Is It Objective?

Feature Selection IS the Set of Features For the Domain

Feature Selection IS An Answer To Competition

Feature Selection IS Subjective

Patches And/Or Donations

What If There Are Multiple Committers?

UppercuT – Custom Extensions Now With PowerShell and Ruby

Extension Points

Customize UppercuT Builds With PowerShell

Customize UppercuT Builds With Ruby

UppercuT and Albacore.NET

Warmup – Getting Started

Templating

Run Warmup

Conclusion