Rob Reynolds - The Fervent Coder : chocolatey

Chocolatey Automatic Packages

Rob Reynolds — Thu, 03 Jan 2013 07:15:47 GMT

I updated three packages this morning. I didn’t even notice until the tweets came in from @chocolateynuget.

How is this possible? It’s simple. I love automation. I built chocolatey to take advantage of automation. So it would make sense that we could automate checking for package updates and publishing those updated packages. These are known as automatic packages. Automatic packages are what set Chocolatey apart from other package managers and I daresay could make chocolatey one of the most up-to-date package manager on Windows.

Automatic Packages You Say?

You’ve followed the instructions for creating a Github (or really any source control) repository with your packages. All you need to do now is to introduce two new utilities to your personal library, Ketarin and Chocolatey Package Updater (chocopkgup for short).

Ketarin

Ketarin is a small application which automatically updates setup packages. As opposed to other tools, Ketarin is not meant to keep your system up-to-date, but rather maintain a compilation of all important setup packages which can be burned to disc or put on a USB stick.

There are some good articles out there that talk about how to create jobs with Ketarin so I am not going to go into that.

Ketarin does a fantastic job of checking sites for updates and has hooks to give it custom command before and after it has downloaded the latest version of an app/tool.

Chocolatey Package Updater

Chocolatey Package Updater aka chocopkgup takes the information given out from Ketarin about a tool/app update and translates it into a chocolatey package that it builds and pushes to chocolatey.org. It does this so you don't even have to think about updating a package or keeping it up to date. It just happens. Automatically, in the background, and even faster than you could make it happen. It's almost as if you were the application/tool author.

How To

Prerequisites And Setup:

Optional (strongly recommended) - Ensure you are using a source control repository and file system for keeping packages. A good example is here.

Optional (strongly recommended) - Make sure you have installed the chocolatey package templates. If you’ve installed the chocolatey templates (ReadMe has instructions), then all you need to do is take a look at the chocolateyauto and chocolateyauto3. You will note this looks almost exactly like the regular chocolatey template, except this has some specially named token values.

#Items that could be replaced based on what you call chocopkgup.exe with
#{{PackageName}} - Package Name (should be same as nuspec file and folder) |/p
#{{PackageVersion}} - The updated version | /v
#{{DownloadUrl}} - The url for the native file | /u
#{{PackageFilePath}} - Downloaded file if including it in package | /pp
#{{PackageGuid}} - This will be used later | /pg
#{{DownloadUrlx64}} - The 64bit url for the native file | /u64

These are the tokens that chocopkgup will replace when it generates an instance of a package.
Install chocopkgup (which will install ketarin and nuget.commandline). cinst chocolateypackageupdater.
Check the config in C:\tools\ChocolateyPackageUpdater\chocopkgup.exe.config (or chocolatey_bin_root/ChocolateyPackageUpdater). The PackagesFolder key should point to where your repository is located.
Create a scheduled task (in windows). This is the command (edit the path to cmd.exe accordingly): C:\Windows\System32\cmd.exe /c c:\tools\chocolateypackageupdater\ketarinupdate.cmd
Choose a schedule for the task. I run mine once a day but you can set it to run more often. Choose a time when the computer is not that busy.
Save this Ketarin template somewhere: https://github.com/ferventcoder/chocolateyautomaticpackages/blob/master/_template/KetarinChocolateyTemplate.xml
Open Ketarin. Choose File –> Settings.
On the General Tab we are going to add the Version Column for all jobs. Click Add…, then put Version in Column name and {version} in Column value.
Click [OK]. This should add it to the list of Custom Columns.
Click on the Commands Tab and set Edit command for event to “Before updating an application”.

Add the following text:

chocopkgup /p {appname} /v {version} /u {preupdate-url} /u64 {url64} /pp {file} 
REM /disablepush

Check the bottom of this section to be sure it set to Command.
Click Okay.
Note the commented out /disablepush. This is so you can create a few packages and test that everything is working well before actually pushing those packages up to chocolatey. You may want to add that switch to the main command above it.

This gets Ketarin all set up with a global command for all packages we create. If you want to use Ketarin outside of chocolatey, all you need to do is remove the global setting for Before updating an application and instead apply it to every job that pertains to chocolatey update.

Create an Automatic Package:

Preferably you are taking an existing package that you have tested and converting it to an automatic package.

Open Ketarin. Choose File –> Import…
Choose the template you just saved earlier (KetarinChocolateyTemplate.xml).
Answer the questions. This will create a new job for Ketarin to check.
One important thing to keep in mind is that the name of the Application name needs to match the name of the package folder exactly.
Right click on that new job and select Edit. Take a look at the following:
Set the URL appropriately. I would shy away from FileHippo for now, the URL has been known to change and if you upload that as the download url in a chocolatey packages, it won’t work very well.
Click on Variables on the right of URL.
On the left side you should see a variable for version and one for url64. Click on version.
Choose the appropriate method for you. Here I’ve chosen Content from URL (start/end).
Enter the URL for versioning information.
In the contents itself, highlight enough good information before a version to be able to select it uniquely during updates (but not so much it doesn’t work every time as the page changes). Click on Use selection as start.
Now observe that it didn’t jump back too far.
Do the same with the ending part, keeping in mind that this side doesn’t need to be too much because it is found AFTER the start. Once selected click on Use selection as end.
It should look somewhat similar to have is presented in the picture above.
If you have a 64bit Url you want to get, do the same for the url64 variable.
When all of this is good, click OK.
Click OK again.

Testing Ketarin/ChocoPkgUp:

We need to get a good idea of whether this will work or not.
We’ve set /disablepush in Ketarin global so that it only goes as far as creating packages.
Navigate to C:\ProgramData\chocolateypackageupdater.
Open Ketarin, find your job, and right click Update. If everything is set good, in moments you will have a chocolatey package in the chocopkgup folder.
Inspect the resulting chocolatey package(s) for any issues.
You should also test the scheduled task works appropriately.

Troubleshooting/Notes

Ketarin comes with a logging facility so you can see what it is doing. It’s under View –> Show Log.
In the top level folder for chocopkgup (in program data), we log what we receive from Ketarin as well and the process of putting together a package.
The name of the application in ketarin matches exactly that of the folder that is in the automatic packages folder.
Every once in awhile you want to look in Ketarin to see what jobs might be failing. Then figure out why.
Every once in awhile you will want to inspect the chocopkgupfolder to see if there are any packages that did not make it up for some reason or another and then upload them.

Conclusion

Automatic chocolatey packages are a great way to grow the number of packages you maintain without any significant jump in maintenance cost by you. I’ve been working with and using automatic packages for over six months. Is it perfect? No, it has issues from time to time (getting a good version read or actually publishing the packages in some rare cases). But it works pretty well. Over the coming months more features will be added to chocopkgup, such as been able to run its own PowerShell script (for downloading components to include in the package, etc) that would not end up in the final chocolatey package.

With full automation instead of having packages that are out of date or no longer valid, you run the small chance that something changed in the install script or something no longer works. The chances of this are much, much lower than having packages that are out of date or no longer valid.

It takes just a few minutes longer when creating packages to convert them to automatic packages but well worth it when you see that you are keeping applications and tools up to date on chocolatey without any additional effort on your part. Automatic packages are awesome!

Chocolatey featured on LifeHacker!

Rob Reynolds — Wed, 12 Sep 2012 14:44:21 GMT

Chocolatey was just featured on LifeHacker! http://lifehacker.com/5942417/chocolatey-brings-lightning-quick-linux+style-package-management-to-windows

I was ecstatic to hear about this, of course now I need to write an actual comparison between chocolatey and other windows package managers.

Comments on Reddit: http://www.reddit.com/r/commandline/comments/zqnj6/chocolatey_brings_lightning_quick_linuxstyle/

Chocolatey - Guidance on Packaging Apps with Both an Install and Executable/Zip Option

Rob Reynolds — Sat, 25 Feb 2012 17:35:00 GMT

One of the thoughts I've been considering recently with chocolatey is consistency with packages and naming conventions as chocolatey continues to grow. It's fine to name packages by the app/tool name, that's both intuitive and expected. What I am more interested in is when an application has multiple installation options (ie. an MSI and a ZIP). It can become confusing for people to install these when they don't know what they are getting if they call a package that has both. If you start with one that has a .zip and later they release an MSI (nodejs anyone?), what do you call the package that installs the MSI? Do you keep around the executable? Do you rename the original package in response to the other option? Is there a third option?

One Option

If there is only one option available, you are fine to make the package name the same as the application/tool. This makes it intuitive and reduces confusion.

Multiple Options

To start putting together guidance on this and alleviate confusion, I see that we would move forward in these cases with three packages. One with no suffix, one with ".app" suffix, and one with ".tool" suffix. This brings in chocolatey's idea of applications versus tools ( https://github.com/chocolatey/chocolatey/wiki/ChocolateyFAQs - apps are installed on the system, tools are not).

If you would take a quick look at Instant EyeDropper ( http://chocolatey.org/packages?q=instanteyedropper ), you will notice there are three packages here.

InstantEyeDropper is what will ultimately be a virtual package
InstantEyeDropper.app is the package name for a package that uses a native installer (i.e. MSI, exe)
InstantEyeDropper.tool is the package name for a package that has an executable / downloads & unpacks an archive / etc

InstantEyeDropper right now is taking a dependency on InstantEyeDropper.app (which makes it a meta package). When virtual packages (see Virtual Packages below) are ready, that dependency will be removed and the chocolateyinstall.ps1 file will look something like the following (this is not definitive of what it will look like though):

Install-VirtualPackage '7zip.tool' '7zip.app'

You will notice I put the ".tool" ahead of ".app". In the end, I think the behavior of a virtual package should default to a command line version. Why? There are folks that do not have administrative access to their machines. Chocolatey is really nice for them because they can install and use chocolatey without ever needing to assert administrative privileges. Marcel Hoyer (https://twitter.com/pixelplastic) first proposed the idea of being able to use chocolatey without administrative privileges. Him and I took pains to make chocolatey work for these scenarios. This did complicate chocolatey a little bit for the package maker, but in the end I think it is a really good thing. As a person inspecting a package to decide whether to install or not, they can see every point that the package maker mentioned they needed administrative privileges.

That said, the default will be the one on the leftmost side. You are beholden to the community in justifying why you didn't put the command line version first if you decide not to in the virtual package. But chocolatey won't constrain you on that because you may have a really good reason.

App Now has Multiple Options

When an application/tool moves to where it has multiple options, like an installer it didn't used to have, that's when it is time to break the package out to a virtual (meta for now until virtual is available) and create the other two packages with the correct suffixes as outlined in the guidance above.

Virtual Packages

For those confused about the idea of a virtual package, it allows folks to say I need to take a dependency on a PDFReader. PDFReader becomes a virtual package that does nothing other than point to all of the different pdf readers available. When someone installs the package that has a dependency on PDFReader, chocolatey looks at the virtual options and sees you have adobereader installed (one of the options in the list). So it moves on because you have met the virtual package requirements. If you have foxitreader installed, it moves on. Otherwise it picks the first item in the virtual tree and installs it as the default. More information? https://github.com/chocolatey/chocolatey/issues/7

Virtual Packages vs Meta Packages

A meta package is one that points to other packages. If you think of a package that does nothing more than take on dependencies to other packages, that is a meta package. A virtual package is like a meta package, except it has the concept of optional dependencies.

Ending Thoughts

This seems to be on the surface the best way to provide an intuitive user experience. There may be some things we learn along the way and adjust this as we go. If you are a package owner and you have packages that have both options, you may want to start getting them into this format. I myself have some work to do in this aspect.

Thoughts?

Original Discussion: http://groups.google.com/group/chocolatey/browse_thread/thread/a1d208b022dfc30f