Rob Reynolds - The Fervent Coder : Tools, SidePOP

Gems - Package Management For .NET

Rob Reynolds — Fri, 16 Jul 2010 04:27:00 GMT

The Ruby community has enjoyed a great user experience with a package management system they use called Gems. A gem is a package (or a library), compressed with some additional metadata, and can be either source files or binaries. Let’s focus on binary gems. We have the same concept in .NET (DLLs/EXEs). You may have references to other DLLs. When you want to update a reference you are using on a project, you may also need to update its dependencies as well. And so on and so forth. A package management project is meant to help make that easier. It’s actually really hard to explain what gems or package management without just showing you. So take a look here:

I type:

gem install sidepop

And that’s it. It looks and sees that I have a dependency on log4net. Notice how it nicely just pulls down log4net version 1.2.10 as well?

Background

I can count on one hand all of the package management projects that have been started for .NET. Dru Sellers, Chris Patterson, and I have talked about package management stuff from time to time. Dru and Chris have been a part of one project (Nu) that has been started several times to start to answer this question. We’ve participated on the mailing list for HornGet. At one point I casually asked why we couldn’t just use gems. Other people out there have probably stated the same thing. But no one has really carried the idea forward. Until now.

Yesterday Dru asked for a gem-ify feature for UppercuT. We started talking and looking at how easy it is to create a gem. Then we figured out how to make the executables piece work as well. This is where you can install a gem and then call the executable from the command line anywhere and get output. From the thread on ChuckNorris where we talked about this:

Here is something you might find pretty interesting: http://rubygems.org/gems/roundhouse

If you have ruby installed, you can install roundhouse now from the command line:

gem install roundhouse

This gives you the opportunity to type this anywhere:

rh <options>

Why Should I Care?

If you work with Open Source, you know how much of a pain it can be to update your references. You update one library, say NHibernate, and find out that you also need to update your references to Castle. And possibly, you might then need to update your references to log4net. It can be a painful process. This is the start of answering that question.

Why Should I Get Excited?

It sounds like Jeremy Miller among others are getting excited about this. And why not? We’ve been trying to answer the gems question since Ruby made it so easy.

The implications of this are awesome! I still haven’t fully grokked what we’ve just opened up. But it’s huge!

It doesn’t get us all of the way there to updating the references in our source code folder. That’s where projects, like Nu, are going to start showing up that leverage the idea of using the gems infrastructure to get the libraries from the ruby folders to your source code folder. You are going to see UppercuT come back soon with taking care of getting your gem built with the proper version.

This is the start of something very cool.

Before you comment about “cluttering” the ruby community, please be sure to read this (we’re with you on this): http://devlicio.us/blogs/rob_reynolds/archive/2010/07/19/gems-for-net-community-response.aspx

How To - Gems and .NET and How To – Gems & .NET - Dependencies (References)

Walkthrough – Create Gems Even Easier With a Conventional Build (UppercuT)!

The Future is Now!

Chuck Norris Welcomes You

Rob Reynolds — Sat, 17 Apr 2010 14:57:21 GMT

Chuck Norris doesn’t program with a keyboard. He stares the computer down until it does what he wants.

All things need a name. We’ve tossed around a bunch of names for the framework of tools we’ve been working on, but one we kept coming back to was Chuck Norris. Why did we choose Chuck Norris? Well Chuck Norris sort of chose us. Everything we talked about, the name kept drawing us closer to it. We couldn’t escape Chuck Norris, no matter how hard we tried. So we gave in.

Chuck Norris can divide by zero.

What is the Chuck Norris Framework?

@drusellers and I have been working on a variety of tools:

WarmuP - http://github.com/chucknorris/warmup (Template your entire project/solution and create projects ready to code - From Zero to a Solution with everything in seconds. Your templates, your choices.)
UppercuT - http://projectuppercut.org (Build with Conventions - Professional Builds in Moments, Not Days!) | Code also at http://github.com/chucknorris/uppercut
DropkicK - http://github.com/chucknorris/dropkick (Deploy Fluently)
RoundhousE - http://projectroundhouse.org (Professional Database Management with Versioning) | Code also at http://github.com/chucknorris/roundhouse
SidePOP - http://sidepop.googlecode.com (Does your application need to check email?)
HeadlocK - http://github.com/chucknorris/headlock (Hash a directory so you can later know if anything has changed)

Others – still in concept or vaporware

What’s in a Name?

People ask why we choose such violent names for each tool of our framework? At first it was about whipping your code into shape, but after awhile the naming became, “How can we relate this to Chuck Norris?” People also ask why we uppercase the last letter of each name. Well, that’s more about making you ask questions…but there are a few reasons for it.

Project managers never ask Chuck Norris for estimations…ever.

The class object inherits from Chuck Norris

Chuck Norris doesn’t need garbage collection because he doesn’t call .Dispose(), he calls .DropKick()

Call To Action

So what are you waiting for? Join the Google group today, download and play with the tools.

And lastly, welcome to Chuck Norris. Or should I say Chuck Norris welcomes you…

Lessons In Building An Email Parser

Rob Reynolds — Fri, 18 Dec 2009 09:11:00 GMT

The Classic Infinite Email Loop

When building an email parser, one must think about validating an email address that sends you a message. Why? Just because an email message has a from address does not already mean that it is a valid email address. When you're building an auto-responder, not validating an email can cause an email loop. Some of you can already see where I’m going with this.

I was building something in Bombali that would respond to emails when receiving them. When Bombali received a message from an address, Bombali would send a response. No checking on the address. In the world of computers, this is a classic mistake. In my head I hear the word "classic" as if it came from SNL and the "Classic Peg" skits. Classssssss-ic (with laughing and shoulder shrugging) Mistake. Right? Well this is all new to me, I’ve never been able to do this programmatically until recently. So Bombali received a message with a no reply from address. It sent a response. Then it got a response from the postmaster saying that this was an invalid address. I bet you’re guessing what happens next? Yes, Bombali responded to the postmaster. And then the postmaster responded back. Over a matter of about 30 minutes there was quite a bit of traffic back and forth until I deleted the emails before Bombali could check them. At least I was smart enough to set the email checking to every half minute.

How did I get past this? Validation. An authorized list form of validation. Interested? Read on dear reader.

SidePOP XmlConfigurator

I was exposing my configuration in the last post, so I built an XmlConfigurator that does this for you and gives you back a list of Email Watchers (renamed SidePOPRunner).

private void configure_mail_watcher()
{
    EmailWatcherConfigurator configurator = new SidePopXmlConfigurator();
    foreach (EmailWatcher emailWatcher in configurator.configure())
    {
        emailWatcher.MessagesReceived += runner_messages_received;
        emailWatcher.start();
    }
}

Subscriber for Receiving Email

The code above and the code below show how easy it is for your application to receive email. I map the email to a local object and then I pass that off to a function to parse and send a response.

private void runner_messages_received(object sender, MessageListEventArgs e)
{
    IEnumerable<SidePOPMailMessage> messages = e.Messages;

    foreach (SidePOPMailMessage message in messages)
    {
        Email mail_message = Map.from(message).to<Email>();
        parse_and_send_response(mail_message);
    }
}

So now I am receiving email, but I need to parse it do determine what to do. So let’s show the parser first before the method for sending a response.

Really Simple Email Parser

This is a really simple parser. It’s not by any means what some people would use in production and that’s fine. It works for what Bombali needs.

public MailQueryType parse(Email message, IList<IMonitor> monitors, IDictionary<string, ApprovalType> authorization_dictionary)
{
    MailQueryType query_type = MailQueryType.Authorizing;

    string user = message.from_address.to_lower();
    bool user_is_authorized = false;

    if (authorization_dictionary.ContainsKey(user))
    {
        ApprovalType user_is_approved = authorization_dictionary[user];
        if (user_is_approved == ApprovalType.Approved)
        {
            user_is_authorized = true;
        }
    }

    if (user_is_authorized)
    {
        query_type = MailQueryType.Help;
        string subject_and_body = message.subject + "|" + message.message_body;

        if (message_contains_status(subject_and_body)) query_type = MailQueryType.Status;
        if (message_contains_config(subject_and_body)) query_type = MailQueryType.Configuration;
        if (message_contains_down(subject_and_body)) query_type = MailQueryType.CurrentDownItems;
        if (message_contains_approve(subject_and_body)) query_type = MailQueryType.Authorized;
        if (message_contains_deny(subject_and_body)) query_type = MailQueryType.Denied;
        if (message_contains_version(subject_and_body)) query_type = MailQueryType.Version;
    }

    return query_type;
}

I have a MailQueryType so that I separate what how to respond from actually responding. What you can see is that I’m already checking to see if we have an approved user before I set a different MailQueryType. The mail parser just sends back how Bombali should respond to the message.

Sending a Response

In subscribing to receiving email, I passed the message to the method below. The second line of the method shows me passing the message off to the mail parser to find out how to respond. The switch could probably be replaced by a Strategy pattern, but for now it works and it’s all I need.

private void parse_and_send_response(Email mail_message)
{
    string respond_to = mail_message.from_address.to_lower();
    MailQueryType query_type = mail_processor.parse(mail_message, monitors, authorization_dictionary);
    Log.bound_to(this).Info("{0} received a message from {1} of type {2}.", ApplicationParameters.name, respond_to, query_type.ToString());

    string response_text = string.Empty;

    if (query_type == MailQueryType.Authorized || query_type == MailQueryType.Denied)
    {
        string[] body_words = mail_message.message_body.Split(' ');
        foreach (string body_word in body_words)
        {
            if (body_word.Contains("@"))
            {
                respond_to = body_word.Replace(Environment.NewLine, "");
                break;
            }
        }
    }

    switch (query_type)
    {
        case MailQueryType.Denied:
            authorization_dictionary.Add(respond_to, ApprovalType.Denied);
            return;
            break;
        case MailQueryType.Authorized:
            authorization_dictionary.Add(respond_to, ApprovalType.Approved);
            response_text = string.Format("Congratulations - you have been approved!{0}Send 'help' for options", Environment.NewLine);
            break;
        case MailQueryType.Help:
            response_text =
                string.Format(
                    "Options - send one:{0} help - this menu{0} status - up time{0} config - all monitors{0} down - current monitors in error{0} version - current version",
                    Environment.NewLine);
            break;
        case MailQueryType.Status:
            TimeSpan up_time_current = up_time.Elapsed;
            response_text = string.Format("{0} has been up and running for {1} days {2} hours {3} minutes and {4} seconds.", ApplicationParameters.name,
                                          up_time_current.Days, up_time_current.Hours, up_time_current.Minutes, up_time_current.Seconds);
            break;
        case MailQueryType.CurrentDownItems:
            response_text = string.Format("Services currently down:{0}", Environment.NewLine);
            foreach (IMonitor monitor in monitors)
            {
                if (monitor.who_to_notify_as_comma_separated_list.to_lower().Contains(respond_to))
                {
                    if (!monitor.status_is_good) response_text += string.Format("{0}{1}", monitor.name, Environment.NewLine);
                }
            }
            break;
        case MailQueryType.Authorizing:
            response_text =
                string.Format("Bombali notified admin to add you to approved list. If you are added, you will receive a response.");
            break;
        case MailQueryType.Version:
            response_text = string.Format("Bombali is currently running version {0}.", Version.get_version());
            break;
        default:
            response_text = string.Format("{0} has not been implemented yet. Please watch for updates.", query_type);
            break;
    }

    SendNotification
        .from(BombaliConfiguration.settings.email_from)
        .to(respond_to)
        .with_subject("Bombali Response")
        .with_message(response_text)
        .and_use_notification_host(BombaliConfiguration.settings.smtp_host);

    if (query_type == MailQueryType.Authorizing)
    {
        SendNotification
        .from(BombaliConfiguration.settings.email_from)
        .to(BombaliConfiguration.settings.administrator_email)
        .with_subject("Bombali Request")
        .with_message(string.Format("{0} reqests approval. Send approve/deny w/email address. Ex. 'deny bob@nowhere.com'", respond_to))
        .and_use_notification_host(BombaliConfiguration.settings.smtp_host);
    }

}

You’ll notice that I’m authorizing any new email addresses that send an email to Bombali before allowing them access to the application. I do that by having an administrator contact I can send a request for approve or deny to before allowing access. I also send an email back to the sender letting them know they are in the process of being approved. At most I would send one email to a bad address. As administrator, the email can be validated by a human. And I only respond to an authorized list. I could also build in something to validate the email by checking for a response from a post master telling me the address doesn’t exist.

So remember readers - always check your sources. Otherwise you might get in an infinite loop. And that would be bad.

How To Check Email Programmatically - SidePOP

Rob Reynolds — Tue, 08 Dec 2009 06:46:00 GMT

Background

Sending email has long been easy to do with the .NET Framework. There really hasn’t been a facility for checking email though. I recently had a need to be able to check email with and send a response with Bombali (a monitoring tool). I went searching for examples or a solution I could use. I found a few articles on this from a few years ago including the .NET POP3 MIME Client. I never found anything that was free and I never found anything that was easy to configure. Those who know me know I have a tinge of the NIH (not invented here), especially when there are no free alternatives. So I created my own based on the articles I had read. And in the spirit of chucknorris, I named it SidePOP!

Configuration

Configuration for receiving email should be easy. Feast your eyes on all that is required for configuration (and for those of you who dislike XML, it’s not required – you can configure all in code):

<configSections>
  <section name="sidepop" type="sidepop.configuration.SidePOPConfiguration, sidepop"/>
</configSections>

<!-- 110 is normal POP3 SSL uses port 995-->
<sidepop>
  <accounts>
    <add hostName="mail.somewhere" userName="yep" password="" minutesBetweenChecks=".1" />
    <add name="Number1" description="Main account" enabled="true" hostName="mail.somewhere.net" hostPort="110" useSSL="false" userName="" password="" minutesBetweenChecks="1" timeoutInMinutes="1"  />
  </accounts>
</sidepop>

You see there are two items here. The first one has the required items in it. The second one is the entire set of items to configure. Notice that we are asking for a password in plaintext. This is where you might think about encrypting this section somehow.

Code Setup

Configuring SidePOP to run is a little more verbose that I might like right now. Looking below you see that it’s still not that much code. I hope to set up a facility to automatically look at the configuration file and configure SidePOP (for those that like XML). If someone wants to send me a patch as well, that would be awesome.

private void configure_mail_watcher()
{
    foreach (AccountConfigurationElement account in SidePOPConfiguration.settings.accounts)
    {
        if (account.enabled)
        {
            SidePopRunner runner = new SidePopRunner(new DefaultPop3Client(account.hostName, account.hostPort,
                                     account.useSSL, account.userName,
                                     account.password), account.minutes_between_checks);
            runner.MessagesReceived += runner_messages_received;
            runner.run();
            Log.bound_to(this).Info("{0} is configured to watch for messages with user {1} at {2} every {3} minutes.", ApplicationParameters.name,
                                    account.userName, account.hostName, account.minutes_between_checks);
        }
    }
}

private void runner_messages_received(object sender, MessageListEventArgs e)
{
    IEnumerable<SidePOPMailMessage> messages = e.Messages;

    const string subject = "Bombali Response";
    TimeSpan up_time_current = up_time.Elapsed;
    string text_message = string.Format("{0} has been up and running for {1} days {2} hours {3} minutes and {4} seconds.", ApplicationParameters.name, up_time_current.Days, up_time_current.Hours, up_time_current.Minutes, up_time_current.Seconds);

    foreach (SidePOPMailMessage message in messages)
    {
        Log.bound_to(this).Info("{0} received a message from {1}. Responding that service is still running.",ApplicationParameters.name,message.From.Address);
    
        SendNotification.from(BombaliConfiguration.settings.email_from).to(message.From.Address).with_subject(
        subject).with_message(text_message).and_use_notification_host(BombaliConfiguration.settings.smtp_host);
    }
}

When I start the service, I call the method configure_mail_watcher(). Because I can have multiple accounts I can check with SidePOP, I am going to loop through the settings and check each one to ensure they are enabled. If so, then I am going to set up a SidePopRunner and subscribe to runner_messages_received (this may not work with multiple accounts, I haven’t yet tested that). Then I kick off the runner for that account.

I could instead subscribe to runner.MessageReceived (instead of runner.MessagesReceived) for getting an event notification for every message instead of batching it up to a list if I desired.

When a message is received, I find out how long the service has been up. I didn’t show it here, but up_time is a System.Diagnostics.Stopwatch. In the constructor I start it. Now I just capture how long it has been up. Then for each message, I send a response to the sender of the email address telling them how long the service has been up. To add more sophistication, I could very easily add a rules engine here to process the message based on a list of rules looking for keywords in the email.

That’s all I need to do. Now I’m up and ready to rock.

What It Looks Like

Here is what it looks like running with Bombali. I send the service a text message. This is what it logs.

2009-12-07 23:59:05,875 5 [INFO ] - Bombali received a message from someone @ somewhere.net. Responding that service is still running.
2009-12-07 23:59:05,875 5 [INFO ] - Sending email to someone @ somewhere.net with subject "Bombali Response" and message:
Bombali has been up and running for 0 days 0 hours 31 minutes and 22 seconds.

This is what I get from my phone.

Now I know that my service is still up and running with a simple text. All for the low price of a few lines of code and the sidePOP.dll library.

Conclusion

SidePOP gives me a level of sophistication that I didn’t have before. Now that I have access to get email, I could very easily add rules to do different things based on the content of the email received. How often have you wanted your application to check email and just didn’t want to pay the price for the products out there that can do this? Now it’s completely free with SidePOP.

Download SidePOP

NOTE: This is still early. Any part of the configuration could change. Be sure you take a look at the sample app to always have the newest configuration. Register any bugs you find here: http://code.google.com/p/sidepop/issues/list

WARNING: SidePOP will delete your email when it checks it. That’s how it can be sure it’s only dealing with new messages every time. Do not test on an account you care about. You’ve been warned.

SVN here: http://sidepop.googlecode.com/svn/trunk/

Downloads here: http://code.google.com/p/sidepop/downloads/list

Rob Reynolds - The Fervent Coder : Tools, SidePOP

Gems - Package Management For .NET

Background

Why Should I Care?

Why Should I Get Excited?

Related Posts

Chuck Norris Welcomes You

What is the Chuck Norris Framework?

What’s in a Name?

Call To Action

Lessons In Building An Email Parser

The Classic Infinite Email Loop

SidePOP XmlConfigurator

Subscriber for Receiving Email

Really Simple Email Parser

Sending a Response

How To Check Email Programmatically - SidePOP

Background

Configuration

Code Setup

What It Looks Like

Conclusion

Download SidePOP