Images in this post missing? We recently lost them in a site migration. We're working to restore these as you read this. Should you need an image in an emergency, please contact us at imagehelp@codebetter.com
.Net routine file IO best practice

image File IO has shaped the computer industry more than any other technology.  Doubt me?  Consider that in 1978 AT&T decided to no longer share the source code of Unix with universities, upsetting professors like Andrew Tanenbaum.  Andrew taught cources on operating systems and felt it was impossible to teach the subject without the source code.  Why?  Some concepts are harder to grasp in theory but easy in code, like threading.  Other concepts are easy in theory, but hold a world of problems in implementation, like file IO.  To solve this, Andrew created MINUX, a "minimal Unix" system for teaching.  Later, a young lad in Finland named Linus started adding features to MINUX and jokingly referred to as "Linus' Minux", a name that stuck as Linux.

So what's so hard about file IO that requires a working example to learn?  After all, you open a file, read or write some data, then close it - simple right?  Consider what goes on when you call File.Open - the OS uses an index to figure out where the bits to the requested file are stored - and they may not be all stored in the same place.  A magnetic disk begins to spin while a mechanical arm is positioned at the spot the OS designates.  Electric current is generated from the variations in the magnetic field, passed over a controller to the CPU to make sense of, before passing it along to your application. 

That's just the hardware, there may be more software involved in that operation.  Virus scanners may kick in to make sure the file is safe, or network users may be half way though copying the file.  Can your application handle the user snooping around with Notepad while you save some data?  Nothing can tick off a user more than a corrupted save file, loosing hours of work in your application.

"Okay Mike, I get it - shut up and show me the code."

The methods I present here are designed to save and read objects as XML.  I came up with these after going though numerous methods in the .Net Framework (it seems every base class has at least one file IO method these days), figuring out what worked best.

using System;
using System.Text;
using System.IO;
using System.Xml;
using System.Xml.Serialization;
using System.Threading;

public class IOTools {

public static void SerializeObject(String FileName, Type ObjectType, Object Data) {
Double timeout = 5000;

StringBuilder xmlData = new StringBuilder();
XmlSerializerNamespaces xsn = new XmlSerializerNamespaces();
xsn.Add(String.Empty, String.Empty);

StringWriterUTF8 stringWriter = new StringWriterUTF8(xmlData);
XmlTextWriter xmlWriter = new XmlTextWriter(stringWriter);
XmlSerializer xmlSerial = new XmlSerializer(ObjectType);
xmlSerial.Serialize(xmlWriter, Data, xsn);
xmlWriter.Close();

DateTime start = DateTime.Now;
Exception lastEx = null;
Boolean success = false;

while (DateTime.Now < start.AddMilliseconds(timeout)) {
try {
using (FileStream fs = File.Open(FileName, FileMode.Create, FileAccess.Write, FileShare.None)) {
StreamWriter writer = new StreamWriter(fs);
writer.Write(xmlData.ToString());
writer.Close();
success = true;
break;
}
}
catch (Exception ex) {
lastEx = ex;
Thread.Sleep(10);
}
}

if (!success) throw lastEx;
}

public static object DeserializeObject(String FileName, Type ObjectType) {
Double timeout = 5000;
String data = String.Empty;

DateTime start = DateTime.Now;
Exception lastEx = null;
Boolean success = false;

while (DateTime.Now < start.AddMilliseconds(timeout)) {
try {
using (FileStream fs = File.Open(FileName, FileMode.Open, FileAccess.Read, FileShare.Read)) {
StreamReader reader = new StreamReader(fs);
data = reader.ReadToEnd();
fs.Close();
success = true;
break;
}
}
catch (Exception ex) {
lastEx = ex;
Thread.Sleep(10);
}
}

if (!success) throw lastEx;

StringReader stringReader = new StringReader(data);
XmlTextReader xmlReader = new XmlTextReader(stringReader);
XmlSerializer xmlSerial = new XmlSerializer(ObjectType);
return xmlSerial.Deserialize(xmlReader);
}

public class StringWriterUTF8 : StringWriter {
public override Encoding Encoding {
get { return Encoding.UTF8; }
}
public StringWriterUTF8() : base() { }
public StringWriterUTF8(StringBuilder sb) : base(sb) { }
}
}

To minimize the chance of accessing the files while they are in use, the data is buffered in a string (or StringBuilder).  The basic logic boils down to this: try to open the file for 5 seconds, and if you can't throw an exception.  In testing I found a Thread.Sleep of just 10 milliseconds was enough that the process didn't show up in CPU usage (without Sleep, a locked file could cause 65% load for 5 seconds - ouch).

There is some extra XML work going on to save the files without namespaces and in UTF-8 (by default, all strings are UTF-16).  Since my XML files aren't limited to being used by .Net apps this helps compatibility and the framework has a little known feature called "XML Namespace Hell" that's best to avoid.  If you were certain your data would only be read by you or other .Net apps, I recommend replacing the XML with the binary serialization methods of System.Runtime.Serialization for better performance.

Using the methods looks like this:

// Saving as XML
MyCustomClass myClass = new MyCustomClass();
try { SerializeObject(@"C:\data\myclass.xml", typeof(MyCustomClass), myClass); }
catch { /* handle a failure */ }

// Loading from XML
MyCustomClass myClass = null;
try { myClass = (MyCustomClass)DeserializeObject(@"C:\data\myclass.xml", typeof(MyCustomClass)); }
catch { /* handle a failure */ }

These methods work for local drives and network shares alike, File.Open is pretty liberal with the possibilities of a full path.  This is by no means asserted here as the final, full-proof way to handle routine file IO; if you have some field notes learned in battle please share them in the comment!


Posted 04-10-2008 9:46 AM by Michael C. Neel
Filed under:

[Advertisement]

Comments

UNIX Coding School » Blog Archive » unix code [2008-04-10 20:06:13] wrote UNIX Coding School &raquo; Blog Archive &raquo; unix code [2008-04-10 20:06:13]
on 04-10-2008 4:11 PM

Pingback from  UNIX Coding School  &raquo; Blog Archive   &raquo; unix code [2008-04-10 20:06:13]

Paul Kinlan’s Development Blog » Development Link Goodness 11th April 2008 wrote Paul Kinlan&#8217;s Development Blog &raquo; Development Link Goodness 11th April 2008
on 04-11-2008 4:11 AM

Pingback from  Paul Kinlan&#8217;s Development Blog &raquo; Development Link Goodness 11th April 2008

Mind Gravy » Blog Archive » links for 2008-04-11 wrote Mind Gravy &raquo; Blog Archive &raquo; links for 2008-04-11
on 04-11-2008 8:33 AM

Pingback from  Mind Gravy  &raquo; Blog Archive   &raquo; links for 2008-04-11

About The CodeBetter.Com Blog Network
CodeBetter.Com FAQ

Our Mission

Advertisers should contact Brendan

Subscribe
Google Reader or Homepage

del.icio.us CodeBetter.com Latest Items
Add to My Yahoo!
Subscribe with Bloglines
Subscribe in NewsGator Online
Subscribe with myFeedster
Add to My AOL
Furl CodeBetter.com Latest Items
Subscribe in Rojo

Member Projects
DimeCasts.Net - Derik Whittaker

Friends of Devlicio.us
Red-Gate Tools For SQL and .NET

NDepend

SlickEdit
 
SmartInspect .NET Logging
NGEDIT: ViEmu and Codekana
LiteAccounting.Com
DevExpress
Fixx
NHibernate Profiler
Unfuddle
Balsamiq Mockups
Scrumy
JetBrains - ReSharper
Umbraco
NServiceBus
RavenDb
Web Sequence Diagrams
Ducksboard<-- NEW Friend!

 



Site Copyright © 2007 CodeBetter.Com
Content Copyright Individual Bloggers

 

Community Server (Commercial Edition)