Yeah, I had that battle a couple of months ago when the disk holding the corporate CMS ran out of space. It had been allocated 100Gb. I offered to take a half day off so they could go buy a terrabyte hard-drive to beef it up.

Disclaimer: I used to be a systems engineer. Today, I being an outsider, telling systems engineers how to do their jobs. ;)

The problem may not just be the absolute disk space you're using... the problem may also be rooted in that many e-mail systems (notoriously the most popular one, Exchange) is woefully poor at handling very large mailboxes, so IT staff are hesitant to up the ante for users.

Similar and unrelated: not all IT managers are purchasing devices which make it easy to swap additional (or higher-capacity) drives in and out. A SAN is expensive, though it does pay significant dividends for data-intensive companies.

Personally, I don't know why companies don't outsource their mail services to a third party and let the third party deal with the idiosyncrasies of products like Exchange.

But as for the disk space issue -- part of IT infrastructure's job is to monitor the growth of disk usage and to stay ahead of the curve, not be the thought police telling people to stop saving old emails.

That being said, one thing can be done to greatly reduce the size of mailboxes: implement a system which stores attachments outside the mail system. It is attachments, not text emails, which tend to fill up mailboxes.

I'll get off my IT systems soapbox now. :)

I have a 75gb hard drive in my development machine, I know how you feel.

450 GB SAN Drive (HP EVA): $3.760,00+VAT

That's somewhat different to consumer drives($100/TB). Plus, your data will be backed up daily, yielding further costs.

I agree that 75MB is very small for a mailbox, but please don't argue that storage comes at nearly no cost.

Maybe if you took time to clean out your inbox, you wouldn't have this problem.

jk

Yea, how stupid.  I think the last time I had a quota on space was only 10 years ago in Lotus Notes. Since then nobody had an issue (either mom & pop, big fortune 500, or any mid-leve companies I've worked at).

Your company is definitely the exception.  I say that reflects pretty bad on the Admins, they probably have no incling wtf they are doing.

I am also plagued by crappy limits on corp email, however disk space is probably not the driving factor there.  It's risk & liability.  Old emails are not something companies like to keep around, as they can be incriminating far in the future.

shit, one of my emails I got from someone right now in my inbox is 1 mb.  Does that  mean that you only get allowed about 100 emails in YOUR inbox?  hahaah, what a joke.  It's definitely a rant to talk about with your situation.

the answer is simple.... MONEY

a quick count:

small it with approximately 1000 mail box

increase of 200mb mailboxes

200mb x 1000 mailboxes = 200 gb total

raid5 config =  200x5 = 1000gb

mail are on a sql server so we should count that sql space required for 200mb  has a factor of 1.3

1000gbx1.3= 1300

san costs for gb is about 20$

1300x20= 26000$

We get a whopping 50MB for our email account and my work machine has an 80GB HDD.

What's even more hilarious is the rate that our IT dept charges each of our projects for network share space.

But like a previous poster said, its not as simple as buying cheap HDDs, there is 24/7 support/backup in play as well.

>>>A SAN is expensive, though it does pay significant dividends for data-intensive companies.

I'm sorry but you have to be kidding.  I've worked at 2-3 very small shops, 2 mid-level and 2 large. Even the smallest of shops which don't produce a hell of a lot of income knew it was obvious that you need a SAN.

In fact my buddy is an IT Manager for a manufacturing plant.  Not huge, and he has around 2-3 subordinates in his shop.  Shit, they have a SAN.

WHO DOESN'T HAVE A SAN..that's a weak point you're making.  IF you don't have a SAN, that's just horrible management of your resources.  Everyone knows a SAN saves you 10 fold on hardware costs, etc.  Unreal.

>>>450 GB SAN Drive (HP EVA): $3.760,00+VAT

You have to be kidding.

That is the price of "A" server....so you're telling me this is a SAN and that this is a lot of money?  I feel bad for your business then if they can't even afford 3k?  A SAN should have at least 1 Tb + on it.  Shit I can go to Fry's and buy a 1.5 gig drive for $150 and build my own whitebox "server" for under 2k.  I think HP is definitely fing you over.

That's not a SAN.  So you're telling me your "business" rather buy a bunch of new servers every time you need more disk space, than be willing spend $5-10k or so on a decent SAN where they can all share disk space and really not have any issues with storage from that point on?

please....do you guys work in a garage?  Sorry but that's lame.

You can EASILY get a 3tb server for $10k.  EASILY.  So your numbers are a bit off my friend.

Nice rant, but obviously you know nothing about email storage.  Storage within the Exchange server is so much more expensive than your consumer storage options. Here is how to work within the restrictions, create a PST either locally on your machine or ask an admin to set one up on the cheap storage on the network. Then you just migrate your mail over to it as you get it, or when you hit your limits. One thing we have done is always allow our employees to receive mail even if they are over the threshold, they just can't send until they do some clean up.

>>>What's even more hilarious is the rate that our IT dept charges each of our projects for network share space

Oh geez!  that is sad.  Again I've been at a LOT of companies and never have I seen this either, not even at a mom & pop.

I will pray for you today during my lunch break ...haha

That in my opinion is jsut a poor network admin who cannot convince the business to get a decent SAN and process in the IT department.  This is 2009, holy shiza.

I'm really shocked that I'm hearing all this lameness and that people still have to put up with disk space like this when it comes to email and SANs.  We are talking BASICS here.  

You've been kicked (a good thing) - Trackback from DotNetKicks.com

Dave no offense but you know nothing about SANs, email quotas, and or system administration.    

Is 75mb kind of crazy?  Yes.  Are there legitamite reasons to have email quotas?  Yes a million of them.  Is disk space cheap?  Yes but...its not always as easy as just slapping in a new drive or buying a SAN.

If you're going to put a quota, put something realistic.  75mb is NOT realistic.  Not even close.

>>Dave no offense but you know nothing about SANs, email quotas, and or system administration

Mr. Phil.  In fact I DO.  I have implemented RAID arrays (enterprise boxes/severs) including popping in new enterprise disks for expanding arrays and the such.  I know that I can call the bluff that getting a SAN and having to be so anal about disk space is a bit piontless. Youre users are never going to "manage" their space the way "you hope for".  IT's a face.  Get a SAN and be done with it and quit bitching to me saying I know nothing when I've setup and installed HP servers, Disk arrays, and much more in my career besides just being a programmer.

Obviously you assume too much about me Mr. Phil.

>>>Yes but...its not always as easy as just slapping in a new drive or buying a SAN.

I have never seen 1 place I've worked in the past 10 years (when exception to 11 years ago with Lotus Notes) that had a disk quota in Exchange.  And they were some of the best network admins I've ever worked with.

by the way all, Phil and I are good friends...but he sometimes assumes us programmers don't know jack about hardware or networking. ;)

I ran into this at my last corporate gig, who had 30k+ employees.  Their issue was that no matter what they set their limit to, a very significant percentage (75% or so) were always at the limit.  It was cost prohibitive to set a limit where it would be difficult or impossible to hit the limit, 1GB or so.

The limit used to be much higher (~200MB), but they found that a higher limit was filled _faster_, as employees would use Email as their CMS.  Think 10MB Word docs, CCd to dozens of people, each using Reply All with a Tracked Changes doc.  Lowering the limit forced people to use email as a communication medium, not a CMS.

Their recommendation was just to AutoArchive everything older than 2 weeks.  Disk space is cheap, but each of the tens of thousands of desktops had gigs of free space.  That was cheaper.

Pingback from  Arjan`s World    » LINKBLOG for January 6, 2009

It is also about the cost during the discovery process when a lawsuit occurs. Imagine how much less expensive it is for a 10,000-employee business gets sued. If it was company A which keeps "unlimited" mailboxes and tells everyone to just "archive" old messages, then they may have an average mailbox size of about 1 GB. Company B keeps 75 MB quotas, and additionally trains their employees about proper retention practices (i.e., retaining what's required legally, also disposing what's required). Company A is exposed to more liability and has the potential expense of discovery on 10 TB of data. Company B is less exposed, liability-wise, and only has to worry about 750 GB of data.

come on Derik, you don't NEED to store all that data in email form.  If there's important attachments, file them. If there's important emails, save them.

The vast majority of corporate email space is just wasted crap because people won't archive or people attach the same 2mb document and send it to 500 users so there is a copy in every mail box.

I imagine you dislike when people make simplistic assumptions in your domain yes? "if it takes n time to do make 2 more people work on it so it will be faster by that factor".  The same is true of equating the intricacies of network storage and allocation in a large organization to the simple "i can buy a hard drive from best buy for $x"

Yeah, I can go either way - I got to admit I probably have about 50GB that I'm just too lazy to delete because I'm worried about 'just in case' reasons to keep it.

I use to have that problem with my code but I out grew that; maybe someday I'll do the same with me email - maybe when like the code, it becomes a mess to maintain because of the crap I think I might need someday that I don't actually need now.

Kind of a developer/systems guy/it manager here, and, as has been illustrated, it is alot more expensive and trickier than tossing a bunch of spindles at the problem and upping everyone's limits.

Jimmy Bogard's comment is absolutely true--email always expands to the limit in most cases. And your comment about "keeping stuff so it is easy to search for" is a case in point--most non computer types think about people and conversations (email) not files and folders (disk). I can also testify that we have a few users here who are always 1mb short of their special extended limits and are always asking for more.

One thing that has not been mentioned is that you can set the actual limit and the nastygram limit quite differently. I know we have many people with 100mb+ head room over the time when they are sent the note. And no, the nasty gram does not necessarily mentione what the actual limit.

Don't forget that Exchange specifically has storage limits for the entire mail store. Depending on which version and which edition, you can have very little space to spread around before you end up NOT having email. And, from the systems side of things, that is very, very, very bad.

Others have mentioned the backup angle, but have forgotten one key bit--you need to be able to run the backup overnight in most cases. Tape speed still ain't quite disk speed.

Finally, such bad things happen when you hit the limit, and there are times one needs the headroom, that I know we keep something on the order of 20-40% of space reserved for occasions when we need to extend limits.

So yeah, email space is alot like the mythical man month--the resources don't scale linearly.

I am a developer who also did 7 years in a combination of IT support and system administration.

Sorry Dave, without meaning to sound too blunt but your comments do illustrate a lack of understanding about how to run IT systems properly and suggesting installing disks and expanding a RAID array in any way makes you qualified to talk about proper network management reminds me of the people who come to IT support jobs thinking they're qualified just because they built their own PC once.

You mention building white box systems rather than going for something like HP because it's expensive, but this is not how things are done professionally. Going to a big vendor gives you more than just the box, it gives you reliable support and provides external accountability, being able to get a failed disk swapped out within 4 hours from the manufacturer is a big bonus over having to deal with 30 day turn arounds on warranty calls from off the shelf hardware. You can get 3rd party maintenance contracts of course, but what's the point when it only takes the cost upto or higher than the HP product in the first place?

Regarding mail box limits, I have to agree that they do serve a purpose even if you have as much disk space as you ever want. The more space you give users, the more they use and before you know it even 200mb isn't enough for them. As has been mentioned above there are users that will reach their limit whatever their limit.

As Exchange has to do more than just store data, it has to be searchable and such as well there's also the need for faster hardware and the need for more RAM and so on also. So the more space you give users, the more your overall hardware requirements are. Even the network can suffer if people start sending 50mb e-mails to each other more frequently because suddenly they have enough room to store mails that size.

You would not generally use consumer disks either, so to buy server grade disks is much more expensive but also you have to be able to back it up. Eventually you reach your disk limit too and need a new controller or new server, or you have to replace existing disks. Remember with RAID, you're limited somewhat by your smallest disk, if you have a striped RAID5 array, and a 5 320gb disks you can add more disks of this size until you hit capacity but if you stick a 1tb drive in you can usually still only stripe to 320gb of that disk anyway unless you replace every disk.

The best solution is to mail settings in control panel and create personal folders, you can even do this on a specific file server that's backed up if you wish. File stuff in your personal folders if you need to keep it, the only mail folder that need be kept on the exchange server is your Inbox then and if you have 75mb of e-mails in your inbox you either have big attachments that can be saved off or your inbox is simply too messy to be manageable and needs a tidy up anyway! Of course personal folders are equally a drag on your file server but at least they're not bogging the exchange server down and file servers are better optimised to handle large files than e-mail servers which tend to handle primarily lots of small e-mails.

Time and time again I find that people who have problems with storage quotas have them because they're not organising their data sensibly. Where I used to work when I was working in that role we had a 10mb quota and I never once hit it despite also having to hand via personal folders over 5 years worth of e-mails.

Try Gov't in Australia... 75MB for my mailbox on server, then I have a 40MB archive folder that is "forced" on a network share. I can't even create a local archive. (No Outlook, Novell GroupWise... *shudder*) I get warnings when my archive gets full, but no warning on the mailbox. It auto-deletes oldest messages or any message older than 3 mo.

Considering I'm active on 2-3 projects at a time and the pace of gov't projects, (Calendar time) keeping records of e-mail conversations is really flipping fun.

I too go through and delete a ton of emails.  However, there is no way anyone maintains their emails every day perfectly.  And 75mb is not going to let even a secretary store a month's worth of emails let alone a week!  

People who do not clean up their inbox not good.  But I know I do.  But even if I do, there is no way I could do anything with 75mb.  Especially if you are a manager!

@Ian

>>>You mention building white box systems rather than going for something like HP

Obviously I promote using actual Enterprise Servers.  It was just to illustrate a point.  And in fact the RAID array box was an Enterprise box.  I did NOT build that.  I threw it in a rack just like everyone else and at the time I was a DEVELOPER.  My own Network Admin at the time even said there is no use in the company trying to maintain control of the network share (where people were putting docs) because you can bitch all you want all day long and have a day called "clean up day" which they tried and it failed.  People are not perfect.  Disk space has to be expanded.  You will never get everyone to trim as much as you wish.

This could go on and on.  I only built one whitebox server and that was for a forune 500 company 10 years ago when I was fresh out of college.  After that, it's obviously been either HP or Dell servers.    The RAID box I bought and seeked out was 5k at the time built by a company who build enterprise RAID boxes.

Ian, Also the point about the whitebox server was not that I built them.  It's that the quote one of the replies here stated about a SAN and the amount of disk space that person was claiming was definitely not realistic.  That's the point I was making.  I  have not built whitebox servers.  Obviously I've been in environments after that first year after college (11 years ago) that have been racks as far as you can see of HPs, DELLs, SUN, and whatever else.  I'm not that naive.

Disk space might be cheap but you're not looking at the bigger picture. At a small company I consulted for we had to consider remote backup (which was expensive per GB), bandwidth that could backup all the data or the delta nightly and download, how long it would take to recover based on a location fire, a local backup on the otherside of the office, RAID5 disks compatible with the HP servers and warranty, thus space was expensive and restricted. We found that previous mismanagement of space lead to hundreds of 3mb photos on server drives etc..  

I agree outsourcing email is the way to go, though getting hardcore outlook users to move to Google Apps is painful.

Typically, a storage area network (SAN) is part of the overall network of computing resources for an enterprise. Storage Area Network (SAN) allows a network of systems to access the storage over a dedicated storage