Storage growth. How can you hope to control it?

From ZDNet.com

http://www.zdnet.com/article/getting-a-grip-on-storage-growth/

This a subject near and dear to my heart as I am working on this problem in my organization right now.  As the article states, you can always purchase more storage, but with additional storage you increase backup capacity, backup times, HVAC requirements and at some point you will run out of physical space to keep the storage devices.  While working for the DoD we would request storage and by the time we actually received the additional storage we had already allocated all of the new space and had to ask for more, so non-profit, government, many organizations have storage issues.

The article lists two major methods to resolve a storage problem.  The first is reducing the amount of data that needs to be stored.  de-duplication is the methodology they propose to reduce the amount of data.  From personal experience there are multiple de-duplication techniques/abilities out there.  At it's most basic level de-duplication works at the file level.  If the system see's more than one identical file it will create pointers from each duplicate to the original file.  A more advanced technique is block level de-duplication.  This looks at files from the block level so  it chunks the files and if it finds identical chunks, it will create pointers to those individual chunks.  This is a much more space reducing capability if you think about how every Word document will have quite a bit of similar overhead for each document and all of those would be de-duplicated.

The second method listed in this article is to improve the utilization of storage.  The article points out that most applications focus on the maximum capacity needed, but they tend to operate at a much lower requirement.  Thin-provisioning is listed as a solution for this problem.  Thin provisioning is typically seen in a virtualized environment.  You can allocate a maximum storage amount that the system can use, but space is only allocated as it's used.  For example you can thin provision a disk to grow to 100GB but if the application only used 2GB that is all that would be allocated on the physical disk.  Thin provisioning does have some major issues, Mainly you have to pay close attention to how much your disks could grow to, If multiple systems end up using all their allocated storage you can over-allocate and run out of space.

Again from personal experience I'd throw out another method which I am sure would work if I could ever get it implemented is the chargeback model.  The chargeback model charges internal organizations for their storage use.  Each department gets an allocation of space and anything over that amount incurs a charge.  I have always thought that if I could get this in place anywhere I worked that you would see a much more restrained use of storage.  Users tend to believe that storage is free so they have no problems keeping 2-3 copies of the same file, or holding onto a file from 8 years ago, just in case.

There are just a few examples of managing storage within the organization, they do not take into account cloud storage, tiered storage or they myriad of other ways to manage storage, but these are a few good ones to examine if you are looking to keep your data in-house.