VMWare and how it effects Storage

"VMWare Changes Everything"

That's a lovely marketing phrase, but when it comes to storage, it does, and it doesn't. What you really need to understand is how VMWare can effect your storage environment as well as the effects that storage has on your VMWare environment. Once you do, you'll realize that it's really just a slightly different take on what storage administrators have always battled. First some background.

Some Server Virtualization Facts

1. The trend of server virtualization is well under way and it's moving rapidly from test/dev environments into production environments. Some people are implementing in a very aggressive way. For example, I know one company who's basic philosophy is "it goes in a VM unless it absolutely can be proven it won't work, and even then we will try it there first."
2. While a lot of people think that server consolidation is the primary motivating factor in the WMVware trend, I have found that many companies are also driven by Disaster Recovery since replicating VMs is so much easier then building duplicate servers at a DR site.
3. 85% of all virtual environments are connected to a SAN, that's down from nearly 100% a short time ago. Why? Because NFS is making a lot of headway, and that makes a lot of sense since it's easier to address some of the VMWare storage challenges with NFS than it is with traditional fiber channel LUNs.
4. VMWare changes the way that servers talk to the storage. For example, they force the use of more advanced file systems like VMFS. VMFS is basically a clustered file system and that's needed in order to perform some of the more attractive/advanced things you want to do with VMWare like VMotion.

Storage Challenges in a VMWare Environment

1. Application performance is dependant on storage performance. This isn't news for most storage administrators. However, what's different is that since VMWare can combine a number of different workloads all talking through the same HBA(s), the result is that the workload as seen by the storage array turns into a highly random, usually small block I/O workload. These kinds of workloads are typically very sensitive to latency much more than they require a great deal of bandwidth. Therefore the storage design in a VMWare environment needs to be able to provide for this type of workload across multiple servers. Again, something that storage administrators have done in the past for Exchange servers, for example, but on a much larger scale.
2. End to end visibility from VM to physical disk is very difficult to obtain for storage admins with current SRM software tools. These tools were typically designed with the assumption that there was a one-to-one correspondence between a server and the application that ran on that server. Obviously this isn't the case with VMWare, so reporting for things like chargeback becomes a challenge. This also effects troubleshooting and change management as well since the clear lines of demarcation between server administration and storage administration are now blurred by things like VMFS, VMotion, etc.
3. Storage utilization can be significantly decreased. This is due to a couple of factors, the first of which is that VMWare requires more storage overhead to hold all of the memory, etc. so that it can perform things like VMotion. The second reason that VMWare uses more storage is that VMWare admins tend to want very large LUNs assigned to them to hold their VMFS file systems and to have a pool of storage that they can use to rapidly deploy a new VM. This means that there is a large pool of unused storage sitting around on the VMWare servers waiting to be allocated to a new VM. Finally, there is a ton of redundancy in the VMs. Think about how many copies of Windows are sitting around in all those VMs. This isn't new, but VMware sure shows it to be an issue.

Some Solutions to these Challenges

As I see it there are three technical solutions to the challenges posed above.

1. Advanced storage virtualization - Things like thin provisioning to help with the issue of empty storage pools on the VMWare servers. Block storage virtualization to provide the flexibility to move VMWare's underlying storage around to address issues of performance, storage array end of lease, etc. Data de-dupulication to reduce the redundancy inherent in the environment.
2. Cross domain management tools - Tools that have the ability to view storage all the way from the VM to the physical disk and to correlate issues between the VM, server, network, SAN, and storage array are beginning to come onto the market and will be a necessary part of any successful large VMWare rollout.
3. Virtual HBAs - These are beginning to make their way onto the market and will help existing tools to work in a VMWare environment.

Conclusion

Organizations need to come to the realization that with added complexity comes added management challenges and that cross domain teams that encompass VMWare Admins, Network Admins, and SAN/Storage Admins will be necessary in order for any large VMWare rollout to be successful. However, the promise of server virtualization to reduce hardware costs and make Disaster Recovery easier is just too attractive to ignore for many companies and the move to server virtualization over the last year shows that a lot of folks are being drawn in. Unfortunately, unless they understand some of the challenges I outlined above, they may be in for some tough times and learn these leassons the hard way.

--joerg

Block Storage Virtualization

For my first posting I really want to talk about block storage virtualization. I really think that 2008 will be the year that people start to roll this out in production in a serious way. Why? It's the money stupid!

Yes, that's right, with the economy getting tight, I suspect that IT budgets, even those for storage, are going to get slashed. So, how are storage managers going to do more with less? You don't think that with the budget cuts there will also be a reduction in the growth of storage/data do you? Of cource not! The business will simply expect the storage team to do more with less, that's all. Simple really, don't you think?

What this will mean is that storage managers are going to be looking for a way to drive the per-GB cost of storage down even more. For many I think that the answer will be block storage virtualization.

Why? Well, I think that there are a couple of answers to that. First off, one direct way to reduce CAPEX will be to drive down the cost of the array's themselves. How? Easy, more competition. If I virtulize the storage, then the array becomes even more of a comodity than it is today, thus driving down the price. It's basic economics really. The more vendors I allow to bid on my next 100TB storage purchase, the lower the price per GB should be, right?

Also, if the real "smarts" is in the virtualization controller, then I don't need it in the disk array, so I can save money on licencing the software in the array. I no longer need to buy replication software from each storage vendor, I have a single replication mechanism which is probably in the virtualization controller itself. More in this in a later post, I think it's going to have a huge impact on the storage vendors going forward.

I also think that I can achieve some OPEX savings by having more efficent operations and fewer outages. Think about it, if all of my storage admins work with a single tool for provisioning, replication, etc. then I have more people with the same skill set, all working in the same interface. That's got to be more efficient and less error prone than having a couple of folks who know the HDS stuff well, and a couple more that know the EMC stuff well, etc.

You had this option before by just buying all of your storage from a single vendor, the trouble with that way of approaching things was that I also had vendor "lock-in". The vendor knew that they had me by the short hairs. Where this really showed up was not in the per/GB price of my storage, or my storage software. I mean, anyone with two brain-cells to rub together knows that if you are going to get everything from a single vendor you better lock in your discount up front, and it better be big. But trust me, the vendors made up for those big discounts via things you didn't have them locked in on. Professional Services, for example. At any rate, virtualization gets me out from under all of that, makes provisioning something that anyone on the team can do at any time following the exact same processes and procedures. You have to believe that will have a posative effect on you OPEX costs.

So, if 2008 is the year of block storage virtualization, what about file virtualization? We all still do NAS right? More on that next time.

--joerg