Hyper-V Shared Nothing Live Migration In Windows Server 2012– VM Mobility Rules

I see and hear some people shrug at the idea of Shared Nothing Live Migration, dismissing it as marginally useful. Some do state they’ll have it as well but that it’s not that valuable. Well I disagree totally. A lot of the time these remarks are due to a lack of understanding about how several technologies in the Microsoft stack work together. Combine this with tunnel vision and the fear of some vendors and you get a lot of FUD.

I advise you to look beyond the virtualization stack, to the issues that people who are building infrastructure for dynamic, flexible and * cloud  data centers are dealing with.

Look, as “architects” we have to design & build for failure. We all know that it’s just a matter of time before things go BOINK.  So we build in redundancy, some of this within a silo, some of this is between silos. The two approaches compliment each other. What this gives you is options and everybody who knows me, especially those who work  with me has heard my mantras: “Assumptions are the mother of all F* Ups” and “Options, options, options”. Make sure you design & build in options. This way you can maneuver your self out of a bad situation. Don’t ever assume you’re out of options, especially not when you put some in the design on purpose Winking smile. It’s also very useful beyond that because a lot of you might agree with me that silos and fork lift, down time inducing upgrades, migrations, transitions or replacements are expensive and bad. This is where Share Nothing Live Migrations comes into play. You gain mobility over silos. That silo might be a server, a cluster, storage or mixtures of them all.

With Shared Nothing Live Migration we can migrate virtual machines between those silos with nothing more than a network cable.This is huge people. You are no longer trapped in that silo. In this context it provides you with all the options & flexibility mobility gives you. even it the technology itself is not about high availability.

Some very useful scenarios

Migrate virtual machines from an old cluster to a new cluster with out any down time

  1. Migrate virtual machines from stand alone hyper-V hosts to a fail over cluster with out any down time
  2. Migrate virtual machines from one stand alone host to another one for maintenance, again, without any down time
  3. Choose different types if storage & Hyper-V deployment depending in IOPS, redundancy, availability, manageability needs. With Shared Nothing Live Migration you can be confident  that  you can move your virtual machine from one environment to the other when needs change. This is breaking the storage silo boundaries open people! This is huge … think about it.

How it works

The details are for another post but basically is made possible by the combination of Live Storage Migration and Live Migration.

First the Storage is Live Migrated


After the Live Storage Migration is done the state of the virtual machines is copied and synchronized.


This Is Mobility

I hear the competition shrug.  It isn’t high availability. Well indeed no one who understands the feature ever said it was. It’s virtual machine mobility. Look at the scenarios above and you’ll see that this ability could very well be game changer in how we look at storage & design solutions.

Speed & Performance

What did we hear on this front: “it will be too slow to be really useful”. Really? Well let’s see:

  1. The world is converging to 10Gbps and after that 40Gbps and up will come
  2. NIC Teaming in box With Windows 2012 which can provide more bandwidth.
  3. SMB 3.0 Multichannel. This provides multiple channels per connection spreading the load over multiple CPUs
  4. SMB Direct, have you seen the speeds this achieves?

Before you state that this doesn’t work on Live Migration … as confirmed at TechEd 2012 Europe with Jose Baretto this does work when both the source AND the target is an SMB 3.0 share. This means yet another reason to use SMB 3.0 share for your Hyper-V storage needs! So unlike what Tad at vLimited keeps saying, unhindered by any knowledge, it is a very valuable feature and it can be extremely fast given the right connectivity and storage that can handle the IOPS. And no, the fact that it’s unbuffered doesn’t impact this to much. Test this by using xcopy/robocopy /J with a VHD over your infrastructure.


Even if you’re on a budget and cannot go for the RDMA NICs & SMB 3.0 you have several options to get very decent virtual machine mobility and not be stuck in a silo. And for those who want to leverage this feature to create and agile & mobile virtual environment you have some very nice technologies available to optimize to your needs & budgets.


Virtual Machine mobility and storage mobility are very interesting features that provide for a previously unknown flexibility. Windows Server 2012 makes us rethink our storage approaches (I sure am) and I’m very interested in seeing how this will evolve.

Very Educational Microsoft TechEd 2012

Hello from TechEd 2012 Europe at the RAI in Amsterdam. I’ve been extremely busy attending sessions, talking to Microsoft employees and vendor engineers. We’ve had some very interesting discussions and I learned a lot and clarified even more. TechEd has once more proven to be an excellent investment of time and I have been able to get a lot of face time with the right people. To me this is important because that helps me tremendously when designing solutions. Sorry for the low quality pictures.


Bob Combs on stage at the TechEd Europe 2012  educating us on NIC Teaming

You know my mantra “options, options options” as this is what gets you a way out of a pinch. However a lot of options also mean you need to make decisions, and not just when dealing with issues but also at design time. Knowledge and understanding is what help to make the correct of the best decision fast. Attending this conference with its tremendous networking opportunities provides for a very nice and effective setting for passionate discussions and deep dives into scenarios. Challenging vendors, interacting with peers, throwing ideas out there and deep diving into the possibilities and drawbacks with each other is great and helps a lot to understand technologies better. You have to thrown what you have learned out there and discuss it to test your understanding of the subject. Don’t be afraid to do so. We all don’t know things, get stuff wrong, etc. Don’t let  fear stop you from interacting with your peers.


Ben Armstrong in action on Live Storage Migration

It is also great to meet up with my community buddies from all over the word again and I feel privileged to have the opportunity to attend these conferences. For me personally these are priceless and the value to my employers/clients is considerable. There is a tsunami of new technology in the Windows Server 2012 stack and learning to put these into context is both fun and useful. These are very interesting times in the Microsoft Infrastructure ecosystem so life is good!

Answer to Brad at TechEd Europe 2012 Keynote: Pessimists & Tad Don’t like Windows Server 2012

Brad is on stage for the opening keynote asking if the glass is half full or half empty. Well it depends on where you are in the ecosystem. For us the glass is half full and filling up fast.

Some people nag me about the fact that Windows Server 2012 is so different and that it’s wrong to turn the world upside down. Yes, it is different and new in many ways.  There are also many improvements to features that already exist. There is a lot to learn and understand. Why are some people so pessimistic?

Ever since I got my hands on the BUILD Developer Preview bits I have personally invested a lot of my time in Windows Server 2012. With the beta that only increased. Why? Well, that’s the way forward, because that’s where the improvements are. We can’t do tomorrows jobs and meet tomorrows demands with yesterdays technology.


The picture above is basically the pessimists view of the world. Enjoy your cupper but I’m not joining you. Windows Server 2012 rocks and it’s going to do a whole lot for our industry and businesses. But wait a minute, I do understand why Tad is so pessimistic. But that’s about the future of vLimited and being stuck in the past. Listen Tad, you’d better empty that cup because this is where vLimited becomes history rather than write it.

Does that mean I’ll be throwing away Windows 2008 R2? Nope. I expect to deal a lot with it in the next few years but I’m not going to build future infrastructure on the previous version. I will introduce Windows Server 2012 where and when we benefit from it. For me that is from day one the bits RTM. The benefits are so overwhelming we’d hurt ourselves by not doing it. Your mileage may vary. But don’t get stuck in the past  Here’s a link to your escape pod: Microsoft Virtual Machine Converter Solution Accelerator I’m happy it’s here. That’s what people are asking me more and more about, how to move to Hyper-V.

But what’s with the negativism of some? Sure people are still running Windows Server 2000/2003. Sometimes for good reasons, often for (very) bad ones. Are some going to go through all this again with people clinging to Windows2008 R2? No doubt. Been there, seen it. Very predictable. Is Windows Server 2012 going to fail?  No way.  And what I’m seeing in Windows Server 2012 is great technology. Will it be perfect? No. I already have feature requests for vNext Smile. But this is pushing the ball forward, this is ambitious in the best sense of that word.  There will be bugs, there will be challenges and hiccups. That’s part of the business and the realities of life.  But look at all what’s available in there. Don’t just read some industry press articles. Did you test it your self already? Did you do any clustering? Tested all the new functionality in Hyper-V? The innovations in Live Migration options and networking? Looked at the amount of PowerShell support in there? Notice the improvements in Active Directory, DHCP and other core infrastructure services? Have you used Windows Server 2012 at all yet? You didn’t look at SMB 3.0 and all the storage improvements in there did you? Go talk to Jeff Woolsey, he’s passionate about it and for good reasons. Put in some effort, live a little, get out of your comfort zone and you’ll be going places. Don’t be a pessimist. Think positive or you’ll end up like Tad who was the joke of the party at MMS2012


Moving Clustered Virtual Machines to Windows Server 2012 with the Cluster Migration Wizard

As you might remember I did a blog post on transitioning from a Windows Server 2008 R2 Hyper-V cluster to Windows Server 2012 (well I was using the beta at the time, not the RC yet):

  1. Part 1 Upgrading Hyper-V Cluster Nodes to Windows Server 2012 (Beta) – Part 1
  2. Part 2 Upgrading Hyper-V Cluster Nodes to Windows Server 2012 (Beta) – Part 2
  3. Part 3 Upgrading Hyper-V Cluster Nodes to Windows Server 2012 (Beta) – Part 3

Microsoft has now blogged about the process themselves and they use the migration wizard in Failover Cluster Manager to get the job done where I did this using the Import, “register only” functionality.

This is the first step by step that describes the official way. You can read about the process here:

How to Move Highly Available (Clustered) VMs to Windows Server 2012 with the Cluster Migration Wizard

Multi Site SAN Storage & Windows Server 2012 Hyper-V Efforts Under Way

First some stats: 36 pallets of hardware handled over a period of 10 days. 29 of those over a period of 3 days. Most of it didn’t even exist at the beginning of the month, it was just an order. But DELL is a logistical force to be reckoned with. “Easy as DELL” is a reality, the speed at which they respond to request and orders is amazing. For quality/price balance, service, logistics, speed and support, it’s hard to beat them

A lot of people are used to dealing with slower processes and think SANs take at least 2 to 3 months to de delivered after ordering. This means they are caught of guard by this. I’m happy to say I’m not otherwise the data center would have been blocked by a tsunami of packaging material and hardware.

We’ve been busy unloading, unpacking, racking and partially cabling the new hardware coming in for a multi site SAN project. And let’s not forget the labeling. While we are far from finished, this good news. We’re finally busy working on the installation after the long time consuming process of procuring the equipment. That’s never an easy process, let alone a fast one. But I digress.

What are we working with?

  • Dell Compellent SANs (intra and inter site data protection / redundancy)
  • PowerVault MD3600 & MD1200 storage units for disk to disk backup capacity

Now to go from this


to this and beyond  …



Takes quite a while as you can imagine and we still have a ton of stuff to do Smile. I’ll be sharing my experiences and findings via this blog when I can.

My high level design  focuses on scale out to achieve both performance, flexibility and resiliency. We’ll build a modular scale up and scale out solution using commodity hardware and not in a mega redundant, ultra scalable single and very expensive storage solution. You can read more on my views about this subject here Some Thoughts Buying State Of The Art Storage Solutions Anno 2012.For the backup we are following the same approach. We cannot afford to pay the amounts of money that seems to be needed to buy high end backup appliances. We have plans to leverage Windows 2012 to help us achieve this but these are subjects for some other blog posts later.

Windows Server 2012 Hyper-V Supports IPsec Task Offloading

IPsec has been around for a while now. In an ever more security conscious & regulated world you want and/or are required to protect your network communication by
authenticating and encrypting the contents of at least some of your network traffic. Think about SOX and HIPPA and you’ll see that trade or government security requirements are not going anywhere but up for us all. This is not just restricted to military of intelligence organizations.

We’ve seen the ability to offload IPsec traffic to the NIC for a while now. This is great as the IPsec processing is a very CPU intensive workload. Unfortunately it didn’t work for virtual machines . Until now IPsec offloads was only available to host/parent workloads in using Windows Server 2008 R2. The virtualization of high volume network traffic workloads that require encryption means a serious hit on the resources on the host. If you’re willing to pay you might get by by throwing extra host & CPU power at the issue. But what if the load means a single virtual machine with 4 vCPUs can’t hack it? Game over. Sure Windows Server 2012 Hyper-V allows for 32 vCPUs now,  but that is very costly, so this is not a very cost effective solution. So in some cases this lead to those workloads being marked as “unsuited for virtualization”.

But with Windows Server 2012 Hyper-V we get a very welcome improvement, that is the fact that a virtual machine can now also offload the IPsec processing to the physical NIC on the host. That frees up a lot of CPU cycles to perform more application-level work, resulting in better virtualization densities, which means less costs etc.

Let’s take a look where you can set this in the Hyper-V GUI where you’ll find it under the network adaptor /Hardware Acceleration.


IPsec offload is also managed by the Hyper-V switch, this controls whether the offloading will be active or not. This is to prevent that the IPsec offload stopping the services if insufficient resources are available. Please do note that IPsec when required in the guest will be done anyway creating an extra CPU burden. So this does not disable IPsec, just the offloading of it. On top of this and in the gravest extreme you can guarantee that IPsec servers can get the resources they need by sacrificing less important guest if needed. by using virtual machine prioritization. The fact that you can configure the number of security associations helps balancing the needs of multiple virtual machines requiring IPsec offload.

To conclude, this wouldn’t be Windows Server 2012 if you couldn’t do all this with PowerShell. Take a look at  Set-VMNetworkAdapter and notice the following parameter:


This specifies the maximum number of security associations that can be offloaded to the physical network adapter that is bound to the virtual switch and that supports IPSec Task Offload. The thing to notice here is that specify a zero value is used to disable the IPsec Offload feature.


Microsoft Private Cloud Computing–The Book

I’d like to mention the launch today of a new book Microsoft Private Cloud Computing written by a group of experts in this subject.


Take a look at the authors below. You probably know them from the community efforts they put in and might have read previous materials from them or heard them present.They bring a varied background and a mix of experiences and knowledge to the table needed to tackle any aspect of the private cloud and to a write complete book on the subject: hardware (servers, networking, storage), operating systems, hypervisors and the software stack.

Knowing these people and how busy they are this quite an achievement requiring a serious effort.  The technical reviewing was done by Kristian Nese (Cloud and Datacenter Management MVP,@KristianNese), who’s has been working on private clouds from day one.

Well done guys and thank you!