Shared Nothing Live Migration Between Clusters

Q: How can I perform a shared nothing live migration from one cluster to another cluster?

A: Shared nothing live migration allows the migration of a virtual machine (VM), including its storage, memory, and device state, between Hyper-V hosts without any downtime. A logical extension is the desire to perform a shared nothing live migration between Hyper-V clusters (when you don't want to use a cluster migration). However, you can't use shared nothing live migration to migrate between clusters directly. Instead, you need to do the following:

1. Remove the VM as a cluster resource.
2. Use shared nothing live migration to move the VM to a node on the new cluster on cluster storage.
3. Add the VM as a cluster resource on the target cluster.

There's no downtime for any of these actions, because VMs can be added and removed from clusters without incurring downtime. I wrote a script that uses a basic flow to perform this action, as follows:

1. Prompts you for a credential that it will use on the remote cluster
2. Prompts for a node of the remote cluster you want to migrate the VMs to
3. Enables CredSSP delegation to the node specified (you need to have enabled CredSSP on the remote server, using the Enable-WSManCredSSP Server command, before starting this)
4. Starts a loop that prompts for a VM name to be moved and then:

• Finds the cluster group that owns the VM and displays the cluster resources for the VM
• If the VM isn't running on the current node, the VM is live migrated to the current node (which makes things simple)
• The VM is removed from the cluster
• There's a step that can be commented out, which stops the VM—this is necessary if the processors in the source and target cluster don't match (e.g., Intel versus AMD or different versions of Intel/AMD and you don't have processor compatibility enabled)
• The VM is moved to the target node using shared nothing live migration
• The VM is made a cluster resource on the target cluster
• The next VM name is prompted on, or type Q to quit, and the earlier steps then repeat for the next VM

The complete script follows. Note that you need to run the script on a node in the source cluster. In addition, you might want to change the storage location that's used to store the VM on the target from the C:\ClusterStorage\Volume1 target that I hard-coded into the script. Also, make sure the Hyper-V Module for Windows PowerShell and the Failover Cluster Module for Windows PowerShell are installed on the host running the script. This script performs no error checking; it's a very basic script, so if you intend to use it in a major way you should probably add some error checking. My goal here is simply to show the required steps.

#Migrates virtual machines between clusters
#Copyright 2014 John Savill
#
#Assumes virtual switch with same name exists on target
#
#Need to enable CredSSP to solve the double-hop problem associated with cluster commands
#http://blogs.msdn.com/b/clustering/archive/2009/06/25/9803001.aspx documents the double-hop problem related to clustering remote commands
#On the target server must have run Enable-WSManCredSSP Server

#Lets get started

Write-Host 'Please enter credential to be used for remote cluster execution'
$CustomCred = Get-Credential #Get credentials to use with CredSSP on the remote cluster

Write-Host 'Ensure the target Server has been configured to accept CredSSP by running command Enable-ESManCredSSP Server on the server'

$DestinationHost = Read-Host 'Enter the target host that will be used for all migrations >'

Enable-WSManCredSSP Client -DelegateComputer $DestinationHost -Force

$VMName = Read-Host 'Enter VM name to move >'
while ($VMName -ne 'Q')
{

    #Show the cluster resources. Cannot use the $VMName as the cluster group as may not match, for example SCVMM names the cluster group differently
    #Get-ClusterGroup -Name $VMName | Get-ClusterResource
    $ClusterGroups = Get-ClusterGroup | ? {($_.GroupType -eq 'VirtualMachine') -and ($_.State -eq 'Online')} 
    foreach ($ClusterGroup in $ClusterGroups)
    {
        $TempVM = $ClusterGroup | Get-VM 
        if ($TempVM.Name.ToUpper() -eq $VMName.ToUpper())
        {
            $FinalClusterGroup = $ClusterGroup
        }
    }

    $FinalClusterGroup | Get-ClusterResource

    $ClusterGroupName = $FinalClusterGroup.Name

    #Store the host the VM is currently active on if wanted to take the approach of NOT moving it to local node but means requires Constrained Delegation configured
    $NodeName = Get-ClusterGroup -Name $ClusterGroupName | Get-ClusterResource | Where-Object {$_.ResourceType -eq "Virtual Machine"} | Select-Object OwnerNode
    $CurrentNodeName = $NodeName.OwnerNode.Name

    #View the VM
    Get-VM -Name $VMName -ComputerName $CurrentNodeName


    #Move the VM to the current host
    $LocalNodeName = $env:COMPUTERNAME
    if ($CurrentNodeName.ToUpper() -ne $LocalNodeName.ToUpper())
    {
        Write-Host 'Moving VM from '$CurrentNodeName' to current node'
        Move-ClusterVirtualMachineRole -Name $ClusterGroupName -Node $LocalNodeName -MigrationType Live
    }

    #Remove the cluster resources and group which does not stop or remove the actual VM on the host
    Write-Host 'Removing cluster group for VM'
    Remove-ClusterGroup -Name $ClusterGroupName -RemoveResources -Force

    #Sleep to avoid lock from removing cluster resources (not typically required)
    Start-Sleep -Seconds 10

    #Have to shutdown as different processor type (Intel to AMD in my case but normally would not need to stop the VM) or if different versions
    #and processor compatibility not enabled
    Stop-Vm -Name $VMName #This will wait until the VM has stopped
    Start-Sleep -Seconds 10

    #Now migrate it from this node so don't need -ComputerName $LocalNodeName
    Write-Host 'Moving VM to target node'
    Move-VM -Name $VMName -DestinationHost $DestinationHost -DestinationStoragePath C:\ClusterStorage\Volume1\$VMName

    $ScriptBlockContent = {
        param ($VMName)
        Add-ClusterVirtualMachineRole -VirtualMachine $VMName -Name $VMName }

    #Now add as resource on the target cluster
    Write-Host 'Making VM a cluster object on target'
    Invoke-Command -ComputerName $DestinationHost -ScriptBlock $ScriptBlockContent -ArgumentList $VMName -Authentication Credssp -Credential $CustomCred

    #Could automatically start the VM here if you wanted but I don't in this example

    #Get next name
    $VMName = Read-Host 'Enter VM name to move (Q to quit)>'
    $VMName = $VMName.ToString()
}

The following is an example execution moving a single VM.

PS C:\> .\MigrateBetweenClusters.ps1


cfg         : http://schemas.microsoft.com/wbem/wsman/1/config/client/auth
lang        : en-US
Basic       : true
Digest      : true
Kerberos    : true
Negotiate   : true
Certificate : true
CredSSP     : true

Please enter credential to be used for remote cluster execution
Ensure the target Server has been configured to accept CredSSP by running command Enable-ESManCredSSP Server on the server
Enter the target host that will be used for all migrations >: savaushv10
Enter VM name to move >: Test - DC01
Name         : SCVMM DC01 (1)
State        : Online
OwnerGroup   : SCVMM DC01 Resources (1)
ResourceType : Virtual Machine

Name         : SCVMM DC01 Configuration (1)
State        : Online
OwnerGroup   : SCVMM DC01 Resources (1)
ResourceType : Virtual Machine Configuration

Name             : Test - DC01
State            : Running
CpuUsage         : 0
MemoryAssigned   : 883949568
MemoryDemand     : 724566016
MemoryStatus     : OK
Uptime           : 81.20:38:09
Status           : Operating normally
ReplicationState : Disabled

Moving VM from  savdalhv11 to current node
Name      : SCVMM DC01 Resources (1)
OwnerNode : savdalhv10
State     : Online

Removing cluster group for VM
Moving VM to target node
Making VM a cluster object on target
Name           : Test - DC01
OwnerNode      : savaushv10
State          : Offline
PSComputerName : savaushv10

Enter VM name to move (Q to quit)>: Q