How I Reclaimed Terabytes of Disk Space From Hyper-V

I recently worked through a server hardware refresh in my organization. Given the costs involved in a server hardware refresh, my initial plan had been to split the refresh into two parts. The first part of the refresh would focus on the servers themselves. The second part – a storage refresh – would take place at a later point.

Even though I was beginning to run low on storage, I estimated that I should have enough storage remaining to last me for several months. However, I ran into a situation that completely upended my plans.

Context for the Server Hardware Refresh

Before I explain what happened or why a storage refresh is no longer even necessary, I need to give some background about my environment.

I have two production servers (along with several lab servers) that act as Hyper-V hosts and run my production virtual machines. These two servers are set up as Hyper-V replication partners, with virtual machines running on one host and replicated to another every 30 seconds. That way, if there is ever a failure on my primary host or its attached storage, I can simply failover to my replica hardware.

Over the course of the hardware refresh, I replaced both of my production Hyper-V servers. Because of some of the logistics involved in the upgrade process, I had to remove and then reenable the replication process. Doing so, of course, triggered an initial seeding process whereby my production VMs were copied to the replica server. This is where the problems began.

Related:Use Virtual Machine Manager as a File Server Management Tool

Diagnosing and Fixing the Problem

Even though I had about 7 TB of space remaining on the replica volume, that volume ran out of space during the replica seeding process. I had assumed that the existing replication data would simply be overwritten by the new replication data, but clearly that wasn’t the case.

Given that my replica volume was out of space, I had no choice but to disable Hyper-V replication, then format the volume and reenable the replication process. When the initial seeding process had completed, I did not have the expected 7 TB of free space but rather almost 16 TB of space. The important takeaway from this is that even before my hardware refresh, Hyper-V was wasting about 9 TB of space. Presumably, this waste stemmed from replication failures that had occurred in the past.

Out of curiosity, I looked at the storage volume that was attached to my primary Hyper-V server. That storage volume had about 14 TB of free space. Since the volumes attached to the two servers were the same sizes, it presumably meant that 2 TB of space were being wasted. I confirmed this by browsing the volume, in which I discovered multiple HRL files (Hyper-V replication files) from last August.

While I could have just deleted the old HRL files, I decided that the best way to clean things up was to perform a failover to my replica host, turn off Hyper-V replication, format the volume attached to what had been my primary host, and then reinitiate the replication process. After everything was done, I could perform another failover, making what had originally been my primary host the primary host once again.

This alone would have been sufficient, but I decided to go one step further. My production virtual machines are configured to use dynamically expanding virtual hard disks. The nice thing about dynamically expanding virtual disks is that they automatically grow to accommodate any data that is saved to them. The downside is that the virtual disks do not automatically shrink in response to the data removal. However, there is a simple process you can use to compact virtual hard disks. I decided to go ahead and compact my Hyper-V virtual hard disks prior to enabling the replication process.

As you can imagine, this was a highly involved and time-consuming process. Even so, it was well worth the effort. It allowed me to reclaim physical disk space so that I could put off a storage refresh for at least a year.

Reclaim Disk Space in Six Steps

Now that I have explained the process that I used and its outcome, I will walk you through the entire procedure in six steps.

Step 1: Create a backup

The very first thing that I did before anything else was to create a backup of all my data. Even though the procedure should have been low risk, I did not want to take any chances and so protected my data accordingly. If you find yourself in a similar situation, I strongly recommend that you also create a backup.

Step 2: Perform a planned failover

With the data backup completed, the next step is to shut down the virtual machines and performed a planned failover. A planned failover is a failover that occurs without data loss (assuming that both hosts and Hyper-V replication are working properly), whereas an unplanned failover is considered to be lossy.

To perform a planned failover, right-click on the virtual machine and select the Replication | Planned Failover commands from the shortcut menus, as shown in Figure 1.

Hyper-V Reclamation 2-1

Figure 1. Right-click on the VM and select the Replication | Planned Failover commands from the shortcut menus.

After selecting the Planned Failover option, Hyper-V will display a dialog box asking if you want to start the VM on the replica host as well as reverse the replication direction. Typically, if you were performing a planned failover, you would enable both these options, which you can see in Figure 2. In this case, however, both options needed to be deselected. Clicking the Failover button will initiate the failover process.

Hyper-V Reclamation 2-2_0

Figure 2. Deselect both check boxes and click Failover.

Step 3: Disable replication

Once the failover process completes, the next step is to disable Hyper-V replication. To do so, right-click on the VM and select Replication | Remove Replication from the shortcut menus. You must perform this procedure on both hosts.

Step 4: Format the disk

With the replication removed, now delete the virtual machine from what had initially been the primary host and then reformat the volume that contains the virtual machine files.

Step 5: Compact the virtual hard disks

The next step is to compact the virtual hard disks on the remaining host. In the Hyper-V Manager, select the Edit Disk command from the Actions menu. This will cause Hyper-V to launch the Edit Virtual Hard Disk Wizard.

Click Next to bypass the wizard’s welcome screen. When prompted, provide the path to the virtual hard disk file that needs to be compacted and then click Next.

The next screen will prompt you to choose an action to perform on the virtual hard disk. Choose the Compact option, as shown in Figure 3, and then click Next. Click Finish to begin the process of compacting the virtual hard disk.

Hyper-V Reclamation 2-3

Figure 3. Set the action to Compact and click Next.

If space can be reclaimed by compacting a dynamically expanding virtual hard disk, a progress bar will appear that indicates the disk is being compacted. However, if there is no empty space within a virtual hard disk, then nothing will appear to happen after you click Finish. In my case, for example, I attempted to compact four virtual hard disks. Three of those virtual hard disks could not be compacted, but Hyper-V spent about an hour compacting the fourth virtual hard disk and reclaimed 1.4 TB from that disk.

Step 6: Put everything back to normal

The last step in the process is to boot the virtual machine and enable Hyper-V replication.

Once the initial seeding process has completed, you can shut down the VM and perform a planned failover. Unlike earlier, the two checkboxes shown in Figure 2 should be selected. This will cause the server that was originally acting as the primary host to become the primary host once again. It will also cause the virtual machine to be automatically booted once the failover is complete. Because this process reverses the replication direction, the newly designated primary host should automatically begin to replicate the virtual machines to the replica host. No initial seeding process will be required because the replica is already up to date.