VMware ESX offers several options for creating virtual disks (i.e., VMDK files) for your virtual machines (VMs). This article covers options for thin and thick VM disk provisioning, along with general recommendations and guidelines for storing VMDK files on ESX DAS and SANs. It’s especially important to consider the block size when formatting an ESX storage group; otherwise, you might have to back up any existing VMs and reformat the storage group. These disk configuration parameters have a significant effect on functionality, performance, and manageability.
Creating a Disk
When you add a disk to a VM in VMware ESX, you’re prompted with options to create a new virtual disk, use an existing virtual disk, or create a raw device mapping. The first two options are self-explanatory. When a raw device mapping is stored on a SAN, the files aren’t encapsulated inside a VMDK file. If you browse the files on the SAN LUN, the individual files will appear, not just a single VMDK file that encapsulates all the files inside the VM’s disk.
If you have a SAN, you can create a raw device mapping that maps storage to a LUN on the SAN. Raw device mapping can give you better throughput for sequential disk I/O and about the same performance for random disk I/O compared with non–raw device mapping disks. However, the main reason to use raw device mapping is when you need to expose the physical device properties for SAN management software.
Raw device mapping can be configured in either virtual or physical mode. Virtual mode presents the raw device mapping just like any other VMDK file, with all of the same functionality of a regular VMDK file, such as snapshots and cloning. Physical mode exposes all the physical hardware, with the exception of the REPORT LUNS command, which is virtualized so that ESX can track which VM is using the LUN.
Some SAN vendors’ VSS writers that are used for SAN snapshots, backup, or replication require the use of a raw device mapping in physical mode. However, raw device mappings in physical mode prevent you from using VMware snapshots, cloning, and VMotion on the VM. Physical mode is required by some SAN vendors to ensure that the VMs are properly quiesced before a SAN snapshot is taken.
Quiescing is important for VMs running any type of logged data changes, such as Microsoft Exchange Server and Microsoft SQL Server. Quiescing temporarily stops the transaction flow on the VM so that when a SAN snapshot is taken, only fully complete transactions reside in the database. If a VM isn’t properly quiesced, you run the risk of snapping a VM with a database in a “dirty” (partial transactions) state.
Raw device mapping disks can be used instead of a large VMDK file (over 1TB) if you don’t feel comfortable storing that much data in a single VMDK file. Check with your SAN vendor for more information about raw device mapping.
Disk Provisioning
You can choose either thick or thin disk provisioning when you create a VM disk. If you select thick provisioning, ESX will allocate the entire size of the disk on the storage group when it’s created. If you thin-provision a drive, only the space that’s used in the VM disk is allocated. Thin provisioning saves storage space, but you’ll take a performance hit and increase the probability of running out of space on the ESX storage group.
The performance hit of thick versus thin provisioning is based on many factors, including disk speed, array configuration, and degree of disk fragmentation. As a general rule, thick provisioning gives the best performance and reduces the chance of running out of disk space. Although it requires more disk space, it’s a more conservative approach.
If you plan to thin-provision your VM disks, I suggest using it only for VM base images rather than data drives for VMs. If you create several VMs with 1TB thin-provisioned data drives, and users start saving data on these drives, you might very quickly run out of disk space on the ESX storage group. If you’re concerned about disk fragmentation on the storage group, check out Diskeeper’s V-locity or Raxco Software’s PerfectDisk 11 vSphere Bundle disk defragmenting software for ESX.
Disk and Controller Types
When you create a VM, you have the option of creating either IDE or SCSI disks. You should use SCSI disks whenever possible, because they provide better performance than IDE disks. You might have to use IDE disks if the VM’s application doesn’t support SCSI disks.
You must select the SCSI controller type when creating disks. The Windows OS determines the type of SCSI controller to use, based on compatibility and performance. Table 1 lists controller types by Windows OS.
Table 1: SCSI Controller Type Based on Windows OS
In addition to the SCSI controllers listed in Table 1, you can also use the VMware Paravirtual driver. However, you should use the Paravirtual driver only under the following circumstances:
- Drives are stored on a SAN and not using DAS.
- The VM will require more than 2,000 I/O operations per second of disk throughput.
- The VM will run Windows Server 2008 R2, Windows Server 2008, Windows Server 2003, or Red Hat Enterprise Linux (RHEL) 5.
- The drive isn’t a boot disk.
- The VM isn’t fault-tolerant.
- The VM won’t be used as part of a Microsoft cluster.
As a general rule, using the VMware Paravirtual driver gives you about 10-percent better disk throughput and 15-percent lower CPU utilization than using the LSI Logic SCSI controller when the VMDK file is stored on a SAN.
Independent Disks: Persistent vs. Nonpersistent
After selecting the disk type (IDE or SCSI) and the controller type, you have the option of creating an independent disk. You might use an independent disk if you don’t want the disk to be affected by snapshots.
For example, suppose you have an Exchange Server VM with the three disks C, D, and E. The OS is stored on C, the Exchange log files on D, and the Exchange data files on E. You could make C an independent disk. When you snapshot the VM, the C disk is unaffected by any snapshots and will speed up the process of taking the snapshot. If you need to revert back to the snapshot, you don’t have worry about also rolling back any files on C, because only files on D and E are affected.
In general, you should leave the Independent check box blank, unless you need this feature. If you decide to create an independent disk, you have two additional options:
- Persistent—Any changes are immediately and permanently written to disk, even when a snapshot is taken of the VM.
- Nonpersistent—Any changes to the disk will be discarded when the VM is rebooted or reverted back to a snapshot. The only time you might want to use this setting is in a lab environment if you want the VM to go back to its original state when it’s rebooted. Because this is such a dangerous hacking tool, I suggest reviewing the drive configuration of all VMs on a regular basis to verify that they aren’t configured with any nonpersistent disks.
Storage Group Block Size
One item that haunts administrators who are new to ESX is the block size when a storage group is formatted. The block size determines the largest VMDK file that can be created on the storage group. The block size is independent of the total size of the storage group. Table 2 shows the relationship between block size and storage group file size.
Table 2: Relationship Between Block Size and Storage Group File Size
The default block size is 1MB, so you can only create a VMDK file that’s 256GB less 512 bytes. If you need to create a VMDK file that’s larger than 256GB, you must back up your data, format the storage group with a larger block size, and restore the VMDK files—which isn’t a lot of fun in a production environment.
If you want to retain a VM’s snapshot functionality, make sure to create a VMDK file that’s 2GB smaller than the maximum allowed file on the storage group. For example, if you have storage that’s formatted with a 1MB block size, you shouldn’t create a file larger than 254GB. If you create a VMDK file that’s 256GB on the storage group, you’ll receive an error that the VMDK file exceeds the maximum file size on the storage group when you attempt to snapshot the VM. When a snapshot is created, the original VMDK file creates a 2GB stub file. So, if you try to snapshot a VM that has a 256GB VMDK file, ESX tries to create a VMDK file that’s 258GB and the snapshot fails.
Choose Your Options Carefully
You have many options for creating VM disks on VMware ESX. Some of these options are available only if your VM disk is stored on a SAN. Probably the biggest pitfall is the storage group block size, because it’s difficult to change. Choose your disk configuration carefully to ensure the best combination of management, cost, performance, and functionality for your VMs. (For information about configuring VM disks on Microsoft Hyper-V, see "Hyper-V Disk Configuration Options.")
