With all the virtualization of network storage, I'm beginning to wonder whether I can count on anything being real anymore. From a Windows administrator perspective, understanding exactly what happens to the hard disks you install in your servers is becoming increasingly difficult. You used to be able to install a hard disk in your server, then reboot, and you instantly had another disk device to configure and use. Today, as storage devices become more virtualized, determining how your system uses storage can be challenging.
RAID technology was the first step in storage virtualization; we no longer dealt with individual disks but with arrays or groups of disks. This concept was more challenging than simple individual disks, but most folks soon learned how to use the arrangement to their advantage. Combining OS and hardware functionality—with both providing some rudimentary form of virtualization—was the next step. We could "hook together" a couple of RAID arrays to create a logical disk that the OS and applications could use. This approach to storage was possible because an abstraction in the OS let us "filter" our view of physical devices into different logical views of storage for practical use. This scenario is the essence of storage virtualization.
Although today's storage-virtualization technology offers some distinct advantages for Windows administrators, the technology also makes planning, designing, and managing storage increasingly complex. Fortunately, Windows Server 2003 has helped simplify these complexities.
What Happened to My Disks?
Storage virtualization's complexity starts with the hardware vendors. Today's high-end storage systems—Direct Attached Storage (DAS), Network Attached Storage (NAS), and Storage Area Networks (SANs)—incorporate some level of storage virtualization. Even a simple locally attached RAID controller in a server virtualizes its physical disk devices to the OS. Instead of configuring 14 individual disks, for example, an administrator would typically choose to configure the disks into an array with some level of fault tolerance. The RAID controller can virtualize the physical disks into a different view and expose that logical entity to the OS. Even NAS and SAN devices have, at one level, a controller that connects directly to the physical disk devices and provides an abstracted view of those devices. This arrangement isn't rocket science and, for the most part, most of us who are familiar with these technologies are comfortable planning, designing, and managing our servers within this relatively simple paradigm.
More recently, however, hardware and software vendors, including Microsoft, have taken storage virtualization to new levels. Recent offerings in the NAS and SAN markets provide storage enclosures that seem to hide the where and what of physical disk devices. Those of us who need a concrete view of things to understand them have been forced to shift our way of thinking and go through a significant learning curve when it comes to managing these storage systems. For example, the Hewlett-Packard (formerly Compaq) HP StorageWorks HSG80 controller in SAN products such as the MA8000 and EMA12000 used SCSI interfaces that connected to SCSI buses; each bus supported a finite number of physical disks. When planning storage around this beast, you could simply look at the physical disks and SCSI buses; you had concrete rules about how to group the devices into LUNs and present them on the SAN for hosts to use. However, newer solutions provide storage enclosures and array controllers that have their own virtualization methods (such as disk groups that have more flexible configuration boundaries) and new ways of building these enclosures and connecting disk devices (such as Fibre Channel attached disks and controller-based virtualization).
Windows 2003 VDS to the Rescue
Microsoft added a new key component to Windows 2003: Virtual Disk Service (VDS). VDS is the only block-storage interface for the Windows platform that's hardware, software, interconnect (e.g., Fibre Channel, SCSI), and subsystem neutral. Regardless of whether the OS, hardware enclosure/controller, or other storage-virtualization engines (e.g., VERITAS Software's VERITAS Volume Manager) provide block-storage functionality, all the components leverage VDS to manage this function. In VDS, Microsoft attempted to provide a Windows storage-virtualization service that third parties could extend and that the company could easily support. Hardware and software storage-solution vendors need only adhere to VDS specifications, and their solutions will integrate with Windows 2003 and Windows applications.
Although the details about VDS are too lengthy to cover in this article, the key is that providers can use the VDS API to communicate with the OS and applications. As Figure 1 shows, the layered VDS architecture includes VDS and VDS providers, which can come from Microsoft or third-party vendors. VDS communicates with management tools and interfaces that Windows administrators and VDS-compliant providers (hardware and software) use. This architecture exemplifies Microsoft's VDS goals of delivering one management interface for Windows storage virtualization, an extensible framework for third-party storage-solution vendors, and a supportable product component for Microsoft.
What does VDS do? First, it acts as a coordinator for all storage devices and providers that are connected to the OS. For the first time, Windows has one management "traffic cop" for managing storage functionality, whether that functionality is native to the OS or vendor-provided. VDS also serves as an abstraction layer by mapping either physical disks or virtual disks from hardware devices to software-managed disks. VDS also manages and coordinates the various VDS clients (e.g., management applications, command-line tools) that administrators use to manage Windows storage. In addition, VDS provides core file-system functions such as partitioning, formatting, drive-letter assignment, and mount-point mapping. Finally, VDS exposes a rich storage-management API that lets Microsoft, third-party developers, and even Windows administrators extend the VDS functionality.
VDS-Related Features
Windows 2003 delivers three VDS-related functions in addition to VDS. Microsoft supplies two software-storage providers: the Basic Disk Provider and the Dynamic Disk Provider. (For details about the differences between basic and dynamic disk types, see "Choosing Basic vs. Dynamic Disk Storage for Windows Servers," December 2002, http://www.winnetmag.com, InstantDoc ID 27085.) The Basic Disk Provider includes support for basic disk functionality such as partitions and volumes. The Dynamic Disk Provider adds dynamic disk functionality, such as disk striping and spanning, and fault-tolerant functionality, such as RAID 5 and mirroring.
The third item is a collection of administrative tools that provides an interface to VDS. One of these tools is the familiar Microsoft Management Console (MMC) Disk Management snap-in that most administrators know and love. Windows 2003's Disk Management snap-in isn't visually different from what you're used to in Windows XP and Windows 2000, but Microsoft has overhauled the tool to directly interface with VDS (the tool is a real VDS client).
Another familiar tool is the Diskpart command-line tool. Diskpart shipped with XP and is also available for Win2K. In Windows 2003, Microsoft adds new commands to the tool that enable SAN functionality (such as the ability to specify parameters for automount LUNs and to mark LUNs as inactive) from the command line.
The last tool, Diskraid, is new to Windows 2003 and ships as part of the upcoming Windows 2003 resource kit. Like Diskpart, Diskraid is a command-line tool that lets administrators perform LUN operations (e.g., create, delete) from the command line. These tools interface directly with VDS as clients and extend the manageability of Windows storage.
The VDS Unification Point
As vendors that provide Windows storage solutions make huge strides in functionality and features, storage virtualization becomes increasingly complex to understand and administer. This trend could complicate the administration and manageability of Windows systems unless we can find a way to ensure that everyone is on the same page where storage management and virtualization are concerned. Windows 2003 VDS provides this unification point by offering Windows administrators and third-party vendors an open and extensible framework while providing Microsoft with a high degree of stability and supportability.
