Question: What are the benefits and issues that can come from booting your SQL Servers from the SAN?
Answer: Booting servers from the SAN (called SAN booting) is something that Windows and Storage folks like to do as they can buy servers without hard drives installed. Basically the storage team builds another LUN for the server and the servers OS is installed on that LUN instead of being installed on local disks.
Related: Can't I just build my own SAN using commodity hardware?
The benefits of booting your server's from the SAN come from the manageability side of things. Basically in the event that the SQL Server's hardware fails getting the server up and running on a new piece of hardware is very quick and easy. The LUNs are allocated to the new world wide names for the HBAs and the server can now boot up as if it was the old physical server. There will be a bunch of driver changes that need to be made because the cards are different (different MAC addresses for the NICs, different world wide names for the HBAs, PCI slots are different, etc). But these changes will basically just take a reboot after the Windows OS deals with the driver changes. Once the system reboots once or twice the SQL Server will be back up and running with all the drives in the right location and with SQL accepting requests and doing what it needs to do.
The down side of booting off of the SAN is that SAN storage is very expensive. Another downside is that if you were using local disks, you probably would be using a basic RAID 1 array to host the OS. When using SAN storage, using RAID 1 or RAID 10 for the Windows OSs would be quite pricey, so the LUNs that the Windows OSs boot off of are probably RAID 5 or RAID 6 to save money. This means that when the Windows page file is being accessed, specially written to it'll be to a slower than expected volume. This means that in the event that SQL Server begins paging data to disk it will not be the normally slow speed that is expected, it'll be even slower. This is doubly true when you keep in mind that there are probably many servers all booting off of the same disks on the storage array. If a server was to be rebooted during the day all of the servers that have OSs which are on the same physical disks will see a performance hit to the OS drive while the machine is booting.
When recovering from a major power outage where the data center looses power you'll see a massive boot storm on the array which could actually slow down the boot process of all your servers as they boot up.
When working in a virtual environment if your virtual hosts are clustered then all your virtual servers are already booting off of the storage array. Keep this in mind when planning your deployment.
Related: Should I Be Using SAN Snapshots as a Backup Solution?
Denny