Architecting SharePoint: Virtualization and Cloud Decisions

When choosing how to design your SharePoint environment, you need to decide whether to virtualize SharePoint. You also need to decide whether to use a hosted environment, such as Microsoft SharePoint Online.

I'll discuss the reasons for and against virtualizing SharePoint, as well as reasons for and against using SharePoint Online.

I'll also cover how to prove your single-farm or multi-farm design with a working model. (If you haven't yet decided whether to use a single-farm or multi-farm architecture, see "Architecting SharePoint: Choosing a Farm Design.")

The Pros and Cons of Virtualizing SharePoint

I've worked with organizations that virtualize by default, unless it can be proven that a dedicated physical environment is required. However, virtualization has some drawbacks. Like any decision you make it, the decision to virtualize must be rooted in factual requirements.

This is why I'm big on doing your homework, collecting the facts, working with stakeholders, documenting what's agreed on, and escalating issues when required for decisions and conflict resolution.

When virtualization technology was first introduced, I recommended it for development environments and functional testing.

I never recommended it for physical environments and load testing because, when you virtualize, you're creating a very dense environment with several virtual machines (VMs) running side by side on several powerful hosts.

Today, VM technology is greatly improved and therefore the only risks you have are host density and capacity management.

On the positive side, a virtualized environment lets you perform backups and restores quickly as well as migrate VMs between hosts to balance workloads. In addition, you can provision VMs without having to wait for hardware to be ordered and delivered.

That often isn't the case with physical environments.

I've worked with organizations in which there was a two-to-three-month wait for hardware, followed by a couple more months of waiting for the Windows OS, SQL Server, and SharePoint to be loaded on the hardware.

Another benefit of using a virtualized environment is that you'll have higher asset utilization. In contrast, physical environments tend to have lower asset utilization, which can make the financial types nervous because you're carrying a cost structure that doesn't appear optimized for the intended workload and operating budget.

On the negative side, a virtualized environment can be problematic because it's dense (i.e., many VMs are running side by side). If not sized properly and proactively managed for capacity needs, this density can lead to overhead. Specifically, when you virtualize, you might need additional servers to match the capacity delivered by a physical environment.

This can be due to the way the VM service offering is modeled and managed. In addition, many virtualized environments are a shared service designed to house many applications and aren't specific to SharePoint.

Therefore, the VM options available might or might not suite SharePoint. However, if you're able to deploy a virtualized environment built specifically for SharePoint and you have full control over it, you shouldn't experience that type of problem.

You might want to use virtualized farms if:

• Your company policy mandates that you use virtualization to optimize data center space and leasing costs.
• Your organization has sufficient IT staffing, and those IT staff members have the skills and experience required to manage a virtualization environment.
• A physical environment can't be justified because of financial or capacity constraints.
• Your organization wants to establish quality assurance (QA) and development environments. 

You shouldn't use virtualized farms if:

• Your company policy mandates that you can't use a virtualization environment.
• Your organization doesn't have sufficient IT staffing to manage a virtualization environment or the IT staff members don't have the necessary skills and experience.
• The virtualization environment can't support the capacity requirements.
• The virtualization environment wouldn't be robust enough.

If you decide to use a virtualized environment, the following criteria need to be met:

• There needs to be a governing body that's directly accountable for the virtualization solution and empowered to manage it effectively (e.g., capacity planning, monitoring).
• There needs to be a formal process for assessing the viability of the virtualization solution. Specifically, you need a process for assessing workload and fit, based on the solution's capabilities and proven performance. You need to make sure that it has sufficient networking capacity (e.g., bandwidth and switch capacity), storage capacity, and I/O operations per second (IOPS). You also need to make sure that it has elasticity built into it to avoid complex capacity and sizing gating processes.
• There needs to be a formal process for tracking performance and service availability metrics on a daily basis. The metrics need to be published in reports.
• There needs to be a technical design and operational plan for the virtualization solution. This information needs to be published in a readily available document.
• The virtualization solution's costs need to be published and well explained along with the benefits of the solution from a financial perspective.

To virtualize a SharePoint farm, you can use Microsoft's virtualization products or other company's virtualization products (e.g., VMware products). The TechNet article "Use best practice configurations for the SharePoint 2013 virtual machines and Hyper-V environment" describes the configuration options for Hyper-V and those options' potential impact on the performance of the virtualization host computer, the VMs, and the SharePoint 2013 farm.

Once your virtualization solution is designed, it's crucial that you test it for capacity, performance, and resilience. If you don't, you could end up trying to troubleshoot performance problems in production.

If you need help determining what to monitor during this testing, you can use the free Performance Analysis of Logs (PAL) tool, which you can download from CodePlex. This powerful tool analyzes Performance Monitor log files using known thresholds.

You can also Visual Studio 2013 to load test various aspects of a SharePoint farm, such as its page load and the time it takes to upload documents, download documents, and get search results. The MSDN Create and run a load test web page discusses how to use Visual Studio 2013 to perform load tests.

The Pros and Cons of Using SharePoint Online

SharePoint Online, which is part of Microsoft Office 365, is a hosted service that can help organizations adopt SharePoint when an on-premises solution isn't viable or an externally hosted solution just makes sense.

Once you get past the hype of the "cloud," hosted solutions can make a lot of sense for organizations that don't have the capital budget for SharePoint and its dependent technologies. (Note that SharePoint Online mainly reduces the upfront capital costs. The operational costs for SharePoint Online are often higher than the operational costs for on-premises solutions, although this varies widely based on a lot of variables.)

SharePoint Online can also make sense for organizations that want to establish an offering that enables staff to collaborate on the Internet but can't leverage their existing environment due to capacity, security, costs, or compliance requirements.

With SharePoint Online, you can deploy a SharePoint offering quickly and with minimal effort. Specifically, there are collocated virtualized Active Directory (AD) servers that connect your environment with Microsoft for authentication, profile import, and other services.

You might want to use SharePoint Online if:

• Elasticity is needed to meet demand for capacity and features.
• Your organization's capital budget is insufficient to establish its own physical or virtual SharePoint environment.
• Your organization needs an agile solution to meet business demand and wants to avoid additional capital acquisitions, builds, and operational staffing.
• Your organization's track record for designing, deploying, and operating complex environments such as SharePoint is poor.
• Your organization doesn't have sufficient data center space. 

You shouldn't use SharePoint Online if:

• Your organization's capital and operational budgets are sufficient to establish its own physical or virtual SharePoint environment.
• Your organization's security and FOIP requirements don't permit you to use hosted solutions.
• The risks and costs of using a hosted solution are less attractive than those for an on-premises solution.

For more information about SharePoint Online, see the SharePoint Online Service Description web page.

Proving Your Architecture

Now that you have designed your future environment on paper, it's time for you to test concepts and prove the design with a working model.

But first you need to check to see whether your organization has a policy regarding proof of concepts (POCs), pilots, and solution certification. If such a policy exists, you need to research it.

Here's a general approach you can follow to prove your design, align stakeholders, and mitigate risks:

Perform POC testing with a limited and controlled audience. Why perform POC testing? The main reason is that it lets you test functionality and performance, catching problems sooner rather than later. You don't want to release a solution to a broad audience and experience functional or performance problems. POC testing is also beneficial because it:

• Makes sure that the solution's operational capabilities are in place. Not having them in place can lead to unmet expectations and complaints.
• Lets the IT staff members work with SharePoint technologies that they might have limited experience with. They'll be able to develop or hone their operational skills in a controlled environment. You must get them involved early in the process to pave the way for a successful transition of the final solution to the production environment.
• Lets the IT staff members test and work with customizations that aren't proven or customizations with which they have limited experience.
• Serves as the first pass at certifying the solution for production.

POC testing needs to be performed in an environment separate from production.

Perform pilot testing. Once a design has proven itself in POC testing, you need to perform pilot testing using a limited and controlled audience. With pilot testing, you're testing a refined solution. You perform pilot testing for the same reasons you perform POC testing. In addition, pilot testing lets you:

• Scale up the solution to make sure it performs as it should in production environment.
• Test the build process to ensure repeatability quality.
• Ramp up the IT staff members' experience with the solution in a production environment.
• Certify that the solution will meet your organization's requirements before the organization invests in a full deployment. 

For the pilot testing, you need to:

• Test the build process in a production environment.
• Test the operational processes in a production environment.
• Test connectivity to products and services (e.g., virus scanning, backups) 
   

The Final Step

After performing the POC and pilot tests, the next step is full deployment of the solution. If the POC and pilot tests were performed properly and the results were well documented, your build process in production should be much easier than if you hadn't done the testing.

I'm not implying your installation will be hassle free, but there should be fewer surprises related to the solution and its operational readiness. Generally, the final build uncovers environmental problems, such as problems with security (e.g., service accounts, Service Principal Names—SPNs), hardware availability, and data center readiness (e.g., space, power).

Generally, organizations have a checklist for transitioning the architecture (the end solution) to the IT staff members after the build is complete.

This checklist includes transferring project documents (e.g., requirements, designs, test reports), providing training, functional and performance testing, operational toolset testing, and providing administration IDs.

If the architecture design process is well executed, the IT staff will be involved much earlier in the process. In other words, the end solution won't be thrown over the fence to the IT staff members, who must then fend for themselves. I witnessed one case in which a problem-ridden solution was forced on the IT staff. It took three years and millions of dollars to stabilize the solution.

Advice from the Trenches

"Do your homework" is sage advice when you're designing a SharePoint architecture. Here's some more advice from someone who has worked the trenches:

• Don't take on a project like this if you're new to the organization. If you must take on such a project and you're new to the company, make sure you get a coach who can help you both from an organizational and technical perspective.
• Make sure that you have an executive sponsor who can ensure budgetary support and cooperation among the stakeholders and third parties. (For more information about executive sponsors, see "Architecting SharePoint: How to Create a Team to Design SharePoint Farms.")
• Watch out for fiefdoms, the people who lead them, and their minions. When you do encounter them, work with your sponsor and trust your gut.
• Get to know and work with the technology managers for all the areas involved in the SharePoint architecture project (e.g., AD, servers, storage, networking).
• Develop trusted relationships. Know who you can trust. Working in large organizations for a long time takes its toll on people—expect weird and territorial behavior.
• Get product training and build a lab so you have hands-on experience. Your goal is to understand the product so you can be a great architect. Read all you can and experiment in a multi-server lab. (For information about building a lab, see "Architecting SharePoint: Building a Solid Foundation for SharePoint Farms.")
• Leverage the SharePoint community. There are several sites as well as discussion groups on social networks such as LinkedIn.
• Leverage Microsoft's free SharePoint training.