|
Executive Summary: Learn more about the behind-the-scenes work Microsoft's huge Windows Server 2008 team has done to make this OS strong and stable, as our Microsoft insider-on-the-outside, Paul Thurrott, interviews the Windows Server 2008 project managaer, Alex Hinrichs. |
Last month, I presented part one of an exclusive, behind-the-scenes look at the creation of Windows Server 2008 with Alex Hinrichs, the Windows Server 2008 project manager (see “What You Need to Know About How Windows Server 2008 Developed,” December 2007, InstantDoc ID 97400). Hinrichs runs the Windows Server ship room and manages the development of this increasingly complex product line. This month, I continue my interview with Hinrichs. Here, then, is more of what you need to know about the development of Server 2008.
Stepping Into the Ship Room
“It’s real simple,” Hinrichs told me. “You walk in each morning and find out what is the state of the main build. We build it overnight. Did the build break any of the BVTs \[build verification tests\] or the thousand or so self-host tests we run? We look at the state or health of the daily build.” Build breaks are very rare, Hinrichs said. In such a case, errant code from further down the tree can break dependencies or functionality in the main build, causing a temporary build cessation while the suspect code is found. Although this calamity occurred many times during the development of Windows Server 2003, it’s only happened once or twice to Server 2008.
“The day-to-day focus of \[the ship room\] until fall \[2006\] was Windows Vista,” Hinrichs said. “Here on Server, we kind of rode along for the ride. But once Vista was done, there was a changing of the guard almost immediately, and I took over ship room.” Now, he said, Server 2008 and Vista SP1, which are being developed in tandem, are the focus. “Starting from Vista RTM \[release to manufacturing, in early November 2006\], we flipped the switch pretty quick,” he added. “It was like two aircraft carriers passing each other. But working from the Vista RTM code gave us a very stable code base, so we were able to go full bore from the get-go. We have been able to create a stable main build every day since then, since most of the problems are trapped before they even get here.”
Locking Down the Code
In keeping with their habits of learning from the past, the Windows Server team carried over another bit of wisdom from their experiences on Windows 2003: They locked down the code for this product fairly early in development. “We locked down the feature list at end of 2006,” Hinrichs said, noting that a change-management process was then set up so that anyone who wanted to make a change request would have a formal process to follow. “We have a board consisting of me, Iain \[McDonald\], and Bill Lang \[Iain’s boss\],” Hinrichs told me. “When someone wants to make a functional change to the product, they meet with us. We weigh the consequences of those changes.”
Most changes aren’t accepted, as the Server team is leery of feature creep, which it feels created problems on early Windows Server versions. Now, developers can’t just check in new code on a whim: Microsoft has quality gates in place, up and down the code tree, to monitor what’s coming in. “We worked hard to maintain the feature list we finalized and have only taken in changes that were either blocking deployments or because the business case was so compelling we simply had to,” Hinrichs confided.
This raises an obvious question: Which features were added after the late 2006 functional freeze and which were not added? Hinrichs wasn’t particularly keen on discussing what wasn’t added to Server 2008, for obvious reasons, but one change the team made does stand out: Very late in the development of Server 2008, the team decided that it simply had to add Windows PowerShell to the product, even though the existence of two scripting and command-line environments, one of which (PowerShell) wouldn’t be fully supported with built-in management tools for all of the product’s capabilities, would be confusing to some customers.
“This is precisely the type of thing we reject,” Hinrichs admitted. “You don’t want to randomly add some new administrative tool to the product so late in the game, and there’s no way to create the necessary PowerShell scripts after Beta 3 \[which was when PowerShell was added to Windows Server\]. We already had command-line tools and had significantly improved the traditional command-line environment in Windows Server 2008. So it was a tradeoff. We’re committed to PowerShell, and love it, and we wanted it in the box. But we didn’t want to slip the release date just to add some administrative scripts for PowerShell. We’re not going to add gravy.”
On the flipside, after Microsoft revealed the Server Core installation option for Server 2008, its customers immediately began providing the company with feedback about other roles they wanted to see added. The number one requested role was Web Server. But because Server Core doesn’t support the Microsoft .NET Framework in this release, there was no way to create a version of the Microsoft IIS Web server for that environment that supported dynamic capabilities such as ASP.NET. The Windows Server team decided instead that it would support a Web Server role in Server Core and that Server Core version of IIS won’t support any .NET-oriented features in the first release. The result is a low-impact version of IIS that’s super secure. Customers were ecstatic, as what they were looking for in this space was a low-end Web server solution that could compete more effectively with Linux-based solutions.
“The feedback from Beta was pretty clear,” Hinrichs said. “Customers told us, ‘Wow, this Server Core thing is fantastic, I love it, but you missed the most important role.’ So we went and took care of that.”
Adding IIS to Server Core was a big undertaking but worth the effort because IIS impacted so many customer deployments. PowerShell, by comparison, is something the company is moving toward supporting more pervasively across Windows Server but felt it wasn’t absolutely critical to have a complete set of admin scripts in the initial release (they can ship after the fact via the Web). It would be nice, but it’s not critical. It’s worth noting that Microsoft will indeed support PowerShell with a full suite of administrative scripts via a Webbased library concurrently with the release of Server 2008. (Hinrichs noted that an amazing community of PowerShell users has already sprung up; this group, too, will no doubt supply users with useful and compelling scripts for that environment.)
Working with Customers
One of the most amazing changes in the manner that Server 2008 was developed is the way that the team has integrated customer feedback so thoroughly into the process. “The old way of doing this was that customers would file beta bugs, and then we’d have call downs and onsite customer visits,” Hinrichs told me. “We still do those things, of course. But on top of that, we can now process CEIP \[customer experience improvement program\] data, which is information that comes back from the servers that are installed around the world. This information, which is strictly opt-in, is pumped back to Microsoft so we can study it.”
It’s a considerable amount of data. Microsoft received CEIP data from over 165,000 installations of Windows Server 2008 Beta 3, which shipped in late April 2007. The company knows how many customers have installed Beta 3 and even exactly what roles they installed. And yes, I can tell you what those are: Percentage-wise, the most popular Server 2008 roles are Active Directory (AD), Web Server, File Server, and Terminal Services.
Server 2008 also supports the notion of features, which are functional components that aren’t as broadly defined as roles. The most popular features so far are PowerShell and, surprisingly, Desktop Experience, the latter of which allows the Server 2008 desktop to look more like Vista. Hinrichs told me that people had complained about the feature in Windows 2003, so the team carved it off of the default installation so that those who wanted it could install it separately. These people are typically enthusiasts, he said.
Continue to next page
The Windows Server team also sees which combinations of roles are getting installed on the same server and which architecture (x86 or x64) people are installing. “We know that x64 is what all customers are buying now and in the future,” Hinrichs said. “The deployments are all x64. There is some testing on 32-bit \[x86\], but we also know that many of these are in virtual machines \[which tend to be 32-bit only\]. So we put our testing and development wood on x64. The CEIP really helps us know that we have the right focus and priorities.”
Customer feedback also helped Microsoft determine the feature set for Server 2008. “That was actually the origin of Server Core,” Hinrichs told me. “Customers were saying that there are all these services running they don’t need, all these extra components. They were installing QFEs \[hot fixes\] that had nothing to do with server roles they’re running. That was the genesis of componentization: Cut the dependencies and arrive at a smaller, more pared-down version of the OS. It runs lean and mean and doesn’t need a GUI, Internet Explorer, or the .NET Framework.” That said, Microsoft is also evaluating changing the supported component mix in Server Core for the future based on customer feedback.
Read-Only Domain Controller (RODC) is another example of a Server 2008 feature that arose out of customer feedback, this time from enterprise customers. “They said they loved Active Directory but had branch offices all over the world,” Hinrichs said. “Replicating across the WAN was painful. Maybe they had a head office in New York and an oilrig in Kazakhstan. They don’t want all of their passwords replicated to that remote location. Read-Only Domain Controller caches passwords only for the people who log on in that location. They’re not giving up the goods if they get compromised.”
Calm Before the Storm
As a long-time Microsoft observer, I noted to Hinrichs that the Server 2008 development process, although lengthy, seemed to lack the drama and disappointments that marred Vista. He wasn’t particularly interested in making direct comparisons with the Windows client team, but he did tell me that the calmness exuded by the server team is a direct result of the quality and maturity of the team they have in place.
“It may seem calm on the outside,” he said, “but that’s because we have a lot of senior people on Windows Server, including some folks who shipped many versions of Server over time. They are very effective at planning and managing their businesses, and they deliver what they promise. The seniority factor really helps.”
“The other thing is that Server people are Server people,” he added. “We just love working on Server. We haven’t had a hard time retaining people at all.”
Hinrichs also credits the team’s clear focus on server roles as a factor in the clarity of the product’s development. “This isn’t BS,” he said. “We get a lot of steadiness from Bob \[Muglia\], Bill \[Lang\], and Iain \[McDonald\]. They’re steady, and their world is steady. We go to ship room, we set milestones, and we figure out how to work with the teams. We tried hard to be consistent, and maintain a consistent rhythm.”
Part of this rhythm involves some of the little things that other organizations simply don’t get. For example, employees aren’t asked to work most weekends. “We tried things that way, once,” he said. “It didn’t work.”
The result is indeed a smoother running organization. When Microsoft shipped Server 2008 to almost 1 million people, with 1,000 external real-world deployments and 600 internal deployments, the Windows Server team sat back and wondered whether the complaints would come pouring in. It never happened. “The builds are just so solid,” Hinrichs told me. “There’s just nothing major wrong. We haven’t had a million customer problems.”
Hinrichs told me that the executive in charge of the Technology Adoption Program (TAP) is on call with all of the Fortune 500 companies that are deploying pre-release Server 2008 versions in production. They have his cell phone number and can call him 24 x 7,” he said. “Remotely or via a plane trip to visit them on-site, he will fix whatever problems they’re having. He’s only been woken up once in the last six months. That’s it. That was not the case with previous Server releases.”
One area where Server 2008 has, perhaps, veered a bit off course, however, is with Windows Server Virtualization technologies, code-named Viridian. This feature is now due in public beta form when Server 2008 ships in first quarter 2008 and will appear in final form within 180 days of that date. Hinrichs, curiously, wasn’t interested in taking the easy out on this one, even though Viridian was actually being developed outside of the Windows Server team for much of its existence and was only recently pulled into the core OS team.
“It was a parallel project, but we’re just glad we have a CTP \[Community Technology Preview, which appeared in Release Candidate 0\] so that folks can have an early look,” he said. “Some parts of Windows Server 2008 have been stable for a very long time, like IIS, which began a Go Live program before Beta 3. Active Directory is the same thing.”
Final Thoughts
Although many have noted that the gestation time for Server 2008 was lengthy even by Microsoft standards, the resulting product appears to address customer needs and achieve a level of stability and reliability that has thus far largely escaped the Windows client product. It seems that the extra time has been well spent—though Server 2008 will arrive in the market five years after the previous major Windows Server release, it’s coming with an array of customer-driven functionality such as Server Core, RODC, a more modular IIS 7.0, and integrated virtualization support. Microsoft often talks up its interactions with customers, but in this case, this partnership appears to have worked wonders with the development of Server 2008.
