Riding the bleeding edge of technology is a typical part of a Microsoft IT administrator's job. Although many Exchange admins will likely migrate gradually to Exchange Server 2007 as early adopters work out the product's kinks, at Microsoft the Exchange 2007 production rollout to 70,000 users is in full swing. The aggressive rollout is backed by more than 18 months of testing by both the Exchange product team and IT, in keeping with Microsoft's "dogfooding" philosophy of deploying its own products in house before releasing them to the public (for more information, see the Web-exclusive article "Eating Its Own Dog Food," March 2005, InstantDoc ID 45597). Derek Ingalls, general manager, and other members of the Exchange IT staff spent all of 2006 and much of 2005 intensively engaged in testing Exchange 2007 on a 5,500-mailbox Exchange server dedicated to dogfooding. I spoke with Derek about how his team made the shift from Exchange Server 2003 to Exchange 2007 and how their dogfooding experiences can guide other IT pros on the path to Exchange 2007.
What overall process did you follow in dogfooding exchange
2007?
This upgrade was much different for us than previous Exchange upgrades. For
us, Exchange 2003 was a lot like a service pack upgrade. We upgraded the entire
environment in a weekend. But the Exchange 2007 upgrade process was quite a
cycle. From the time we built the first server and put the first production
mailboxes on it, we had milestones all along the way. As you know, Exchange
2007 consists of server roles, and not all the roles were done initially. Our
first milestone was having the first Client Access server, then production mailboxes
on a Mailbox server, then the first Edge server, and so on.
What was the most difficult part of the transition to
exchange 2007?
We had some significant battles about storage. Because Exchange 2007 isn't as
disk-I/O intensive as previous versions, you can use larger, slower, and less-costly
disks for storage. And because high-availability features in Exchange 2007,
such as Cluster Continuous Replication (CCR), mean you aren't so dependent on
a single piece of storage, you can consider using less-reliable storage. When
we were on Exchange 2003, we used a clustered SAN and had four nines of absolute
availability. The notable exceptions to this availability were always related
to storage outages.
The Exchange team took that problem to heart and wanted to reduce \[Exchange users'\] dependency on SANs specifically. So understanding that customers will want to deploy a number of different storage scenarios in Exchange 2007, they needed us to validate the "crazier" scenarios—such as using DAS or Serial Attached SCSI (SAS).
But microsoft isn't saying that Das is the preferred
storage medium for exchange 2007—just that it's an option, right?
Right. There was some controversy in IT about switching from a SAN to the cheaper
disks because of our users' expectations of high availability. But because we
decided to increase mailbox size from 200MB to 2GB, for compliance as well as
dogfooding reasons, we needed to back up 10 times the amount of data that we
were backing up in Exchange 2003. Ironically, in the middle of this controversy,
we lost a SAN array with 8,000 users on it. It completely died, and we had to
reinstall Exchange and recover 800 200MB mailboxes. That was an exceptionally
painful exercise. And the Exchange development team guys told us, "If you'd
been using CCR, it would have been a two-minute outage, not a two-day outage."
They were right. That's really what helped us turn the corner and embrace this
scenario. It was an unfortunate event but exactly what we needed.
Another challenge we had was that when we switched to 2GB mailboxes, we also told users, "no more PSTs." That decision caused probably the biggest backlash that we've ever seen. But when we offered to let the users who wanted PSTs out of the pilot program, they said, "no way—I love my 2GB mailbox!" We use the new records management features to set limits and actions to take on different folders, depending on the classification of the mail—for example, whether it needs to be archived or deleted after a certain period of time. With 2GB, users have essentially the experience of a bottomless mailbox.
During the course of the dogfooding, what exchange 2007
features did iT staff most appreciate?
The advances with Windows PowerShell (aka Exchange Management Shell) and the
ability to do so much via the command line were a huge win for us from an administrative
perspective. When we heard about Monad (the prerelease version of PowerShell)
initially, everybody in IT thought that this new way of doing administration
would be a headache. But after we started using PowerShell, we absolutely loved
it.
I think PowerShell represents tremendous potential for the user community because it provides a consistent way of doing things. I think there will be a community of sharing scripts, cmdlets, and knowledge, so that people won't have to reinvent processes for automated ways of moving mailboxes or certain sorts of tasks.
What's an example of a problem that would be easier for
you to identify using Powershell?
The state of services and databases is one. Another example: One of the more
problematic roles for us, because it was brand new, was the Edge server role.
When troubleshooting issues with the Edge server, we found that our traditional
methods—using Performance Monitor or Queue Viewer through Exchange System
Manager (ESM)—were less accurate than we would have hoped. But we were
able to find the answers through PowerShell; it gave us a more granular view
of what was happening on the Edge server.
What's your advice to your iT peers in other organizations
to help them get over their fear of Powershell?
Actually, all I think they need to do is take a look at it. After less than
30 minutes of playing with PowerShell, our IT folks were saying, "Wow, this
is incredibly powerful and extremely beneficial."
