Your organization has decided to migrate from a Microsoft Mail (MS Mail) system to Outlook 97 or Outlook 98 and Exchange Server. You've probably developed your migration strategy, but have you thought about the effect Outlook will have on your desktop environment? Will it cause any problems with your organization's mission-critical applications? For example, every time you run Outlook, will it crash your financial system? In this article, I'll discuss why you need to formally test Outlook in a corporate environment and how to plan for testing. In a subsequent article, I'll suggest a testing method.
When a New Zealand local government authority approached me to test Outlook with its corporate applications (e.g., the organization's custom PowerBuilder applications and financial system), I set out to learn other organizations had approached this type of testing. After hours of searching, I realized that many organizations migrate from MS Mail to Exchange Server and Outlook but only glance at Outlook before implementing the migration. Instead, these organizations focus testing on migrating to Exchange Server. Although back-end testing is important to a successful migration, Outlook is the user interface. If it conflicts with your current corporate applications, it can jeopardize your whole migration.
Armed with the small amount of information I could glean, I developed a strategy to test Outlook with my corporate applications. You can apply my process to any organization to whatever depth you think is necessary. The process needs to be flexible, because testing a shrink-wrapped product with other applications is not as simple as you think.
Why Test Outlook?
The most important reason for testing Outlook with your corporate applications is to limit surprises—a good idea any time you're introducing software or hardware into a corporate environment. Conflicts with existing applications can lead to expensive downtime and loss of productivity. It's foolhardy to think that your installation will work the first time. Mainframes and minicomputers have had procedures for testing and implementing new hardware and software for decades, but the PC/LAN environment is playing catch up. You can limit your vulnerability by identifying risks, modifying your migration plan to take them into account, and working to resolve obvious problems.
Testing provides peace of mind for you, your managers, and your users. Bolstered by successful test results, they not only will be more positive about implementing a new mail system but also will be more likely to buy in to the system. Management and user backing is crucial for your migration's success.
Identify Your Testing Process
Every organization is different; a good testing approach for one organization might not suit another. You need a flexible framework that you can adapt to your organization. When you develop your process, be aware of dependencies that will affect that process. You need to work with people involved with your migration to ensure that you're testing an appropriate Exchange Server and Outlook configuration. Figure 1 shows where the testing process fits in the overall project.
Decide Which Applications to Test
Identify which applications you need to test in conjunction with Outlook You won't need to test some applications (e.g., Word and Excel) individually, because they're unlikely to cause a conflict. First, list all applications in your environment—your financial system, terminal emulation packages that connect to your UNIX server or mainframe, custom-built applications, commercial applications for specific requirements—that everyone in the organization uses.
Next, consider the effect Outlook will have on each application. For example, if the application uses your current mail software, will Outlook affect its functionality? Effects will depend on factors specific to your environment, such as
- Platform (e.g., Windows NT Server or a UNIX server)
- Application purpose (e.g., a financial, inventory, or payroll system)
- Importance to your organization
- Number of users who need access to the application to do their job
- Application type (e.g., custom-built, off-the-shelf, or modified)
- Mail or workflow integration
Grouping applications by type and platform can be useful. For example, users might use a common interface such as a specific terminal emulator to access different applications on the same server. In that case, you don't have to test each application individually. Many applications won't interface with your mail system, either directly or through workflow, but you still need to test these applications with Outlook to ensure that conflicts or performance problems won't occur.
Decide Who Will Be Involved
Testing requires the participation of people in several roles:
A testing coordinator. The coordinator manages the testing process. Coordinators determine which applications to test with Outlook, develop the test templates and the test plan, and document the tests' outcomes. Choose a neutral person as testing coordinator to ensure unbiased results.
A technical consultant. A technical person must be available during testing to handle problems that arise and to reload or configure software and thereby protect valuable testing time.
Testers. People need to try out each application. In my testing, a group of people participated in testing. Because they used the applications daily, they knew whether the application was behaving normally. Sometimes, having two or more people involved ensures more thorough testing.
The testing coordinator, with the organization's managers and the testing consultant, chooses the testers to ensure that the most appropriate people from a cross-section of the organization participate in the tests.
Determine Testing Method
Include both users and technical staff in the testing process. When you have established who will be involved with the testing, interview each person to develop test templates for defining areas of the application you need to test. A test template is similar to a test script in software development, because it provides a step-by-step list of tests to complete for each application. With the people involved in the testing, develop a test template for each application you identified for Outlook testing. Figure 2 shows a generic test template.
For this type of testing, limit the test templates to the applications' interoperability with Outlook in three broad areas: performance, compatibility, and integration.
Performance. Performance testing monitors the effect of Outlook on the desktop environment as compared with the previous mail system. This testing doesn't monitor network, server, or workstation performance. Look at memory usage and any other specific requirements your organization finds important. Performance monitoring is especially important when you're testing Outlook with resource-intensive applications.
Compatibility. Does the application function correctly after you've installed Outlook? Does the order in which you load the application and Outlook affect functionality? Do you retain the same level of functionality you had before you installed Outlook (e.g., can you still print and access CD-ROM towers and network drives)? Can you use Outlook to email files created within the application to people using other email software and to other Outlook users? Can Outlook users open files from this application when non-Outlook users send the files as attachments?
Integration. If your application uses mail as a transport mechanism (e.g., your request-tracking system can send notifications directly via email from within the application), does this capability still work? Does the workflow still function? Will Outlook's support for long attachment filenames have any effect on workflow applications?
Develop a Test Plan
After you develop test templates, develop a test plan, such as you see in Figure 3. The test plan documents in more detail all aspects of the Outlook testing.
Develop a Test Log Template
You need to document all test results uniformly. Develop a test log template that includes the following information:
- When you conducted the test
- Who did the testing
- What machine and environment they used
- Which tests they performed (e.g., performance, compatibility, integration)
- Any problems encountered during testing and their resolutions
Because the tests will vary with each application, design the log template so that you can exclude sections.
Proceed with Testing
Now that you know what you're going to do, who's going to participate, and how you're going to proceed, you're ready to start the process. In Part 2, I'll look at setting up the test environment, carrying out the testing phase, and documenting conclusions and recommendations.
