As I undertake this discussion of testing Windows Azure applications, I realize that you might be brand-new to Azure. Additionally, you might be unfamiliar with the concept of unit testing and the related notions of Inversion of Control (IoC), dependency injection, and test-driven development (TDD). When you combine all these tools, the volume of information can prove overwhelming.
In this article, then, my aim is to provide a clear overview of the various testing technologies (unit testing frameworks, mocking frameworks, and automated unit-test generation tools). To this end, I'll focus on how you can leverage the tools to test your Azure applications and how to apply various testing technologies in a practical way that does not require you to refactor any Azure code that you may have already written (or inherited). At the same time, I'll try to provide a good road map for improving the testability of your projects through better design, wherever possible.
The main reason to consider unit and integration testing of your Azure projects is to save time. Read through the following list of rationales for testing, and I'm sure you'll agree that any small investment in testing can pay big dividends in time savings:
- to prevent situations in which you have to wait for deployments to Azure or for the cloud emulator to fire up
- to avoid simple mistakes and regression failures
- to prevent troubleshooting of problems related to temporary environmental issues (such as network outages)
- to generate builds that can quickly verify continued functionality without impeding the code-build-debug cycle
- to create an environment in which you can quickly test newly added functionality
Over time, building your suite of unit tests will help you avoid the accumulation of technical debt.
Unit Test Types
I'll focus primarily on unit tests and the robust relative integration tests. Let's get some basic terminology out of the way before we dive in.
Unit tests are narrowly focused and are designed to exercise one specific bit of functionality. A unit test is commonly referred to as the Code Under Test (CUT). Any dependencies taken by the CUT are abstracted away either by hard-coding values (i.e., providing stub implementations) or by dynamically substituting production implementations with "mock" implementations from the test. As an example of the latter process, a unit test of a business layer can examine the validation logic for creating a new domain entity by simulating the actual interaction of the entity with the database through a Data Access Layer (DAL).
Integration tests are broader tests that, by their nature, test multiple bits of functionality at the same time. In many cases, integration tests are akin to unit tests for which the stubs use production classes. For example, an integration test might create the domain entity, write this entity to a real database, and verify that the correct result was written.
Azure Applications from a Dependency Perspective
When you consider all the aspects of unit tests, it should become clear that we must look at Azure applications from the perspective of their components and the dependencies that they take. Azure applications can consist of web roles plus their related websites, web services, and RoleEntryPoint code. They can also contain worker roles and the RoleEntryPoint code that defines the worker logic.
For Azure applications, the most common dependencies are those to the data sources, such as Windows Azure Tables, BLOBs, queues, AppFabric Service Bus Durable Message Buffers, Microsoft SQL Azure, cloud drives, AppFabric Access Control Service, and various third-party services. When you build tests for Azure applications, you want to replace these dependencies so that your tests can focus more narrowly on exercising the desired logic.
You typically employ the following items to test your Azure applications:
- a unit testing framework to define and run the tests
- a mocking framework to help you isolate dependencies and build narrowly scoped unit tests
- tools that provide increased code coverage to aid automatic unit test generation
- other frameworks that can help you use testable designs by leveraging dependency injection and applying the IoC pattern
I'll touch on all these items, but I'll pay particular attention to the unit testing and mocking frameworks.
Selecting a Unit Testing Framework
Unit testing frameworks assist you as you work with coding, managing, executing, and analyzing unit tests. You definitely don't want to build one of these yourself, as plenty of good frameworks are already available. We'll focus on the Visual Studio Unit Testing Framework (aka MSTest), because it's included in Visual Studio 2010, feels familiar, and does a good job. Other popular .NET unit testing frameworks include NUnit, xUnit, MbUnit, and MSpec.
MSTest is included with all versions of Visual Studio except the Express and Test Professional editions. In general, you create MSTest projects by first adding a new project to your solution. Within that test project, you then add classes that are adorned with MSTest attributes. These attributes indicate the classes that are used for testing and the methods within them that are test methods and that can be executed by MSTest from within Visual Studio. Various windows in Visual Studio let you execute unit tests that are defined within the project and review the results after you execute the tests.
In MSTest, unit test method bodies are commonly authored by using an AAA pattern: Arrange, Act, and Assert. In the Arrange phase, you build any objects and configurations that are required by the CUT. In the Act phase, you perform the actual, narrowly scoped test on the code. In the Assert phase, you verify that the outcome is what you want. There are alternatives to the AAA pattern, such as Record/Replay, but I won't discuss those here, as they are beyond the scope of this article.
The Premium and Ultimate editions of Visual Studio include enhanced unit test tools that integrate with MSTest. These tools let you analyze the amount of code that is exercised by your unit tests (aka code coverage) and visualize the result by adding color coding to the source code.
Selecting a Mocking Framework
A mocking framework helps you remove dependencies from your CUT. This enables the test to focus on only that logic while predictably controlling the behavior and status of any dependencies. For example, if a portion of your CUT includes logic to read records from SQL Azure, you don't want your CUT to test SQL Azure itself; you simply want to test how your logic handles the records it receives or any errors that are thrown. Mocks let you substitute this logic, and mocking frameworks are what make this possible.
By employing unit testing, you can test in isolation only the functionality of interest. However, you will often have code under test that cannot be tested in isolation because it was not written with testing in mind. To test such code in isolation would require refactoring it. Also, the code might rely on other libraries that are not easily isolated themselves, such as those that interact with external environments. A mocking framework helps you isolate both types of dependencies without having to maintain cumbersome parallel libraries of your code that you would end up using only for testing. Figure 1 shows a good sampling of the mocking frameworks that are currently available.
| Mocking framework |
|---|
| Moles |
| Moq |
| NMock3 |
| EasyMock.NET |
| NSubstitute |
| FakeltEasy |
| TypeMock Isolator (paid) |
| JustMock (paid) |
Which framework should you choose? The main difference between most mocking frameworks is the syntax that's used to create or configure the mocks in your unit test code and also the ability of the frameworks to substitute your logic on members of the mocked type. Most mocking frameworks create mock types on the fly by deriving from the type that you indicate for the members that you want to modify. A few mocking frameworks have support for handling sealed classes, non-virtual methods, static members, delegates, or events that require alternative techniques (such as leveraging the .NET Profiling API). It's important that you choose a mocking framework that includes these features because you'll likely require such features if you try to mock anything in the CLR.
Among mocking frameworks, Typemock, Telerik JustMock, and Microsoft Research Moles all provide forms of these advanced features. I'll use JustMock for my examples in this article. Note that although Typemock and JustMock offer free versions, only the commercial versions include the advanced features. Moles provides these features for the great price of free. In addition, all three products integrate with Visual Studio 2010.
Testing Web Role Websites
With the background of the process in place, let's take a look at actually building tests for an ASP.NET MVC 3 website hosted in an Azure web role. Testing MVC 3 at its core is about testing the controller actions because those actions describe the website's behavior. This process is less about performing web UI tests (and, therefore, less about examining output HTML) because you can verify which view is returned, what data the view will get (by examining the view data dictionary), and any redirects to other controllers or views that occur.
From a high level, this amounts to defining controller unit tests. You typically define one test method per verb (get, post, and so on.) per action (LogOn, LogOff). When the test is finished, the folder structure of your test project will mirror the MVC website by having a Controllers folder, a Models folder, and so on.
For example, assume that you have a website that accesses a back-end book catalog stored in Windows Azure Tables. Your goal is to provide a Create, Read, Update, Delete (CRUD) interface similar to the one shown in Figure 2.
Figure 2: Sample web role-hosted MVC UI to test
This interface is driven by the Index controller action defined in the HomeController. This action responds to get requests by querying the Azure table for the list of books, then passing that list of books to a view that renders it as a table. Figure 3 shows the implementation of this action.
In this example, the Index controller action makes direct queries to the Azure Table service. As a result, the action takes dependencies on the service configuration for the connection string that's accessed via RoleEnvironment. RoleEnvironment is the CloudStorageAccount class for parsing a valid connection string into the endpoint and credentials provided to the TableDataServiceContext-derived BooksTableServiceContext. The BooksTableServiceContext is what is actually queried to get the list of books.
When you create an MVC project, the dialog box offers you the option of automatically adding a test project. Accepting this option creates a test class that has test methods for the HomeController. This resembles the project shown in Figure 4.
When it runs, this auto-generated code fails and throws an exception. This is because the service configuration is not available. So how can we remove this dependency on the external configuration? As you can see from the comments in Figure 3, our goal is to build a unit test that can successfully return a list of books without forcing us to resort to the Cloud Emulator or to make calls to the real Azure Table service. I'll take this one step further and demonstrate how to build a unit test that defines a successful outcome as having the expected number of books in the list passed to the view. Figure 5 shows the result of creating mocks for these tests by using Telerik JustMock.
As we walk through this unit test phase by phase, notice that the bulk of the work is done within the Arrange phase. This is typical because unit tests isolate everything but the code under test. The mock framework lets us create mocks for the RoleEnvironment's static methods and inject our own hard-coded connection string in place of any call to the GetConfigurationSetting method that uses the StorageConnectionString parameter.
In others words, this detour applies to the code that runs within the Index controller action. Similarly, the mock framework lets us detour calls to CloudStorageAccount.Parse method for any string and, instead, return our fixed account. The code also detours access to the Books property of any instance of the BooksTableServiceContext, so that it always returns a hard-coded list created by the local private GetTestBookList method. In other words, the real Azure Table Service is never queried at all—not within the unit test and not within any of the controller actions under test.
By having this setup in place, the Act phase simply invokes the Index controller action. After the Act phase is completed, we expect the Model property of the returned ViewResult to contain the four books that are created by GetTestBookList, and show that we defined two asserts to verify this. As long as they are both true, the test will succeed.
Testing Worker Roles
The ability to create unit tests that mock dependencies extends to any code item, and worker roles are no exception. The core item to test in a worker role is defined in the implementation of RoleEntryPoint, typically in the OnStart and Run methods. So how can you build units test that employ a worker role without having to fire up the emulator? By using mocks, of course!
Let's see how this plays out in testing the Run method of a worker role. In the Books application, for example, we have a worker role that periodically deletes old books (see Figure 6). Our unit test verifies that this routine correctly deletes the two old books that are provided by the mocked Azure table. Figure 7 shows the complete implementation of the unit test.
The main challenge in testing the Run method is that this method is usually implemented as an infinite loop. Therefore, we need some deterministic way to let the expected number of iterations occur. Then, we have to stop the execution and verify the result. This is why we execute Run on a separate thread, and why we signal a ManualResetEvent when the expected number of iterations has passed. One slight complication to this test is that any mocking that's done in the Arrange phase must be performed within the new thread.
Why Mocking Makes Sense
At this point, you should have a clear understanding of how mocking within your unit tests enables you to dramatically speed up the execution time of your unit and integration testing by keeping all external data sources local and in-memory. This encourages you to run the tests more frequently and, by extension, helps you build a higher quality product. By leveraging mocks when you test your Azure applications, you gain the ability to test without having to rely on cloud services or network connectivity. And that keeps your unit test fast, lightweight, and more likely to see frequent use.
