Review: SmartBear Software's TestComplete 8 Enterprise

Rating: 4 out of 5 stars
Website: http://www.automatedqa.com/products/testcomplete
Price: $1,999

I have been a proponent of AutomatedQA's TestComplete product for years. DevProConnections readers may recall that the earlier edition of TestComplete won an honorable mention in the 2009 Readers' Choice Award. With the passage of time, the product has continued to improve with features and capabilities that keep it current for testing the latest generation of Microsoft technologies.

Now a member of the SmartBear Software family, AutomatedQA's TestComplete 8 has expanded beyond Win32 and .NET to support a bevy of Windows-hosted programming technologies. Considering the companies that are now stewards of the related technologies, SmartBear's stewardship of TestComplete is simply a sign of the times.

Key Differentiation
Several features make TestComplete 8 stand out among the crowded arena of both free and commercial test suites. Perhaps the most notable is its adoption of Keyword Testing. Instead of requiring you to learn an arcane scripting language that is sometimes more complex than the language of the original application being tested, TestComplete helps testers and developers assemble tests in a series of checkpoints that are graphically displayed in a step-by-step logical hierarchal tree. Scripts can be visually recorded to aid with UI as well as business logic testing. Keywords can introspectively examine the states of the application's various properties, methods, and events via "white-box testing".

This deeper analysis is available to both 32-bit and 64-bit versions of Windows running .NET, WPF, ActiveX, Visual C++, Java, web pages (via Internet Explorer 8 and Firefox 3.6), Nokia Qt, Embarcadero's Delphi and C++ Builder, and Microsoft Office applications that use the IAccessible interface. Adobe Flex applications are limited to just the public properties. And for those who prefer scripts over visual keyword interfaces, keyword tests can be exported to scripts for such needs as incorporation into continuous testing frameworks.

Another attractive aspect of TestComplete is the full catalog of test types it offers, including the expected functional, unit, and regression tests as well as data-drive, keyword-driven, and white-box tests. Additionally, scalability, load, and stress tests can be combined with distributed tests to measure the performance of an application designed for high-volume performance. Coverage tests can be constructed with SmartBear's AQtime profiling tool for extensive code analysis. Finally, manual tests can be constructed for those scenarios that can't be automated. The manual test instructions can be imported from Microsoft Word and Excel documents, databases, and text files and can be exported to Word and HTML format for internal team or client review.

More Features
In addition to TestComplete's ability to record and play back test scenarios (which, by the way, focus solely on the task at hand and will not be interrupted by other system-processing events such as email alerts, IM pop-ups, and other distractions that could halt execution of other less-intelligent testing utilities), it also integrates with source-control systems such as CVS and Microsoft Visual SourceSafe, allowing tests to accompany source code and be versioned accordingly. And for teams that have standardized on Microsoft Visual Studio Team System, TestComplete can hook directly into the team build process with its results published to Microsoft Team Foundation. This avoids having to jump through hoops and trying to shoehorn the system into playing nice with an established team development environment. It just works.

Most Notable
Some of my favorite aspects of the TestComplete 8 release are its ability to easily edit and reorder the keyword tests. If I prefer, I don't have to plow through reams of lines of code to make a simple parameter or value change. And if I do need to dive into the test scripts for a granular degree of customization, I can do so in the five scripting syntaxes supported by the product (VBScript, Jscript, and DelphiScript rank high on my list). The script editor is a full-featured IDE, complete with code snippets, templates, breakpoints, and reusable function libraries along with parameter tool tips and context-sensitive help with plenty of copy-and-paste code samples.

Another cool feature is the ability to alias a control that has been given the time-pressured and/or sloppy name of ListBox1. Should a test fail with this control name, finding it, let alone associating it with a meaningful operation, could be a test of one's anger management skills. Fortunately, TestComplete 8 allows such ambiguous names to be renamed in the test scripts without refactoring the original application source code. Testers can get their work done and harangue the developers later for their naming convention shortcomings.



Database connectivity is also a breeze; as long as there is an ODBC entry, pulling data into TestComplete is as easy as point and click. About the only thing lacking is an explicit wizard for working with LINQ-oriented interactions. It would be really cool if such a wizard existed for Parallel LINQ (i.e., P-LINQ) data traversals. Perhaps such a feature will exist in a future release.



What's Missing?
The most obvious Microsoft technology not yet supported in the product is Silverlight 4, though the SmartBear/AutomatedQA folks tell me this should be in the next point release. Additionally, Adobe AIR and updated support for Adobe Flex applications will be included, as well as some load- and performance-testing enhancements.

A Sophisticated Testing Package
TestComplete 8 is a culmination of refined testing expertise, codified in a package that greatly eases the burden of developers and testers to construct, execute, and identify application failures before it's too late and costly to adequately do so. The product is easy enough for anyone with the most basic of application testing experience to use and sophisticated enough to serve the seasoned professional.

Numerous screencasts, some more than one hour long, serve as an effective advertisement as well as helpful online training for those evaluating or already using the product.

TestComplete 8's pricing—$1,999 USD for a named license and $4,499 USD for a floating license—may seem like a major investment, and it is. But considering that the costs involved in repairing program flaws not caught by the extensive tests that TestComplete has to offer may be far more significant, it's an investment worthy of serious consideration.


Mike Riley ( [email protected]) is an advanced computing professional specializing in emerging technologies and new development trends. He is also a contributing editor for DevProConnections. Follow Mike on Twitter @mriley.