Testing any type of mobile application can be challenging. There are thousands of potential device types and models you have to test for. Factor in different screen sizes, operating systems, browsers, and so on, and it becomes even more difficult to ensure that your tests adequately cover the experiences your users are likely to have in the real world.
But if you're testing mobile games, you face an additional set of challenges that extend above and beyond the basic complications of mobile software testing. Indeed, in many respects, mobile games are one of the most difficult types of mobile apps to test. To meet the special challenges, you need to optimize your mobile testing strategy and methodology for testing games in particular.
This article explains how to do that by walking through the top three challenges of testing mobile games, then discussing solutions for each one.
Mobile Game Testing Challenge 1: High Frame Rate
Frame rate is the frequency at which the content on a screen is updated. Frame rates are measured in terms of the number of frames that an application can display per second.
Most mobile apps don't require an especially high frame rate to deliver a good user experience. If your app hosts mostly static content like text and images, the ability to update screen content multiple times per second is not very important.
But with mobile games, this is not the case. Mobile games typically require frame rates of at least 25 frames per second to provide a highly responsive user experience and to display changes to users in real time.
From a mobile testing perspective, frame rate requirements create a challenge because most conventional testing frameworks aren't capable of representing changes to frames as quickly as the frames actually changed. With a traditional testing strategy that requires non-optimized test data to move over the network in order to display the results of the test, you'll usually achieve a frame rate of around 5 frames per second, at best. If the content inside your game actually changes at a rate of 25 frames or more per second, seeing test results at a much lower rate isn't sufficient if you want to simulate a real-world user experience accurately.
Solutions: Network optimization or local device testing
There are two ways to solve the frame rate challenge of mobile game testing.
One is to find a way to optimize network traffic so that it keeps pace with the actual rate of change within a dynamic mobile game. This can be done by gracefully reducing the quality of some of the content that is moved across the network during tests, which allows the most important traffic to flow at very low rates of latency. As a result, it can be rendered in under 35 milliseconds instead of about 170 milliseconds — a crucial difference if you need to display dozens of frames each second.
A second approach to representing changes to mobile game content in real time is to use on-premises devices for testing. That way, your test data doesn't have to flow across the network at all, eliminating the network latency issues described above.
Mobile Game Testing Challenge 2: Lack of OIDs
In most mobile testing scenarios, engineers can leverage object identifiers (OIDs) that are built into applications as a means of telling test scripts what to test.
But with mobile games, there are usually not many unique OIDs to work with. The reason why is that content in games is highly dynamic, so you can't assign standard OIDs to the elements that appear in a mobile game in the same way that you could assign OIDs to images, text and video in a traditional app.
Solution: Automated object identification
To solve this challenge, mobile game testing teams need a tool that can automatically identify and label elements in games. Then, tests can be run against those elements.
Companies such as GameDriver provide this. Through partnerships with such companies, we can provide a one-stop solution for running automated tests for mobile games that would otherwise be difficult to orchestrate, due to a lack of native OIDs.
Mobile Game Testing Challenge 3: Varying Screen Resolutions
Ensuring that software performs adequately across multiple screen resolutions is a key challenge for most mobile software tests. But with games, handling screen resolution variations is particularly crucial because in some cases content may be rendered inaccurately on a given device if the screen resolution is different from what developers expected it to be. The result is distorted graphics and characters — a major problem from the user experience perspective.
Solution: AI-driven rendering analysis
Previously, the only way to deal with this challenge was to review renderings manually. That approach, of course, is time-consuming and does not scale.
There is a different strategy. Use an AI engine that automatically compares renderings of content, including dynamic content that appears in mobile games, to detect differences across different game versions or screen resolutions. This approach makes it fast and easy to identify instances where content does not render in a consistent way during testing. From there, developers can address the issue so that users don't experience rendering problems when the game is deployed.
Conclusion: A Hands-On Approach to Mobile Game Testing
The bottom line: Conventional mobile testing strategies aren't fully suited to meet the unique requirements of mobile game testing. To ensure that mobile games actually deliver the user experience your developers intend, you need to leverage special tools and strategies — such as network traffic optimization and AI-assisted content analysis.
Frank Moyer is CTO of Kobiton.
