When you release an app, you must be 100% sure it's going to work smoothly and flawlessly. And, of course, it should meet the expectations of your users too! Otherwise, your project may fail, which isn't the desired outcome, right? So you need to pay due attention to app quality assurance (QA) testing.

But what is the right approach to testing your platform? Which techniques to select? Is it worth checking product performance manually or using automation? Or maybe choose both options at once?

In quality assurance (QA), testing automation is one of the greatest champions of speed. But, when it comes to functionality and first impressions, there is no substitute for manual testing. The fact is, even as automated testing software becomes more refined, manual mobile app testing still plays a huge role in shipping quality applications. 

Both categories of mobile app testing have distinct advantages and disadvantages, but it’s particularly important for mobile app development companies to practice manual testing to deliver an exceptional user experience.


Click "Learn More" to drive your apps & games business with ASO World app promotion service now.

Mobile app testing basics

In the broadest terms, there are two approaches to testing a Mobile Application. You can have humans use the application under different situations to see how it responds, or you could have an application drive the software into interesting places and look for expected results.

This second term is commonly referred to as "test automation", or, perhaps "automated testing." It might be more accurate to refer to it as "automated test execution and evaluation", or "automated checking", but the earlier terms have taken hold.

Both of these approaches can be valid in different circumstances, and that validity can change over time in different moments of an app’s lifecycle. In practice most organizations use a blend of the two, perhaps pushing some of the human explorations to customers. 

What is automated and manual testing? 

Ensuring high-quality software is a complex process, which takes a lot of time and costs some money. However, such expenses cannot be avoided (provided you want to succeed in the long run). 

It is important to draw up a competent testing strategy that would allow you to get the maximum effect with minimal effort and understand whether the site or application meets the requirements set.

Manual testing

Automated testing is the process in which pre-scripted tests are executed on an app before it is released into production. Automated testing helps you find flaws in your app quicker. Test automation can be used to run tests that are repetitive, that need to be run periodically and that can help you find bugs in the early stages. 

While the aim of manual testing is to test the so-called “user experience”, automated testing aims to test all the functionalities that characterize an app.

Sixteen years after I was first told that human testing would "go away," and nearly a decade after I first heard human explorers called "dinosaurs," manual testing remains the most common testing approach for both mobile and desktop applications. 

By manual testing, I mean a human actually using the application through the front end. There are a variety of places this manual testing might happen. Most programmers at least simulate an application, if not put it on a phone, before passing it on to someone else to test. Some companies employ testers to go deeper, looking beyond the happy path or into different models of devices. 

If the software is internal, the company may have the people who will use the software actually perform User Acceptance Testing, which is more focused on "can I do my job with this software." Some companies release the software early to "beta" testers, which might be employees, using a tool such as Microsoft AppCenter. 

Finally, companies like Applause and Testio exist to take that "beta" version and crowdsource it, providing dozens to thousands of eyeballs to look at the software, in a variety of configurations, over a short period of time.

Regardless of who how or when testing manually gives you the real feeling of how it is to actually use the application. Manual tests can see if the buttons are in the right position if they are big enough if they overlap if the colours look good together etc. Computers turn out to be particularly bad at evaluating if a picture on a screen "looks right." 

However, there are some actions computers can test for easily. For example, when you type in username and password and submit, you should go to screen that shows your name and that you have been logged in, and leaving the password wrong should yield a particular error text. 

The challenge of mobile testing is less likely to be which of the two paradigms to accept, but how much, when, and who.

Why do I need manual testing?

Manual testing provides some feedback on usability and appearance along with functionality. The tester plays the role of a user trying out everything in that application, doing typical actions a user would, to see if or when the app crashes. With manual testing, you often get feedback on performance, battery drainage or overheating early enough to fix them before release. The feedback is often "free" just because the tester was paying attention.

How many devices do I need for testing?

You could try to manually test every device supported with every supported operating system version. The last time we checked three years ago there were over 24,000 different Android devices, and we stopped counting. Realistically, most organizations test with the newest version supported, one release back, and the oldest version supported. Between Android and iOS, tablet and phone, most midsize and larger companies we work with end up with a test lab of 10-20 devices.

Automated testing

What is automated testing?

When people use the phrase Automated testing, they generally mean having a tool, like Selenium or Appium, drive the user interface of the application, checking expected results along the way. 

This checking comes from a series of commands and inspection points, that can be stored either in something like a spreadsheet, or, more likely, an actual computer program. 

Thus the tests are pre-scripted; every test might be a computer program. Each test might click or type a dozen times, and have another dozen verification points. Once the tests exist, if the application behavior has not changed, automated testing can find defects in the application very quickly, typically within minutes of a commit to version control. 

Test automation can be used to run tests that are repetitive, that do not require human discernment, that needs to be run periodically and that can help you find bugs in early stages. While the aim of manual testing is to test the “user experience”, automated testing aims to test all the functionality that characterizes an app. 

Automated testing will click the button that "looks wrong" and is in the wrong place and not register a problem (unless you thought to check in advance)--it is more likely to find errors, incorrect search results, and so on.

Why do I need automated testing?

As the application grows, the time to test it grows also. Automated testing brings that time down for frequent release. That makes automated testing key to speed the testing process, decrease cost, and radically reduce time-to-feedback for major errors from days to minutes. Test automation allows you to:

Test functionalities that are repetitive and therefore error-prone if performed manually; test cases that have a predictable outcome;

Easily set up and run complicated and tedious test scenarios

Most important: you can test on a higher number of mobile devices simultaneously, saving time. Using simulators or the cloud you can do this without buying or managing the devices!

How many test cases do I need for a basic set of tests?

This will vary widely on the application, and how big the "test case" is. If test cases are simple dom-to-database tests that check one logical operation, then a typical function might have four to ten tests, and a typical application might have four to ten features. 

If the application is coded with two different programming languages, one for iOs and one for Android, you may need to double that, or else write an abstraction layer and have one set of business scenarios and two implementations that vary by the operating system. 

Manual testing for mobile apps

Benefits of manual testing

• Testing the user interface accurately and quickly
• Experiencing and testing the app as a real user to replicate the final experience
• Identifying and reproducing reported issues easily
• Tackling complicated issues and use cases
• Testing small changes and projects faster and more cost-effective

Drawbacks of manual testing

• The human error affects accuracy
• Tests have to be repeated manually each time
• Tedious testing tasks are much harder
• Huge coverage requires a lot of resources, like tester headcount and access to a wide array of devices

Manual testing in mobile apps is used for:

• Exploratory Testing: As an approach, exploratory testing values the tester as an integral part of the test process. This needs a manual tester to benefit from their insight, creativity, and knowledge.
• Ad-hoc Testing: By definition, ad-hoc is an informal testing type. It’s aimed at breaking the system, therefore, it cannot be performed in a planned manner. It needs a tester to behave irregularly to try and break the system.
• Usability Testing: The manual tester can get a feel of the final user experience. Using the app like it’s designed to be used will help determine how user-friendly the app is and human input is key in this.

 

Automated testing for mobile apps

Benefits of automated testing

• Performing tedious and demanding tests faster
• Testing many devices
• Automated tests can be used repeatedly
• Fast, efficient, and accurate results
• Multiple automated tests can be done at the same time
• For large mobile apps, automated tests are more cost-efficient

Drawbacks of automated testing

• No insight into user experience from a visual point of view
• The cost to get automated tests up and running is high due to code needed
• Wasteful in smaller-scale situations
• Still some human error since automation test is programmed by a human and requires its own testing
• Needs specific configurations and code for each mobile app

Automated testing in mobile apps is used for:

• Regression Testing: One of the automated testing’s greatest strengths is the ability to have a test script ready to run whenever. One of the best ways to utilize automated testing is with constant small changes when you want to make sure they won’t have any negative effects.
• Performance Testing: Unlike usability testing, performance is more of a numbers game. Testing how fast, responsive, and stable an app is on different devices or under different conditions is something automated tests thrive at executing. The scale and accuracy of the results make automated tests essential for performance testing.
• Repeated Testing: After an initial cost of coding and configuring a test script, over the long term automated testing becomes very cost-efficient. Being able to repeat tests with no extra tester resource costs makes it much easier to test and get results.