LTS QA

The booming of the smartphone has opened the door for global businesses to interact with consumers more effectively and frequently through thousands of applications. Since mobile apps became a significant channel to connect with consumers, executives of firms have spent more effort on enhancing the quality of applications. However, firms have to face many mobile application testing challenges.

According to a report of Capgemini about Quality Assurance and Testing in 2017-18, 47% of the respondents stated that they lack an appropriate testing process or method. Meanwhile, 46% of the companies surveyed don’t know which are the right tools to perform mobile testing. Shortage in testing devices is also a crucial issue that occupies 40%. 

3 Common mobile application testing challenges

1. Lack of efficient testing process

What is the effective testing process for mobile applications in such a high-competitive market like smartphone apps? There are three factors you need to consider, let’s take a look at them below:

Test Strategy

A thorough strategy for your testing project is significant. Some of the aspects that you should plan are Test methodology, Automation testing, and Test environment.

Firstly, when it comes to testing methodology, one of the most favored ones is the Agile approach. In Agile, the development process breaks into repetitive loops, and testing goes in parallel with development.

The second factor you should think about is how to set up the test environment. You can choose from setting up on real mobile devices, on simulators, or clouds. 

The final thing is automation testing. Although test automation can reduce the time and effort to perform repetitive test cases such as regression testing; some of the tests still need to be run manually. One of the efficient ways to apply automation test is when you run one test case on various devices, like in this video: [Demo video] Automation test on 10 mobile devices at the same time

Continuous Testing

Mobile applications are built and updated regularly. As a result, traditional manual testing cannot keep up with the pace of releasing new versions. Continuous testing will run automation tests regularly to get immediate feedback after new updates are released. Moreover, testing apps in parallel with the development process will decrease the risk of failing at the end of the project.

Select Test Types

For mobile application testing, you should execute both functional and non-functional tests. Functional testing includes testing the function of the apps (path testing, boundary values, data lifecycle), application lifecycle, network, and display. Non-functional testing requires testers to perform some special testing, such as: Typical Interrupts Testing, Testing for Power Consumption, Testing for Different Displays, Testing for Device Input Sensors, and Testing for Screen Orientation Change.

2. Choosing from numerous testing tools

What makes mobile testing more complicated is the complexity of mobile testing tools in the market. Each tool has different features that can test a certain type of mobile apps. Companies have to know exactly what they are looking for in the testing tool to choose the one that has appropriate features, such as:

• Fees: Open-source tools and paid tools
• Type of application that the tools can test: Native apps, Web, hybrid apps
• The operation system: iOS, Android, Windows

3. Shortage in testing devices

In 2019, the shipment of smartphones around the world reached 1,375 billion units, in which Android devices accounted for 76% market share and iOS devices took 13%.  Each operating system has various versions, which means mobile apps have to run in numerous environments. This leads to obstacles in setting up mobile testing devices because the testing team cannot access all types of devices available. The solution is you can combine using different test environments such as Real Devices, Emulators / Simulators, and Clouds to perform testing. Each type has its advantages and drawbacks:

Environment	Advantages	Drawbacks
Real devices	Show how the app actually works Can do specific testing for mobile such as Interrupt testing	Not all the target devices are available
Emulators	No need to look for rare devices Simulate hardware and software	Time-consuming to adjust No testing of mobile-specific factors (battery consumption, interrupts, etc.) Not suitable for all types of testing (e.g. UI testing)
Clouds	Unlimited availability of devices	Not always suitable due to security concerns

Nevertheless, the quality assurance team cannot guarantee that if a tested application works well on a given device, it will work 100% on another device. Even though it’s from the same product family, the screen resolution, CPU, Memory, and hardware could be different.

If you want to have more advices about how to improve the efficiency of mobile application testing, you can contact us for a mobile application testing service

Smartphone applications are now capable of acting as sources of entertainment (gaming, music, movies), social media updates and even personal management tool. This means mobile apps are expected to perform much more complicated tasks; leading to focus on several areas in mobile application testing. With this trend, mobile application testing tools are also getting more and more diverse in scope.

Therefore, it is crucial to understand the strengths and weaknesses of each of these tools in order to choose the suitable one for specific tasks.

 

Appium

 

 

 

 

 

 

Appium is an open source testing tool for assessing Android and iOS applications. Developers can test mobile applications, web mobile, and hybrid applications by using this software.

To run the test, Appium uses the WebDriver interface which supports C#, Java, Ruby and many other languages that belong to the WebDriver library. The tester is also able to check initial applications written with the Android and iOS SDKs, mobile web apps, and hybrid apps that contain web views. As a cross-platform tool, it allows developers to reuse the source code between Android and iOS.

 

Robotium

 

 

 

 

 

 

Robotium is an open source tool that allows testing Android applications of all versions; it supports the testing of native and hybrid applications. It uses JavaScript to prepare and execute test scripts. Therefore, Robotium is really popular in the case of automated black box testing for Android applications.

Moreover, it automates many of Android’s operations and creates solid test cases in a minimum of time.

 

Special Features

Multiple Android activities can be handled in parallel.

Robotium can create powerful test scripts in minimal time, without having a deep knowledge of the project.

You can even run test cases on pre-installed applications.

 

Espresso

 

 

 

 

 

 

Espresso is one of the most popular mobile testing frameworks. Created by Google and integrated with Android Studio, this mobile application testing tool is familiar with anyone who develops native Android applications. Like TestComplete, this framework has several options for test script generation, but with Espresso, you can create Android UI tests only.

 

Special Features

A platform-specific solution

Supports all Android instrumentation

Supports manual creation of tests using Kotlin and Java

Has a simple and flexible API

Espresso UI tests can be executed on emulators as well as real devices

 

MonkeyTalk

 

 

 

 

 

 

Next, MonkeyTalk automatically tests the functionality of Android and iOS applications.

Even non-technical people can run tests on this application because it requires no in-depth knowledge of programming and scripting. The scripts of MonkeyTalk are easy to understand, therefore, tester can also generate XML and HTML reports. Besides, it takes screenshots when the failure occurs. In addition, MonkeyTalk supports emulators, network devices and tethered.

 

EarlGrey

 

 

 

 

 

 

EarlGrey is a native iOS UI automation test framework that enables developers to write clear and concise tests, developed and maintained by Google.

With this framework, testers have access to advanced synchronization features. For example, EarlGrey automatically synchronizes with the UI, network requests, and various queues; while still allows the developer to manually implement customized timings.

 

Special Features

Synchronization: From run to run, EarlGrey 2.0 ensures that you will get the same result in your tests, by making sure that the application is idle. These tasks are executed by automatically tracking UI changes, network requests, and various queues. In addition, EarlGrey 2.0 also allows you to manually implement custom timings.

White-box: EarlGrey 2.0 allows you to query the application under test from your tests.

 

Conclusion

Test automation is a complex process, and its adoption requires all the team members to put in a great deal of effort and time. The success of automated tests, however, mainly depend on the mobile testing tools you choose.

While looking for the right tool or framework for writing test scripts, pay attention to its features. Be sure to pick a reliable solution that allows different options for test creation, supports multiple scripting languages and mobile platforms.

It is clear that the mobile application or mobile devices is very different from the desktop one. So, we should take this feature into the testing process.

Devices feature

1. Various mobile devices with different screen sizes and hardware configurations such as hard keyboard, virtual keyboard (touch screen) …
2. Many mobile device manufacturers such as HTC, Samsung, Apple,…
3. Various mobile operating systems such as Android, Symbian, Windows, IOS…
4. Different versions of operating systems like iOS 5.x, iOS 6.x, BB5.x, BB6.x etc.
5. Regular updates of the version – (such as android- 4.2, 4.3, 4.4, iOS 5.x, 6.x) – with each update need to ensure that no application functionality is affected .
6. Mobile devices have smaller phone screen sizes than desktops.
7. Devices have less memory than desktop computers.
8. Mobile devices typically use a 2G, 3G, 4G or WIFI network connection, while desktop computers often use broadband or dial-up connections.
9. Automated testing tools may not work on mobile applications

Devices limit

1. CPU processor limit of devices
2. Limited RAM
3. Depends on the source
4. Limited battery life
5. And importantly, now in companies the equipment for testing is very scarce.

How to choose mobile devices

Compared to emulators or simulators, real devices are always the best choice for testing mobile applications. But it is not easy to choose appropriate devices. Here are some suggestions for mobile devices selection

・Perform an analysis to identify the most popular and used utilities in the market. Besides, if you must test on various devices, hiring devices from mobile stores is also an economical option.

・Choose devices with different screen resolutions, different operating systems,…

・Check other factors such as compatibility, memory size, connectivity,…..

For more information about Testing services, please contact us

———————————————————–

Lotus Quality Assurance (LQA)

Tel: (+84) 24-6660-7474

Email: [email protected]

Website: https://www.lotus-qa.com/

Before starting testing mobile applications, Tester needs to know a few things to be able to test better:

1. Analyze similar mobile applications

Try analyzing some other applications similar to yours. For example, if you have to test a file-sharing application on a mobile phone, look for some other similar applications and observe its features.

2. Keep the emulators ready

Sometimes it takes a lot of time to borrow or require mobile devices for testing. In this case, to take advantage of the time you might want to test a few cases on the emulator.

You can refer to http://responsivepx.com/ and http://quirktools.com/ to check how your website looks on the desktop and on various types of mobile devices.

3. Analysis of device-related issues

Once the target device has been identified, explore the issues related to that device. This will help you understand what you are and will encounter with a device or application.

4. Use an emulator but don’t fully trust it

You may need the emulators during testing, but keep in mind that you cannot perform 100% test cases on emulators. In addition, the response time in the emulator is very different from the actual device, so you may overlook some errors and these are the weaknesses of real devices.

5. Determine performance criteria

For any mobile application, performance is the biggest concern. Make sure you have performance parameters or requirements that you can rely on during testing. Memory is also one of the weaknesses of mobile devices, and application behavior depends on these conditions

Those are some information we find useful before testing mobile devices.

For more information about Testing services, please contact us

———————————————————–

Lotus Quality Assurance (LQA)

Tel: (+84) 24-6660-7474

Email: [email protected]

Website: https://www.lotus-qa.com/

As we all know, technology is growing and mobile phones, smart devices are the trend, the future of the world. To meet such a large demand, applications must be of good quality, with rich and various content to meet the needs of users. That is also a big challenge for developers and software testers, especially for testers who are familiar with web testing but not yet aware of mobile testing.

So how is mobile testing different from web testing? 

1. Mobile testing has more flatforms

Mobile application testing will be much more complicated due to the variety of mobile devices. Ensuring that mobile applications work on all types of devices (smartphones, tablets or phablets) provided by a number of major vendors (such as Samsung, Sony, Nokia, HTC, Apple …) and on Operating systems (iOS, Android, Windows, Blackberry …) is really a challenge.

Therefore, testers need to have as many test cases as possible for each version, and perform as many tests on different devices as possible for good mobile application.

2. Screen size is constantly changing

Major suppliers tend to change the size of their phones, tablets or phablets to fit users’ needs and tastes, as well as compete with other vendors. Then, when testing, we need to have test cases for screens of different sizes, to check for different screen sizes, whether the interface has broken layout or not; size buttons, the text box, radio button … have been changed or not.

3. User experience is needed for tester

For developers they only care about how the system functions work properly but for testers it is required user experience to support the users. An application can be considered a failure if it is difficult to use or can not be used in different situations, especially mobile applications need to support users to be able to use in all circumstances.

4. Users have more interactions than web applications

When testing the web, the testers only interact with the system through a keyboard or mouse. But for mobile apps, we will interact with more ways such as touching, waving, eye movement, sound …..That’s why when testing, we will need to grasp catch those test cases.

5. Data security and privacy

Mobile applications such as pictures, videos are needed to be inaccessible to many other flatform feature. For mobile testing, there are also test cases of data access and privacy like camera access, photo data access, etc.

6. Excessive reliance on emulators and simulators leads to a lack of real-life device experiences

When we test websites, the differences between browsers are quite small. However, for mobile apps, we often test in emulators and simulator environments. Those environments are not like real devices. Therefore, there are test cases that cannot be performed in emulators and simulator environments. As a tester, we need to control such cases.

7. Cases about installation, removal, update versions of application

Mobile applications are installed, removed and updated more frequently so that when testing, we need to catch how the next platform version changes and what it can affect the application.

What if the users have more than 1 device? What if those devices have different versions of the application? Compatibility, simultaneous support for multiple versions, data storage and the ability to install/ upgrade multiple times,… play an important part in application testing. 

8. Does the app work well when being interrupted

For mobile applications, users will often be interrupted by other applications such as SMS, incoming calls, notifications. After finishing those interruptions, how will the app work save actions in progress? These cases are often very rare when we test the web.

9. Phone’s specific functions testing

There are many factors that need to be taken into account when testing.

・How much data is this application consuming？

・How much battery does this application consume? 

・Does this application work well if the battery is low? 

・How much junk data does it generate? 

—————————————————————

For more information about testing services, please refer to Testing services

Lotus Quality Assurance (LQA)

Tel: (+84) 24-6660-7474

Email: [email protected]

Website: https://www.lotus-qa.com/

Mobile application testing is 1 type of mobile testing. For more information, refer Mobile testing in series of Mobile testing tutorial

1. Mobile application categories

For mobile applications, it can be classified into three categories:

・Type 1: Native apps – apps written specifically for a platform like iOS, Android, or Windows Phone in their respective languages.

・Type 2: Web application (Web App) – web-based application, mobile device users will use different browsers such as Chrome, Firefox, Safari to access the web server to use, such as m.facebook.com.

・Type 3: Hybrid App – a combination of native application and web application, can be run both offline and online and often uses web-making techniques such as HTML5, CSS.

In view of this

・ Native applications can only run on a certain operating system while mobile web applications can run on all mobile browsers that support HTML and Javascript.

・The native application is written in platforms like the SDK while the mobile web application is written with web technologies like HTML, CSS, ASP.NET, JAVA, and PHP.

・ For a native application, it is necessary to install but with mobile web applications, there is no need to install.

・ The original application can be updated from the app store while the mobile web application is updated centrally.

・ The original application may work without an Internet connection, but for mobile web applications, an Internet connection is always required.

・ Native apps work faster when compared to mobile web apps.

2. Some special test cases for testing mobile applications:

・Battery consumption: It is important to track the battery drain when running applications on mobile devices.

・Application running speed: Track time response time on different devices with different memory capacities, different network speeds …

・Memory requirement: When downloading and installing an app, running the app ..

・Application’s function checking: To ensure the application does not crash when losing network connectivity or other external impacts.

3. Some automated testing tools for mobile applications

3.1. IOS has popular automatic testing tools like:

3.2. Android operating system has some automated testing tools such as:

For more information about testing services, please refer to Testing services

---

Lotus Quality Assurance (LQA)

Tel: (+84) 24-6660-7474
Email: [email protected]
Website: https://www.lotus-qa.com/