Functional testing of an Android application


Download Functional testing of an Android application


Preview text

Linköping University | Department of Computer Science Bachelor thesis, 16 ECTS | Information technology 2016 | LIU-IDA/LITH-EX-G--16/073--SE
Functional testing of an Android application
Funktionell testning av en Androidapplikation Sebastian Bångerius Felix Fröberg
Supervisor : Simin Nadjm-Tehrani Examiner : Nahid Shahmehri
Linköpings universitet SE–581 83 Linköping
+46 13 28 10 00 , www.liu.se

Abstract
Testing is an important step in the software development process in order to increase the reliability of the software. There are a number of different methods available to test software that use different approaches to find errors, all with different requirements and possible results. In this thesis we have performed a series of tests on our own mobile application developed for the Android platform. The thesis starts with a theory section in which most of the important terms for software testing are described. Afterwards our own application and test cases are presented. The results of our tests along with our experiences are reviewed and compared to existing studies and literature in the field of testing. The test cases have helped us find a number of faults in our source code that we had not found before. We have discovered that automated testing for Android is a field where there are a lot of good tools, although these are not often used in practice. We believe the app development process could be improved greatly by regularly putting the software through automated testing systems.

Acknowledgments
We want to thank our fellow course mates in the parallel project during the sixth semester of our program. Chi Vong, Albin Odervall, Philip Montalvo, Anton Tengroth and Jacob Nilsson were excellent to work with and together we created the application we are testing in this thesis. We also want to thank Fabian Johannsen and Mattias Hellsing for their opposition on this thesis. Lastly, we would like to thank our supervisor Simin Nadjm-Tehrani as well as Mikael Asplund for the feedback that we have received during the work on this project.

Contents

Abstract

i

Acknowledgments

ii

Contents

iii

List of Figures

iv

List of Tables

v

Listings

0

1 Introduction

1

1.1 Why automate testing? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Aim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.4 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.5 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Theory

3

2.1 Testing methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2.2 Android in general and Android testing frameworks . . . . . . . . . . . . . . . 7

2.3 Related Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3 Application and testing method

9

3.1 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.2 Testing approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

4 Test results

13

4.1 Map testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

4.2 Contacts testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4.3 Login testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4.4 Stress testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

5 Discussion

16

5.1 Test results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

5.2 Our work in relation with existing studies . . . . . . . . . . . . . . . . . . . . . . 17

5.3 Method review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

6 Conclusion

20

Bibliography

22

A Espresso Test Source Code

24

iii

List of Figures
2.1 General overview of a testing process . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3.1 Screenshots of the map and contacts activity of the application . . . . . . . . . . . . 9 3.2 Flowchart of the map test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.3 Flowchart of the contacts test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 3.4 Flowchart of the login test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
iv

List of Tables
2.1 Myers’ Principles of testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
v

Listings
2.1 Espresso code example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 4.1 The space bug . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 4.2 The name bug . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

1 Introduction
As we continue deeper into the "Internet age" we become more and more connected every day. New hardware like smart phones, tablets and smart watches have already popped up, and it is hard to know what the future has to offer when it comes to computational devices for both personal and professional use. On these devices there is a lot of software for communication, shopping, etc. and more are created and updated daily. According to International Data Corporation the number of mobile applications was estimated to increase by 1600% between 2010 and 2015 [8] and the Google Play application market currently holds over one million applications1. These systems are becoming a larger and larger part of our lives, and as we depend more on them it is utterly important that they do still work the way they are intended to. There are a lot of aspects that define the dependability of a system, including but not limited to software, hardware and cognitive usability. In this thesis we will focus on the testing of the first subject, ignoring the latter two. Today manual testing is more widely used, even though automated solutions achieve better coverage statistics [10]. We clearly see the need for understanding how to use automated testing and what benefits can come from doing so. For this thesis we will test some functionalities of an Android application using automated testing tools publicly available.
1.1 Why automate testing?
When publishing a new application or making changes to an existing one, it is desirable to know that the application does indeed (still) work the way it is intended to. Since the number of possible inputs to the application quickly becomes unmanageable, it would be more or less impossible to test all combinations. Therefore, it is important that there exists a set of methods to test the functionality of the application without having to test all possible combinations of input. To achieve reasonable input coverage, manual tests are far from optimal. For the purpose of scalability, a methodology of automation is suitable. A good idea might be to use an existing framework that automatically finds views and tests different inputs to a computer program [9].
1https://play.google.com/intl/nl_us/about/apps/index.html. (Accessed june 14th 2016)
1

1.2. Aim
1.2 Aim
The purpose of this thesis is to give the reader a better view of what test automation means. This thesis will give the reader an introduction to software testing in general and then focus on software testing for Android. We will study some common misconceptions and problems faced by Android developers and developers in general. After this, we will show how this knowledge can help software developers and testers to increase the efficiency of the testing. After reading this thesis one should be able to start software testing of an Android application in an efficient manner.
1.3 Problem statement
The questions that we want to answer during this project are: 1. How can we use functional testing to find and remove errors and faults in our mobile application? 2. What are some problems that software testers face, and how can we avoid them when testing our own application?
1.4 Approach
In order to give an answer the questions in the problem statement, we chose three functions of our application to put under functional testing. We have also performed a stress test of the entire application, with the goal of finding errors that the functional tests did not find. To perform the functional tests we have used Google’s framework Espresso, while the stress tests have been performed using the Google program Monkey. We have also looked at a number of existing papers to try to relate our findings to existing scientific literature.
1.5 Delimitations
As the time and resources for this project are quite limited, we have chosen to focus our work on the Android platform, though some of the concepts mentioned will be valid for other platforms as well. Since some of the important functions of the application in the parallel group project were finished late during the work on this thesis the tests will not be as extensive as they could have been otherwise. This will not be a problem since the main purpose of the tests are to show the applicability of relevant concepts discovered through literature study.
2

2 Theory
Before introducing the testing part of this thesis, one must have some basic knowledge in the field of Android software testing. Therefore, this section is dedicated to presenting some basic background information that will be needed to understand the rest of this report.
2.1 Testing methods
The main reason to start software testing is to find faults that can be fixed to increase the reliability of the system under test (SUT)[3]. The faults might cause errors that lead to failures [2]. In this thesis the errors are divided into two categories. The first category, nonfunctional errors, consists of events such as memory leaks or runtime errors etc. These are events where the software does not work and might lead to a failure such as a crash. The second category, functional errors, has more to do with the expected behavior of the app; the software runs, but not the way it is intended to work. An example of a functional error is that images, buttons or other graphic elements are not displayed correctly, either at different times or on different devices. Another one might be that a text piece is displayed in the incorrect setting or to specific users that should not see the text [3].
Figure 2.1 represents a generalized flow of a testing process. The core of the testing framework is the "Test System" block placed in the middle. This consists mainly of an input generator. This input generator can find nonfunctional errors such as crashes but it can not find
Figure 2.1: General overview of a testing process
3

Preparing to load PDF file. please wait...

0 of 0
100%
Functional testing of an Android application