Proceedings of the Fourth International Workshop on

Download Proceedings of the Fourth International Workshop on

Preview text

Electronic Communications of the EASST Volume 33 (2010)
Proceedings of the Fourth International Workshop on Foundations and Techniques for Open Source Software Certification
(OpenCert 2010)
GUI Inspection from Source Code Analysis
Joa˜o Carlos Silva , Jose´ Creissac Joa˜o Saraiva 18 pages

Guest Editors: Luis S. Barbosa, Antonio Cerone, Siraj A. Shaikh Managing Editors: Tiziana Margaria, Julia Padberg, Gabriele Taentzer ECEASST Home Page:

ISSN 1863-2122


GUI Inspection from Source Code Analysis
Joa˜o Carlos Silva 1,2 Jose´ Creissac 1 Joa˜o Saraiva 1
1Departamento de Informa´tica, Universidade do Minho, Braga, Portugal 2Departamento de Tecnologia, Instituto Polite´cnico do Ca´vado e do Ave, Barcelos, Portugal

Abstract: Graphical user interfaces (GUIs) are critical components of today’s software. Given their increased relevance, the correctness and usability of GUIs are becoming essential. This paper describes the latest results in the development of our tool to reverse engineer the GUI layer of interactive computing systems. We use static analysis techniques to generate models of the user interface behavior from source code. Models help in graphical user interface inspection by allowing designers to concentrate on its more important aspects. One particular type of model that the tool is able to generate is state machines. The paper shows how graph theory can be useful when applied to these models. A number of metrics and algorithms are used in the analysis of aspects of the user interface’s quality. The ultimate goal of the tool is to enable analysis of interactive system through GUIs source code inspection.
Keywords: Source Code, Reverse Engineering, Graphical User Interface, Metrics, Properties

1 Introduction
Typical WIMP-style (Windows, Icon, Mouse, and Pointer) user interfaces consist of a hierarchy of graphical widgets (buttons, menus, textfields, etc) creating a front-end to software systems. An event-based programming model is used to link the graphical objects to the rest of the system’s implementation. Each widget has a fixed set of properties and at any time during the execution of the GUI, these properties have discrete values, the set of which constitutes the state of the GUI. Users interact with the system by performing actions on the graphical user interface widgets. These, in turn, generate events at the software level, which are handled by appropriate listener methods.
In brief, and from a user’s perspective, graphical user interfaces accept as input a pre-defined set of user-generated events, and produce graphical output. From the programmers perspective, as user interfaces grow in size and complexity, they become a tangle of object and listener methods, usually all having access to a common global state. Considering that the user interface layer of interactive systems is typically the one most prone to suffer changes, due to changed requirements and added features, maintaining the user interface code can become a complex and error prone task. Integrated development environments (IDEs), while helpful in that they enable the graphical definition of the interface, are limited when it comes to the definition of the behavior of the interface
A source code analysis tool can minimize the time necessary by a developer to understand and evaluate a system. In this paper we present GUISurfer, a static analysis based retargetable framework for GUI-based applications analysis from source code. In previous papers [SCS06a,

1 / 18

Volume 33 (2010)

GUI Inspection from Source Code Analysis

L anguage independent


A stA nalyser.hs GuiX .hs SlicingX .hs

GUI model State M achine Event Flow Graph

GUI analysis

GUI Intermediate R epresentation

GUI code slicing

A bstract Syntax Tree


Source code GUI layer
Business layer D ata layer

L anguage dependent


Figure 1: GUISurfer Architecture and Retargetability

SCS06b, SCS09] we have explored the applicability of slicing techniques [Tip95] to our reverse engineering needs, and developed the building blocks for the approach. In this paper we explore the integration of analysis techniques into the approach, in order to reason about the GUI models.

2 GUISurfer tool
GUISurfer’s goal is to be able to extract a range of models from source code. In the present context we focus on finite state models that represent GUI behavior. That is, when can a particular GUI event occur, which are the related conditions, which system actions are executed, or which GUI state is generated next. We choose this type of model in order to be able to reason about and test the dialogue supported by a given GUI implementation.
Figure 1 presents the architecture of the GUISurfer tool. GUIsurfer is composed by three tools: FileParser, AstAnalyser, and Graph. These tools are configurable through command line parameters. Below we outline some of the more important parameters for each tool.
The FileParser tool is language dependent and is used to parse a particular source code file. For example, the command FileParser allows us to parse a particular Login Java class. As a result, we obtain its AST.
The AstAnalyser tool is another language dependent tool used to slice an abstract syntax tree, considering only its graphical user interface layer. Part of this tool is easily retargetable, however most of the tool needs to be rewritten to consider another particular programming language.
The AstAnalyser tool is composed of a slicing library, containing a generic set of traversal functions that traverse any AST. This tool must be used with three arguments, i.e. the abstract

Proc. OpenCert 2010

2 / 18


syntax tree, the entry point in source code (e.g., the main method for Java source code), and a list with all widgets to consider during the GUI slicing process. The command AstAnalyser main JButton lets us extract the GUI layer from’s abstract syntax tree, starting the slice process at the main method, and extracting only JButton related data. Executing the command generates two files initState.gui and eventsFromInitState.gui which contain the initial state and possible events from the initial states, respectively.
Finally, the Graph tool is language independent and receives as arguments the initState.gui and eventsFromInitState.gui files, and generates several metadata files with events, conditions, actions, and states extracted form source code. Each of these types of data is related to a particular fragment from the AST. Further important outputs generated by the Graph tool are the GuiModel.hs and GuiModelFull.hs files. These are GUI specifications written in the Haskell programming language. These specifications define the GUI layer mapping events/conditions to actions. Finally, this last tool allows us also to generate several visual models through the GraphViz tool, such as state machines, behavioral graph, etc.

3 GUI Inspection from source code
The evaluation of an user interface is a multifaceted problem. Besides the quality of the code by itself, we have to consider the user reaction to the interface. This involves issues such as satisfaction, learnability, and efficiency. The first item describes the users satisfaction with the systems. Learnability refers to the effort users make to learn how to use the application. Efficiency refers to how efficient the user can be when performing a task using the application.
Software metrics aim to measure software aspects, such as source lines of code, functions invocations, etc. By calculating metrics over the behavioral models produced by GUISurfer, we aim to acquire relevant knowledge about the dialogue induced by the interface, and, as a consequence, about how users might react to it (c.f. [TG08]). In this section we describe several kinds of inspections making use of metrics.
The analysis of source code can provide a mean to guide development and to certificate software. For that purpose adequate metrics must be specified and calculated. Metrics can be divided into two groups: internal and external [ISO99].
External metrics are defined in relation to running software. In what concerns GUIs, external metrics can be used as usability indicators. They are often associated with the following attributes [Nie93]:
• Easy to learn: The user can do desired tasks easily without previous knowledge;
• Efficient to use: The user reaches a high productivity level.
• Easy to remember: The re-utilization of the system is possible without a high level of effort.
• Few errors: Errors are made hardly by the users and the system permits to recover from them.
• Pleasant to use: The users are satisfied with the use of the system.

3 / 18

Volume 33 (2010)

GUI Inspection from Source Code Analysis

However, the values for these metrics are not obtainable from source code analysis, rather through users’ feedback.
Internal metrics are obtained by source code analysis, and provide information to improve software development. A number of authors has looked at the relation between internal metrics and GUI quality.
Stamelos et al. [SAOB02] used the Logiscope1 tool to calculate values of selected metrics in order to study the quality of Open Source code. Ten different metrics were used. The results enable evaluation of each function against four basic criteria: testability, simplicity, readability and self-descriptiveness. While the GUI layer was not specifically targeted in the analysis, the results indicated a negative correlation between component size and user satisfaction with the software.
Yoon and Yoon [YY07] developed quantitative metrics to support decision making during the GUI design process. Their goal was to quantify the usability attributes of interaction design. Three internal metrics were proposed and defined as numerical values: complexity, inefficiency and incongruity. The authors expect that these metrics can be used to reduce the development cost of user interaction.
While the above approaches focus on calculating metrics over the code, Thimbleby and Gow [TG08] calculate them over a model capturing the behavior of the application. Using graph theory they analyze metrics related to the users’ ability to use the interface (e.g., strong connectedness ensure no part of the interface ever becomes unreachable), the cost of erroneous actions (e.g., calculating the cost of undoing an action), or the knowledge needed to use the system (e.g., the minimum cut identifies the set of actions that the user must know in order to to be locked out of parts of the interface).
In a sense, by calculating the metrics over a model capturing GUI relevant information instead of over the code, the knowledge gained becomes closer to the type of knowledge obtained from external metrics. While Thimbleby and Gow manually develop their models from inspections of the running software/devices, an analogous approach can be carried out analyzing the models generated by GUISurfer. Indeed, by coupling this type of analysis with GUISurfer, we are able to obtain the knowledge directly from source code.

4 An Agenda application
Throughout the paper we will use a Java/Swing interactive application as a running example. This application consist of an agenda of contacts: it allows users to perform the usual actions of adding, removing and editing contacts. Furthermore, it also allows users to find a contact through its name.
The interactive application consists of four windows, namely: Login, MainForm, Find and ContactEditor, as shown in Figure 2. The initial Login window (Figure 2, top-left) is used to control users’ access to the agenda. Thus, a login and password pair has to be introduced by the user. If the user introduces a valid login/password pair, and presses the Ok button, then the login window closes and the main window of the application is displayed. On the contrary, if the user introduces an invalid login/password pair, then the input fields are cleared, a warning message is

Proc. OpenCert 2010

4 / 18


Figure 2: A Java/Swing application

produced and the login window continues to be displayed. By pressing the Cancel button in the Login window, the user exits the application.
The Java fragment defining the action performed when the Ok button is pressed is as follows:
private void OkActionPerformed(...) {if (isValid(user.getText(),pass.getText()))
{new MainForm().setVisible(true); this.dispose();}
else javax.swing.JOptionPane.showMessageDialog (this,"User/Pass not valid","Login",0);
where the method isValid tests the username/password pair inserted by the user. Authorized users can use the main window (Figure 2, top-right) to find and edit contacts (c.f.,
Find and Edit buttons). By pressing the Find button in the main window, the user opens the Find window (Figure 2, bottom-left). This window is used to search and obtain a particular contact’s data from his name. By pressing the Edit button in the main window, the user opens the ContactEditor window (Figure 2, bottom-right). This last window allows the editing of a contact’s data, such as name, nickname, e-mails, etc. The Add and Remove buttons enable editing the e-mail addresses’ list of the contact. If there are no e-mails in the list then the Remove button is automatically disabled.
Until now, we have informally described the (behavioral) model of our interactive application. Such descriptions, however, can be ambiguous and often lead to different interpretation of what the application should do. In order to unambiguously and rigorously define an application, we can use a formal model. Moreover, by using a formal model to define the interactive application, we can use techniques to manipulate and inspect such application.
Figure 3 shows a formal model to specify the behavior of our running example: a graph. A graph is a mathematical abstraction and consists of a set of vertices, and a set of edges. Each edge connects two vertices in the graph. In other words, a graph is a pair (V,E), where V is a finite set and E is a binary relation on V. V is called a vertex set whose elements are called vertices. E is a

5 / 18

Volume 33 (2010)

GUI Inspection from Source Code Analysis





Cancel/cond1/[1] Ok/cond2/[2,3]


Search/cond1/[]Show /cond3/[]




Open MainForm window




Close Find window init/condInit1/[6,7,8,9,10,11,12,13,14,15]



Open Find window Open ContactEditor window Exit/cond1/[1]







Close ContactEditor window

Remove/cond3/[3,4] Add/cond1/[1,2]

Cancel/cond5/[7] Ok/cond6/[8]



Cancel/cond5/[7] Ok/cond6/[8]


Figure 3: Agenda’s behavior graph

Proc. OpenCert 2010

6 / 18


collection of edges, where an edge is a pair (u,v) with u,v in V. Graphs are directed or undirected. In a directed graph, edges are ordered pairs, connecting a source vertex to a target vertex. In an undirected graph edges are unordered pairs of two vertices.
If some edge (u,v) is in graph, then vertex v is said to be adjacent to vertex u. In a directed graph, edge (u,v) is an out-edge of vertex u and an in-edge of vertex v. The number of out-edges of a vertex is its out-degree, and the number of in-edges is its in-degree.
A path is a sequence of edges in a graph such that the target vertex of each edge is the source vertex of the next edge in the sequence. If there is a path starting at vertex u and ending at vertex v we say that v is reachable from u.
Graphs are a commonly used to represent user interfaces. Vertices represent the possible GUI states, and the transitions between vertices (edges) define the events associated to the GUI objects.
The model in figure 3 was automatically extracted by GUIsurfer. Associated to each edge there is a triplet representing the event that triggers the transition, a guard on that event (here represented by a label identifying the condition being used), and a list of interactive actions executed when the event is selected (each action is represented by a unique identifier which is related to the respective source code).
Using this model it becomes possible to reason about characteristics of the interaction between users and the agenda application.

5 GUI Inspection through Graph Theory
This section describes some examples of analysis performed on the Agenda application’s behavioral graph (cf. figure 3) from the previous section. We make use of Graph-Tool for the manipulation and statistical analysis of the graph.

5.1 Graph-tool
Graph-tool is an efficient python module for manipulation and statistical analysis of graphs (cf. It allows for the easy creation and manipulation of both directed or undirected graphs. Arbitrary information can be associated to the vertices, edges or even the graph itself, by means of property maps.
Graph-tool implements all sorts of algorithms, statistics and metrics over graphs, such as degree/property histogram, combined degree/property histogram, vertex-vertex correlations, assortativity, average vertex-vertex shortest distance, isomorphism, minimum spanning tree, connected components, dominator tree, maximum flow, clustering coefficients, motif statistics, communities, centrality measures. Now we will consider the graph described in figure 4 (automatically obtained from figure 3) where all vertices and edges are labeled with unique identifiers.

5.2 GUI Metrics
To illustrate the analysis, we will consider three metrics: Shortest distance between vertices, Pagerank and Betweeness.

7 / 18

Volume 33 (2010)

GUI Inspection from Source Code Analysis



10 27



0 23










7 16182022











13 6



4 9 1114

10 12


Figure 4: Agenda’s behavior graph (numbered)

5.2.1 Shortest Distance
The Graph-Tool enables us to calculate the shortest path between two vertices. As examples the obtained results for the shortest path between vertices 11 and 6 is (cf. figure 4):
shortest path vertices: [’11’,’10’,’13’,’8’,’7’,’5’,’4’,’6’] shortest path edges:

We obtain the vertices sequence from vertex 11 to vertex 6. And we have also access to the
edges sequence. This is useful to calculate the number of steps to execute a particular task. Now let us consider another inspection. The next result gives us the shortest distance (mini-
mum number of edges) from the Login window (vertex 11) to all other vertices. Each value gives

Proc. OpenCert 2010

8 / 18


the distance from vertex 11 to a particular target vertex. The index of the value in the sequence correspond to the vertex identifier. As example the first value is the shortest distance from vertex 11 to vertex 0, which is 6 edges long.
shortest distance from Login [6 5 7 6 6 5 7 4 3 5 1 0 2 2]
Another example makes use of MainForm (vertex 7) as starting point. Negative values (-1) indicate that there are no paths from Mainform to those vertices.
shortest distance from MainForm [2 1 3 2 2 1 3 0 -1 1 -1 -1 -1 -1]
This metrics are useful to analyze the complexity of an interactive application’s user interface. Higher values represent complex tasks while lower values are applications with simple tasks. The example also shows that they can be used to detect parts of the interface that can become unavailable. In this case, there is no way to go back to the login window once the Main window is displayed. The application must be quit.
This metrics can be used to calculate the center of a graph. The center of a graph is the set of all vertices A where the greatest distance to other vertices B is minimal. The vertices in the center are called central points. Thus vertices in the center minimize the maximal distance from other points in the graph.
Finding the center of a graph is useful in GUI applications where the goal is to minimize the steps to execute a particular task (i.e. edges between two points). For example, placing the main window of an interactive system at a central point reduces the number of steps an user has to execute to accomplish tasks.

5.2.2 Pagerank
PageRank is a distribution used to represent the probability that a person randomly clicking on links will arrive at any particular page [Ber05]. A probability is expressed as a numeric value between 0 and 1. A 0.5 probability is commonly expressed as a ”50% chance” of something happening.
PageRank is a link analysis algorithm, used by the Google Internet search engine that assigns a numerical weighting to each element of a hyperlinked set of documents. The main objective is to measure their relative importance.
This same algorithm could be applied to our GUI’s behavioral graphs. Figure 5 gives pagerank for each Agenda vertices. The size of a vertex corresponds to its importance within the overall application behavior. This metric is useful, for example, to analyze whether complexity is well distributed along the application behavior. In this case, the Main window is clearly a central point in the interaction.

5.2.3 Betweenness
Betweenness is a centrality measure of a vertex or a edge within a graph[Sa09]. Vertices that occur on many shortest paths between other vertices have higher betweenness than those that do not. Similar to vertices betweenness centrality, edge betweenness centrality is related to shortest

9 / 18

Volume 33 (2010)

Preparing to load PDF file. please wait...

0 of 0
Proceedings of the Fourth International Workshop on