In this post I will try to summaries my experiences with web test automation. I will refer to selenium and Java examples, although most of the issues raised in the examples are also applicable with the use of other tools as well. I will start by discussing high level strategic issues and then drill down to a more technical level.
Background
Web as GUI testing is comprised of three major elements:
- Object identification – find the object that the action or analysis will work on.
- Action – once the object was identified perform an action on it.
- Analysis – verify objects content and state.
Most of the complexity lies in the identification and analysis.
Business logic level automation
You’re probably aware of the pitfalls involved in the ‘record and replay’ approach. ‘Record and replay’ can be used in order to extract code snippets. It is not recommended as an automation strategy.
Why does the ‘record and replay’ fail?, it look so nice in the demo of the sales person.The reason is lack of design.
What is business logic level automation?
The idea is to separate your automation into two layers. The first layer is the building blocks infrastructure layer that uses interfaces such as GUI in order to expose the application logic. The second layer is the test layer where the building blocks layer is used to build tests.
You know that you have separated the tests well, if the tests that are written do not have any UI specific orientation.
Why is it a good practice?
- Organization/systems do not tend to change business logic very often. So, if you did the modeling correctly there is a chance that your tests will be valid five years from now.
- Most of the changes that are performed focus in a small layer of the connectivity.
Bad code example:
Window(“App”).button(“Add”).click()
Dialog(“Add user”).wait()
Dialog(“Add”).textField(“Name”).set(“guy”)
Dialog(“Add”).button(“Add”).click()
Good code example:
app.addUser(“guy”)
app.checkUserExist(“guy”)
db.checkUserExist(“guy”)
Functional testing using the user interface
There is a tendency to mix ‘user interface testing’ and ‘functional testing using the user interface’. The first is very hard to automate as it involves user experience. SeeTest from Experitest is a new tool that do just that.
Usually most of the automation focus should be ‘functional testing using the user interface’.
HTML Objects Identification Strategy
The HTML DOM is a tree of objects, you have the root tag (HTML) then you can have a child (BODY) …
In order to identify an object I have to scan the DOM tree model. There are a few ways to identify a node in the tree:
1. Tag name.
2. Tag value
3. By it relation with other nodes.
4. By attribute of the node.
Usually a combination of more than single method will be used.
In order to observe the HTML DOM I recommend Firebug ( https://addons.mozilla.org/en-US/firefox/addon/1843 )
After installing it (Firefox plug-in) you will see a bug icon on bottom right corner (see the red box). Pressing it will launch the plug-in for the current browser page. Then you can press the ‘Inspect’ button, when you move the mouse over the page you will see that page items becomes highlighted and the HTML section of the item can be seen.
So when looking at the Google main page the text box tag looks as follows:
<input
value=""
title="Google Search"
size="55"
name="q"
maxlength="2048"
autocomplete="off"
/>
You can see a tag called input and six attributes: value, title, size, name maxlength and autocomplete.
HTML DOM is a tree and the best way to identify object in such tree is by using XPath. “XPath is a language for finding information in an XML document” (http://www.w3schools.com/Xpath/ ).
Following are few examples to identify the input tag:
- //input[2] – the second input tag in the page.
- //input[@name=’q’] – an input tag with attribute name that equals to ‘q’.
- //td[input[@name='btnG']]/input[@title] – this one is little more complex. First it identifies a ‘td’ tag. The identification is done by a child tag ‘input’ that has ‘name’ attribute with ‘btnG’ value. This ‘td’ tag should have additional child with ‘titile’ attribute.
You can use selenium IDE to verify your XPath.
When entering the search string use the ‘Find’ button in order to identify the searched object in the HTML page. You will see a green rectangle around the found object.
When working with selenium RC (remote control) the code that uses the XPath will appear as follows:
selenium.type(“xpath=//input[@name=’q’]”, “jsystem”);
An identification strategy could appear as follows (from the highest priority and down):
- Use object visible name (like button name…). If does not exist or case uniqueness problem:
- Use unique ID tag. If does not exist:
- Use other unique tag. If does not exist:
- Zoom on possible container (like table or div) and then identify by index.
- Identify by index.
The guidelines used when building the identification strategy is to build a strategy that requires minimal maintenance.
Objects map
A good practice is to save all of the objects identification strings as metadata. I use the property files. For example:
map.properties file:
google.textfield= xpath=//input[@name=’q’]
code:
selenium.type(map.getProperty(“google.textfield”), “jsystem”);
State less page navigation
You should define your application home. The application home should be the starting point of every building block. A good application home can be the landing page after to login page.
Following is an example for building block implementation:
public void addUser(String username){
goHomeAndLoginIfNeeded();
navigateToPosition(Position.ADD_USER);
_addUserStrict(username);
}
Every action (addUser for example) should start with the goHomeAndLoginIfNeeded method. This method with navigate to the home URL it will check if login requires and if yes will login.
Automation sanity
The idea of ‘automation sanity’ is to create small and non-context tests that will verify all the object in the GUI map exists. The main idea is to find objects identification problems in the right context. When a functional test is executed and fails because of an identification problem the investigation can be painful and waist lot of time.
