How to Use XPath in Selenium - A Complete Guide.pdf

Navigating the complex web world as an automation tester aims to ensure the delivery of an application that is bug-free and seamless and offers exceptional user experiences.

1. How to Use XPath in Selenium - A
Complete Guide

Navigating the complex web world as an automation tester aims to ensure the delivery of an
application that is bug-free and seamless and offers exceptional user experiences. However,
this is often complex to achieve. One common challenge on this journey is locating web
elements accurately and efficiently to interact with them. Selenium WebDriver helps identify web
elements efficiently with various tools and locators like CSS Selector, XPath, and many more,
which enable enterprises to navigate through the XML elements. Today, several enterprises are
leveraging XPath owing to its greater flexibility and compatibility with old browsers, making it
one of its most powerful and versatile locators. This provides a way to navigate elements and
attributes in an XML document, allowing automation testers to locate and interact with web
elements on a page. This blog provides a holistic XPath tutorial on how you can use XPath in
Selenium, along with examples and tips on handling dynamic elements.
1. What is XPath?
XPath is a query language designed to navigate through elements and attributes in XML
documents. In web development, XPath is utilized to locate and interact with elements within an
HTML document's structure. This capability is particularly beneficial for web scraping and
automated testing, as it allows precise identification of web elements based on various criteria
such as attributes, text content, and hierarchical relationships.

2. 2. How to Write XPath in Selenium
In Selenium WebDriver, XPath expressions are crafted to pinpoint web elements for interaction.
The basic syntax of an XPath expression is:
Xpath=//tagname[@attribute='value']
Where:
●​tagname: The name of the HTML tag you wish to target.
●​@attribute: The attribute within the tag.
●​'value': The specific value of the attribute.
Example:
To locate an input field with the attribute id equal to username:
Xpath=//input[@id='username']
This expression selects all <input> elements with an id attribute of 'username'.
3. The Difference Between Static and Dynamic XPath
Understanding the distinction between static and dynamic XPath is crucial for effective test
automation:
Static XPath: These are absolute paths that directly point to an element's location in the DOM
hierarchy. They are straightforward but fragile; any change in the UI can break the path.
Example:
/html/body/div[1]/div[2]/input
This path starts from the root and traverses down to the desired element.
Dynamic XPath: These are relative paths that use flexible criteria to locate elements, making
them more resilient to changes in the UI. They often employ functions like
contains(),
starts-with()
, or axes like following-sibling to identify elements based on partial
matches or relationships.
Example:
//input[contains(@id, 'user')]

3. This expression selects any <input> element with an id attribute containing the substring
'user'.
Dynamic XPath expressions are generally preferred in test automation due to their robustness
against changes in the application's UI structure.
4. Examples
Using contains() Function:
To locate a button element with a class name that includes the substring 'submit':
//button[contains(@class, 'submit')]
Using starts-with() Function:
To find an input element whose name attribute starts with 'email':
//input[starts-with(@name, 'email')]
Using Axes:
To select a label element that is immediately preceding an input element:
//input[@id='password']/preceding-sibling::label
Mastering XPath expressions allows you to create more reliable and maintainable Selenium test
scripts, ensuring robust interaction with web elements even as the application's UI evolves.
What is XPath in Selenium?
XPath, or XML Path Language, is a language for navigating an XML document and selecting
nodes. In the context of Selenium WebDriver, XPath is used as a locator to find web elements
on a page. It is a powerful tool that can navigate the webpage's HTML structure, making it
extremely useful when other simple locators like ID or class fail to find the elements reliably.
Let’s consider the types of XPath in Selenium in this XPath tutorial.
Types of XPath in Selenium
There are primarily two types of XPath used in Selenium:
Absolute XPath

4. It's a direct path from the root element to the desired element. It starts from the root node and
ends with the desired node, providing a complete path. However, it's brittle and can break with
small changes in the web page's structure.
Here's a Selenium XPath example for how you might use an absolute XPath in your Selenium
code. Suppose you have the following HTML structure:

<html>
<body>
<div>
<p>Hello, world!</p>
</div>
</body>
</html>

To find the <p> tag using an absolute XPath, you would start at the root <html> tag and provide
the full path to the <p> tag:

WebElement paragraph = driver.findElement(By.xpath("/html/body/div/p"));

In this case, the XPath "/html/body/div/p" represents the absolute path from the root <html> tag
to the desired <p> tag.
In several scenarios, however, absolute XPath is not recommended unless necessary because
it's brittle, and any change in the HTML structure may cause your test to fail.
Relative XPath
It starts from any node and ends with the node you want to select. It's more flexible and
preferred in most cases, as it's not affected by changes in other parts of the HTML structure. A
relative XPath allows you to locate elements starting from any location within the HTML
document, not just the root. The relative XPath expression usually starts with //.
Here's an example of using relative XPath in Selenium with Python:

```python
from selenium import webdriver
# Create a new instance of the Firefox driver
driver = webdriver.Firefox()
# Navigate to a website
driver.get("https://example.com")

5. # Find an element using relative XPath
element = driver.find_element_by_xpath("//div[@id='myDiv']/p[2]")
# Perform actions on the element
element.click()
# Close the browser
driver.quit()
```

In the above example, we first import the necessary modules from the Selenium library. Then,
we create a new instance of the Firefox driver and navigate to a website (in this case,
"https://example.com").
Next, we find an element using a relative XPath expression. The XPath used in this example
selects the second `<p>` element inside a `<div>` element with the id "myDiv". You can modify
the XPath expression to suit your specific needs.
After finding the element, you can perform various actions on it, such as clicking it, entering text,
or retrieving its attributes.
Finally, we close the browser using the `quit()` method to clean up and release the resources
used by the driver.
Remember to have the Selenium library installed and the appropriate web driver executable
(e.g., geckodriver for Firefox) set up in your system's PATH for this code to work.
Using relative XPaths is generally recommended over absolute XPaths because they are more
resilient to page structure changes.
In this XPath tutorial, we’ll now consider the types of XPath locators.
Types of XPath locators
XPath locators in Selenium WebDriver are used to identify elements on a web page. These
locators allow complex and flexible navigation of the web page's Document Object Model
(DOM). There are several types of XPath locators, each useful in different situations.
● XPath locator by ID: This locator allows you to identify an element by its id attribute.
Example:

driver.findElement(By.xpath("//*[@id='username']"));

Note: XPath is a wildcard in the snippets helping to select unknown XML nodes

6. ● XPath locator by class name: This locator can identify elements based on their class
attribute.
Example:

driver.findElement(By.xpath("//*[@class='login-button']"));

● XPath locator by name: This locator identifies elements by their name attribute.
Example:

driver.findElement(By.xpath("//*[@name='password']"));

● XPath locator by tag name: This locator can identify elements by their HTML tag name.
Example:

driver.findElement(By.xpath("//p"));

● XPath locator by text: This locator identifies elements based on their inner text.
Example:

driver.findElement(By.xpath("//*[text()='Submit']"));

● XPath locator using contains: This locator can identify elements based on a substring of
one of their attribute values.
Example:

driver.findElement(By.xpath("//*[contains(@href,'google.com')]"));

● XPath locator using starts-with: This locator identifies elements whose attribute values start
with a particular string.
Example:

driver.findElement(By.xpath("//*[starts-with(@id,'user')]"));

7. ● XPath locator using ends-with: This locator can identify elements whose attribute values
end with a particular string.
Example:

driver.findElement(By.xpath("//*[ends-with(@id,'name')]"));

To enhance this XPath tutorial, consider what chained XPath is in Selenium.
What is chained XPath in Selenium?
Chained XPath in Selenium is a concept where multiple XPaths are used in conjunction to
locate an element that might not be uniquely identifiable by a single XPath expression. In other
words, instead of writing one absolute XPath, we can separate it into multiple relative XPaths.
This approach can be specifically useful when dealing with complex or dynamic web structures
where elements are not easily accessible through single, unique identifiers. Chaining XPaths
can provide more precision and robustness in element location strategy, thus making the
automation scripts more stable.
Let us consider a scenario where we need to locate an element that doesn’t have unique
attributes to create a precise XPath. Nonetheless, its parent elements do have unique attributes.
Here's how you can use chained XPath in Selenium:

WebDriver driver = new ChromeDriver();
driver.get("http://www.yourwebsite.com");
// Let's assume there is a div with id="parent" and it has a child button with text "Submit"
WebElement parentDiv = driver.findElement(By.xpath("//*[@id='parent']"));
WebElement submitButton = parentDiv.findElement(By.xpath(".//button[text()='Submit']"));
submitButton.click();

What are XPath Axes?
XPath Axes are used for finding dynamic elements when normal XPath element search
methods like name, ID, class name, etc., aren't possible. XPath Axes navigate through
elements in the XML structure of a webpage. They allow you to locate elements based on their
relationship with other elements, like parent, sibling, child, ancestor, or descendant. Here are
some of the commonly used XPath Axes:
ancestor: Selects all ancestors (parent, grandparent, others.) of the current node.

8. child: This selects all children of the current node.
descendant: Selects all descendants (children, grandchildren, others.) of the current node.
following: This selects everything in the document after the closing tag of the current node.
following-sibling: This selects all siblings after the current node.
parent: This selects the parent of the current node.
preceding: This selects all nodes that appear before the current node in the document,
excluding ancestors and attributes.
preceding-sibling: This selects all siblings before the current node.
self: This selects the current node.
attribute: This selects the attributes of the current node.
These axes provide a flexible way to traverse the DOM and locate elements based on their
relationships with other elements, making XPath a very powerful tool for web scraping and
testing tasks.
Let’s walk through a few of these XPath methods:
1. Following
The `following` axis in XPath is a powerful tool for navigating XML trees in Selenium tests. It
selects all the nodes in the document, post the closing tag of the current node, no matter where
they are nested or at what level.
Let's say you have a web page with multiple sections and want to identify an element that
appears after a particular section. With the `following` axis, you can effectively locate that
element without navigating through the entire tree structure.
Here's an example of how you might use the `following` axis in a Selenium script:

```java
driver.findElement(By.xpath("//div[@id='main-section']/following::div"));
```

In this code, `//div[@id='main-section']/following::div` would select all `div` elements in the
document that come after the `div` element with the id 'main-section'.

9. So, if you have a dynamically changing structure on your webpage, the `following` axis can be a
helpful way to locate elements relative to others. It's particularly useful in instances where you
need to find elements that appear after a specific point in the document, regardless of their
nesting or hierarchical level.
2. Ancestor
The `ancestor` axis method in XPath is particularly useful when dealing with complex XML
documents or web pages with deeply nested elements. This method allows you to select all
ancestor elements (parent, grandparent, others) of the current node in reverse document order
(from the closest ancestor to the furthest).
To illustrate, consider the following HTML snippet:

```html
<html>
<body>
<form id="loginForm">
<div id="credentials">
<input id="username" type="text"/>
<input id="password" type="password"/>
</div>
<input id="submitButton" type="submit"/>
</form>
</body>
</html>
```

If you wanted to find the `form` element that encloses the `input` element with the id
"username", you could use the `ancestor` axis method as follows:

```python
form = driver.find_element_by_xpath("//input[@id='username']/ancestor::form")
``

The XPath expression `//input[@id='username']/ancestor::form` selects the `form` ancestor of
the `input` element with the id "username". In this case, there's only one such `form`, but if there
were more, this expression would select all of them. If you wanted to select only the closest
`form` ancestor, you could use the following XPath expression:

```python

10. form = driver.find_element_by_xpath("//input[@id='username']/ancestor::form[1]")
```

This XPath expression would select the first `form` ancestor of the `input` element with the id
"username", where the first ancestor is the closest one.
Understanding the `ancestor` axis method is crucial when writing robust Selenium test scripts,
as it enables you to locate elements in a more precise and flexible way, especially when dealing
with dynamic or complex web pages.
3. Child
The `child` method in XPath axes is used to select all children of the current node. This is one of
the most commonly used XPath axes methods for locating web elements in Selenium
WebDriver. It enables testers to directly access the child nodes of a specific element, helping in
the navigation of an HTML document from parent to child.
To put it in perspective, consider an HTML structure where a 'div' element with id 'content' has
multiple 'p' elements as its children.

```html
<div id='content'>
<p>Paragraph 1</p>
<p>Paragraph 2</p>
<p>Paragraph 3</p>
</div>
```

To select all 'p' elements (children of the 'div' element), the XPath would be

`//div[@id='content']/child::p`.

Breaking this down:
- `//div[@id='content']`: This selects the 'div' element with the id 'content'.
- `/child::p`: This selects all 'p' elements that are children of the previously selected 'div' element.
Hence, the `child` method helps directly access the child elements of a specific node, which can
be very handy while writing Selenium scripts, particularly in cases where the parent element is
easily locatable, and the structure from the parent to child is stable.

11. 4. Parent
The Parent Axis method in XPath is another crucial aspect of identifying elements in relation to
other elements in the XML document or DOM structure. As the name suggests, this method is
used to select the parent of the current node.
The basic XPath syntax in Selenium for the parent axis is as follows:

`//tag[@attribute='value']/parent::tagName`

Let's break down this syntax:
- `//tag[@attribute='value']`: This portion of the XPath identifies the current node in the
document. The 'tag' represents the HTML tag of the element (like 'div', 'a', 'span', etc.), the
'attribute' refers to the attribute of the element (like 'id', 'class', etc.), and the 'value' represents
the value of that attribute.
- `/parent::tagName`: This part of the XPath is used to select the parent of the current node. The
'tagName' is the HTML tag of the parent element you want to select.
For example, let's say you have a 'div' element with the id 'username', and you want to select its
parent element, which is a 'form'. Your XPath using the parent axis would look something like
this:

`//div[@id='username']/parent::form`

This XPath selects the 'form' element which is the parent of the 'div' element with the id
'username'.
In the context of Selenium, the parent axis comes in handy when the child elements have some
unique attributes that can be easily located, and you want to interact or check something with
their parent element.
However, it's worth noting that parent-child relationships in HTML are not always
straightforward, especially with complex, nested structures, so understanding the DOM structure
well is crucial to use the parent method in XPath effectively.
How HeadSpin helps streamline Selenium utilization
HeadSpin offers a robust AI-driven testing Platform with easy integration with multiple
automation frameworks to improve testing efficiency and ship faster to the market.

12. HeadSpin expands beyond the traditional capabilities of Selenium and adheres to the W3C
WebDriver specification, each HeadSpin host operates a Selenium server supporting custom
features. Moreover, HeadSpin's cloud-based Selenium load balancer accommodates extra
capabilities, facilitating device selection and redundancy management.
Following are the key capabilities of HeadSpin that enable testers and QA teams to leverage
Selenium for unique testing needs—
●​Support for Selenium Webdriver: HeadSpin's platform fully supports Selenium
WebDriver. This allows developers and testers to write Selenium scripts and execute
them on real devices in HeadSpin's device cloud, thereby facilitating automated testing
across various device and OS combinations. ●​Parallel testing: With HeadSpin, you can conduct concurrent testing across multiple
devices, significantly reducing test times and accelerating your development process.
This is particularly useful in Selenium Grid setup, where you might want to run your tests
on different browser and operating system configurations simultaneously. ●​Integration with CI/CD pipelines: HeadSpin can seamlessly integrate into your existing
CI/CD pipelines. This means your Selenium tests can be automatically triggered each
time there's a commit or before a release, ensuring your application remains bug-free.
●​Real device testing: HeadSpin allows Selenium tests to run on real, physical devices,
providing more accurate results compared to emulators or simulators. This helps in
identifying real-world issues that users might face.
●​Detailed reports: After test execution, HeadSpin provides detailed reports, complete
with videos, logs, network stats, and performance metrics, helping you identify,
understand, and fix issues quickly.
●​Global test coverage: HeadSpin's device cloud spans 150+ locations worldwide,
enabling you to conduct Selenium testing from a global user perspective. You can
understand and address region-specific issues, ensuring a top-notch user experience
everywhere.
Conclusion
This XPath tutorial explains why XPath is critical in Selenium testing, offering a robust method
for locating elements within the web page's DOM. Understanding and utilizing XPath's
capabilities, such as functions, axes, and expressions, can greatly improve the effectiveness
and efficiency of your Selenium tests, as shown in this XPath tutorial. Moreover, with platforms
HeadSpin, you can enhance your testing capabilities further, leveraging its unique features
aligned with Selenium standards. As the world of web development continues to evolve, it's
essential to stay updated with these tools and methodologies to ensure high-quality, reliable
web applications.
Article Source:

This article was originally published on:

13. https://www.headspin.io/blog/using-xpath-in-selenium-effectively