How to find elements by XPath in Selenium?

To find an element by XPath in Selenium, use the find_element method with the By.XPATH locator strategy. To match multiple elements, use find_elements instead. Both methods accept an XPath string that identifies the element or elements you want on the page.

find_element vs find_elements

  • find_element(By.XPATH, ...) returns the first element in the page that matches the XPath selector. It raises a NoSuchElementException if no match is found.
  • find_elements(By.XPATH, ...) returns a list of every matching element. If nothing matches, it returns an empty list instead of raising an error.

Note: find_element_by_xpath() used to be the standard way to do this, but it was deprecated in Selenium 4.0 and removed by Selenium 4.3.0, along with all find_element_by_* methods. They were replaced with the single find_element() / find_elements() methods paired with a By locator strategy, to keep the API consistent across Selenium's language bindings.

XPath syntax basics

  • // matches an element anywhere in the document, regardless of its position in the DOM tree.
  • / matches a direct child of the current node.
  • [@attribute='value'] filters elements by an exact attribute value, for example //input[@id='email'].
  • [contains(@attribute, 'value')] filters elements where an attribute contains a substring, which is useful for matching one of several space-separated CSS classes, for example //a[contains(@class, 'btn')].

Example: Selenium find element by XPath in Python

Here is a complete example code that navigates to the ScrapingBee website and extracts some data using XPath:

from selenium import webdriver
from selenium.webdriver.common.by import By

driver = webdriver.Chrome()

try:
    # Open ScrapingBee's website
    driver.get("https://www.scrapingbee.com")

    # Match all img tags on the page
    all_img = driver.find_elements(By.XPATH, "//img")

    # Match all a tags that contain the class "btn"
    all_btn = driver.find_elements(By.XPATH, "//a[contains(@class, 'btn')]")

    # Match the first h1 tag on the page
    first_h1 = driver.find_element(By.XPATH, "//h1")

    # Get the text of the h1 tag
    first_h1_text = first_h1.text

    print(first_h1_text)
    print(len(all_img))
    print(len(all_btn))
finally:
    driver.quit()

Common errors when finding elements by XPath in Selenium

  • NoSuchElementException: find_element found no match. Check the XPath syntax, or confirm the element is present in the DOM at the time you query it (it may load asynchronously).
  • InvalidSelectorException: the XPath string is malformed, for example unbalanced brackets or quotes.
  • Unexpected empty lists with find_elements: an empty list means no elements matched. Unlike find_element, this does not raise an exception, so check the list length before indexing into it.

Related Selenium web scraping questions: