How to extract text from webdriver elements found through xpath using Selenium and Python

Question

I have working code:

options_elements = issn_dropdown.find_elements_by_xpath("//ul[contains(@id,'my_id')]//li")
options = [x.text for x in options_elements]

options_elemets is an array of 5 selenium.webdriver.remote.webelement.WebElement, each of them contains text. The result option is exactly what I need. Why can't I change it to:

options = issn_dropdown.find_elements_by_xpath("//ul[contains(@id,'my_id')]//li/text()")

? Test just fails on this line with no particular error displayed.

Answer 1

text() in the xpath returns text node, Selenium doesn't support it. Your first code block is the way to go.

Answer 2

You saw it right. find_elements_by_xpath("//ul[contains(@id,'my_id')]//li/text()") would just fail using Selenium.

We have discussed this functionality in the following discussions

• Treatment of DOM Text Nodes in mixed content nodes
• Enhancement request: access to text nodes
• Add support for interacting with text nodes

The WebDriver Specification directly refers to:

• Contexts in which this element can be used
• Element nodes are simply known as elements

and none of them precludes a WebElement reference from being a Text Node.

---

So the bottom line was, as the specification does not specify local end APIs. Client bindings are free to choose an idiomatic API that makes sense for their language. Only the remote end wire protocol is covered here.

@Simon Stewart concluded:

One option might be to serialize a Text node found via executing javascript to be treated as if it were an Element node (that is, assign it an ID and refer to it using that). Should a user attempt to interact with it in a way that doesn't make sense (sending keyboard input, for example) then an error could be returned.

Hence, your first option is the way to go:

options_elements = issn_dropdown.find_elements_by_xpath("//ul[contains(@id,'my_id')]//li")
options = [x.text for x in options_elements]