Getting specific elements in selenium

Question

I am trying to get the elements displayed as N06D-X N07X R01A-C01 S01G-X01 in the following image:

Now, I got something like the WebDriver in this way:

who = driver.find_element_by_tag_name("span").find_elements_by_tag_name("p")

and get an output like this:

[<selenium.webdriver.remote.webelement.WebElement (session="1c044455cf883fdedf6845bcd456bfab", element="0.23338884730774767-2")>]

I am working on Mac Catalina and when I type: who.text it returns an empty list for some reason. I got quite similar troubles with Bs but I solved them with .string rather than .text. Here .string does not work on WebDriver elements.

The question is: how can I get the items N06D and so on with selenium?

Answer 1

Seems you were pretty close enough.

[<selenium.webdriver.remote.webelement.WebElement (session="1c044455cf883fdedf6845bcd456bfab", element="0.23338884730774767-2")>]

represents the element where as you were looking for the text within the element.

To extract the texts e.g. N06D-X, N07X, etc from all of the <p> tags using Selenium and python you have to induce WebDriverWait for visibility_of_all_elements_located() and you can use either of the following Locator Strategies:

• Using CSS_SELECTOR and get_attribute("innerHTML"):

  print([my_elem.get_attribute("innerHTML") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "li.data-list__property#who-atc-codes span.data-list__property-value p")))])
  

• Using XPATH and text attribute:

  print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//li[@class='data-list__property' and @id='who-atc-codes']//span[@class='data-list__property-value']//p")))])
  

• Note : You have to add the following imports :

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  

---

Outro

Link to useful documentation:

• get_attribute() method Gets the given attribute or property of the element.
• text attribute returns The text of the element.
• Difference between text and innerHTML using Selenium

Answer 2

you dont search in whole website but in previously found object

li_object = driver.find_elements_by_id('who-atc-codes')
lst = li_object.find_element_by_tag_name("span").find_elements_by_tag_name("p")

for p in lst:
    print(p.text)
    print(p.get_attribute('innerHTML'))

or you can try

span_object = li_object.find_element_by_tag_name("span")
print(span_object.get_attribute('innerHTML'))