Menu
I want to scrape ads in websites but many of them are dynamic and they are DOM objects. For example in this snippet
I can get the iframe tag by Selenium but I cannot go any further. I think it is because of the XPATH. In this case the XPATH of the <html> inside the iframe is /html which is the same as the main page <html>.
This is the line of code that use:
element = WebDriverWait(self.driver,20).until(EC.presence_of_all_elements_located((By.XPATH, '/html')))
Any suggestions?
649
asked Mar 23 '26 09:03
By default the selenium.webdriver object is set to the default page which it has parsed. To get the iframe data you will have to switch to the given iframe.
driver = webdriver.Chrome(executable_path=path_chrome)
# find the frame using id, title etc.
frame = driver.find_elements_by_xpath("//iframe[@title='iframe_to_get']")
# switch the webdriver object to the iframe.
driver.switch_to.frame(frame[i])
Always remember, if iterating over the iframes then to SWITCH BACK to the default webpage. Otherwise you won't be able to switch to other iframes in same code.
driver.switch_to.default_content()
Below mentioned functions are deprecated now. So i have updated the answer.
driver.switch_to_frame('Any frame') #deprecated
driver.switch_to_default_content() #deprecated
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
