Webdriver in Python How to wait for the HTML table to load

Question

I am using Selenium Webdriver with Python. I am having trouble in how to wait for the table to complete loading.

The webpage displays a table of columns (Name, Variable, Feed etc). Each row has data for each column. E.g. Lucy, Name, datafeed. I am checking if the data is correct in each row in the if statement. I need to wait for all of the rows in the table to be loaded to avoid Element is no longer valid.

What is the best way to wait for the whole table to be loaded? I have tried using WebdriverWait. I think my syntax is in the wrong place.

My code is:

def is_mappings_details_saved(self, name, dataset, feed, variable):
            try:
                #self.driver.implicitly_wait(10)
                wait = WebDriverWait(self.driver, 20)
                table_id = wait.until(EC.presence_of_element_located((By.ID, 'data_configuration_mappings_ct_fields_body')))
                rows = table_id.find_elements(By.TAG_NAME, "tr")
                for row in rows:
                    time.sleep(1)
                    # Get the columns
                    col_name = row.find_elements(By.TAG_NAME, "td")[1]  # This is the Name column
                    col_variable = row.find_elements(By.TAG_NAME, "td")[2]  # This is the Variable column
                    col_feed = row.find_elements(By.TAG_NAME, "td")[3]  # This is the Feed column
                    col_data_category = row.find_elements(By.TAG_NAME, "td")[4]  # This is the Data Category column
                    col_dataset = row.find_elements(By.TAG_NAME, "td")[6]  # This is the Dataset column
                    col_datamap = row.find_elements(By.TAG_NAME, "td")[7]  # This is the Datamap column
                    if (col_name.text == name) and (col_variable.text == variable) and (col_feed.text == feed) and (col_data_category.text == "Main") and (col_dataset.text == dataset) and (col_datamap.text == self.datamap_name):
                        return True
                return False
            except NoSuchElementException, e:
                return False

The error is:

raise exception_class(message, screen, stacktrace)
StaleElementReferenceException: Message: Element is no longer valid

The line highlighted at the error is:

rows = table_id.find_elements(By.TAG_NAME, "tr")

I wait for the table to load and then find the rows. Do i not need to find the rows if i have the whole table?

The HTML snippet is:

<table id="data_configuration_mappings_ct_fields_body" cellspacing="0" style="table-layout: fixed; width: 100%;">
<colgroup>
<tbody>
    <tr class="GOFU2OVFG GOFU2OVMG" __gwt_subrow="0" __gwt_row="0">
    <td class="GOFU2OVEG GOFU2OVGG GOFU2OVHG GOFU2OVNG">
    <td class="GOFU2OVEG GOFU2OVGG GOFU2OVNG">
        <div __gwt_cell="cell-gwt-uid-418" style="outline-style:none;">
            <span class="linkhover" title="Name" style="white-space:nowrap;overflow:hidden;text-overflow:ellipsis;empty-cells:show;display:block;color:#00A;cursor:pointer;">Name</span>
        </div>
    </td>
    <td class="GOFU2OVEG GOFU2OVGG GOFU2OVNG">
        <div __gwt_cell="cell-gwt-uid-419" style="outline-style:none;">
            <span class="" title="Name" style="white-space:nowrap;overflow:hidden;text-overflow:ellipsis;empty-cells:show;display:block;padding-right: 1px;">Name</span>
        </div>
    </td>
    <td class="GOFU2OVEG GOFU2OVGG GOFU2OVNG">
        <div __gwt_cell="cell-gwt-uid-420" style="outline-style:none;">
            <span class="" title="crm" style="white-space:nowrap;overflow:hidden;text-overflow:ellipsis;empty-cells:show;display:block;padding-right: 1px;">crm</span>
        </div>
    </td>
    <td class="GOFU2OVEG GOFU2OVGG GOFU2OVNG">
        <div __gwt_cell="cell-gwt-uid-421" style="outline-style:none;">
            <span class="" title="Main" style="white-space:nowrap;overflow:hidden;text-overflow:ellipsis;empty-cells:show;display:block;padding-right: 1px;">Main</span>
        </div>
    </td>
    <td class="GOFU2OVEG GOFU2OVGG GOFU2OVNG">
        <div __gwt_cell="cell-gwt-uid-422" style="outline-style:none;">
            <span class="" title="TITLE + FNAME + SNAME" style="white-space:nowrap;overflow:hidden;text-overflow:ellipsis;empty-cells:show;display:block;padding-right: 1px;">TITLE + FNAME + SNAME</span>
        </div>
    </td>
    <td class="GOFU2OVEG GOFU2OVGG GOFU2OVNG">
        <div __gwt_cell="cell-gwt-uid-423" style="outline-style:none;">
            <span class="" title="CRM" style="white-space:nowrap;overflow:hidden;text-overflow:ellipsis;empty-cells:show;display:block;padding-right: 1px;">CRM</span>
        </div>
    </td>
        <td class="GOFU2OVEG GOFU2OVGG GOFU2OVNG">
        <td class="GOFU2OVEG GOFU2OVGG GOFU2OVBH GOFU2OVNG">
    </tr>
    <tr class="GOFU2OVEH" __gwt_subrow="0" __gwt_row="1">
        <td class="GOFU2OVEG GOFU2OVFH GOFU2OVHG">
            <div __gwt_cell="cell-gwt-uid-417" style="outline-style:none;">
                <input type="checkbox" tabindex="-1"/>
            </div>
        </td>
    <td class="GOFU2OVEG GOFU2OVFH">
        <div __gwt_cell="cell-gwt-uid-418" style="outline-style:none;">
            <span class="linkhover" title="DOB" style="white-space:nowrap;overflow:hidden;text-overflow:ellipsis;empty-cells:show;display:block;color:#00A;cursor:pointer;">DOB</span>
        </div>
    </td>
    <td class="GOFU2OVEG GOFU2OVFH">
    <td class="GOFU2OVEG GOFU2OVFH">
    ... etc     
    </tr>
</tbody>
</table>

Thanks, Riaz

Answer 1

I have managed to solve it. Just wait for the table container to load. Not wait for rows and columns. I also put self.driver.implicitly_wait(20) in the base class

class BasePage(object):

def __init__(self, driver):
    self.driver = driver
    self.driver.implicitly_wait(20)

class MappingsPage(BasePage):
    def is_mappings_details_saved(self, name, dataset, feed, variable):       
         table_id = WebDriverWait(self.driver, 20).until(EC.presence_of_element_located((By.ID, 'data_configuration_mappings_ct_fields_body')))
         rows = table_id.find_elements(By.TAG_NAME, "tr")
         for row in rows:  
         # Get the columns
         col_name = row.find_elements(By.TAG_NAME, "td")[1]  
        col_variable = row.find_elements(By.TAG_NAME, "td")[2]  
    etc...