Selenium: Navigating non-scrollable tables in Airtable?

Other Questions Airtable
1

I’m trying to scrape data from an Airtable page but I’m stuck. The table doesn’t seem to scroll normally. I’ve tried using JavaScript in the console and various Selenium methods, but nothing has worked so far.

Here’s what I’ve attempted so far:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains

def scrape_airtable():
    driver = webdriver.Chrome()
    driver.get('https://airtable.com/some-embed-url')
    
    try:
        table = driver.find_element(By.ID, 'data-grid')
        ActionChains(driver).move_to_element(table).perform()
        table.send_keys(Keys.PAGE_DOWN)
        
        # Other attempts:
        # driver.execute_script('arguments[0].scrollTop = 500', table)
        # driver.find_element(By.CLASS_NAME, 'scroll-bar').click()
    except Exception as e:
        print(f'Error: {e}')
    finally:
        driver.quit()

scrape_airtable()

I also tried this JavaScript: document.querySelector('#view-container').scrollTop = 300.

Any ideas on how to scroll this tricky table? I think I need to target a specific element, but I’m not sure which one. Your help would be greatly appreciated!

2

hey there! i’ve had similar issues with airtable. have u tried using the ElementScrollInterceptor? it can bypass some weird scrolling mechanics. also, check if there’s an iframe - sometimes that’s the culprit. if nothing else works, u might need to use a headless browser like playwright. good luck!

3

I’ve dealt with Airtable’s quirky tables before, and it can be a real headache. One thing that worked for me was using Selenium’s wait functions combined with JavaScript execution. Here’s a snippet that might help:

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# Wait for the table to load
table = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.CLASS_NAME, 'dataGrid'))
)

# Use JS to scroll
driver.execute_script('''
    var grid = document.querySelector('.dataGrid');
    grid.scrollTo(0, grid.scrollHeight);
''')

# Wait for new content to load
WebDriverWait(driver, 10).until(
    EC.staleness_of(table.find_element(By.CSS_SELECTOR, 'div[role="row"]:last-child'))
)

This approach waits for elements to load, scrolls using JavaScript, then waits again for new content. It’s been pretty reliable for me. Just remember to adjust the class names if needed - Airtable likes to change things up sometimes.

4

I’ve encountered similar challenges with Airtable’s non-standard scrolling. One effective approach is to locate and interact with the scrollbar element directly. Try this:

scrollbar = driver.find_element(By.CSS_SELECTOR, '.scrollableScrollbar')
ActionChains(driver).click_and_hold(scrollbar).move_by_offset(0, 100).release().perform()

This simulates dragging the scrollbar. If that doesn’t work, you might need to use JavaScript to manipulate the table’s internal state:

driver.execute_script("""
    var grid = document.querySelector('.gridView');
    grid.__tableDragManager.handleScrollToPosition({top: 1000});
""")

Adjust the ‘top’ value as needed. Remember, Airtable’s structure can change, so you may need to inspect the page to find the correct selectors.