You can wait for the page to load in Selenium via multiple strategies:
- Explicit wait: Wait until a particular condition is met. For E.g. a particular element becomes visible on the screen
- Implicit wait: Wait for a particular time interval
- Fluent wait: Similar to explicit wait but provides additional control via timeouts and polling frequency
By default, the web driver waits for the page to load (but not for the AJAX requests initiated with the page load) and you can instruct it to explicitly wait for an element by making use of the WebDriverWait and the expected_conditions module.
Here is an example where the driver goes to the ScrapingBee homepage and waits for an element with the ID of wrapper:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.common.by import By
DRIVER_PATH = '/path/to/chromedriver'
driver = webdriver.Chrome(executable_path=DRIVER_PATH)
driver.get("http://www.scrapingbee.com")
# Timeout in seconds
delay = 3
try:
# Wait for the element with the ID of wrapper
wrapper = WebDriverWait(driver, delay).until(
EC.presence_of_element_located((By.ID, 'wrapper'))
)
print("Element is present in the DOM now")
except TimeoutException:
print("Element did not show up")