Python 错误:索引错误:使用硒抓取时列出索引超出范围

Question

我收到此错误：

Traceback (most recent call last):
  File "scrap.py", line 40, in <module>
    phone3 = driver.find_elements_by_css_selector('#mobile')[0].text
IndexError: list index out of range

一次又一次

我注意到当此脚本单击第三个按钮时，我收到此错误！

我试图修复此错误，但我无法修复此错误:(

有没有解决方案可以解决这个问题！

也请告诉我为什么我会收到此错误

这是我试图抓取的网站

https://www.zameen.com/EstateAgents/Islamabad-3-2.html

这是我的代码：

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import csv
import time
csv = 'islamabad.csv'
with open(csv,'w') as s:
    print('Sheet Maked!: ' + csv)
driver = webdriver.Chrome()
for i  in range(2,154):
    driver.get('https://www.zameen.com/EstateAgents/Islamabad-3-' + str(i) + '.html')
    name1 = driver.find_elements_by_css_selector('#agent_search_listing_section .title')[0].text
    name2 = driver.find_elements_by_css_selector('#agent_search_listing_section .title')[1].text
    name3 = driver.find_elements_by_css_selector('#agent_search_listing_section .title')[2].text
    name4 = driver.find_elements_by_css_selector('#agent_search_listing_section .title')[3].text
    name5 = driver.find_elements_by_css_selector('#agent_search_listing_section .title')[4].text
    name6 = driver.find_elements_by_css_selector('#agent_search_listing_section .title')[5].text
    name7 = driver.find_elements_by_css_selector('#agent_search_listing_section .title')[6].text
    name8 = driver.find_elements_by_css_selector('#agent_search_listing_section .title')[7].text
    name9 = driver.find_elements_by_css_selector('#agent_search_listing_section .title')[8].text
    name10 = driver.find_elements_by_css_selector('#agent_search_listing_section .title')[9].text
    time.sleep(1)
    button1 = driver.find_elements_by_id('agent_phone')[0].click()
    time.sleep(2.5)
    phone1 = driver.find_elements_by_css_selector('#mobile')[0].text
    #
    time.sleep(1)
    cross = driver.find_element_by_id('popup_close').click()
    time.sleep(2)
    button2 = driver.find_elements_by_id('agent_phone')[1].click()
    time.sleep(2.5)
    phone2 = driver.find_elements_by_css_selector('#mobile')[0].text
    time.sleep(1)
    cross = driver.find_element_by_id('popup_close').click()
    time.sleep(2)   
    button3 = driver.find_elements_by_id('agent_phone')[2].click()
    time.sleep(2.5)
    phone3 = driver.find_elements_by_css_selector('#mobile')[0].text
    print(len(phone3))
    time.sleep(1)
    cross = driver.find_element_by_id('popup_close').click()
    time.sleep(2)
    button4 = driver.find_elements_by_id('agent_phone')[3].click()
    time.sleep(2.5)
    phone4 = driver.find_elements_by_css_selector('#mobile')[0].text
    time.sleep(1)
    cross = driver.find_element_by_id('popup_close').click()
    time.sleep(2)
    button5 = driver.find_elements_by_id('agent_phone')[4].click()
    time.sleep(2.5)
    phone5 = driver.find_elements_by_css_selector('#mobile')[0].text
    time.sleep(1)
    cross = driver.find_element_by_id('popup_close').click()
    time.sleep(2)
    button6 = driver.find_elements_by_id('agent_phone')[5].click()
    time.sleep(2.5)
    phone6 = driver.find_elements_by_css_selector('#mobile')[0].text
    time.sleep(1)
    cross = driver.find_element_by_id('popup_close').click()
    time.sleep(2)
    button7 = driver.find_elements_by_id('agent_phone')[6].click()
    time.sleep(2.5)
    phone7 = driver.find_elements_by_css_selector('#mobile')[0].text
    time.sleep(1)   
    cross = driver.find_element_by_id('popup_close').click()
    time.sleep(2)
    button8 = driver.find_elements_by_id('agent_phone')[7].click()
    time.sleep(2.5)
    phone8 = driver.find_elements_by_css_selector('#mobile')[0].text
    time.sleep(1)
    cross = driver.find_element_by_id('popup_close').click()
    time.sleep(2)
    button9 = driver.find_elements_by_id('agent_phone')[8].click()
    time.sleep(2.5)
    phone9 = driver.find_elements_by_css_selector('#mobile')[0].text
    time.sleep(1)
    cross = driver.find_element_by_id('popup_close').click()
    time.sleep(2)
    button10 = driver.find_elements_by_id('agent_phone')[9].click()
    time.sleep(2.5)
    phone10 = driver.find_elements_by_css_selector('#mobile')[0].text
    time.sleep(1)
    cross = driver.find_element_by_id('popup_close').click()
    with open(csv,'a') as s:
        for item in range(1):
            s.write(name1 + ',' + phone1 + 'n')
            s.write(name2 + ',' + phone2 + 'n')
            s.write(name3 + ',' + phone3 + 'n')
            s.write(name4 + ',' + phone4 + 'n')
            s.write(name5 + ',' + phone5 + 'n')
            s.write(name6 + ',' + phone6 + 'n')
            s.write(name7 + ',' + phone7 + 'n')
            s.write(name8 + ',' + phone8 + 'n')
            s.write(name9 + ',' + phone9 + 'n')
            s.write(name10 + ',' + phone10 + 'n')

Answer 1

您需要将显式硬编码等待替换为硒等待条件。如果您使用find_element_by则无需索引，并且有时电话号码针对不同的 id，因此您可以使用 css Or 语法和要匹配的 id 列表。浏览您的代码并切换这些硒等待。也许使用函数来管理给定元素的等待。

示例 使用 CSS 或等待

phone1 = WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.CSS_SELECTOR , '#mobile, #phone'))).text

和进口

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait 
from selenium.webdriver.support import expected_conditions as EC