如何使用selenium从网站上抓取评分和所有评论

Question

我想在页面上抓取评分和所有评论。但找不到路径。

enter code here
import urllib.request
from bs4 import BeautifulSoup
import csv
import os
from selenium import webdriver
from selenium.webdriver.support.select import Select
from selenium.webdriver.common.keys import Keys
import pandas as pd
import time
chrome_path =r'C:/Users/91940/AppData/Local/Programs/Python/Python39/Scripts/chromedriver.exe'
driver = webdriver.Chrome(executable_path=chrome_path)
driver.implicitly_wait(10)
driver.get("https://www.lazada.sg/products/samsung-galaxy-watch3-bt-45mm-titanium-i1156462257- 
s4537770883.html?search=1&freeshipping=1")
product_name = driver.find_element_by_xpath('//*[@id="module_product_title_1"]/div/div/h1')
print(product_name.text)
rating = driver.find_element_by_xpath("//span[@class='score-average']")
print(rate.text)
review = driver .find_element_by_xpath('//* 
[@id="module_product_review"]/div/div/div[3]/div[1]/div[1]')
print(review.text)

Answer 1

我相信print(product_name.text)的执行是正确的，对吧？

driver.find_element_by_xpath("//span[@class='score-average']")有问题，我在HTML源代码的任何位置都找不到score-average。

所以试试这个：

driver.find_element_by_css_selector("div.pdp-review-summary")
print(rate.text)

您可以尝试以下代码来获得审查：

wait = WebDriverWait(driver, 10)
driver.get("https://www.lazada.sg/products/samsung-galaxy-watch3-bt-45mm-titanium-i1156462257- s4537770883.html?search=1&freeshipping=1")
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[class$='pdp-review-summary__link']"))).click()
ActionChains(driver).move_to_element(wait.until(EC.visibility_of_element_located((By.XPATH, "//h2[contains(text(), 'Ratings & Reviews')]")))).perform()
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.item-content")))
for review in driver.find_elements(By.CSS_SELECTOR, "div.item-content"):
print(review.get_attribute('innerHTML'))

进口：

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains

Answer 2

也许您的路径有问题？(很抱歉，我没有在窗户上测试(。在内存中，Windows路径使用字符，而不是/字符。此外，您可能需要在驱动器路径(C:\(之后添加两个backtick。

c:\Users91940AppDataLocal...