Scrapy从选择器列表返回单个项

Question

我试图获得所有亚马逊畅销书的数据，并处理我使用的scrapy，我能够获得数据的整个选择器列表，但同时迭代数据列表的结果仍然只返回单个数据项。

def parse_page(self, response):
product_data = response.xpath("//div[@id='gridItemRoot']") #THIS RETURNS A SELECTOR LIST
for data in product_data:
product_name = data.xpath("//div[@class='a-section a-spacing-mini _cDEzb_noop_3Xbw5']//img/@alt").get()
product_rank = data.xpath("//span[@class='zg-bdg-text']/text()").get()

# It only generates a single result
yield {
"name": product_name,
"rank": product_rank
}

我尝试不迭代选择器列表，而是将选择器直接传递给方法并产生结果，但也返回单个元素。

def parse_page(self, response):


# in previous applications all the results were scraped without iterating over any selectorlist just like following
product_name = response.xpath("//div[@class='a-section a-spacing-mini _cDEzb_noop_3Xbw5']//img/@alt").get()
product_rank = response.xpath("//span[@class='zg-bdg-text']/text()").get()


yield {
"name": product_name,
"rank": product_rank
}

Answer 1

您需要使用相对xpath表达式。

def parse_page(self, response):
product_data = response.xpath("//div[@id='gridItemRoot']") #THIS RETURNS A SELECTOR LIST
for data in product_data:
product_name = data.xpath(".//div[@class='a-section a-spacing-mini _cDEzb_noop_3Xbw5']//img/@alt").get()
product_rank = data.xpath(".//span[@class='zg-bdg-text']/text()").get()

# It only generates a single result
yield {
"name": product_name,
"rank": product_rank
}

如果在xpath表达式的开头没有.，它将始终获取相对于根元素的第一个匹配，对于每次迭代，根元素始终是相同的元素。