刮擦请求优先级循环

Question

for each in player_url:
yield scrapy.Request(each, callback=self.player_data2, priority=1)
yield scrapy.Request(match_data.get('table_url'), callback=self.parse_table_url)

这是行不通的

如果不完成 forloop，则调用下一个请求

Answer 1

您的代码正在 for 循环中生成具有相同优先级(优先级 1(的scrapy.Request。如果您希望首先处理第一个请求，请分配不同的优先级。

检查以下内容：

priority = -1
for each in player_url:
yield scrapy.Request(each, callback=self.player_data2, priority=priority)
priority -= 1 # so that next request gets lesser priority
yield scrapy.Request(match_data.get('table_url'), callback=self.parse_table_url, priority=priority) # Assiging priority to this also so that this gets executed at last

负数可用于显示较低的优先级。 希望这有帮助！