"AttributeError: 'SelectorList' object has no attribute 'get'"在刮云中

Question

我正在使用Scrapy设置一个刮板，在我的笔记本电脑上运行良好。但是当我在刮擦云上尝试同样的蜘蛛时，这条消息就会出现：

File "/usr/local/lib/python2.7/site-packages/scrapy/spidermiddlewares/depth.py", line 58, in <genexpr>
    return (r for r in result or () if _filter(r))
  File "/tmp/unpacked-eggs/__main__.egg/ccv_spiders/spiders/gmitem.py", line 31, in parse
    data["marque"] = caritem.css("div.make::text").get().strip().split(" ", 2)[1]
AttributeError: 'SelectorList' object has no attribute 'get'

这是我的代码：

def start_requests(self):
        for item in self.data:
            request = scrapy.Request(item['gm_url'], callback=self.parse)
            request.meta['item'] = item
            yield request
    def parse(self, response):
        item = response.meta['item']
        item['results'] = []
        for caritem in response.css("div.car-item-border"):
            data = AuctionItem()
            urllot = "https://www.website.com/img/auctions/byLot/"
            urlbase = "https://www.website.com/img/auctions/car/thumb/"
            data["marque"] = caritem.css("div.make::text").get().strip().split(" ", 2)[1]
            data["model"] = caritem.css("div.make::text").get().strip().split(" ", 2)[2]
            data["model_year"] = caritem.css("div.make::text").get().strip().split(" ", 1)[0]
            data["price_str"] = caritem.css("div.price::text").get().strip().replace(",", " ")
            if caritem.css("div.price::text").get().find("Estimate"):
                data["sold"] = True
            else:
                data["sold"] = False
            data["auction_house"] = caritem.css("div.auctionHouse::text").get().split("-", 1)[0].strip()
            data["auction_country"] = caritem.css("div.auctionHouse::text").get().rsplit(",", 1)[1].strip()
            data["auction_date"] = caritem.css("div.date::text").get().replace(",", "").strip()
            if caritem.css("div.view-auction a::attr(href)").get().find("/auction-cars/show-backup-image"):
                data["auction_url"] = caritem.css("div.view-auction a::attr(href)").get()
            else:
                data["auction_url"] = None
            data["image_urls"] = caritem.css("img.img-responsive::attr(src)").get()
            if urllot in data["image_urls"]:
                data["image_cloud"] = caritem.css("img.img-responsive::attr(src)").get().replace(urllot,"https://res.cloudinary.com/ccv/image/upload/auctions/")
                data["image_cloud"] = re.sub(r"(?<=[A-Z])/*(?=d)", "-", data["image_cloud"])
            elif urlbase in data["image_urls"]:
                data["image_cloud"] = caritem.css("img.img-responsive::attr(src)").get().replace(urlbase, "https://res.cloudinary.com/ccv/image/upload/auctions/")
            item['results'].append(data)
        yield item

我的 Python 版本有任何问题吗？它适用于我的笔记本电脑上的Anaconda和Python 3，我不明白为什么它似乎通过"/usr/local/lib/python2.7..."使用python 2.7。

此外，我的 JSON 输出没有显示任何结果的数组。

Answer 1

这完全与库的版本有关。

get和getall方法都是由Parcel(Scrapy的解析库(在1.2.0版本中首次引入的，如果您使用的是Scrapy 1.5.2或更低版本，则不会授予该版本。

您可以使用extract_first和extract作为替代品，或将Scrapy升级到1.6+。