所以我用django和mongoengine设置了这个模型。
class Product(Document):
product_id = IntField()
title = StringField(max_length=255)
sources = ListField(ReferenceField(Source, dbref = True))
class Source(Document):
source_id = IntField()
source_type = StringField(choices=settings.PARENT_TYPE_CHOICES, max_length=50)
name = StringField(max_length=255)
url = URLField(max_length=2000)
meta = {"allow_inheritance": True}
在我的管道中,我保存了以下数据:
class SaveItemPipeline(object):
def process_item(self, item, spider):
product = item["product"]
product["sources"] = self.create_sources(product)
saved_product,created = Product.objects.get_or_create(**product)
return item
def create_sources(self,product):
temp_sources = []
for source in product["sources"]:
print source
if source["source_type"] == "user":
temp_source,created = UserSource.objects.get_or_create(**source)
elif source["source_type"] == "store":
temp_source,created = StoreSource.objects.get_or_create(**source)
elif source["source_type"] == "collection":
temp_source,created = CollectionSource.objects.get_or_create(**source)
temp_sources.append(temp_source.id)
return temp_sources
Howerver,当我运行刮板时,在保存时它给了我这个错误:
引发验证错误(消息, 错误=错误, field_name=field_name) mongoengine.errors.ValidationError: [ObjectId('55787a07516ddcf4d93cd4c6'), ObjectId('55787b07516ddcf5aff06fa9'), ObjectId('55787b07516ddcf5aff06faa')] 不是有效的 ObjectId
顺便说一下,用户来源和存储来源...都从源代码继承,所以它们只是子类。但是,我在这里做错了什么,我不明白为什么在创建产品时会给我这个错误。
谢谢!
你可以使用这个
class Source(Document):
source_id = IntField()
class Product(Document):
sources = ListField(ReferenceField(Source, dbref = True))
src, created = Source.objects.create(source_id=1)
pd, _ = Product.objects.create(sources=[src])
它对我有用。我正在使用 mongoengine 0.8.7,pymongo 2.8