Scrapy没有从json中识别密钥



我正试图从网站上抓取与圣经评论有关的信息。以下是我为此编写的代码。start_urls是我试图抓取的json文件的链接。我选择了['0']['father']['_id']来获取评论者的名称,但是,出现了以下错误。我该怎么办?

错误:TypeError: list indices must be integers or slices, not str

代码:

import scrapy
import json
class catenaspider(scrapy.Spider): #spider to crawl the url
name = 'commentary' #name to be called in command terminal
start_urls = ['https://api.catenabible.com:8080/anc_com/c/mt/1/1?tags=[%22ALL%22]&sort=def']
def parse(self,response):
data = json.loads(response.body)
yield from data['0']['father']['_id']```

再次阅读文档。

import scrapy

class catenaspider(scrapy.Spider):  # spider to crawl the url
name = 'commentary' # name to be called in command terminal
start_urls = ['https://api.catenabible.com:8080/anc_com/c/mt/1/1?tags=[%22ALL%22]&sort=def']
def parse(self, response):
data = response.json()
yield {'id_father': data[0]['father']['_id']}
# if you want to get all the id's
# for d in data:
#     yield {'id_father': d['father']['_id']}

最新更新